Measuring Fleet Health Beyond Vanity Traffic

When you run several websites, the easiest number to watch is traffic. Sessions and page views are visible in every analytics dashboard, they go up and down satisfyingly, and they make for simple comparisons between sites. They're also one of the least useful measures of whether a site is actually healthy.

At Taciturn Studios we try to look past raw traffic to a handful of signals that tell us whether each site is findable, useful, sustainable and worth the time it takes. This post describes what we watch and why.

The problem with vanity metrics

Eric Ries popularized the term "vanity metrics" in The Lean Startup (2011) to describe numbers that look good but don't help you make decisions. Total page views are a classic example. A spike might come from a single post shared somewhere unexpected, from bot traffic, or from visitors who leave in seconds. A slow decline might hide a growing core of loyal readers.

Traffic matters, but only in context. The more useful question is: Is this site doing its job for the people it's meant to serve, and is it sustainable for us to keep running?

Signal 1: Indexation and search health

For content sites, being findable through search is fundamental. We check Google Search Console regularly for each site, with attention to:

• Page indexing. How many pages are indexed versus excluded, and why. A sudden rise in pages marked "Crawled, currently not indexed" or "Duplicate" can signal quality or technical problems before traffic drops.
• Sitemap status. Whether the sitemap is being read successfully and matches the pages that actually exist.
• Soft 404s. Pages that return a normal "200 OK" status but look empty or missing to Google. A common cause is a site that serves a generic page for any URL, including ones that don't exist. Unknown URLs should return a real 404.
• Queries and impressions. Which searches show our pages, even when they don't get clicks. Impressions without clicks often point to titles or descriptions that don't match what people want.

These numbers are slower and less exciting than traffic, but they explain traffic. If indexing is healthy and impressions are growing, traffic usually follows.

Signal 2: Page experience

Speed and stability affect both readers and search. Google's Core Web Vitals measure loading (Largest Contentful Paint), responsiveness (Interaction to Next Paint, which replaced First Input Delay as a Core Web Vital in March 2024) and visual stability (Cumulative Layout Shift). Search Console's Core Web Vitals report shows how real visitors experience each site.

We don't chase perfect scores. We look for regressions: a site that used to be fast and isn't anymore, often because of a new script, a heavy image or a third-party widget.

Signal 3: Content quality, measured honestly

Some quality problems can be measured mechanically, and we do measure them:

• How many articles are genuinely substantial once navigation and boilerplate are removed.
• Whether any articles contain repeated paragraphs or padding.
• Broken internal links and pages returning errors.
• Articles that haven't been reviewed in a long time on topics where facts change.

These checks don't prove an article is good, but they flag ones that are clearly not. When we found filler on some of our own pages, these checks are what surfaced it. We describe how we avoid producing it in the first place in A Calm Approach to Shipping Content at Scale.

Signal 4: Conversion paths

Every site exists for a reason: to sell a book, collect email signups, offer a tool, or support itself through advertising. Instead of total traffic, we look at whether visitors complete the paths that matter:

• For book and author pages: clicks through to retailers, and signups for the reader list.
• For tool sites: completed uses of the tool, such as a calculation run or a result exported.
• For content sites: whether readers move to a second page, and whether returning visitors grow over time.

A site with modest traffic and a healthy conversion path is often more valuable than a popular one where almost nobody does anything.

Signal 5: Support load and maintenance cost

This is the signal most dashboards ignore. How much time does each site take to keep running? That includes reader emails, broken features, dependency updates, policy changes and content upkeep.

We keep rough notes on time spent per site. A site that needs constant attention for little return is a candidate for simplification or retirement, however good its traffic looks. We discuss that decision in When to Kill a Side Project Cleanly.

Signal 6: Compliance basics

For a fleet, small compliance problems multiply. We regularly verify that each site has accurate legal pages, correct analytics installation, a valid ads.txt file where advertising runs, and disclosures for affiliate links. These are pass-or-fail checks, and we treat them as such. Our approach to shared standards is laid out in Fleet Sites and Shared Standards Without Blandness.

Putting it together: a simple scorecard

Rather than one big dashboard, we use a scorecard per site with a few rows:

• Search health: indexing trend, sitemap status, soft 404s.
• Page experience: Core Web Vitals status.
• Content: count of substantial articles, flagged issues.
• Conversion: the one or two actions that matter for that site.
• Cost: approximate maintenance time.
• Compliance: legal pages, analytics, ads.txt, disclosures.

Each row gets a simple pass, watch or fail. The scorecard tells us where to spend the next hour, which is the only thing a metric really needs to do.

Why this matters beyond our studio

Authors and small publishers face the same trap. Social followers and page views are easy to watch and easy to feel good or bad about. The questions that actually guide decisions are harder: Are the right people finding you? Are they doing what you hoped? Is the effort sustainable? Our post on the indie author tools we actually use applies the same thinking to book marketing, and How Taciturn Studios Picks Which Products to Ship shows how these signals feed into decisions about what to build next.