Vanity Metrics vs. Actionable Metrics: How to Tell the Difference
Vanity metrics feel like progress and produce none. Here is a practical test for telling the difference — and a reference table of the most common offenders.
The term "vanity metric" was popularised by Eric Ries in The Lean Startup, but the concept predates the book by decades: it describes any metric that makes you feel good about your business without helping you understand it or improve it. Page views. Total downloads. Gross signups. Press mentions. Twitter impressions. These numbers go up, and going up feels like winning, and that feeling is precisely the danger — because the effort spent celebrating and reporting vanity metrics is effort not spent on the numbers that actually drive decisions.
The distinction between vanity and actionable is not always obvious, and some metrics genuinely belong in both categories depending on context. What follows is a framework for making the call, and a reference table covering the metrics most commonly debated.
The three-question test
Apply these three questions to any metric you are considering tracking. If it fails more than one, it is almost certainly a vanity metric.
Question 1: Can this metric go down? A metric that only ever increases — total registered users, cumulative page views, all-time downloads — contains no information about whether things are getting better or worse right now. Actionable metrics can go in either direction. The ability to decline is what makes a metric meaningful as a signal.
Question 2: Does a change in this metric prompt a specific action? If this metric dropped 20% next month, would you know what to do differently? If the answer is "we'd investigate further," the metric is too high-level to be directly actionable. If the answer is "we'd increase outbound sales calls" or "we'd fix the onboarding flow at step three," it is actionable.
Question 3: Is this metric hard to inflate without improving the underlying reality? Total registered users can be inflated by buying email lists or launching on Product Hunt. Weekly active users cannot be inflated without genuinely bringing users back to the product. Metrics that are easy to inflate without creating real value are vanity metrics by design.
A vanity metric is not a metric that looks good. It is a metric that makes you feel like you are improving when you might not be. The feeling is the trap.
The reference table
| Metric | Verdict | Why |
|---|---|---|
| Total registered users | Vanity | Only goes up. Tells you nothing about engagement, value delivered, or whether the product is working. Replace with monthly active users or activation rate. |
| Monthly active users (MAU) | Actionable | Can go up or down. Reflects whether users are returning — the prerequisite for retention and revenue. Particularly powerful when tracked by cohort. |
| Page views / website sessions | Vanity | Can be inflated by bot traffic, social spikes, or any paid spend. Without a conversion rate attached, it describes attention, not traction. Use traffic-to-trial conversion rate instead. |
| Trial-to-paid conversion rate | Actionable | Measures whether your acquisition funnel is working. A drop triggers specific investigation into onboarding, pricing, or messaging. Hard to inflate without improving actual conversion. |
| Social media followers | Vanity | Follower count can be bought, follows a power law unrelated to business quality, and is a poor predictor of any downstream business outcome. Track referral rate or share of voice in qualified communities instead. |
| Email list size | Depends | Vanity if unqualified and unengaged. Actionable if segmented by engagement level and correlated with downstream conversion. List size matters only when open rates and click rates are healthy. |
| Net Promoter Score (NPS) | Depends | Actionable when tracked consistently over time with segment-level breakdowns and a follow-up process for detractors. Vanity when surveyed once, celebrated, and filed away. The score is less valuable than the trend and the qualitative follow-through. |
| Revenue (gross) | Depends | Actionable when broken down by cohort, channel, and segment. Potentially misleading when presented as a single blended number that masks high churn behind strong new-customer acquisition. Always pair with net revenue retention. |
| Press mentions / media coverage | Vanity | Rarely correlated with business outcomes at the early stage. Measure whether press drives trials or signups — if it does not, it belongs in the marketing archives, not the business dashboard. |
| Day-30 retention rate | Actionable | Directly measures whether new users found enough value to remain active one month after signup. One of the most honest leading indicators of long-term retention and LTV available to early-stage teams. |
| App store rating | Vanity | Reflects a biased, self-selected sample of your most and least satisfied users. Useful for qualitative feedback; useless as a business metric. Track Day-30 retention and churn rate to measure true product satisfaction. |
| Customer Acquisition Cost (CAC) | Actionable | Directly actionable — a rise triggers investigation into channel efficiency, conversion rate, and people costs. Meaningful only when calculated with full cost inclusion (people + ad spend + tools) and tracked by channel, not just blended. |
The "depends" category deserves attention
Several metrics in the table — NPS, revenue, email list size — sit in a "depends" category that is worth unpacking. These are metrics that are actionable in one context and vanity in another. The difference is almost always about how they are tracked, segmented, and followed up on, not about the metric itself.
NPS is the clearest example. A single blended NPS number, surveyed annually and presented in a board deck, is a vanity metric. It tells you something (a score), it feels meaningful (a number above 40 is "good"), but it does not tell you which customers are detractors, why they are detractors, or what you should do to convert them. NPS tracked monthly by customer segment, with a consistent follow-up process for every detractor response, is an actionable metric. The same number, entirely different utility.
Almost any metric becomes more actionable when it is segmented. Blended revenue broken down by acquisition channel, customer segment, and contract size generates entirely different insights from the blended total. The work of making a metric actionable is often the work of breaking it into the segments where the variation lives — because the variation is where the decision-making opportunity is.
The organizational cost of vanity metrics
Vanity metrics are not harmless. They take time to produce, time to report, and — most damagingly — they give teams permission to feel good about work that may not be advancing the business. A team that celebrates 100,000 total registered users is a team that is not asking why only 4,000 of them are active. That question, unasked, is the difference between a product that solves the retention problem in Month 6 and one that discovers it in Month 18 when the growth trajectory has already been written.
The practical fix is not to ban all high-level metrics from the dashboard — it is to pair every easy-to-celebrate metric with the harder metric that provides the context it lacks. Total registered users, paired with monthly active users and activation rate. Website traffic, paired with trial conversion rate. Gross revenue, paired with net revenue retention. Each pairing creates the tension that prevents the comfortable metric from producing complacency.
Some investors, particularly at the earliest stages, will ask for total registered users, app downloads, or social media following as proof of traction. It is worth understanding why: these metrics are easy to compare across companies and stages, and investors without deep domain knowledge use them as rough proxies. Provide them when asked — but always lead with the actionable metrics that tell the real story. A founder who explains "we have 8,000 registered users, 3,400 of whom are monthly active, with 72% Day-30 retention" demonstrates a level of operational clarity that a simple user count cannot convey.
The shift from vanity to actionable metrics is as much a cultural habit as a technical one. In FabricLoop, teams that have made the shift often maintain a single, shared note that lists the official metrics — the ones that appear in the weekly review, the ones that have owners, the ones that generate tasks when they move in the wrong direction. That list acts as a filter: when someone proposes adding a new metric to the review, the question "does this pass the three-question test?" has a shared reference point. The list does not have to be long to be powerful — often six to eight metrics, consistently tracked, is enough to keep a growing team pointed in the right direction.