Vanity metrics are the enemy

Oct 30, 2017

Vanity metrics are the enemy

It’s likely some metrics in your product analytics are useless. These’ll be the vanity metrics any stock analytics software spews out and which people readily regurgitate. Vanity metrics are surface-level data. They’re large numbers like downloads, page views, or sign ups. They don’t explain or demonstrate anything about the inner workings of your product. They’re almost universally applicable.

“Millions of pageviews so far this quarter,” cry content creators.

“100,000 sign ups this month,” acquisitions announce.

“We had thousands more downloads this week,” executives tell funders.

As a product person you know these vanity metrics are useless for product development. You know they don’t help you answer the pressing questions about why people do or don’t use your product. But people like vanity metrics because they’re easy to track and sound good when you share them. It’s true that vanity metrics have their place. They read well in sales copy and impress in board meetings. They’re good for building an audience and brokering partnerships. They just don’t explain anything for product people.

As a data-informed product person vanity metrics are the enemy.

Product metrics can’t afford to be surface-level

Metrics shouldn't be surface level
Metrics shouldn't be surface-level.

A guaranteed way of killing any product is using vanity metrics as a measure for progress. There might be thousands and thousands of user sign ups but how many people complete onboarding? After two weeks, how many people are still coming back to your product? How long do people use your product for at any one time?

At a minimum your analytics should be answering the following:

  • When did the user sign up for our product/service?
  • Have they made a purchase?
  • How many active minutes do they spend with us? (although this one is will depend on the nature of your product)
  • Where do they spend the most time? Where do they spend the least? What proxies predict this behavior?
  • How does their behavior change over time?
  • When or where do they stop using our product?

These are the important questions about actual usage that your metrics need to answer. These are the clarity metrics.

Clarity metrics expose the inner workings of your product. They’re often unique to your product (or a unique combination of metrics) and are the real data that drive growth. Companies and product teams adept at using data to inform product development come up with ‘success metrics’ or ‘growth insights’. These insights are made up from various clarity metrics.

Deep insight comes from deep metrics
Deep insight comes from clarity metrics.

A famous growth insight is Facebook’s ‘seven friends in 10 days’. Facebook discovered, through various interconnecting metrics, that people who have seven friends 10 days after signing up are more likely to continue using Facebook. This is what drove Facebook to 1 billion users.

Now I’m not saying actively tracking clarity metrics ends up with a magic growth insight for explosive product growth. Far from it. And there’s a lesson here about causation and correlation. But finding the clarity metrics unique to your product can give you insight like nothing else. Those insights can help you deeply understand the core reasons people do or don’t use your product. The right analytics tool to help you narrow your focus is essential.

Expose and fix sticking points

Clarity metrics help you pinpoint the sticky bits of your user experience. The data can help you solve those sticky problems and boost your acquisition and retention. I’m not saying you’ll reach 1 billion users. But you might just out do your competition who’re stuck staring at vanity metrics, ignoring all the people choosing your product over theirs.



Get more product goodness like this from Prodlytic