Kartik Kant / All lessons Chapter 5 · Finding your first users
Lesson 5.3

Which numbers are worth watching

Once people start using your app you can measure almost anything, which is exactly the problem. Most dashboards are full of numbers that feel informative and change nothing.

A small number of measurements tell you whether this is working. The rest are decoration, and some of them actively mislead.

The test for whether a number is worth having

One question: if this number changed, would you do anything differently?

If yes, it's worth watching. If no, it's entertainment.

Total signups since launch is entertainment. It only goes up, including when everyone who ever joined has left, so it can rise while your app quietly dies. Total downloads is the same. So is follower count, mostly.

The numbers that pass are ones where a change tells you something is wrong and roughly where.

The handful worth having early

Five things cover most of what you need before you're big.

How many people arrive, and where from. Not a vanity count, but broken down by source, so you know which of your channels is doing anything.

What fraction reach the point where your app becomes useful. Back in lesson 3.2 you named that moment. This measures whether your first minute works, and it's usually lower than people expect.

How many come back. The single most important one, and it gets its own section below.

What fraction pay, if you charge.

And how many leave, meaning cancel or simply stop appearing.

That's enough to know whether the thing is working and which part is broken when it isn't.

Look at groups, not totals

The single most useful habit here, and it's the difference between seeing reality and being fooled by it.

Say you have 500 active users this month, up from 400. Looks like growth.

It might be. Or you might have gained 300 and lost 200, meaning you're leaking badly and hiding it with new arrivals. The total can't tell you which.

The fix is to group people by when they joined, and follow each group over time. Everyone who signed up in January, then how many of those were still around in February, in March. Then do the same for February's group.

That view shows the truth immediately. If each group flattens out at some level above zero, you have something real, a core of people who genuinely stuck. If each group slides toward nothing, you're refilling a leaking bucket, and more marketing just pours water in faster.

Most analytics tools do this for you and call it cohort retention. It's worth learning to read that one chart properly, more than any other.

Steer by the early ones

Some numbers tell you what already happened. Revenue, cancellations. True, and too late to act on.

Others tell you what's coming. Whether new people are reaching the useful moment. Whether week-one retention is drifting down. Those you can still do something about.

Watch both, but make decisions on the early ones.

Numbers and people together

Analytics tell you what happened and never why.

You'll see that 60 percent of people drop off on one screen. That's the what. The why comes from watching someone use it, or asking a few people who left.

The pairing is what works: the number tells you where to look, the conversation tells you what to fix. Either alone leads somewhere unhelpful. Numbers alone produce confident changes to the wrong thing.

A quick story. My app was built entirely on a retention bet. Ninety days of daily use, streaks, progress you could see. The whole idea was that people would come back because they could watch themselves change.

I never wired up analytics. Not a single event.

So the product's central claim, the one thing the entire design existed to deliver, had no measurement attached to it whatsoever. Even if I had launched, I couldn't have told you whether it worked.

The thing about analytics is that you can't add them retroactively. Data you didn't collect is gone. Half an hour before launch would have covered it, and I didn't spend it because it was less interesting than building.

If you want to set up measurement

Work this chapter with Claudea prompt to run on your own project
I'm launching an app and I want to measure the right things without drowning in dashboards. I'm new to this, so keep it simple. What my app does: [describe it] The moment it becomes useful to someone: [describe it] Whether I charge: [yes and how, or no] Please: 1. Tell me the five or six events I should record, and no more. 2. Recommend a simple, privacy-friendly analytics tool and roughly how to add it. 3. Explain how to look at retention by group rather than as a total, and what a healthy chart looks like versus a leaking one. 4. Tell me which numbers I'll be tempted to celebrate that don't actually mean anything.
Open Claude ↗

That last one is worth asking. Vanity metrics are seductive precisely because they always look like progress.


Next: Why people leave →

Go deeper: Chapter 49, Metrics That Matter

Useful? Share this lesson