How to Read a Statistic Without Getting Fooled

Imagine a headline that reads: "New study: people who eat breakfast are 40% less likely to be overweight." That number sounds solid. It has a decimal point's worth of confidence. But a careful reader does not swallow a statistic the way they swallow a cookie. They chew it first. Chewing a statistic means asking three questions before you decide what it means.

The first question is: who got counted? Every statistic comes from a sample — a group of people, animals, or things the researchers actually measured. The sample is almost never everyone. So you have to ask whether the sample looks like the larger group the headline is talking about. If the breakfast study only surveyed adults who go to the gym, the 40% figure tells you something about gym-goers, not about humans in general. When a sample does not represent the bigger group it claims to describe, we call that selection bias. Selection bias does not mean the researchers lied. It means the math is honest but the headline is wearing a costume.

The second question is: how was the thing measured? "Overweight" sounds obvious, but somebody had to draw the line. Did they weigh people on a scale, or did people report their own weight on a form? Self-reported numbers are famously shaky — people round down. "Eating breakfast" is even slipperier. Does a granola bar in the car count? Coffee with sugar? The definition the researchers used — called the operational definition — decides what the statistic is actually about. Two studies using different operational definitions of the same word can produce numbers that disagree, and neither one is wrong. They are answering different questions.

The third question is the one most people skip: compared to what? A risk that is 40% lower sounds dramatic until you ask what the original risk was. If 10 out of 1,000 non-breakfast-eaters became overweight in the study, then a 40% reduction means 6 out of 1,000 breakfast-eaters did. That is a real difference, but it is four people out of a thousand, not a transformation. The 40% is a relative risk — a comparison between two groups. The 4-in-1,000 gap is the absolute risk — the actual size of the change. News headlines almost always quote the relative number, because it sounds bigger. Both numbers are true. Only one of them tells you how much the finding should change your day.

Notice what these three questions are not. They are not a way to dismiss every statistic you dislike. A careful reader is not a cynic who decides numbers are meaningless. The point is the opposite: statistics are useful precisely because they can be examined. A number you have chewed on — checked the sample, checked the measurement, checked the comparison — is worth more than a number you swallowed whole, even if you end up believing the same thing either way. The work is what makes the belief earn its place.