You will meet too much false precision

Precise numbers and claims - as though there is no margin for error - are all around us. When someone tells you that 54.3% of people with some disease will have a particular outcome, they're basically predicting the future of all groups of people based on what happened to another group of people in the past. Well, what are the chances of exactly that always happening, eh?

If our fortune teller was quoting the mean of a study here, it could be written like this: roughly 67.5% (95% CI: 62%-73%). The CI stands for "confidence interval" and it gives you an idea of how much imprecision or uncertainty there is around the estimate. The confidence level - 95% here, which is common - is chosen when a confidence is calculated. The 95% level means the significance level is at 0.05 (or 5%) - more about that here.

The chances of the result always being precisely 67.5% can be pretty slim or very high, depending on lots of things. If there is a lot of data, the confidence interval would be narrow: say, 66% to 69%.

We give ranges for estimates all the time. If someone asks, "How long does it take to get to your house?", we don't say "39.35 minutes". We say, "Usually about half an hour to 45 minutes, depending on the traffic." There's still a very small chance you could make it in 25 minutes – or longer than 45 minutes.

In a systematic review, you will often see an outcome of an individual study shown as a line. The length of that line is showing you the width of the confidence interval around the result. It looks something like this:

This is called a forest plot. Find more from Statistically Funny on this in The Forest Plot Trilogy.

What a confidence interval isn't: it doesn't mean people's outcomes will definitely be between those upper and lower boundaries. It's where the mean is expected to be likely to fall (or median, or whatever other statistic is being measured).

If the statistical estimates are made with Bayesian methods, the range you will see around an estimate isn't a confidence interval: it's a credible interval. I explain a bit about Bayesian statistics in this post. Unlike a confidence interval, a credible interval has incorporated extra data about the probability of the result falling inside the interval.

Update [4 June 2016]: The American Statistical Association (ASA) issued a statement encouraging people to consider estimates like confidence intervals instead of only looking at p-values and statistical significance. I've written an explanation of that in this post: 5 Tips for Avoiding P-Value Potholes.

Heaven's Department of Epidemiology

Watch out for risk's magnifying glass - and cut your risk of being tripped up by 82%!

Whenever you see something tripling - or halving - a risk, take a moment before you let the fear or optimism sink in.

Relative risks are critically important statistics. They help us work out how much we might benefit (or be harmed) by something. But it all depends on knowing your baseline risk - your risks to start with.

If my risk is tiny, then even tripling or halving it is only going to make a minuscule difference: a half of 0.01% isn't usually a shift I'd even notice. Whereas if my risk is 20%, tripling or halving could be a very big deal. Unless you know a great deal about the risks in question - or your own baseline risk, you need more than a relative risk to make any sense out of data.

There's a good introduction to absolute and relative risks at Smart Health Choices. 

This is one of the 5 shortcuts to keep data on risks in perspective, at Absolutely Maybe.

Cartoon and content updated on 3 June 2017: This post was originally the cartoon only, from my blog post for the British Journal of Sports Medicine.