Today, I am thrilled to have my husband guest posting! I may be biased, but I actually do think most people would agree that he is not only funny and jolly, but pretty brilliant as well. Anyone who's been reading this blog for any amount of time knows that John is a board certified internal medicine physician; what you may not know is that he also holds a MSIS degree and has a strong background in research, statistical methods, and general awesomeness. John blogs over at CKM Beat.
100% of Men Reading This Post Are Male
and Other Ridiculous Statistics
- 100% of Dead People Were Once Alive
- Buying Two Tickets Doubles Your Chance of Winning the Lottery
- 0% of Children Surveyed Were Over Age 18
- There's a 5% Chance the Truth is False
- 100% of Correlations Are Unproven
- 3 Out of 4 Children Like Macaroni and Cheese
Right there you have 6 absolutely ridiculous, perfectly true statements.
It's been said that there are lies, damn lies, and statistics. And now you see why. Statistics are incredibly powerful when used wisely. Unfortunately, they are rarely used wisely.
Proper use of statistics generally doesn't make headlines. No one cares when taking a medication results in an extra 1 in 10,000 people having a heart attack, but when the news reports that the medication "doubles the risk of a heart attack" everyone calls their doctor. The numbers of heart attacks haven't changed, but the statistics used to report the numbers dramatically affect how the populous responds to the information.
The six examples above illustrate 6 common ways statistics are abused.
- 100% of Dead People Were Once Alive: Here we have re-stated a simple truth, using a powerful number in statistics to inflate the importance of the information. Politicians may be the most common offender in this category.
- Buying Two Tickets Doubles Your Chance of Winning the Lottery: This statement is statistically true, but the assumptions on which it has been based are omitted. Certainly, two tickets increase your chances of winning the lottery, but your chances are still incredibly slim. This is a common tactic in sales and marketing, especially with up-selling.
- 0% of Children Surveyed Were Over Age 18: Here we have an example of how framing affects the way statistics are interpreted. This could just as easily have read "100% of Children are Under Age 18," but by framing it (in this case, inverting it), it makes the numbers inarguable, but makes a totally different point than the second statement. This is how advocacy groups can look at the same data as researchers and come up with totally different "conclusions" that are both still statistically accurate... they chose to frame the statistics in a different way than the researchers do.
- There's a 5% Chance the Truth is False: Here's one that most people just don't realize. The standard in research is that we can tell with 95% certainty that our conclusion is true. Therefore we say it is true with 95% confidence. Of course, this means that there is a 5% chance it's not true. All those conclusions you read in various publications? Well, there is, literally, a 5% chance that the "truth" is false.
- 100% of Correlations Are Unproven: It's extremely difficult in research to prove cause and effect; the majority of research actually studies the likelihood that two events are related. That's why most research reports a correlation, not a causation between two variables.
- 3 Out of 4 Children Like Macaroni and Cheese: Well, this is totally true based on the survey conducted in our family... 3 out of our 4 children do like macaroni and cheese. Of course, it's not fair that a small study of children selected for convenience's sake should be representative of all children. Yet the statement is still statistically true. This is a favorite of people with an agenda- ever notice how every toothpaste has 9 out of 10 dentists recommending it?
Any time you have a strong reaction to a statistic, the first thing you should do is question if the statistic was manipulated to incite that reaction. Researchers spend years developing their research methods only to have politicians, advocacy groups and the press manipulate their statistics for personal gain. The best way to gauge this is by looking at the original research rather than headlines or news articles. Even if you don't subscribe to the journal where the original research has been published, most university libraries will assist the public to find an article.
More importantly, any research that leads to meaningful conclusions will be able to be replicated, and further studies will back up the original claims... or not. Unexpected correlations are often found unintentionally in research studies; while they are often reported as a shocking! new! finding! the truth is that they need additional dedicated research before any conclusions can be drawn.
Finally, a recent study demonstrated that 100% of people that comment on my wife's blog are awesome. Who can argue with those statistics?