Lies, Damn Lies And Statistics: Collective Statistical Illiteracy

Everyone knows the supposed origin of the phrase. But as you can see here it goes back to Medicine:

“Look at the dozens of operations by me this year without a death,” says the operator. His less enthusiastic neighbor thinks of the proverbial kinds of falsehoods, “lies, damned lies and statistics” and replies “reports of large number of cases subjected to operations seldom fail to beget a suspicion of unjustifiable risk”.

In “Some Surgical Sins”, John B. Robert A.M,  M.D.,  Chairman’s Address on Surgery and Anatomy, 45th Meeting of the AMA, June 1894.

It looks like not much has changed since then.

A new international report shows how little statistical literacy doctors have. It connects directly to rational ignorance, a topic I wrote about not long ago. The new report demonstrates that many doctors, patients, journalists, and politicians alike do not understand what health statistics mean or draw wrong conclusions without always noticing.

Collective statistical illiteracy refers to the widespread inability to understand the meaning of numbers. [Note: Numeracy is a prerequisite to being statistically literate.]

The report comes just in time to be read before you can turn to “Its Great! Oops, No It Isn’t: Why Clinical Research Can’t Guarantee The Right Medical Answers”, a new book by Ronald Gauch, dissecting medical research methodology and explaining why correct answers in clinical research are so hard to achieve. In the book the author challenges the notion that medical research is too complex for the average citizen to comprehend. He is right. But that comprehension first requires statistical fluency!

The report provides evidence that statistical illiteracy:

  • is common to patients, journalists, and physicians;
  • is created by the nontransparent framing of the statistical information, either because of lack of understanding or intentionally to manipulate or persuade people; and
  • can have serious consequences for health.

As e-Patient Dave wrote yesterday the report is directly relevant to e-patients and shows how statistical illiteracy of the public perpetuates the old paradigm of medicine:

The causes of statistical illiteracy should not be attributed to cognitive biases alone, but to the emotional nature of the doctor–patient relationship and conflicts of interest in the healthcare system. The classic doctor–patient relation is based on (the physician’s) paternalism and (the patient’s) trust in authority, which make statistical literacy seem unnecessary; so does the traditional combination of determinism (physicians who seek causes, not chances) and the illusion of certainty (patients who seek certainty when there is none).

The authors show that information pamphlets, Web sites, leaflets distributed to doctors by the pharmaceutical industry, and even medical journals often report evidence in nontransparent forms that suggest big benefits of featured interventions and small harms. Because of  statistical illiteracy the public is susceptible to political and commercial manipulation of their anxieties and hopes, which undermines the goals of informed consent and shared decision making.

What can be done? The authors advocate for teaching, starting in primary and secondary education and continuing in medical school, the methods of statistical thinking and transparent representations. This early teaching first requires familiarizing children with the concept of probability. It also requires transforming the meaning of statistical literacy as the art of solving real-world problems. The authors also note that a major precondition for statistical literacy is transparent risk communication. To change the current situation the authors recommend using:

They also note that

Statistical literacy is a necessary precondition for an educated citizenship in a technological democracy. Understanding risks and asking critical questions can also shape the emotional climate in a society so that hopes and anxieties are no longer as easily manipulated from outside and citizens can develop a better-informed and more relaxed attitude toward their health.”

This last point is worth expanding. It doesn’t apply just to health statistics. For example the NetGen people have had a completely different experience than their parents had with playing outdoors. Thanks to the abuse of crime statistics by many media, a large percentage of parents (the Baby Boomers) have kept their children indoors due to the perceived risk of letting their children play freely outside.  In turn, this has helped shape the NetGeners who are completely used to communicate with others from their computers, in a perceived safe environment. Since statistics are used all the time, here is a good example of clear statistics, using all the prescribed tools from the report:

As of 2006, it is estimated that at least 9.25 million people are currently imprisoned worldwide.

In absolute terms, the USA currently has the largest inmate population in the world, with more than 2½ million or more than 1 in 100 adults in prison and jails. Although the United States represents about 3% of the world’s population, over 25% of the people incarcerated around the world are housed in the American prison system. Pulitzer Prize winning author Joseph T. Hallinan wrote in his book Going Up the River: Travels in a Prison Nation, “so common is the prison experience that the federal government predicts 1 in 11 men will be incarcerated in his lifetime, 1 in 4 if he is black.”

Prison population per 100,000 inhabitants:

USA: 756                                    France: 85

I wish medical statistics were always presented in such a concise and clear manner.

Print

Posted in: general | understanding statistics

 

 

Comments

10 Responses to “Lies, Damn Lies And Statistics: Collective Statistical Illiteracy”

  1. [...] Read Gilles Frydman’s entry: Lies, Damn Lies And Statistics: Collective Statistical Illiteracy [...]

  2. Sarah Greene says:

    Statistics are well-reported in Tara Parker-Pope’s Well column on Tuesday, wrt to the Jupiter study on statins (http://www.nytimes.com/2008/11/18/health/18well.html?ref=health). Quoting:

    “Only 1.8 percent of the subjects who took a placebo had a major cardiovascular problem during the study period. Among statin users, 0.9 percent did. In other words, the absolute risk of a serious cardiovascular problem (as opposed to the relative risk) was reduced by less than one percentage point.

    “Absolute differences in risk are more clinically important than relative reductions in risk in deciding whether to recommend drug therapy,” The New England Journal of Medicine noted in an editorial accompanying a report on the study.

    An important indicator of the usefulness of a drug is the “number needed to treat,” a measure of how many people needed to take a pill for just one person to be helped. There is disagreement about what Jupiter showed. The New England Journal editorial concluded that treating 120 people for about two years would help one person. The study authors, using different criteria, came up with a figure of 95.

    Some researchers think the number is actually much lower. Extrapolating the data to five years, the study’s authors concluded that just 25 healthy people would need to take a statin to prevent one serious heart problem.”

  3. [...] the study results were unclear and another example of statistical illiteracy [...]

  4. [...] issue is of course the general lack of statistical literacy, as we wrote in the e-patients blog here and there. As a report said “Statistical literacy is a necessary precondition for [...]

  5. [...] while a poor font choice can ruin a meeting, a poor interpretation of statistics or data can kill you (or someone else, for that matter). Proper use of data is a viciously complicated topic, and I am [...]

  6. Joe McCarthy says:

    I highly recommend Head First Statistics, by Dawn Griffiths, as a valuable tool for making statistics more accessible and understandable to a broad audience.

  7. [...] Lies, Damn Lies And Statistics: Collective Statistical Illiteracy (Nov. 2008) [...]

  8. [...] issue is of course the general lack of statistical literacy, as we wrote in the e-patients blog here and there. As described [...]

Leave a Reply