e-Patient Training Topic: National Article Reports Relative Risk, Not Raw Data.

Important update: it turns out the writer did get it right, and this was an editing error at the Boston Globe. See my comment August 17.

—–

As empowered, engaged patients we have a responsibility to evaluate the articles we read. A case in point is this week’s Associated Press article Any Spread Of Breast Cancer Raises Risk Of Return. It’s about a paper in this week’s New England Journal of Medicine that found a 50% increase in the chance of cancer recurring, and quotes an expert as saying

“I think it will influence treatment. If we’re considering treating the patient, we probably should.”

Can you spot the flaw in that reasoning, if all you have is that number? That’s all the article reported, and that’s a problem.

The problem is relative risk reduction. As we discussed last November in Making sense of health statistics, relative risk is an unsound basis for policy decisions, because you can’t tell the actual size of the impact. That affects you as a patient and you as a citizen thinking about policy.

In the study, did the 50% change mean represent a change from 200 patients per thousand to 300 per thousand, or from 2 per thousand to 3 per thousand? (Specifically, if we spend a zillion dollars based on this result, will we affect 1/1oth of the US population, or 1/1,000th?

Think about it for your own body. Your health provider suggests a treatment and says it’ll reduce your chance of a problem by 35%. (That’s the number commonly cited for statins like Lipitor.) Sounds impressive, right? But that’s a relative risk number. The raw data showed that among the general population, 1.59% had a coronary event, and among Lipitor users, 1.02% had one anyway. So for the $25 billion we spend on Lipitor in the US, we affect one patient in 200.

So now imagine your provider says “There’s a one in 200 chance this prescription will make a difference for you. Want it?”

I think there are two levels of concern for us as patients engaged in the new world of healthcare:

  • For you personally, ask your providers for the absolute numbers. They may need to go digging, because they themselves may not have been given the numbers. But you have every right to know, eh?
  • At a national level, the same thinking must be applied to policy decisions.

I wrote a note to the author of the AP article asking if she can get the absolute numbers. (As usual, the original article in NEJM isn’t open for us to evaluate.)

Related resources:

  • Health News Review – a superb resource for understanding how to evaluate health news. (Thanks to Ted Eytan for the tip last year.)
  • Evidence-based medicine – my post last November citing the Lipitor numbers, as reported in Andy Kessler’s book The End of Medicine.
Print

Posted in: e-pts resources | general | news & gossip | policy issues | understanding statistics

 

 

Comments

6 Responses to “e-Patient Training Topic: National Article Reports Relative Risk, Not Raw Data.”

  1. ePatientDave says:

    e-Patient training topic: national article reports relative risk, not raw data. http://is.gd/2gHKn

  2. Lodewijk Bos says:

    Once again, you are right on spot!
    Lodewijk

  3. ePatientDave says:

    e-Patient Training Topic: National Article Reports Relative Risk, Not Raw Data. http://is.gd/2gHKn <=impacts ppl & policies

  4. Apologies for the odd background formatting on this comment – something went weird when we upgraded recently. We’ll investigate.
    ————
    Important update: AP writer Marilynn Marchione did get it right, reporting both absolute and relative numbers:

    Among women in the study who were given no additional treatment, 86 percent of those with no cancer in lymph nodes were free of cancer five years later. Only 76 percent of those with micro tumors and 77 percent of those with isolated cancer cells were cancer-free.”

    That’s perfect: in my question above (“does this mean a change from 200 per thousand to 300 per thousand, or from 2 to 3?”) the answer is:

    • It’s the difference between 14 per hundred and 23-24 per hundred.
    • So in a population of 1,000 patients, we’re talking about 90-100 additional recurrences.

    To me that’s substantial. But whatever your view, the absolute numbers let you make an informed decision. But when the Globe edited the story, they cut out the good stuff and only printed the relative numbers. (Here’s one site that ran the fulll article as written: http://www.wtop.com/?nid=106&sid=1738698)

    Kudos and brickbats:

    • Kudos to Marchione for doing it right
    • Brickbat to me for not checking the original source! (Another lesson for e-patients)
    • A brickbat to the Globe, apparently, for editing out the good stuff. :–)

    Next assignment for e-patients in training: What would your next step be?

    • I apologized to Marchione and thanked her for getting it right.
    • Made note to self: “glad I wrote to her respectfully!”
    • Started getting in touch with the Globe’s editors, to go over the same stuff
    • Updated my erroneous blog post
    • Tweeted it to my community.
  5. Ben says:

    An excellent point! Many people fundamentally misunderstand statistics… Saying that a drug reduced by 35% a certain symptom in a study does NOT mean it will necessarily do so in any INDIVIDUAL.

    I liken it to the coin toss experiment. Most patients have a sample size of one (themselves), and on a single coin toss the statistical chances of heads/tails is 50%/50% BY DEFINITION. You need to perform more experiments to see if the coin is weighted and even then you can only use your findings to speak about the group of experiments, not each individual one… :)

  6. Update a week later: I did write to the Globe’s science & health editor with the same info I sent to the AP writer – haven’t heard anything back.

Leave a Reply