The discovery of practice variation: follow the data

It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.

Sherlock Holmes, in Scandal in Bohemia

I’ve been reading Jack Wennberg’s new book Tracking Medicine, which is about his lifetime of work in understanding the reality of how medicine is practiced, as a route to helping us achieve the best care possible for each of us. My first post about this was three months ago, en route to a seminar on SDM (shared decision making); my first post after the seminar was shortly after. The whole subject has bent my thinking about healthcare so severely that it’s taken me this long to decide what to say next.

Key findings:

  • Your doctors, with the best of intentions and the best of training, may unwittingly be prescribing treatments that aren’t necessary for you, or not prescribing things that are. (“You” includes anyone you’re caring for.)
  • This realization was developed not through people’s opinions but by looking at cold hard numbers. After controlling for all variables, the odds of a given patient getting a given treatment vary by hundreds of percent from region to region.
  • For whatever reason, decades of efforts to change this have been fruitless, so the risks from unnecessary treatments continue and inappropriate care continues.
  • Part of the reason is that we’re in denial (patients and providers alike), and part of the reason for that seems to be that the causes are unconscious. (See below.)
  • This doesn’t mean your doctors are incompetent – the forces at play seem to be universal. The problem is that virtually nobody realizes it’s happening – neither we as consumer/patients nor the physicians.
  • E-patient takeaway: If you want the best care for yourself and your family, do what you can to understand these issues and have empowered, engaged, participatory conversations with your providers.

Wennberg’s method was to crawl through databases. From the start of his book:

Early in my career, I was hired as director of a federally sponsored program whose goal was to ensure that all Vermonters had access to recent advances in the treatment of heart disease, cancer, and stroke.… As the results came in, however, rather than evidence for underuse… we found extensive and seemingly inexplicable variation in the way health care was delivered from one Vermont community to another.

This “practice variation” is described in my previous post. Today’s post presents a  top-level introduction to what has taken thirty years of analysis and testing to confirm. It’s hard to imagine this is true; read the book. In short, Wennberg and his colleagues have established that healthcare decisions in reality can be categorized in three groups:

  • Effective care: situations where there’s no debate – all eligible patients should get this. Example: if you have a broken hip, you should have a hip replacement.
    • This is about 15% of Medicare spending.
    • As a separate cause for concern (not cited in this book), other studies have found that doctors only recommend “the standard of care” about half the time. (Isn’t that amazing and eye-opening?)
    • Note: this is only 15%. 85% of the time, the decision is open to consideration, and you should be presented with options.
  • Elective or “preference-sensitive” care: there’s more than one option, and outcomes vary depending on which option you choose.
    • This is about 25% of Medicare spending.
    • Includes decisions about some surgery, and some screening tests.
    • These decisions are preference sensitive – they have different quality of life (QOL) implications, so the “right”decision for proper care cannot be made without knowing the patient’s preference. You should be asked.
    • Example: if you have an enlarged prostate, one option is to just keep an eye on it (“watchful waiting” or “active surveillance”). The surgical option has significant risk of side effects: impotence, leakage, other issues. Years of research has shown that the importance of each side effect varies widely by patient. The correct decision can’t be made by the doctor alone.
    • But many (perhaps most?) clinicians don’t present us with the range of options – they make the choice for us (perhaps with the best of intentions) and then ask our consent. (This is the “informed consent” form we’re asked to sign.)
    • Most patients don’t know about this, so they don’t ask and they don’t get involved with decisions. But when the options and trade-offs are presented, people often opt not to have surgery. That’s informed choice, vs informed consent.
    • Here’s something to think about: some people at the Foundation for Informed Medical Decision Making (FIMDM) propose that operating on a patient without knowing their preference can be as much of a medical error as operating on the wrong limb! (In both cases you may be cutting something that shouldn’t be cut – a clear mistake.)
  • Supply-sensitive care: 60% of Medicare spending (sixty percent!)
    • As described in the previous post, in a majority of cases your likelihood of having a treatment recommended is proportional to how available it is in your area not related to your need. Even the chance that your death will occur in an ICU is proportional to the supply of ICU beds in your area.
    • Yes, there is vast historical evidence for this. (See Sherlock Holmes above.)

“Supply-sensitive care” smacks of Parkinson’s Law (“Work expands so as to fill the time available for its completion,”), or, according to Wikipedia, a generalization: “The demand upon a resource tends to expand to match the supply of the resource.” Indeed, a similar law, Roemer’s Law, is widely accepted by people I’ve talked to: “A bed built is a bed filled.” (No matter how much hospital capacity you build, it’ll get used up.)

And the doctors who recommend that we be hospitalized to the gills generally don’t realize they’re doing it.

Think this is a complaint about American healthcare? It’s not. For instance, in 1992 it was noted in the Netherlands:

e-Patient take-aways – Things for engaged / activated patients to realize:

I can’t overstate the importance of realizing this as we approach any decision about a treatment, especially surgery. It’s especially important to realize that your own physician, with the best of intentions, may not be aware of the invisible influences driving the treatment recommendations of his or her peers in the local community.

It seems clear to me that we must, must, must create discussion tools – an index card, a flyer, a website – from a respected source to help clinicians listen when we ask, “Is this treatment necessary?” and ask for help in researching the rationale for the decision, including researching the relative frequency of recommending it locally compared to other hospital referral regions.

We need to educate clinicians, patient advocates, insurance companies and health plans, and each other about this issue. What’s at stake is patient safety: every hospitalization and treatment carries a risk of harm as well as the possibility of improvement. I’ve heard from many people that their physicians are sometimes offended when patients ask. (Of course, many other physicians aren’t.)

Our society’s definition of participatory medicine talks about patients becoming “responsible drivers” of their care. Clearly, shared decision making is essential: we need to stand up for our right to be responsible for what happens to our bodies.


Posted in: practice variation | shared decision making




16 Responses to “The discovery of practice variation: follow the data”

  1. Nice mention of Roemer’s rule, Dave! Great post.

  2. So, where, as a patient, can you go, enter a condition, and get the information you need to determine:

    * Whether there is a single standard of care and,
    * if not, what are the options and ramifications, and
    * what are the associated costs (to you or your insurer)?

    If it was easier for patients to find this out, it might make for better care.

    • Keith, I couldn’t agree more with the need for this resource. FIMDM and some others have developed decision aids to walk patients through the alternatives; creating one is a fair amount of work, as you might imagine.

      One thing I’d love to see the Society for Participatory Medicine create is a list of patient communities – or patient/provider communities, for that matter.

    • Dr David says:

      I think asking the right doctors the right questions is likely much more important than trying to determine the standard of care. However, some sources you might want to try as a patient to get a general understanding of current treatment standards would be:

      UpToDate for patients is not a bad site for cancer

  3. […] post by e-Patient Dave appeared on on March 11, […]

  4. FIMDM says:

    Sadly there is no good single source…

    Medline Plus is a go to for evidenced-based unbiased information. Consumer Reports Best Buy Drugs is terrific for drug decisions. For those who are so inclined, Google Scholar and Pubmed gets you deep into the primary literature. I agree that patient communities are a great way to go too…

    Hope that helps.

    • So, let’s give that a try. I went to Medline Plus (which is the National Library of Medicine’s consumer-level magazine and online site), and searched for the only thing I know a little about: “kidney cancer treatment options”.

      Oo, I love this: the kidney cancer page under Treatment has links to separate types of info, on pages for Patients, Professionals, and Clinical Trials.

      The list of options on the patient page isn’t detailed but could be a discussion starter, to ask one’s doctor about other options. And though the info on the page for professionals is somewhat over my head, I can sure understand the survival percents, and “the only systemic therapy that has been associated with inducing durable complete remissions.”

      On the other hand, the patient/consumer version on my treatment doesn’t give any hint of that – you have to look into the professionals’ version. (But it’s there, available if I want it!)

      A big concern to me is that the information (even on the physician’s page) is WAY out of date. It cites 15% response rate; the SELECT trial reported last year showed 30% response. Hm. I wonder how we get that fixed.

      Anyway, yeah, I’d give Medline Plus a Thumbs Up as a first stop in researching treatment options!

      p.s. I know urology is urology, but I have to wonder, even in consumer-level diagrams is it really necessary to label the penis??

      • Susannah Fox says:

        Quick note: the links on keep breaking, including ones that I’ve fixed in my own posts. I can’t click through, for example, to see the diagram you’re referencing, but I think I can imagine it. I don’t know what’s going on, but apologies to everyone who might be frustrated by this glitch.

        Regarding your point about labeling the penis: major body parts may indeed be obvious, but the labels may be targeted at low-literacy or non-English-literate audiences. They may know the word in English but not how it’s spelled, for example.

  5. Dr David says:

    e-patient Dave,

    Although in general I agree with Dr Wennberg, and feel that patients participating more effective in shared decision making would be likely to reduce variation, for an interesting and alternative view re variability, see

  6. kgapo says:

    Hi Dave,

    Indeed the findings of Dr. J. Wennberg are shocking! Tracking Medicine is the first book I read this year just after coming back from Salzburg!

    The findings of Dr. Wennberg are not a characteristic of American healthcare. They have universal value. Since his remarks could easily be applied to depict today’s ailments of Greek healthcare.

    When I was about to finish Tracking Medicine, I heard via a tweet about Dr. Jerome Groopman’s fascinating book on “How Doctors Think” which I am about to finish this week.

    Well, both “Tracking Medicine” and “How Doctors think”, are somehow complementary as they give hints on why certain things happen in today’s healthcare.

    Moreover, reading both books you come up a more educated patient, since you have learnt that things in medicine may not be what they apparently are and that you, as patient, have a duty to help, to guide the physician treating you to paths that may help me come up with a correct diagnosis.

    I will also write a post shortly about both books and how they complement each other.

  7. […] The discovery of practice variation: follow the data […]

  8. jonmcrawford says:

    My wife complains constantly (and correctly) about the internet, mainly because there is no good way to determine the age of the information being presented. Perhaps we need standards on the tech side of things that tell the age of information/websites/reports in the same way for all resources.

    This kind of feeds into Paul Levy’s discussion of current healthcare data also (, without up to date stats from patient encounters, up to date info for medical studies to show the emerging results, then e-patients are left muddling through potentially incorrect and misleading information that harms rather than hurts their efforts.

    • Hey Jon –

      This is exactly where the power of a smart patient community saves the day. Individually, we’re often adrift. With peers to chat with, it’s all very different.

      Hence, in our definition of participatory medicine (on the website of the Society for Participatory Medicine), it says “networked patients.”

Leave a Reply