Dear White House: The Personal Data Challenge

Gary Wolf of Wired has posted a whizbang write-up that came out of a whirlwind one-hour 12-way Skype chat about personal health data. Sound frenetic? It was. (I participated. It was, well, 12-way.)

I can’t imagine how to model what happened, except to say that it was wired.

It grew out of a request from the people at the Community Health Data Initiative, which (as we reported here last month) is opening vast amounts of HHS data for innovators to get at. (And innovators are doing so, fast, as that post describes.) Here’s how Gary started his write-up:

Earlier this summer Alexandra Carmichael, who is the founder of CureTogether as well as our director here at the Quantified Self, was in Washington for a meeting in President Obama’s Office of Science and Technology Policy (OSTP). When Alex came back she posted a report about her meeting here on the blog, and invited us to answer some questions from one of the President’s policy advisers. …

Specifically, we in the chat were asked three questions:

  1. How to deal with or integrate personal health data into the data warehouse they’re building
  2. What should the government do or worry about in the health data space?
  3. What are the top things to focus on that work to increase national health?

To understand Gary’s takeaways, you’ll have to read his whole post – take a few minutes, it’s stimulating.  This extract summarizes why my antennas are twitching big-time about this:


Personal health data is a major, positive force for transforming the health care system.

Three important trends converge here:

  1. A consensus that self-care is important in achieving national health goals.
  2. The failure of traditional medical record keeping to accurately describe patient experiences; much less the full range of living conditions relevant to health.
  3. The rise of new systems to collect and understand personal data. …


Ain’t that the truth??

The great technical challenge is, he continues, that since personal health data (of heaven-only-knows-what) has no universal structure, it “looks messy from a traditional clinical or laboratory point of view.” But (I’ll break his counterarguments into a numbered list),

  1. Powerful techniques for analyzing messy data already exist, and they are rapidly improving.
  2. Tools for collecting personal data are also getting better.
  3. Self-tracking is becoming more popular.
  4. These factors are mutually reinforcing: stronger analytical techniques; more accurate and consistent collection; and vastly increased volumes of data create a new landscape of public health knowledge.

Landscape? I’d even say ecosystem – as innovators mash up data heaps in unpredictable ways, extracting new metrics and new knowledge, the ecosystem becomes richer and thus more attractive to more innovators.

I want this stuff. I love this stuff.

“Virtual warehouse”? A question and suggestion –

The one thing I didn’t understand in Gary’s post is the idea of a “virtual warehouse” for the data. I know something about databases but I’m no wizard at it. If “virtual warehouse” means the data will be stored in all sorts of places, then I see another vital enabling role for government to play: provide (or support creation of) a replicable platform, so people and communities can “tear off a copy” and run with it, without centralized management of who does what.

Patient communities spring to mind: what if every ACOR group and every other patient community had a freely available, open platform for storing whatever sort of data they find useful?

Providing a platform like this would parallel how HHS  has enabled wildfire entrepreneurial use of data via CHDI, also without centralized management. And best of all, it would empower patients (the most under-utilized resource in healthcare) to create their own value and pursue whatever information they find useful. That’s participatory, it’s empowering, it’s enabling. It’s e.

I love this stuff. And I bet Doc Tom Ferguson would, too.

(Speaking of which, I just discovered a gem in Doc Tom’s attic. Look for it later this week.)


Posted in: e-pts resources | medical records | policy issues | positive patterns | pt/doc co-care | reforming hc




18 Responses to “Dear White House: The Personal Data Challenge”

  1. Jon Lebkowsky says:

    I can imagine all sorts of issues emerging from this discussion – about data standards and formats, for instance, and security/privacy issues. Also structured vs unstructured data, and how to convert the former to the latter.

    What are the next steps?

  2. Gary Wolf says:

    Jon, re standards, formats, security, structured and unstructured content; I didn’t want to publicly post this until we worked out the details, but the current plan is to refer everybody to you for further explanation.

    Seriously, I’ll hazard a quick answer, but we’ll see what emerges from further discussion. My guess is that creating a system that permits anonymous contributions is not the hardest part of this. I think that the structured vs. unstructured data problem will be solved more easily than many think, through better techniques of coping with semi-structured data. The real tricky stuff, I suspect, is going to be in extracting meaning from the data. There are all kinds of advances waiting to happen; but the first step is to acknowledge that personal data is huge, and hugely valuable, and start gathering it.

  3. "Dear White House: The Personal Data Challenge" (my take on @agaricus #quantifiedself post) WHOOSH!

  4. Annie Stith (@Gr8fulAnnie) says:

    Hey, all. Patient here. Some of this is over my head (we need a glossary for the alphabet soup of acronyms), but here’s what I get from this post, and my comments.

    As always, I appreciate the acknowledgment that patient input is vital for ANY system to be most valuable. I am a proactive patient, and much of what I do for myself outside the constraints of “traditional” medicine ends up as my handwritten notes to my PCP, but the info goes no farther. While the notes are in the file of that particular doctor to whom I provide them, they’re not passed on (for example) with referrals to specialists. I would SO appreciate and take the time to input into any system where these non traditional self-care notes would be available as part of my medical e-file (which I envision as a COMPLETE medical file from all physicians and hospitals where I’ve been treated, plus pharmaceutical records and my own input).

    I’ve also worked with databases, and while the task is certainly gargantuan in terms of historical data to be input, the development of the database(s), forms, and reports just don’t seem all that complicated to me. I can see an input screen with date, Dx code(s), Rx’s written, treatment code(s), etc., and a simple yes/no for unstructured data (such as comments) with a text field for all marked “yes.” Forms and reports can be used to meld info from different databases so long as there’s a consistent patient ID used in all records. (I know that’s a very simplistic view considering the amount of data and sources and potential number of databases, but, truly–is the idea itself really more complicated than that?)

    The most complicated issue I see is privacy and enforcement. That’s a bit less of a concern if preexisting conditions can no longer disqualify for treatment by a new health insurance plan, but I still wouldn’t want employers or hackers to ne able to access my info.

    That’s this patient’s view, anyway.


    • Mark Boguski says:

      Annie, you can store all of your notes in one or more “casebooks” at Resounding Health. Your casebooks are confidential and visible only to you, by default. You can elect to share your insights, observations, discoveries, etc. with (including your doctor) by emailing, creating PDFs or embedding links to them in other sites. Future versions of casebooks will allow you to link your notes directly to your electronic health record. For further info, see

      • Mark Boguski says:

        Hi Annie,
        I see you created a casebook, indeed a “master casebook” to link others to. This is a particularly powerful way to organize your information because the bookmarks associated with casebooks can be any URL on the web and/or links to other casebooks, drug package inserts, etc. internal to the RH site. Furthermore, you can rate each of these bookmarks and comment on why you rated a link the way you did.

        You’ve taken the first step in writing “Annie’s Textbook of Personalized Medicine” and I’m sure that others will benefit from your research, interpretations and recommendations.
        Best regards,
        Mark & Alan

        • Annie Stith (@Gr8fulAnnie) says:

          Hey, Mark & Alan!

          Yes, I started a Casebook. In fact, I started two. I thought having a main casebook as an Index was the best way to start as I have multiple conditions having to do with both my physical and mental health. And I am a HUGE believer in sharing information, probably in part because there wasn’t a place where info was gathered at all when I received some of my Dx’s. (Fibromyalgia was still a “syndrome” and considered by many doctors to be a “garbage can diagnosis.” Chronic Myofascial Pain hadn’t yet been identified or named at all.)

          Unfortunately, as I’ve found so often, I’m not going to be able to use the site. I’m part of the movement towards relying on a Smartphone for my internet access without having a computer, as well. The Remarkable Health website is either not mobile and/or BlackBerry “friendly.” (I wanted to add more conditions to the Index Casebook, but the frame on the page won’t allow me to scroll down far enough to access the search box now that I’ve added the first subject.)

          If this website does become mobile and BlackBerry “friendly” in the future, I would greatly appreciate being able to use the site.

          Meanwhile, I have found the website “Patients Like Me,” where I can at least share my experiences with other FMers, and allow specific doctors access by invite.

          Thanks so much for your input.


  5. Mark Boguski says:

    If one replaces the word “data” with “information,” we already have a system that can not only “warehouse” user-contributed info, but automatically interprets and relates to existing medical knowledge thus allowing users to identify emergent or undiscovered connections. I’ve described this in several other posted comments on this forum. See the comments on user-generated “casebooks” ( and “crowdsourcing” new discoveries (

  6. RT @ePatientDave "Dear White House: The Personal Data Challenge" (my take on @agaricus #quantifiedself post) WHOOSH!

  7. Mark Boguski says:

    Here’s a model by which community health data could be analyzed to discover new uses for existing medicines and also create economic value: This approach could also used to collect data validating (or not) the “off-label” use of FDA-approved drugs.

  8. Dave,

    This Quantified Self initiative needs to piggyback off HITECH meaningful use rules requiring patients to have electronic access to parts of their medical record, e.g., summary health records, discharge summaries.

    The health care industry will be transformed (disrupted) by patients once they understand the power of their own data – as you wrote “gimme my damn data”.

  9. Dear White House: The Personal Data Challenge (my take on @agaricus #quantifiedself post) WHOOSH!

  10. @agaricus @accarmichael see @VinceKuraitis comment on White House challenge – traction=tie in w/HITECH/#meaningfuluse

  11. Gary Wolf says:

    Vince’s comment and Dave’s pointer has been very valuable to me, as it pointed me toward Vince’s extensive HITECH commentary, and gave me a new understanding of the pathologies of the health care industry. Here is the most important quote, to me, from Vince’s recent post on HITECH. Citing an earlier post by e-patient Dave, Vince says this: The $45 B question — would it be more effective and efficient to focus significant funding on “patient-pull” vs. “provider-push”?

    Yes, Quantified Self readers, who are following the conversation here, that is a “B” for billion. On the subject of how much it costs to get the health care industry to change, I know very little. For you guys, is $45 billion a lot or a little?

    Vince’s extremely clear and thorough analysis of HITECH is a bit discouraging, in my reading – and again, I am no expert, and had to look up most of the acronyms. Whether $45 billion is a lot or a little, it seems to have been insufficient to push things along far enough.

    Given that providers are embedded in a system with so many perverse incentives, the answer to Vince’s 45B question seems fairly obvious: patient-pull is the right way to go. But even this way of framing it may be a bit too influenced by industry-think. Health data is a subset of personal data in general. And most of the crucial individual and social decisions that affect health occur among people who are not (yet) patients. (Even among self-defined patients, many of the things affecting the outcome of treatment are not “health care” things in the sense that these would be defined by a conventional health record.)

    So the context for catalyzing this change is not – or rather, not only – the health care industry. This is good news. The big cultural change that is driving people to want to capture and understand their personal data is already happening, and it is lot less expensive to influence and accelerate a desirable trend than to lure or bully administrators and managers whose professional responsibilities make them necessarily conservative. (To put the the most generous spin on it.)

    An investment of just a few million dollars – say a tenth of a percent of that 45 billion – invested on the user side might even be enough. A billion dollars? It would be impossible spend, and would have to be returned to the US Treasury.

Leave a Reply