Before you read this post, think of a time when you had a crush on someone. Think about that swirl of emotions, the highs and the lows. That’s where I was a couple weeks ago, except it wasn’t about a person.
I fell hard for Watson, IBM’s hot new outboard brain. I’d heard he was smart, kind of a know-it-all, but also trainable, a good listener, and maybe the answer to people’s prayers and complaints about the rising flood of information.
At least, that’s what I hoped as I found a seat in Martin Kohn’s Health Foo session. (What’s Health Foo? Read this.)
Kohn began with an explanation about why Watson’s first public task was to compete – and win – on the game show Jeopardy. Playing Jeopardy was not a long-term business model, he said with a smile, but a means to an end: mastering a complex English-language challenge.
Kohn dazzled me and many others with Watson’s voracious appetite for information. My tweet, “Doctors have only an estimated 5 hrs of reading time per month; IBM’s Watson can read 72,000 hours per day” was an instant hit as people re-tweeted it and began firing back questions. Tim O’Reilly tweeted, “In the 3 seconds of a Jeopardy question, Watson could read and understand 200 million pages of text,” which set off another flurry of interest.
But I was too much in thrall to answer questions on Twitter. He (and yes, the developers call Watson “he” too) can dynamically evaluate the quality of information sources, ie, “The New York Times is more often my source for the right answer than Wikipedia, I’ll prioritize that source next time.” He also develops a confidence-level threshold that changes dynamically according to a risk assessment, such as the state of the Jeopardy game or the amount of money at stake.
Joy was welling up inside me as I heard Kohn explain that the health care version of Watson will never say, “This is the answer.” Instead he returns prioritized possibilities. Watson will be an enabler of the information-sorting process, not an instigator (which would answer most of the push-back questions pouring in via Twitter).
As Kohn put it, Watson is the friendly, diplomatic voice in your ear reminding you to think more broadly. Kohn compared Watson to the experienced nurses he worked with as a doctor-in-training, who quietly asked, “Have you considered X? Would you like me to start Y?”
Kohn also shared that one of IBM’s goals is to empower knowledgeable patients with information because they will then be more likely to understand and follow a care plan.
That was it. I had a crush. I was ready to introduce Watson to my friends, the next step in any courtship.
Luckily a big group of them was there with me. People started asking questions, such as, “Watson has the potential to break the traditional medical journal publishing model. He reads all the journals so doctors don’t have to. Will this spawn new subscription models?” Kohn acknowledged, but didn’t answer that intellectual property question since it was not in IBM’s business plans to disrupt the publishing industry. He agreed that Watson helps overcome the flaw of availability (you only consider the options that come to mind) and the flaw of self-reinforcing bias (an enlarged ego can cloud decision-making) by taking in all of the published medical literature.
Wait, what was that? I stopped dreaming and looked up. Another question from the group: “You say that Watson will take in all published studies – what about unpublished studies, such as those submitted to the FDA during a drug-approval process?” And another: “What about the medical knowledge not captured in published journal articles – will Watson be equipped to take in that information?” And another: “You say that Memorial Sloan-Kettering is your first client. Their clinicians do not tell kidney cancer patients about a certain treatment – will their version of Watson be taught this bias?”
Uh-oh. The answers to these questions – and others – whipsawed me between hope and disappointment. Hope that Watson is potentially what we’ve been waiting for – a learning system to assist medical decision-making. Disappointment that Watson is potentially reinforcing the traditional model of “doctor knows best,” instead of the new, participatory medicine model of “doctor knows a lot, but let’s work on this together.”
Here’s the thing about Health Foo: You can’t get through your slides without being interrupted by such questions. My sense is that the questions stem from curiosity and collegiality. We want you to succeed. We want to fall in love with you and your ideas. But we also want you to be the best you can be and will let you know if we see a flaw, like, um, an amazing tool that is going to be trained according to the old model of health care, not the emerging model.
And so, within the space of an hour, I fell in and out of love with Watson. Like many a jilted lover, I gathered with friends to hash out why I was so upset, even angry. I had set Watson on a pedestal, if only for about 15 minutes, and when reality came rushing in, I felt betrayed. Why couldn’t he be perfect? By the end of our conversation I had come back to a feeling of hope. Watson may not be perfect, but he’s on the path. Maybe he has a younger brother.