In one of my visits to the hospital where our EHR system was being deployed the chief doctor turned to me and said: "My experience allows me to predict exactly what will be the patient's condition in the next few hours just by glancing at him, however, I am not able to pass this skill to my young doctors. What I need from you, software engineering people is to help me solve that problem". That sounded to me like a classic problem for computers to crack and bring value: to “connect the dots” between so many unrelated data points simultaneously in real time, in order to detect small anomalies indicating that a patient begins to fail, elusive even from a careful human observer.
The year was 1998 and although predictive analytic has been a hot topic over the years and the vast improvement in computational power, predictive analytic products are not widely being used by health delivery organizations and are still being considered by many as gadgets. Treatment decisions made on incomplete information and educated guesses are quite common and computers, despite the promise, has done little to change that reality.
So what went wrong? why computers haven't been able to crack this "classic problem" despite the large investments in this field?
To turn an idea into a product ,predictive analytic entrepreneurs must pass through what I call the "funnel of diminishing value", which rips down the product scope and decreases its value as it passes through.
The funnel starts by the nature of the field: prediction. Machine learning engineers earn the highest in the industry and the process of finding and tuning an algorithm is lengthy. That is not unique to healthcare, what is unique however is that patient privacy regulation makes it harder to obtain data which is so crucial for the training and fine tuning of the algorithms.
Next in the funnel is what I call: discovering the oblivious. That means, alerts generated by the algorithm reflect a risk to the patient's condition the clinical team is already aware of, and so they do not bring any additional value.
The next hurdle down the funnel relates to the output of the alerts. Doctors seek to base their decisions on data which they can review, draw conclusions from and defend. Unfortunately, this is not how neuron networks, one of the popular models used by prediction algorithms works. Neuron networks alerts do not provide a good description on their triggering rational the same way you can't explain why you recognize the face of your good friend. The care team may be reluctant to act on such unexplained alarms, especially giving the existence of false-positive alerts that are likely to be produced by any algorithm.
Next in turn is the product implementation. The data the algorithm is based on is generated from medical devices, information systems and manual documentation. Assuming the organization has made the investment to capture the data, those sources can omit different values in different timing or nor exists at all in the environment. The better the algorithm is the more data it needs, however the more data it needs the sensitive it gets. The implementation itself may prove a challenge and what works in the lab may not work in the field. This is a key issue to decision makers which needs to invest capital without a clear outcome.
The final scope reducer is the one people less like to talk about. Insight and value are not the same. The assumption along the way has been that alerts can be translated to actions, and that an intervention if given in a timely manner can change the fate of the patient. This is however not an accurate story. In many situations there is no intervention that can dramatically impact the fate of the patient and the patient condition is bound to worsen no matter what the clinician will do.
Predictive analytic products going through this funnel need to get to the other side with a value which is greater than the R&D investment and the licensing costs a health organization will be willing to pay. Over the years the funnel of diminishing value had crunched many good ideas which sounded great in theory.
So what do I predict? Will predictive analytics manage one day to be part of the main stream health system? After all being said and done it seems like one should avoid that bet.
But when pushed to make a bet I think back of the chief doctor's simple request which represents the real need, and think that the day will come and his request will be fulfilled.