Skip to main content

Why the success of predictive analytics in healthcare is hard to predict



In one of my visits to the hospital where our EHR system was being deployed the chief doctor turned to me and said: "My experience allows me to predict exactly what will be the patient's condition in the next few hours just by glancing at him, however, I am not able to pass this skill to my young doctors. What I need from you, software engineering people is to help me solve that problem". That sounded to me like a classic problem for computers to crack and bring value: to “connect the dots” between so many unrelated data points simultaneously in real time, in order to detect small anomalies indicating that a patient begins to fail, elusive even from a careful human observer.

The year was 1998 and although predictive analytic has been a hot topic over the years and the vast improvement in computational power, predictive analytic products are not widely being used by health delivery organizations and are still being considered by many as gadgets. Treatment decisions made on incomplete information and educated guesses are quite common and computers, despite the promise, has done little to change that reality.

So what went wrong? why computers haven't been able to crack this "classic problem" despite the large investments in this field?

To turn an idea into a product ,predictive analytic entrepreneurs must pass through what I call the "funnel of diminishing value", which rips down the product scope and decreases its value as it passes through.

The funnel starts by the nature of the field: prediction. Machine learning engineers earn the highest in the industry and the process of finding and tuning an algorithm is lengthy. That is not unique to healthcare, what is unique however is that patient privacy regulation makes it harder to obtain data which is so crucial for the training and fine tuning of the algorithms.

Next in the funnel is what I call: discovering the oblivious. That means, alerts generated by the algorithm reflect a risk to the patient's condition the clinical team is already aware of, and so they do not bring any additional value.

The next hurdle down the funnel relates to the output of the alerts. Doctors seek to base their decisions on data which they can review, draw conclusions from and defend. Unfortunately, this is not how neuron networks, one of the popular models used by prediction algorithms works. Neuron networks alerts do not provide a good description on their triggering rational the same way you can't explain why you recognize the face of your good friend. The care team may be reluctant to act on such unexplained alarms, especially giving the existence of false-positive alerts that are likely to be produced by any algorithm.

Next in turn is the product implementation. The data the algorithm is based on is generated from medical devices, information systems and manual documentation. Assuming the organization has made the investment to capture the data, those sources can omit different values in different timing or nor exists at all in the environment. The better the algorithm is the more data it needs, however the more data it needs the sensitive it gets. The implementation itself may prove a challenge and what works in the lab may not work in the field. This is a key issue to decision makers which needs to invest capital without a clear outcome.

The final scope reducer is the one people less like to talk about. Insight and value are not the same. The assumption along the way has been that alerts can be translated to actions, and that an intervention if given in a timely manner can change the fate of the patient. This is however not an accurate story. In many situations there is no intervention that can dramatically impact the fate of the patient and the patient condition is bound to worsen no matter what the clinician will do.

Predictive analytic products going through this funnel need to get to the other side with a value which is greater than the R&D investment and the licensing costs a health organization will be willing to pay. Over the years the funnel of diminishing value had crunched many good ideas which sounded great in theory.

So what do I predict? Will predictive analytics manage one day to be part of the main stream health system? After all being said and done it seems like one should avoid that bet.
But when pushed to make a bet I think back of the chief doctor's simple request which represents the real need, and think that the day will come and his request will be fulfilled.

Comments

Popular posts from this blog

FHIR Status Check

 More than 2 years have past since I wrote my article  on the Fast Healthcare Interoperability Resources (FHIR®) standard and it’s time to do a quick status check and revisit the predictions I made back then. The FHIR standard continues the strong trajectory of adoption and is now used across the globe. The application programming interface (i.e. the FHIR API) is available in most major EHR systems today. According to the US Office of the National Coordinator Health Information Technology an estimated 85% of hospitals have FHIR in their systems. The NHS has been quick to adopt FHIR and the adoption curve in the UK is high. The NPfIT (NHS Care Record Service) HL7 V3 interfaces are being redeveloped in FHIR®, and new NHS specifications such as the CareConnect standard for secured Transfer of Care  are being specified in FHIR® by default. Despite the industry enthusiasm about the potential of FHIR still the old and faithful HL7 v2 remains the predominant interoperability standard in use t

Home grown and die slow systems

The atmosphere got tensed as the meeting went on. We were all started feeling a bit uncomfortable. The meeting took place at headquarters of one of the prestige hospitals in the US. A hospital which constantly ranked in the top 10 best hospitals in the world. A drop of sweat sprout on the brow of the IT executive which was leading the meeting. The participants nodded their heads as they realized that despite the massive investments in developing their home grown systems over the years they are lacking some basic features which typically found in commercially available products, and many of the modules in used are becoming old and need to be replaced. That same realization repeated itself in similar discussions I had with other health organizations around the globe which took the "home grown" path. It seems there was a paradox between how wealthy is the organization and how poor its IT systems are. Selecting a commercially available product is a frustrating task. Typic

Hospital CXO - A new well deserve seat at the executive table

I once asked the CEO of a large hospital what will be the main KPI to measure how well the hospital is doing. The CEO immediate response was that the best indicator would be how satisfy the patients are. In a stressful and overloaded environment such as a hospital, it is easy to focus on the disease and forget about the patient. A chief experience officers (CXO), or other similar titled leaders, is a new member in a hospital C-suite. The role of a CXOs have gained scope and respect in the C-suite as studies show how experience affects all aspects of care. There is a growing body of evidence supporting that the association between better patient experience and health care quality. For example, a study found that a higher CMS star rating was associated with lower patient mortality and readmission's.  One of the drivers accelerating the adoption was the US government decision to start mandate measuring patient’s perception of their care and tied reimbursement to those scores. Beyond t