Skip to main content

Why the success of predictive analytics in healthcare is hard to predict



In one of my visits to the hospital where our EHR system was being deployed the chief doctor turned to me and said: "My experience allows me to predict exactly what will be the patient's condition in the next few hours just by glancing at him, however, I am not able to pass this skill to my young doctors. What I need from you, software engineering people is to help me solve that problem". That sounded to me like a classic problem for computers to crack and bring value: to “connect the dots” between so many unrelated data points simultaneously in real time, in order to detect small anomalies indicating that a patient begins to fail, elusive even from a careful human observer.

The year was 1998 and although predictive analytic has been a hot topic over the years and the vast improvement in computational power, predictive analytic products are not widely being used by health delivery organizations and are still being considered by many as gadgets. Treatment decisions made on incomplete information and educated guesses are quite common and computers, despite the promise, has done little to change that reality.

So what went wrong? why computers haven't been able to crack this "classic problem" despite the large investments in this field?

To turn an idea into a product ,predictive analytic entrepreneurs must pass through what I call the "funnel of diminishing value", which rips down the product scope and decreases its value as it passes through.

The funnel starts by the nature of the field: prediction. Machine learning engineers earn the highest in the industry and the process of finding and tuning an algorithm is lengthy. That is not unique to healthcare, what is unique however is that patient privacy regulation makes it harder to obtain data which is so crucial for the training and fine tuning of the algorithms.

Next in the funnel is what I call: discovering the oblivious. That means, alerts generated by the algorithm reflect a risk to the patient's condition the clinical team is already aware of, and so they do not bring any additional value.

The next hurdle down the funnel relates to the output of the alerts. Doctors seek to base their decisions on data which they can review, draw conclusions from and defend. Unfortunately, this is not how neuron networks, one of the popular models used by prediction algorithms works. Neuron networks alerts do not provide a good description on their triggering rational the same way you can't explain why you recognize the face of your good friend. The care team may be reluctant to act on such unexplained alarms, especially giving the existence of false-positive alerts that are likely to be produced by any algorithm.

Next in turn is the product implementation. The data the algorithm is based on is generated from medical devices, information systems and manual documentation. Assuming the organization has made the investment to capture the data, those sources can omit different values in different timing or nor exists at all in the environment. The better the algorithm is the more data it needs, however the more data it needs the sensitive it gets. The implementation itself may prove a challenge and what works in the lab may not work in the field. This is a key issue to decision makers which needs to invest capital without a clear outcome.

The final scope reducer is the one people less like to talk about. Insight and value are not the same. The assumption along the way has been that alerts can be translated to actions, and that an intervention if given in a timely manner can change the fate of the patient. This is however not an accurate story. In many situations there is no intervention that can dramatically impact the fate of the patient and the patient condition is bound to worsen no matter what the clinician will do.

Predictive analytic products going through this funnel need to get to the other side with a value which is greater than the R&D investment and the licensing costs a health organization will be willing to pay. Over the years the funnel of diminishing value had crunched many good ideas which sounded great in theory.

So what do I predict? Will predictive analytics manage one day to be part of the main stream health system? After all being said and done it seems like one should avoid that bet.
But when pushed to make a bet I think back of the chief doctor's simple request which represents the real need, and think that the day will come and his request will be fulfilled.

Comments

Popular posts from this blog

Clinical data warehouses: Sometimes it's worth being lazy

If you are a health organization you probably have or thinking of having a clinical data warehouse. A Clinical Data Warehouse (CDW), sometimes called Clinical Data Repository (CDR), is a database that consolidates data from a variety of clinical sources to form a unified view of a the data for various purposes. Typical data types which are often found within a CDW include: clinical laboratory test results, vital signs, patient demographics, administered meds, hospital admissions, ICD-9 codes and more. Developing and sustaining an effective CDW operations unit is a substantial effort and long-term commitment. The main challenge in designing a CDW is defining its scope and the use cases it should support. In theory a CDW can serve as a basis for reporting, studying and planning. The use case that is often mentioned in relate to CDW is supporting clinical trials. This would allow for researchers to have all the information from a study in one place as well as let other researchers benefit

The big battle: Best of breed vs One stop shop

The world of economics has decided on this debate a long time ago: monopolies are bad, diversity is good. No matter what a monopoly promises, you can rest assure that over time the lack of competition will cause prices to go up and quality to go down. When it comes to healthcare IT, however, there is one unique factor that flips the coin – interoperability. Despite various attempts the healthcare industry has yet to solve the interoperability challenge in a satisfactory manner which will enable a full continuum of care across different health information systems within a health delivery organization. Taking a common scenario of prescribing a medication order in theatres using a surgical system to be later administered in a ward requires significant investment to achieve, even using the modern Fast Health Interoperability Resource (FHIR) protocol. The investment required to streamline the data flow across systems raise in an exponential order with every new system that is thrown to t

Are we ready for a Cloud First hospital?

I will start this article by defining what I mean by the term a Cloud First hospital. The term cloud has been a buzz word in the past decade which led many organizations to declare their support for the cloud, sometime without understanding its true meaning. For the purpose of this article I am proposing a simple test to decide whether an organization is a cloud first or not. If you are software vendor you must have an IT department which directly in charge of the system up-time at your clients sites. If you are a health organization then you should never have visited the data center where your data resides. A Cloud First hospital is one which more than 50% of its systems reside in data center that none of its staff members ever visited or not even sure where they are. According to a recent survey by Datica, in the US only 17.7 percent of the respondents say they work with healthcare organizations that have more than 50% of the existing software infrastructure remotely hosted or