(Draft) Machine learning to detect and quantify low-value care#
The authors seek to detect and estimate low value care – care that provides little health benefit in light of its costs : in particular over-testing and under-testing. They focus on Acute Coronary Syndrom (heart attack) which is difficult to detect : the more conclusive test being stress testing and catheterization.
They use a machine learning algorithm to model the individual risk of having a heart attack in the tested population. These estimates of the risk are plugged into cost-effectiveness estimates based on benefits of the subsequent intervention induced by testing positive –revascularization. Taking as a cost-effective threshold 150,000$, they conclude than half of the tests (52%) can be flagged as not cost-effective before they are performed. This contrasts with an average analysis that would conclude to cost-effectiveness at 135,859 per life-year. This concludes to major over-testing.
Conversely, the adverse events in (predicted) high-risk untested patients are greater than their high-risk tested counterparts. This suggests under-testing as well.
The bigger cost loss come from under-testing. However, by investing low-testing hospitals and a natural experiment taking into account low testing induced by week-end admissions, the authors suggest that incentives to reduce care may lead to under-testing for everyone, irrespective of their risk and thus exaggerate inefficiencies due to under-testing.
Various Notes:#
The paper makes a smart usage of the tail of the AUC curve to detect meaningful performance differences between models. When considering the 1% riskiest patients given each model, how many experienced the outcome? I think that it is the same kind of idea than judging models by their calibration.