In my last post related to our bundled payments research series, I discussed how it’s possible to identify the clinical factors that are most predictive for a commercially insured total joint replacement (TJR) patient hospital readmission. That type of identification could be beneficial since one goal of bundled payment models is to reduce costs, including those associated with readmissions.
But even with the ability to identify readmission risk factors, should we? Is using a predictive model better than a random assignment of intervention? And if it is better, is the value of predictive modeling worth the cost of intervention?
To answer these questions, statistical methods are required.
Consider the predictive accuracy of one model: We analyzed the trade-off between sensitivity (the ability to correctly identify high-risk patients) and specificity (the ability to minimize the number of patients incorrectly identified as high risk). The AUC (area under the ROC curve) score was 0.62, which is distinctly better than the 0.50 that represents a random assignment of risk — but still a fair distance from a perfect 1.00 assignment of risk. Nevertheless, this scoring shows that a statistical approach could be used to set a useful threshold for assigning high risk for an intervention program.
To further assess the value of a readmissions model, we then worked from the equation of: Expected Value = Expected Benefit − Expected Cost.
- Expected cost is the average cost per patient intervention multiplied by the number of interventions. The average cost depends on the mix of interventions and their duration.
- Expected benefit is the cost of an avoided readmission multiplied by the number avoided. The number avoided is the product of the number of interventions and the probability of success of each intervention.
To test our calculations, we performed a simplified version of how a hospital might assess the value of a model. Somewhat arbitrarily but informed by experience, we chose a cost per intervention of $500, the cost of an avoided readmission of $10,000, and an expected success rate of 50 percent.
We then set a risk threshold at 0.07 (above which the patient will be classified as high risk). We found that the intervention would target 26 patients per thousand at a total cost of $13,000. We would expect three of these patients to be readmitted without intervention, and with a 50-percent success rate, the intervention would prevent 1.5 readmissions or $15,000. That’s an expected net savings of $2,000.
Based on this simple example, the results suggest the accuracy of the model in identifying high-risk patients is marginally sufficient to support a targeted intervention. But actual readmission cost, success rate, and risk threshold will change the results in every instance, so understanding the true experience of each organization is critical to deciding whether to apply an intervention model.
There is much more, of course, to this approach than we’ve shown here. For details and more insights from our bundled payments research, you may want to download our latest brief, Total Joint Replacements in the Commercially Insured Population: Predicting Risk of Readmission.
Senior Research Fellow, Advanced Analytics