Experience withAccountable Care Organizations offers an encouraging lesson forbundled-payment models: Cost savings grow over time as providerslearn how to work under the new rules. (Photo:Shutterstock)

Important new research published in the Journal ofthe American Medical Association on Tuesday showed that apromising experiment aimed at reducing health care costs through changes tothe payment system delivered underwhelming earlyresults. But there's more to the story, and the research is exactlywhat should be happening as we transition away from fee-for-service payments.

First, some context. As noted previously, improving value inhealth care is not primarily about loading more responsibility onthe consumer. Instead, most of the gains must come from changingprovider behavior, since the bulk of what'sdelivered in health care is what the doctor orders. And part ofthat, in turn, requires paying doctors and hospitals differently,since paying for volume rather than value produces more careinstead of better care.

Related: 3 ways to know if you're overpaying for healthcare

The two main ideas for alternative payment models are bundled payments (which pay a fixed amount perepisode of care) and Accountable Care Organizations (or ACOs, whichpay a fixed amount per person per year). The vast majority of thesenew models remain voluntary, in the sense that a hospital orphysician can choose to participate. The exception is a bundledpayment for hip and knee surgery, in which the government requiredparticipation by hospitals and paid a fixed amount for all thecosts associated with the surgery and any follow-up care in thesubsequent 90 days.

A fascinating feature of this mandatory hip and knee bundledpayment is that the government randomized the local areas in whichthe new model was applied. That randomization, in turn, providesthe basis for the new research.

Amy Finkelstein of MIT, Yunan Ji of Harvard, Neale Mahoney ofthe University of Chicago, Jonathan Skinner of Dartmouth Collegestudied the differences between the hospitals included in the newpayment model and those randomly excluded from it. They found someencouraging results, including that the hospitals participating inthe new model discharged a significantly smaller share of their hipand knee surgery patients to skilled nursing facilities and otherinstitutional post-acute settings. Previous research has shownsubstantial variation in the cost and quality of these facilities,suggesting that many may not provide sufficient benefit for thedollars spent on them, and a key objective of the bundled paymentmodel is to reduce the use of inefficient post-acute care. Manyhospitals respond to the new payment incentives as expected, bytrying to avoid discharges to places like skilled nursingfacilities.

The more surprising, and disappointing, results involved thetotal cost of care. Medicare spending per episode was a bit over$22,400 in the participating hospitals. At those randomly notparticipating, spending was slightly higher, at $22,900 — but theroughly $500 difference was not statistically significant. Andthere were no significant differences in observed quality measures,either. Since the whole point of these new models is to improvequality and reduce spending, achieving neither isdisheartening.

Before we throw up our hands and give up, though, a few crucialcaveats are necessary. Most importantly, the authors only studiedthe first year of the program. Hospitals, though, may take time tolearn how to operate differently under a new model. Experience withAccountable Care Organizations suggests exactly that: Cost savingsgrow over time as providers learn how to work under the newrules.

In any case, the financial incentives in the first year of thebundled payment program were much weaker than in subsequent years.For example, in the first year, hospitals could gain at most 5percent of the target price set for the bundle if their costs camein lower than the target, and they faced no penalty if their costswere higher than the target. Ultimately, however, hospitals will beable to gain or lose as much as 20 percent of the target price. Theresearchers only had the data for the first year, but the resultsmay change materially in subsequent years as experience builds andthe financial incentives become more potent (that same conclusionholds for an early evaluation of a voluntary version of bundledpayments).

So what lessons can we can learn from this research?

One is the benefit of randomization. Other evidence hadsuggested much larger savings from bundled payments, but one cannever be sure that those don't reflect correlations that aren't dueto the program itself. Randomization of the areas in which the newmodel was implemented (along with the stellar research team thatdid the evaluation) allows us to have greater confidence that theresults are causal. The big differences in results highlight thecrucial importance of randomization.

A related lesson is the benefit of continuous learning as weexperiment with new payment models. Redesigning payment models isnecessarily a messy and imperfect process, and a core component ofimproving them over time is continuing to learn and makeadjustments to them. At the same time, we have to be careful not tooverreact to early results, like the research about the first yearof the mandatory bundled program, if there is reason to believe thefindings may evolve as providers gain experience.

Finally, we should consider whether more of these alternativepayment models should be mandatory, because the voluntary versionstypically involve excessively weak incentives to induceparticipation and because we learn less from them (randomization,for example, is dramatically more challenging in a voluntaryprogram). Former Secretary of Health and Human Services Tom Pricecut back mandatory bundles (and no, it couldn't have been becausehe anticipated these preliminary results, which may not turn out tobe definitive in any case). His replacement, Secretary Alex Azar,on the other hand, is more open to mandatory programs.

The bottom line is that although the new research is cautionary,I remain supportive of mandatory bundled payments. I hope SecretaryAzar introduces more of them, but we will need to continue tomonitor their results carefully. Let me be clear: If the ultimateresults from the hip and knee bundle are no different than thosefrom the first year, the program should be discontinued – but weare far away from any such conclusion. I suspect that the data fromsubsequent years will show stronger results.

This column does not necessarily reflect the opinion of theeditorial board or Bloomberg LP and its owners.

Peter R. Orszag is a Bloomberg Opinioncolumnist. He is a vice chairman of investment banking at Lazard.He was director of the Office of Management and Budget from 2009 to2010, and director of the Congressional Budget Office from 2007 to2008.

Copyright 2018 Bloomberg. All rightsreserved. This material may not be published, broadcast, rewritten,or redistributed.