Editor’s note: This post was originally published on Health Affairs Blog on 11/12/2020.
In a recent post, the administrator of the Centers for Medicare and Medicaid Services (CMS) reported that the Medicare Shared Savings Program (MSSP) generated $2.6 billion in gross savings in 2019 and $1.2 billion in net savings after accounting for shared-savings payments to participating accountable care organizations (ACOs). Achieving this level of savings would constitute remarkable progress in a short time. Relative to CMS’s own estimates, total net savings rose by $876 million over 2017 levels. Moreover, compared to savings estimated in prior evaluations based on counterfactual spending instead of benchmarks, the 2019 reported net savings is more than three times the cumulative net savings over the program’s first three full years of operation ($358 million) and more than eight times the net savings produced in 2015 ($145 million). The surge in programwide savings vastly outpaced participation, as the number of participating ACOs increased by only 21 percent from 2015 to 2019.
Based on the administrator’s interpretation of the data, it would appear that the MSSP is now circling the bases after stumbling out of the batter’s box. What changed? Ostensibly, the recent overhaul of the program, “Pathways to Success,” and specifically, the increased downside financial risk imposed on ACOs under Pathways.
While this assessment may seem like welcome news in the long struggle to elicit more efficient care delivery through payment reform, it is not scientific. In fact, the estimates grossly overstate the true savings for reasons that have been well described. Much of the acceleration in savings is an artifact of selective entry and exit of ACOs based on their established spending levels relative to new spending targets (benchmarks). Starting in 2017, benchmark updates began to disadvantage ACOs with high spending for their region and advantage those with relatively low spending.
Touting benchmark-based calculations as proof of program success despite countervailing evidence is not just harmless promotion. It can impede clear and productive policy discussions by promulgating misconceptions (such as downside risk as a game-changer), obfuscating persistent problems with the MSSP and diverting attention from the need for reform.
Pathways has not been the success it claims to be. Rather, it erred in trying to push the pace of savings without sufficiently attending to the various program parameters that shape ACOs’ incentives to participate and lower spending. In this two-part blog post, we provide a more rigorous and nuanced interpretation of MSSP performance data, describe problems with the program’s design, and lay out key considerations for improving the MSSP over the long haul. Our objective is to help get a repeatedly derailed conversation back on track.
Benchmark-Based “Savings” Do Not Measure Program Success
Benchmarks are not counterfactuals. This bears repeating. Benchmarks are not counterfactuals. In other words, benchmarks do not tell us what Medicare spending for ACO patients would be in the absence of ACO participation in the MSSP. Nor should they necessarily. The purpose of setting benchmarks, along with risk-sharing provisions around those spending targets, is to establish incentives for ACOs to participate and save. Benchmarks are an element of model design, not a metric of model success. Estimation of actual savings requires a rigorous investigation using quasi-experimental designs that establish a counterfactual under reasonable assumptions. Unfortunately, for the life of the MSSP, comparisons of ACO spending with benchmarks have been widely interpreted as quantifying program savings, causing a great deal of confusion. In the first few years of the program, these calculations underestimated actual savings for a variety of reasons. More recently, they have overstated savings because of selective participation.
Before 2017, the basis for an ACO’s benchmark in the MSSP was the ACO’s historical spending level during a baseline period. In 2017, CMS began setting benchmarks based on a blend of ACO-specific historical spending with average spending in the ACO’s region. Pathways then accelerated the timeline for this blending and required downside risk earlier in ACO contracts. As we discuss below, regional benchmarks and downside risk hold conceptual appeal; indeed, they are hallmark features in Medicare Advantage (MA) plan payment. But in the context of a voluntary program with overall weak incentives to lower spending, the combination of regional benchmarks and downside risk has been costly. Providers with spending above the regional average (who face penalties if spending remains unchanged) have disproportionately exited the MSSP, while those with spending below the regional average have disproportionately entered or remained in the program.
This pattern of selective participation is costly to Medicare for two reasons. First, it increases bonus payments to low-spending ACOs by more than Medicare can recoup in penalties from high-spending ACOs because there are fewer high-spending ACOs in the program. Second, it means Medicare forgoes savings from providers who are less efficient (have higher spending levels) upon entering the MSSP and thus have greater savings potential. Indeed, ACOs with relatively high initial spending have achieved greater true savings (net of regression to the mean) than those with relatively low initial spending. If the objective of the MSSP is to lower spending, participation by providers with high spending is critical to its success.
Benchmarks that subsidize already efficient providers may not be wasteful in the long term if they, in concert with other program parameters, motivate less efficient providers to improve and eventually participate and encourage more efficient providers to become even more efficient. Thus, the costs of selective participation must be considered in the context of the design and trajectory of the program as a whole. But careful inspection of the MSSP (as we provide below) should raise concerns that it may be unraveling into unproductive subsidies in its current configuration.
What are the true savings achieved by the MSSP in 2019? Have they grown from the early estimates generated by quasi-experimental studies? It is very hard to say. Evaluations using approaches that establish plausible counterfactuals based on control group trends have become very challenging after eight years of MSSP operation. There has been substantial attrition; only 123 (36 percent) of the 339 ACOs entering the program in 2012–14 are still participating in 2020. The control group has been contaminated by a proliferation of other alternative payment models. Increases in coding intensity have complicated risk adjustment, as has the transition from ICD-9 to ICD-10. And as provider consolidation has continued unabated, organizational membership has evolved, complicating interpretation of organizational changes in spending over time.
Did the MSSP save Medicare $1.2 billion in 2019? No—not even close. Might the true net savings be close to zero? Quite possibly. Is the MSSP headed in the right direction? No. Can reforming MSSP design make the program more successful? Yes.
Promise And Problems In The MSSP
Critics have often pointed to the modest net savings generated in the first few years as evidence that the ACO model “doesn’t work.” This reflexive assessment is misleading and counterproductive to the policy debate. For several reasons, the MSSP is more accurately characterized as a program of great potential limited by substantial, but addressable, problems.
Early Net Savings Understate Progress
First, gross savings are a better measure of behavioral change than the net savings. Quasi-experimental evaluations have found that ACOs reduced spending by a total of $1.6 billion from 2013 to 2015 before accounting for bonus payments, thus suggesting that at least some participants are making meaningful inroads in the effort to lower wasteful spending. Second, direct net savings to Medicare underestimate total net savings because of spillover effects. These include more efficient care delivery for non-ACO patients served by ACOs and reductions in MA plan payments that result from declines in fee-for-service spending.
Success Is A Long-Term Concept
Third, substantial savings in the short term are unnecessary for the MSSP to “work” over the long term. As growth in Medicare fees is slowed—with the Medicare Access and CHIP Reauthorization Act of 2015 (MACRA) instituting a nominally flat trajectory in the physician fee schedule through 2025—the MSSP becomes an increasingly attractive option to providers over time. Population-based payment models give providers greater flexibility in selecting care inputs by decoupling revenue from the provision of specific services. As fees fall in real dollars, taking advantage of this flexibility as an ACO to deliver more efficient care eventually dominates the alternative of delivering less efficient care reimbursed as a non-ACO provider. As participation increases and most or all of Medicare spending becomes covered by the MSSP, it gives Medicare a mechanism by which it can control long-term spending growth. Thus, a 2030 version of the MSSP may “work” not because spending falls substantially below benchmarks but because the MSSP can set benchmark growth at a desirable rate.
However, CMS has not coherently articulated a vision for the MSSP in 2030 and beyond. Long-term thinking makes it clear that participation matters. On that score, the MSSP has not done well of late. Coincident with the implementation of Pathways, the number of ACOs participating in the MSSP has fallen from 561 in 2018 to 517 in 2020. Although the number of beneficiaries attributed to participating ACOs rose slightly from 10.5 million in 2018 to 11.2 million in 2020, this represents a declining share of the traditional fee-for-service Medicare program (from 26.3 percent to 25.7 percent), which has grown with the aging of the population. These participation trends under Pathways contrast starkly with the program’s steady rapid expansion in the preceding five years.
Incentives To Save Have Been Weak
Fourth and finally, the MSSP has been plagued by weak incentives since its inception. That participating ACOs have produced any savings at all in response to the program is arguably astonishing. The primary reason why incentives have been so weak is the mechanism by which benchmarks are periodically “rebased.” Every three years, an ACO’s benchmark is reset to its most recent level of spending, thereby linking the ACO’s benchmark growth to its own performance. Consequently, effective efforts to lower spending are penalized with lower subsequent benchmarks. This form of rebasing not only limits the time ACOs have to accrue returns on upfront investments, it also obliterates incentives to engage in strategies that must be continued to maintain lower spending.
For example, consider an ACO in a two-sided contract using case managers to follow patients into skilled nursing facilities (SNFs) to reduce excessively long stays. A static calculation may conclude that successful implementation of this strategy in one year generates a bonus for the ACO in that year. But a dynamic calculation additionally considers the strategy’s subsequent downward effect on the ACO’s benchmark for its next performance period. If the ACO does not continue its effort to shorten SNF stays, spending on postacute care would revert to its prior level (above the new benchmark), causing the ACO to be penalized. The short-term profits would thus be offset by a future loss, in the form of a penalty payment or the cost of indefinitely maintaining the effort to avoid the penalty.
The main advantage of transitioning benchmarks to a risk-adjusted regional or national spending average is that it decouples an ACO’s benchmark growth from its savings, which greatly strengthens the incentive to lower spending. In isolation, regionalizing benchmarks is therefore an attractive program refinement. However, in the context of a voluntary program with downside risk, continued rebasing of the historical component of benchmarks, inadequate risk adjustment, and an alternative (traditional fee-for-service) that is not yet less appealing, the participation consequences of over-aggressive benchmark regionalization may be crippling. A critical question, the answer to which is unknown, is whether the participation response to regionalized benchmarks would have been as strong and selective if downside risk and rebasing of the historical benchmark component were eliminated and methods of risk adjustment improved.
ACO incentives to save also are weakened by low shared-savings rates. Particularly for ACOs that are large health systems, the incentives to provide less low-value care or substitute lower-price alternatives are weakened by the loss of fee-for-service profits that occur when providers deliver fewer or lower-margin services. When ACOs receive only 50 percent of the savings (or a lower share if their quality score is imperfect), those lost fee-for-service profits typically swamp the bonuses gained. For some tracks, Pathways lowered the savings rate even more. Together, rebasing and low shared-savings rates allow Medicare to share in the savings sooner but ultimately limit the savings that Medicare can ever share by weakening the incentives for ACOs to ever save. Pathways includes a track that allows ACOs to keep 75 percent of savings, but uptake has been limited, most likely because of the moving goalposts (that is, benchmark rebasing) and substantial downside risk requirements (up to 75 percent).
Given the overall weak incentives, it is not surprising that the early bright spots with respect to savings in the MSSP correspond to areas where the incentives to save are stronger. Physician group ACOs have generated greater savings than large health systems, corresponding to the lesser loss of fee-for-service profits that physician groups incur when they lower total spending. In general, providers that deliver less of the spectrum of care have stronger incentives as ACOs because they have more opportunities to lower spending on care that other providers deliver to their patients. Similarly, savings in postacute care have been prominent, consistent with the low rates of SNF ownership by most ACOs; that is, few ACOs incur substantial revenue losses when limiting postacute care in facilities. And the greater savings among ACOs with higher initial spending are consistent with stronger incentives to save when the costs of reducing spending are lower.
Taken together, these considerations suggest that the ACO model has neither flourished nor failed. Such reductionist characterizations only muddy debate and distract from the task at hand. The MSSP is a program that has exhibited promise but has been beset by implementation challenges. It has achieved precisely what it has been designed to elicit—small savings and, more recently, selective participation. The patterns of participation and savings to date nevertheless suggest potential for a successful program if the incentives to participate and save can be strengthened.
The Upside Of Downside Risk Has Been Overstated
The role of downside risk in the MSSP merits special attention, as the rhetoric around Pathways has focused almost exclusively on downside risk as a means to instill greater accountability and speed sluggish savings. The administrator’s recent post links the surge in “savings” directly to the greater proportion of ACOs in contracts with downside risk. Similar conclusions were drawn prior to Pathways based on the observation that ACOs electing to participate in tracks with downside risk had greater savings (relative to benchmarks) than ACOs in one-sided tracks.
All else equal, downside risk-sharing provisions unquestionably strengthen incentives for participating providers to lower spending. However, there is no empirical evidence that downside risk has accelerated savings in the MSSP. In fact, the data suggest that the savings attributed to downside risk may be illusory as well—an artifact of selection. As previously described, ACOs entering tracks with downside risk had lower spending levels relative to their benchmarks before switching tracks; the pattern suggests that these ACOs selected tracks with downside risk to take advantage of higher shared-savings rates available in those tracks, knowing that they faced a low probability of losses. Recent MSSP performance data show that 86 percent of ACOs in tracks with downside risk in 2019 had earned shared savings prior to accepting downside risk, compared with just 46 percent of ACOs remaining in one-sided contracts. Moreover, the potential advantages of imposing downside risk cannot be attained without offsetting consequences. First and most important are the consequences for participation in a voluntary program, as described above (less and selective participation). Second, although ACO contracting appears to have had a limited impact on provider consolidation relative to other drivers, downside risk requirements may increase ACO-related consolidation, as providers with market power can amass the necessary reserves more easily. Third, the costs of obtaining reinsurance or health management services to limit the risk of losses can erode the comparative advantage of a direct contracting model such as the ACO model over Medicare Advantage: the lower administrative costs from not requiring risk-bearing intermediaries. Fourth, from a fiscal perspective, the losses recouped by Medicare through downside risk are at least partially offset by increased payments in the form of advanced alternative payment model bonuses to ACOs that accept downside risk.
Thus, as we consider MSSP reform, it is important to weigh the conceptual merits of downside risk expected from static calculations that is, when all else is held equal) against countervailing dynamic effects of downside risk on other factors. It is important to contemplate the role of downside risk in the context of other program parameters such as benchmark setting and shared-savings rates. In part 2 of this blog post, which will appear in this space tomorrow, we discuss how to strengthen ACO incentives to save in the MSSP while encouraging participation and advancing the program’s broader goals.