Editor’s Note: This paper is part of the USC-Brookings Schaeffer Initiative for Health Policy, which is a partnership between the Economic Studies Program at Brookings and the USC Schaeffer Center for Health Policy & Economics. The Initiative aims to inform the national health care debate with rigorous, evidence-based analysis leading to practical recommendations using the collaborative strengths of USC and Brookings.
Under the Affordable Care Act (ACA), the federal government operates a risk adjustment program in the individual and small group health insurance markets that transfers funds from insurers that attract lower-risk enrollees to insurers that attract higher-risk enrollees. In late December, the Centers for Medicare and Medicaid Services (CMS) published a proposed rule that would change how CMS calculates the enrollee “risk scores” that it uses to quantify differences in claims risk across insurers.
One of the CMS proposals is to begin using a “two-stage” estimation procedure to set risk scores. That procedure, which is described in detail below, would increase risk scores for lower-risk enrollees and reduce them for higher-risk enrollees. As a result, insurers that tend to attract lower-risk enrollees would pay less into risk adjustment, while insurers that attract higher-risk enrollees would receive less.
This analysis makes two main points about how this change would affect market outcomes:
- CMS’ proposal would drive enrollees toward skimpier plans, raise premiums for higher-risk enrollees, and potentially reduce competition: Because higher-risk enrollees tend to prefer insurance plans that offer more robust coverage (e.g., broader provider networks or more lenient utilization controls), the proposal would tend to raise the premiums of these plans relative to skimpier plans, driving enrollees into skimpier plans and raising premiums for the higher-risk enrollees who remained in more robust plans. Increasing the returns to attracting lower-risk enrollees could also spur insurers to stop offering certain plan features valued by higher-risk enrollees or make it harder for the market to support robust competition among insurers.
- Contrary to CMS’ hopes, the change is unlikely to substantially increase enrollment: This policy could, in theory, increase insurance enrollment by reducing the premiums of skimpier plans. However, it is unclear whether these plans’ premiums would actually fall in practice since they would experience an influx of higher-risk enrollees leaving more robust plans. Furthermore, even if the premiums of skimpier plans did fall, this would not reduce post-subsidy premiums for the large majority of individual market enrollees who receive the premium tax credit since the value of the tax credit is linked to the premiums of low-cost plans and would, thus, decline in tandem.
The analysis then examines whether these effects would be desirable. In order for the CMS proposal to improve market outcomes, the current risk adjustment system would need to be overcompensating for differences in claims risk between plans that tend to attract higher- and lower-risk enrollees, and, in turn, tilting the playing field toward plans that particularly appeal to higher-risk enrollees. While CMS suggests this is the case, economic theory and empirical evidence imply that CMS’ current methodology most likely undercompensates for these differences in claims risk, essentially because some aspects of health status are not captured in risk adjustment. The dominance of narrow network, tightly managed plans in the individual market is also consistent with the view that risk adjustment is currently undercompensating, not overcompensating, for differences in claims risk across plans. Ironically, CMS seems to share this view elsewhere in the same proposed rule, where it proposes to strengthen regulations that prohibit insurers from designing their benefits to discriminate against higher-risk enrollees (and certain other groups).
We thus conclude that CMS should abandon its two-stage proposal. While changes to risk adjustment are warranted, they should aim to increase, not reduce, payments to insurers with higher-risk enrollees.
Overview of ACA risk adjustment and CMS’ “two-stage” proposal
People with greater health care needs tend to select insurance plans that offer more robust coverage (e.g., broader provider networks, larger formularies, or more lenient utilization controls), a pattern called adverse selection. Without policies that compensate for adverse selection, insurance markets can function poorly. In the present context, the incentives created by adverse selection can drive enrollees into overly stingy plans by increasing the premiums of robust plans relative to skimpier ones or by causing insurers to refuse to offer plans with certain features. They can also shift premium burdens from lower-risk enrollees onto higher-risk enrollees by increasing the relative premiums of the more robust plans higher-risk enrollees prefer, in effect partially unwinding community rating regulations. In some circumstances, adverse selection can even threaten these markets’ ability to support robust insurer competition.
To avoid these problems, the ACA established a risk adjustment program in the individual and small group markets. Broadly speaking, risk adjustment seeks to sever (or, at least, weaken) the link between who an insurer enrolls and the costs the insurer incurs. To do so, the program collects information from insurers about each enrollee, notably age, sex, and the presence of certain health conditions. CMS uses that information to construct a “risk score” for each enrollee that reflects the claims costs that an insurer covering that enrollee is expected to incur. Simplifying slightly, CMS then uses each plan’s average risk score as its measure of that plan’s average claims risk and transfers funds from insurers with low average risk to insurers with high average risk in amounts commensurate with those differences.
As this description suggests, the keystone of the risk adjustment system is the algorithm it uses to generate risk scores. CMS currently uses a linear regression model to predict the liability an insurer would incur for a particular enrollee given the enrollee’s age, sex, health conditions, and enrollment duration.[1] Those predictions then become each enrollee’s risk score.
In a proposed rule that establishes risk adjustment policies for 2023 and beyond, CMS has proposed to change how it estimates the parameters of that regression model. (CMS has been considering changes like this for some time. It proposed essentially identical changes in last year’s payment rule but did not finalize them. In October 2021, it published a technical paper on these potential changes.)
At present, CMS chooses the parameters of the risk score model via an ordinary least squares regression. Roughly speaking, this approach chooses the parameters that generate the best predictions of claims spending on average across all enrollees, assigning the same weight to predictive accuracy for all enrollees.[2] By contrast, CMS’ proposed “two-stage” estimation procedure would assign greater weight to enrollees with lower expected spending, in effect prioritizing predictive accuracy for these enrollees. More precisely, CMS has proposed first estimating a regression using the current weighting scheme; it then uses the reciprocal of the predicted values from that “first stage” regression as weights (subject to a lower and upper bound on those weights) in a “second stage” regression that delivers the final parameters. Because the reciprocal of the spending predictions generated by the first stage is larger for enrollees with lower expected spending, this approach has the effect of assigning a higher weight to these enrollees.
Before proceeding, we note that the proposed rule also includes several other risk adjustment changes, including changes that allow the effect of certain severe conditions on spending to depend on the total number of conditions an enrollee has and allows the presence of a health condition to have a different effect on spending based on how long a person is enrolled, plus changes to when use of certain medications is used to identify presence of a health condition. These changes do not raise the same concerns as the two-stage proposal and may have the potential to make risk adjustment work better. We do not discuss them further in this piece but have discussed some of them in prior writing.
How would adopting the two-stage estimation procedure affect market outcomes?
The technical paper that CMS published last fall presented evidence showing that adopting the two-stage estimation procedure would affect enrollee risk scores in two ways.[3] First, for reasons that we will return to below, the change would increase risk scores for enrollees with low expected spending, generally people with no or minor underlying health conditions. Second, the change would reduce risk scores for enrollees with high expected spending, generally people with serious health conditions.
This change in risk scores would, in turn, change risk adjustment payments. Insurers that tend to attract low-risk enrollees (typically those offering skimpier coverage) would see their average risk scores rise, while insurers that tend to attract higher-risk enrollees (typically those offering more robust coverage) would see their risk scores fall.[4] As a result, insurers that tend to attract low-risk enrollees would pay less into risk adjustment, and insurers that tend to attract higher-risk enrollees would receive less out.
Economic theory implies that these changes would have three types of effects:
- Shift enrollment toward skimpier plans:The shift in risk adjustment payments described above would raise net-of-risk adjustment claims costs for plans offering more robust coverage relative to those offering skimpier coverage. This, in turn, would raise the relative premiums of plans that offer more robust coverage, driving enrollees away from more robust plans and toward skimpier plans. Reduced payments to insurers that enroll higher-risk enrollees could also lead insurers to stop offering certain plan features that particularly appeal to those enrollees.
- Shift premium burdens toward higher-risk enrollees: A shift in the relative premiums of different types of plans would also change how premium burdens are distributed. Higher-risk enrollees would tend to see increases in premiums relative to lower-risk enrollees since higher-risk enrollees are relatively more likely to be enrolled in robust plans rather than skimpier plans.
- Reduce choice and competition: Reducing risk adjustment transfers could also potentially make it harder for the market to support robust competition among insurers, leading to high premiums, limited plan variety, or both. In particular, it is often reasonable to assume that the lowest-risk enrollees are particularly price sensitive. Recent research has highlighted that, without adequate risk adjustment, this can give rise to extremely intense price competition aimed at attracting these low-risk enrollees. In some cases, that price competition may become so intense that it causes all insurers to incur losses, forcing insurers to exit until just one or two remain.
In the proposed rule and recent technical paper, CMS suggests that adopting the two-stage procedure would have another notable effect: causing more people to take up coverage. This is theoretically possible. If this change to risk adjustment reduced the premiums of skimpier plans, either directly by reducing what those plans paid into risk adjustment or indirectly by creating incentives for those plans to become even less generous, these lower premiums could conceivably attract new people into the market.[5] However, there are good reasons to doubt that enrollment would rise in practice.
To start, it is unclear that the premiums of the lowest-cost plans would actually fall. As noted above, the change to risk adjustment would unambiguously increase the (relative) premiums of more robust plans, causing enrollees to leave those plans. Those enrollees are likely to have higher risk than enrollees previously covered by the skimpier plans, which would put upward pressure on the premiums of skimpier plans that offset the direct downward pressure created by the change in risk adjustment payments.[6] Recent work by one of us (Layton) with others has highlighted this possibility. Using a model calibrated to match empirical estimates of consumer behavior under Massachusetts’ pre-ACA CommCare program, the authors show that weakening risk adjustment typically does not reduce the premiums of less generous plans, precisely because those plans experience an influx of higher-risk enrollees.[7]
Even if the gross premiums of these plans fell, enrollees who receive the premium tax credit—71% of the ACA-compliant individual market as of 2020 and a higher percentage at present (and for as long as the expansion of the premium tax credit included in the American Rescue Plan Act remains in effect)—would experience little or no reduction in net-of-subsidy premiums.[8] This is because the value of the premium tax credit is based on the premium of the second-lowest-cost silver plan, so the tax credit’s value would tend to fall in tandem with the premiums of less generous plans. Enrollment is thus unlikely to meaningfully rise in this group. In fact, enrollment could well fall if insurers’ enhanced incentives to avoid higher-risk enrollees led them to make these plans less generous and, thus, less attractive.
Furthermore, any decline in the value of the premium tax credit caused by reductions in the premiums of skimpier plans would magnify the harm to enrollees in more robust plans. Unlike enrollees in skimpier plans, enrollees in more robust plans would now face higher gross premiums (due both to the mechanical decline in risk adjustment payments to these plans and the departure of relatively low-risk enrollees for skimpier plans), a harm exacerbated by a decline in the premium tax credit.
Would these changes in market outcomes be good or bad?
The preceding section discusses how adoption of the two-stage procedure would affect market outcomes. But it does not, in itself, demonstrate whether these effects would be good or bad. Indeed, if risk adjustment was currently overcompensating for differences in claims risk between plans that tend to attract higher-risk and lower-risk enrollees, then the premiums of plans that offer more robust coverage could currently be too low in relation to the premiums of skimpier plans; this could drive too many people into more robust plans (in the sense that those plans would attract too many people for whom the incremental cost of providing that coverage exceeded its incremental value) or shift the distribution of premium burdens too far toward lower-risk enrollees. In this case, the two-stage procedure—or other steps that reduced risk adjustment transfers—could improve market outcomes.
This is CMS’ core rationale for adopting the two-stage procedure. CMS has documented that its current risk score model underpredicts spending for enrollees with low expected spending.[9] (This existing underprediction is why the two-stage procedure would increase risk scores for low-risk enrollees.) In light of that finding, CMS has expressed concern that it is systematically overestimating differences in claims risk between plans that tend to attract higher- and lower-risk enrollees and, in turn, overcompensating for those differences, thereby threatening the viability of plans that appeal to lower-risk enrollees.
But both economic theory and empirical evidence offer reason to doubt that CMS’ existing risk adjustment methods lead it to overestimate differences in claims risk between plans that attract lower- and higher-risk enrollees. Instead, it appears much more likely that CMS’ is underestimating these differences and, thus, undercompensating for them, a problem that would be worsened by the two-stage procedure.
Economic theory implies that differences in risk scores understate true differences in claims risk
To start, economic theory generally suggests that CMS’ current approach to risk adjustment will tend to systematically underestimate differences in claims risk across plans. The crux of the issue is that there are aspects of health status that CMS does not observe when constructing risk scores. In particular, some aspects of health status do not map to one of the health conditions included in the risk score model, and some of the health conditions that are included in the model are coarse categories that combine disease states of varying severity. These data limitations are likely an important reason that the current risk score model explains less than half of the variation in spending across enrollees.
It is generally reasonable to believe that plans that attract people who are higher risk on aspects of health status that are incorporated into risk scores will also attract people who are higher risk along other dimensions. (For example, plans that attract a disproportionate share of people with diabetes will tend to have a larger share of diabetics with more severe forms of the disease.) If that is true, then the difference in average risk scores between plans that attract (observably) higher- and lower-risk enrollees will be smaller than the true difference in claims risk. It follows that CMS’ current approach will undercompensate for those differences and that skimpier plans will retain some selection-related cost advantages.
Empirical research offers direct evidence that risk scores understate true differences in claims risk
The research on Massachusetts’ pre-ACA CommCare program that was cited above offers direct evidence that differences in average risk scores understate true differences in claims risk across plans. Using discontinuities in the subsidy schedule facing enrollees in this market, the authors estimate how both actual claims risk and risk scores differ between people with lower and higher propensities to select plans that offer more robust coverage.[10] They find that the difference in risk scores between enrollees with the highest and lowest propensities to enroll in more robust coverage is only about two-thirds as large as the actual difference in claims risk between these enrollees, implying that CMS’ current methods likely underestimate differences in claims risk and, thus, fall short of offsetting these differences.[11]
Market outcomes are consistent with the view that risk adjustment is undercompensating for selection
Outcomes in the individual market are also consistent with the view that CMS’ current risk adjustment methods do not fully offset selection against more generous plans. Notably, the typical plan offered through the Health Insurance Marketplace has a substantially narrower provider network than the typical employer plan, and Marketplace plan offerings have gravitated increasingly toward more tightly managed product types. While this pattern could, in principle, be a response to consumer preferences for lower-cost plans, it is consistent with the view that selection pressures are driving plan offerings and enrollment toward these types of plans. At minimum, it suggests that CMS’ current risk adjustment methods are not substantially overcompensating for selection, which would tend to reduce the viability of these plans.
It is notable that, elsewhere in the same proposed rule, CMS itself expresses concern that insurers are distorting their benefit designs in ways aimed at helping them avoid higher-risk enrollees. These concerns are part of CMS’ justification for proposals that would tighten its regulations barring health plan designs that discriminate on the basis of health status (or certain other factors). If risk adjustment were overcompensating for selection, then insurers would already be avoiding these types of plan designs, as it would be highly profitable to attract higher-risk enrollees. Instead, they would design their plans to be “too generous” with respect to benefits that are highly valued by high-risk enrollees. CMS’ stated concerns are, thus, hard to reconcile with the view that risk adjustment is currently overcompensating for selection.
A note on risk score model “fit” as a basis for evaluating the two-stage proposal
Changes to risk adjustment should be evaluated based on how they affect market outcomes. Consistent with the discussion above, this typically depends on how closely risk adjustment transfers track with differences in claims risk across plans. In practice, however, CMS has often focused on risk score model “fit,” or the degree to which the model accurately predicts claims spending at the enrollee level.
Model fit can be a useful heuristic; for example, if adding a new variable to a risk score model improves predictive accuracy at the enrollee level, that will often be true at the plan level as well. On the other hand, as the discussion above makes clear, focusing narrowly on model fit can also be seriously misleading; simply maximizing fit at the enrollee level and then using the resulting risk scores to gauge differences in claims risk across plans will often understate true differences in claims risk across plans.
Regardless, if CMS does wish to use model fit as its main criterion for guiding risk adjustment policy—as much of the discussion in the proposed rule suggests it does—it is worth noting that the two-stage procedure performs poorly in this respect. In particular, the two-stage procedure would reduce the overall predictive accuracy of the risk score model at the enrollee level—at least as measured by R2—even as it increased predictive accuracy for enrollees with low predicted spending.[12]
The proposed rule acknowledges this reduction in overall model fit but dismisses it as “minor” and therefore concludes that this model change presents “limited tradeoffs.” However, if model fit is the right criterion for risk adjustment policy, it is unclear why any reduction in model fit is acceptable. (The small change in overall fit does not even guarantee this policy change would produce only a small deterioration in the functioning of the market; fit is a statistical measure, not an economic one, and the mapping from fit to market outcomes is not at all clear.) Moreover, the reduction in overall fit indicates that the improvement in fit stemming from greater predictive accuracy for enrollees with low predicted spending is smaller than the loss in fit from reduced predictive accuracy along other dimensions, which suggests that the change does, in fact, present important tradeoffs, even measured by fit alone.
Conclusion
CMS’ proposal to begin using a two-stage estimation procedure to set risk scores would reduce what insurers that attract lower-risk enrollees pay into risk adjustment and, correspondingly, reduce what insurers that attract higher-risk enrollees receive from risk adjustment. This, in turn, would drive enrollees out of plans that offer more robust coverage, shift premium burdens onto higher-risk enrollees, and potentially reduce insurer competition. There are strong reasons to believe that CMS’ approach to risk adjustment already underestimates differences in claims risk between plans that attract lower- and higher-risk enrollees and, thus, undercompensates for adverse selection, which implies that these shifts would be undesirable. This is not to say that the current risk adjustment system is perfect. But we believe that improving risk adjustment will require CMS to change its focus. Rather than focusing narrowly on changes that would improve model fit at the enrollee level—and improve fit for low-risk enrollees in particular—CMS should focus on how to ensure that risk adjustment transfers are aligned with differences in claims risk at the plan level, including differences in risk that may not be directly captured in risk scores. There may be several potential paths to that end, one of which we have explored previously. But regardless of the precise path policymakers choose, improving risk adjustment is likely to involve increasing, not reducing, the transfers that plans with lower-risk enrollees make to plans with higher-risk enrollees.
[1] CMS fits different linear regression models for each actuarial value metal tier.
[2] CMS’ current regression approach does weight by the number of months a person is enrolled, so strictly speaking it assigns to the same weight to each enrollment month, not each enrollee.
[3] The effect of the two-stage model on its own is depicted in Figure 2.2 of the technical paper. The incremental effect of the two-stage model over the other main risk adjustment changes included in the proposed rule can be determined by comparing the estimates in Figure 4.2 and Figure 5.1 of the technical paper. CMS notes in footnote 63 of the technical paper that the risk scores it used in constructing the figures were not properly normalized. Normalization would not change the qualitative patterns; however, with appropriate normalization, the increase in risk scores among enrollees with low expected spending would be somewhat smaller than CMS reports, while the decline in risk scores among enrollees with high expected spending would be somewhat larger.
[4] In theory, this could fail to be the case if low-risk-score enrollees who enroll in plans typically preferred by higher-risk enrollees experienced unusually large increases in risk scores or if high-risk-score enrollees who enroll in plans typically preferred by lower-risk enrollees experienced unusually large reductions in risk scores. The CMS technical paper does not provide the information needed to rule out this possibility, but this seems unlikely in the context of this particular proposal. This might be more likely in the context of CMS’ other proposals, which add new variables to the risk adjustment system and thus may help to identify which apparently lower-risk enrollees are actually relatively higher-risk and which apparently higher-risk enrollees are actually lower-risk.
[5] CMS suggests that the policy change might also attract additional enrollees through a distinct mechanism: inducing insurers to newly offer (or retain) some types of plans that particularly appeal to low-risk enrollees. However, this mechanism is less plausible, as it is unclear why these plans could not already be offered (albeit at a higher premium). Whereas unchecked selection can cause plan types that particularly appeal to sicker enrollees to enter a “death spiral” and vanish from the market, there is not an analogous phenomenon for plans that particularly appeal to low-risk enrollees, even in the presence of excessive risk adjustment transfers. This is because net-of-risk-adjustment costs for lower-risk enrollees will still be bounded from above, which places an upper limit on how far the premium of less generous plans could possibly need to rise.
[6] This assumes that risk adjustment is currently undercompensating for differences in claims risk between enrollees who prefer more and less generous plans, so that attracting higher-risk enrollees increases net-of-risk adjustment claims costs. As discussed at length below, this appears to be the case in practice.
[7] While these findings pertain most directly to the individual market, the dynamics in the small group market would likely be qualitatively similar. In any case, selective pressures are generally less intense in the small group market (since many choices are made at the employer level rather than the individual level), so the ability of risk adjustment policy to affect market outcomes—for good or ill—is likely smaller as well.
[8] This 71% estimate reflects data on subsidized enrollment from CMS’ Marketplace effectuated enrollment report and data on total ACA-compliant enrollment from CMS’ risk adjustment summary report. For Massachusetts and Vermont, the risk adjustment data combine individual and small group market enrollment, so we assume that total ACA-compliant enrollment equals total Marketplace enrollment.
[9] This would remain the case after implementing the other methodological changes included in the proposed rule, albeit to a lesser degree.
[10] This work uses an earlier version of the CMS risk score model, but CMS’ approach has not changed significantly in the intervening years, so these results are likely a good guide to CMS’ present risk score model.
[11] Research in the related context of the Medicare Advantage program has reached the similar conclusion that differences in risk scores between traditional Medicare and private Medicare Advantage plans understate true differences in claims risk across these two portions of the Medicare program.
[12] Since CMS’ proposal explicitly prioritizes predictive accuracy in this group over overall accuracy, this is unsurprising. In fact, it is mathematically guaranteed. CMS’ prior method obtains the coefficients that maximize R2 by definition. Thus, any other approach, including the “two-stage” approach, will achieve a lower R2.
Acknowledgements: We thank Richard Frank for helpful comments on a draft of this piece. We thank Conrad Milhaupt for excellent research assistance and Caitlin Rowley for excellent editorial assistance. All errors are our own.