Risk Scoring and the Luck of the Draw

Submitted by jonpearce on Sun, 2011-03-20 17:13

Many recent healthcare initiatives, including the Accountable Care Organizations that are created under health reform, utilize risk adjusters to compensate for the differences in disease status of the members.  These risk adjusters are used to establish payment targets that compare costs incurred by patients in the organization to what the costs “should be” based on those patients’ diagnoses.  The accuracy of the risk adjusters, therefore, is critically important when significant amounts of money are involved.  A small percentage difference between the risk-adjusted payment amount attributed to an organization and the organization’s actual costs may mean the difference between receiving a significant bonus and receiving nothing.  In the future, organizations may be penalized if their costs exceed the risk-adjusted amount.
This situation apparently came into play in the Medicare Physician Group Practice (PGP) demonstration, which took place from 2000 for through early 2008.  In this demonstration, ten participating physician group practices were eligible to receive payments from CMS based on meeting quality measurements and achieving expenditures lower than those computed by risk-adjusted metrics.  As a result of this demonstration, four PGPs earned significant payments from CMS for meeting the quality requirements and having costs significantly lower than their risk adjusted targets.  Of significant interest, at least to this writer, was the fact that all four of these groups “exhibited favorable cost trends prior to the demonstration”, and that “these sites had a cost saving infrastructure in place prior to the demonstration which may be one of the reasons why they elected to participate”[1].  This suggests two possible situations: (a) that these groups knew in general that their costs were low, and were willing to participate in the demonstration based on that fact, or (b) that these groups were able to compute risk-adjusted targets before going into the demonstration, found that they were below the targets, and therefore entered into the demonstration as a “no risk” situation.  In other words, they knew that they couldn’t lose.
No one claims that risk adjusters are perfect.  They under-predict some types of costs and over-predict others.  Like any statistical process, their accuracy improves with size of the population.  Their accuracy is also heavily dependent upon the correctness and completeness of diagnosis information submitted on the claims for those patients.  Under-reporting of diagnoses, especially for chronic care conditions, can severely impact the risk score of the organization.  Organizations that are new to risk score reporting may not be accustomed to the exhaustive requirements of submitting this type of diagnosis information.  Without improvement in reporting, those organizations will never perform well under any risk adjusted payment methodology.
Computing risk scores, however, is not a “black box”.  The hierarchal condition categories (HCC) computation methodology is well-documented and available to organizations wishing to utilize their own data to compute their own risk scores.  This appears to be a prudent step for any organization considering participation in any process in which risk scoring is utilized to evaluate their performance.  Reviewing the computed risk scores from your historical data, as compared to your current costs, may indicate a significant problem of under-reporting of diagnoses, which must be corrected before participation in the actual project begins.  It may show that your organization will be a victim of the inherent in accuracies in the methodology, and that you can’t win no matter what you do.  Or it may show that you’re already ahead of the process.  In any case, it’s important know this before starting out.  If you don't want to try to home-grow this capability, Singletrack Analytics has a suite of tools and reports that can give you a comprehensive look at your risk scoring profile.

[1] “Report to Congress, Physician Group Practice Demonstration Evaluation Report”, HHS 2009