Blog: RAND 4.0 Still Riddled with Methodology Flaws and Incomplete Data
The RAND Corporation’s latest hospital price report is, unfortunately, more of the same. This release comes amid historic financial losses for hospitals and health systems, and rapidly growing costs. Over the last two years of the COVID-19 pandemic we’ve seen how critical hospitals and health systems are to their communities and patients as they have attempted to weather repeated hospitalization surges, pioneered life-saving new treatments and in many cases taken on even more care responsibilities, like rapidly setting up testing and vaccination centers. They have done all of this while facing unprecedented financial challenges throughout the pandemic. Within that context, RAND has doubled down on its misguided prior work, with the same pre-conceived notions and flawed methods.
The underlying premise is fundamentally unsound: as we have previously pointed out, Medicare rates should not be used as a benchmark for commercial coverage. Medicare pays below the cost of providing care to Medicare patients – even the Medicare Payment Advisory Commission (MedPAC) acknowledges this fact.1 In 2020, Medicare paid 84 cents for every dollar spent by hospitals providing care for Medicare patients.
Over time, chronic underpayments threaten the communities that hospitals serve. Medicare underpayments were $75.6 billion in 2020 and have nearly doubled since 2014.2 Armchair critics argue that these underpayments provide incentive for hospitals to become more efficient. In the real world, a record number of rural hospitals shut their doors in 2020, following years of record-setting closures. These closures have significant implications for people’s ability to access the care that they need.3 Moreover, the RAND authors are overgeneralizing based on a limited and biased sample, which is comprised of self-funded employers who self-select to participate, 11 state all-payer claims databases, and health plans who similarly opt in.4 No matter how you slice it, this is a small share of what actually happens in hospitals and health systems in the real world:
- The claims included in the report represent just 1.2% of total inpatient admissions and 0.5% of total outpatient visits in the U.S. over the three-year period.5
- The authors tout that they have increased their sample to 4,102 hospitals. However, they are unable to report inpatient data for over 1,400 hospitals, or more than a third, because they have data covering fewer than 11 inpatient admissions in those hospitals.
- The sample accounts for less than 1% of all inpatient admissions in 30 states and less than 1% of all outpatient visits in 39 states.
- For example, in West Virginia, a state the RAND authors call-out for having above-average commercial rates, RAND’s analysis is based on 0.18% of all inpatient admissions and 0.07% of all outpatient admissions included over the three-year period.
The hospital-level data is similarly underwhelming. For about one-third of hospitals (33.6%) for which inpatient stays are reported, the authors had fewer than 100 inpatient stays over a three-year period upon which to draw their conclusions.
Tellingly, in this year’s iteration, when RAND incorporated more claims compared to previous versions, the average price for hospital services actually declined. This suggests what we have long suspected and pointed out publicly: you just can’t draw credible conclusions from such a limited and biased set of claims.
Similarly, RAND’s effort to link prices and quality falls short. RAND uses just one year of star ratings (2018) and Leapfrog scores (2019) despite the fact that those scores were refreshed several times during the measured period of 2018-2020. In addition, because of normal data reporting lags, some of the data included in star ratings and the Leapfrog scores could be up to five years behind the pricing data. Further, both Star Ratings and Leapfrog scores are a function of the measures included in them and simply do not measure all aspects of patient care, or even the aspects of care that may matter the most to patients.
It’s also telling that once again RAND 4.0 ignores the role another important set of stakeholders – health insurers and third party administrators – play in health spending. There is no discussion of the leverage insurers and third party administrators often have in markets they dominate. This omission occurs because RAND relies on data voluntarily supplied by insurers, who RAND agrees not to identify in its public reports.6 Moreover, RAND also ignores increasingly prevalent insurer policies and procedures that drive up costs for hospitals and may lead to dangerous care delays for patients.
The last two years have shown just how vital hospitals and health systems are to every community. Unlike other entities, our doors are always open, regardless of a patient’s ability to pay. Many hospitals took on public health roles during the COVID-19 pandemic by standing up testing operations and vaccination clinics. This is on top of the non-clinical care hospitals and health systems increasingly provide, often without additional resources, including programs that seek to improve housing stability and food security. No other part of the health care system has stepped up in the same way – and been on the front line – to meet these growing demands.
COVID-19 has changed the health care system, yet RAND’s commentary – already flawed to begin with – remains stuck in the past.
Many hospitals and health systems are now struggling to make ends meet as they continue to care for patients in their community day in and day out. Those already struggling are barely hanging on. Imposing cuts based on flawed Medicare rates would result in the stripping of desperately needed resources and funding from vital programs. Now more than ever, that should be a non-starter.
Aaron Wesolowski is AHA’s vice president of policy research, analytics and strategy. Benjamin Finder is AHA’s director of policy research and analysis.
__________
1 https://www.medpac.gov/wp-content/uploads/import_data/scrape_files/docs/default-source/reports/mar21_medpac_report_to_the_congress_sec.pdf
2 https://www.aha.org/fact-sheets/2020-01-07-fact-sheet-underpayment-medicare-and-medicaid
3 https://www.shepscenter.unc.edu/programs-projects/rural-health/rural-hospital-closures/
4 https://www.rand.org/health-care/projects/price-transparency/hospital-pricing/round4/faq.html
5 AHA Annual Survey, 2018-2020.
6 https://www.rand.org/health-care/projects/price-transparency/hospital-pricing/round4/faq.html