Research Identifiable Files Sourced from Managed Care Data

CMS creates Research Identifiable Files (RIFs) for Encounters and Part D Events by compiling data sent from various Part C (Medicare Advantage) and Part D plan sponsors. As required by contract, Part C and Part D organizations submit administrative data to CMS data systems. While these systems perform validity checks, the final output still depends on the quality of the data from plans.

Over the course of my years working with CMS RIF data, I have personally observed several instances of apparent data quality issues on an organizational and sometimes provider level. I have seen such anomalies in both Encounter and Part D event RIFs, so when I use these files now, I evaluate data quality by disaggregating by plan (and sometimes provider) over time. This is easier to remember when metrics are unexpectedly low, but, as in one example below, sometimes the overall metric may look only slightly low, masking the underlying data quality issue.

Examples

The following three examples highlight likely data quality issues I have observed in Encounter and Part D Event RIFs.

Missing Admission Date

If you’re more familiar with Fee-For-Service (FFS) data, you may have noticed that admission dates are always filled on inpatient (IP) RIF claims. IP Encounter RIFs, on the other hand, do have records with missing admission dates. For example, after excluding chart reviews from the 2023 file, admission date is missing from 9.6% of records.

The admission date is key for an IP stay. It can be used to group records when parts of a stay are broken up into multiple records (more likely for long stays). Claim from date could be used as a proxy for admission date (it is always filled even for Encounter records), but it would be erroneous for multi-record stays. And the simple fact that the admission date is missing makes these records’ overall quality more questionable.

Is there something driving these records or are they randomly distributed? Digging deeper, I noticed the missing information was concentrated in certain CMS contracts (CNTRCT_NUM). When I aggregated plans to the parent organization level (from the Plan Characteristics File), I found 89% of claims with missing admission data are from Humana.

Table 1. Top five data sources of IP Encounter records with missing admission date, 2023

The records also have a disproportionate CLM_FREQ_CD= ‘7- Replacement of prior claim’ (65.7% vs. 24.1% overall) and DRG =’000’ (90.6%). The proportion of these records tick up over the course of the year (January: 2.9%, December: 17.0%). Does this extra information influence the decision about how to handle these records? Do we use claim from date as a proxy, or do we exclude them altogether? It depends on the aims of the research.

Zero Cost Drugs

While working on learning materials for a workshop, I discovered that the 2021 PDE RIF contains 2.1% zero cost events totaling 30.1 million records. In other words, the total drug cost field on these records is zero. Since all events in the file are covered fills, it’s unexpected to see events with zero cost. How can it be that a drug costs nothing to the plan, the patient and all other payers? After discussing it with other experienced PDE users, our best guess is that it is a data quality issue.

A bit more digging revealed these records are not random. Like in example 1, the vast majority are from Humana (95%), and nearly all are generic mail order fills.

Table 2. Top five data sources of zero cost drug fills, 2021

Humana accounts for only 17% of all records in the 2021 PDE, so these zero cost anomalies are highly disproportionate. Due to the concentration in one plan, it seems reasonable to assume this is a plan data quality issue.

Implausible Drop in Record Count by Provider

While studying a primary care provider organization, team discovered a strange drop in evaluation and management (E&M) visits in late 2019 at their Philadelphia locations. No drop was observed during the same period at their Jacksonville locations.

Image 1. Sudden drop in E&M visits in November and December for primary care clinic members in Philadelphia

Note: Clinics were identified using National Provider Identifiers (NPIs).

The clinics in Philadelphia and Jacksonville opened in August and September 2018 respectively, which is why the lines start at a very low percentage and gradually increase. That trend makes sense. What does not make sense is the steep drop in November and December 2019 for Philadelphia.

We could not wrap our heads around such a sudden drop in E&M visits, especially for an organization so focused on primary care. But since we knew they are paid through capitated Part C contracts rather than fee-for-service, they don't necessarily need to submit every claim to the payer to get paid. This led us to believe this was a data glitch. It’s likely the organization missed some record transfers to the payers, which meant the final data sent to CMS was incomplete. The chart below backs this theory up. It shows that the sharp decline in November and December is present for both payers that insure the bulk of clinic members.

Image 2. Sudden drop in E&M present for both Part C payers that insure the bulk of clinic members in Philadelphia

Note: Chart reviews excluded.

Based on these analyses, we felt that the data from November and December was unreliable and we chose to drop these months from our study.

One final thing to emphasize: this example highlights the importance of plotting data over time. If we had not disaggregated by month, we would not have detected this data quality issue.

Summary: Consider Disaggregating by Plan (and Possibly Provider) and by Time

In my experience, CMS Encounter and PDE RIFs contain largely rich, good quality data. However, the examples here show that potentially significant data anomalies may be hiding. Awareness is helpful. Now I know that I can trust but verify, particularly when measures seem unexpectedly low.

Metrics can be disaggregated by plan and even by provider to spot unusual patterns and outliers. And since data quality can change over time, you may also consider disaggregating by month or year.

When questionable variation has been detected, it is up to each researcher to determine how to handle it given their project’s aims. For example, imputing missing values, doing sensitivity analyses or excluding records may help to support proper inference.