Consumer-directed healthcare reform models, such as health savings accounts with high deductibles, depend on giving consumers the information they need to select the providers best suited to their needs and pocketbooks. This requires transparency of cost and effectiveness. In addition, such models are designed to rewards providers with incentives for doing good work, such as pay for performance.

This page does not focus on the debate about whether providers' performance should be evaluated. Rather, it addresses the issue of using insurance claims (administrative) data to evaluate provider effectiveness and improve the quality of care.

Performance measures and claims data


Performance measures often accessible from claims data:
  • Mortality rates
  • Complications
  • Access to appropriate health services
  • Length of stay (duration of treatment)
  • Cost of care
  • With G-codes, compliance to certain practice guidelines can be determine; without G-codes, adherence to certain medication guidelines can still be determined (see http://care.diabetesjournals.org/cgi/content/??abstract/27/12/2800 )

Performance measures NOT accessible from claims include:
  • Biomedical measures (e.g., results of laboratory tests and vital signs)
  • Disease specific symptoms (severity, duration, and frequency, both physical and psychological/cognitive/emotional) based on patient self-report and clinician observation/report
  • Quality of life/morbidity (e.g., pain/discomfort, treatment side-effects, mood, energy level, family and social interactions, sexual function, ability to work, and ability to keep up with routine daily activities, functional status/independence) based on patient self-report and clinician observation/report
  • Safety (avoidable adverse events due to errors and omissions)
  • Patient-reported satisfaction with care
  • Patient knowledge through education of one’s condition and ways to self-manage
  • Changes in lifestyle (e.g., tobacco use, food choices, physical activity, use of alcohol and illicit drugs
  • Reasons for variance from practice guidelines
  • Comorbidity influences.

Both types of data are important in determining provider effectiveness. But without the second data set, you have no way of determining:
  • If the diagnosis is correct (i.e., fits the biological measures and symptoms)
  • How much the patient’s biological measures improved (or worsened)
  • How much the patient’s symptoms improved (or worsened)
  • How much the patient’s quality of life improved (or worsened)
  • Errors and omissions
  • The usefulness of particular guidelines for particular types of patients
  • Whether the patient was satisfied with care
  • If the patient was educated adequately in self-care (well-care)
  • If the patient is complying with the plan of care and if the plan of care is appropriate for the patient
  • If a provider has a good reason for varying from the practice guideline and if such variance improves (or worsens) outcomes
  • The impact of coexisting conditions on outcomes
  • If the data are fraudulent.

In addition, since claims data base all performance measures on diagnostic codes, any analyses are vulnerable to the serious limitations of current-day diagnostic systems (e.g., the ICD), which not detailed enough to describe the nuances of all diseases and conditions. And the DSM, which is used by mental healthcare providers, suffers the same problems, with the addition that it is not designed as a vehicle to tie diagnosis to treatment decisions. Including the second data set enables researchers to examine detailed clinical data in addition to diagnostic codes, which may overcome this limitation.

Examples of how claims data misses the mark

  • Because claims data lack the specificity (“granularity”) needed to assess and address errors, omissions, and the benefit of particular practice guidelines for particular patient types, they are not useful in evaluating determine treatment-related problems resulting in liability ("loss experience") for such things as hypothyroidism (continued and prolonged symptoms even with treatment, and continued arguments over T3, T4 and pig hormone use), diabetes (prolonged issues with infections such as MRSA, and problems of patient adherence to plan of care), hysterectomy (unnecessary procedure), mastectomy (unnecessary procedure), misread MRI, CAT, PET or X-Ray (cancer takes a life early), chronic bed sores (due to lack of good nutrition), and ignorance of time dependent procedures or testing (such as heart catheterization without follow-up procedure because hospital is not certified for further treatment).
  • In a disaster, pandemic or terrorist attack, after action reviews of claims data are useless since they lack the detail to enable the system to improve.
  • Claims data don’t provide enough of the information researchers need to develop and improve evidence-based guidelines.
  • A bias promoted by the insurance industry by encouraging and supporting the adage "less is more" may make claims data unreliable.

Some relevant quotes from the literature


"Administrative data …are informative about major processes of care. However, it is primarily collected for payment purposes, and therefore may be considered suspect. This is evident in the inherent potential for upcoding in claims submissions for the purpose of maximizing reimbursement. Further, administrative data provides only limited clinical information. For instance, accurately capturing data on the rate of surgical site infections is nearly impossible without some medical record abstraction. Without rigorous validation of antibiotic dispensing data with the medical record to confirm that medications were prescribed for surgical site infections rather than as part of routine post-operative prescribing practices, it is not possible to calculate a surgical site infection rate. Therefore, though infection rates may be of interest to consumers, they are difficult to report from administrative data sources. Research based on administrative data requires a rigorous assessment of their quality. This is particularly true given that administrative data were not originally collected for the purpose of health research, so considerable effort must be placed on data validation. … Administrative data, when augmented with data abstracted from medical records, captures such underreported conditions, to the extent that they are documented in medical records. Distinguishing between conditions present on admission and those that occur during the course of a hospitalization is another area needed for data quality assessment...administrative data lacks pathophysiological data and the ability to distinguish between comorbidities and complications …claims do not distinguish reliably between those diagnoses presented at admission and those that arise during hospitalization...Further, the authors note that in some instances a surgeon may not charge a patient for treatment of complications, thus no billing record is generated." http://www.health.state.ri.us/chic/performance/quality/quality7.pdf

"... claims data also have some considerable limitations:
  • You cannot identify a variety of detailed clinical factors that would help stratify patients into clinically meaningful groups.
  • You cannot determine cancer histology because histology is not codable.
  • You cannot reliably gauge cancer stage because codes for regional spread generally do not exist. While codes for distant spread exist (e.g. 197.7 is secondary malignant neoplasm of the liver), they are not consistently used.
  • You cannot reliably identify worsening existing disease other than through inference based on clinical treatment. For example, no code differentiates between mild congestive heart failure from serious CHF. Likewise, codes do not distinguish large tumors from small tumors. Thus, administrative data make it difficult to stratify patients by severity of illness for a specific condition though it is possible to stratify patients based on the number of comorbid conditions (see Klabunde, 2002 for review).
  • Treatment failure is difficult to identify, particularly when studying patients who do not elect to undergo more treatment (Earle, 2002). If a patient receives chemotherapy, has no obvious treatment and then begins chemotherapy again, it is probably reasonable to assume that the cancer recurred. If, however, the patient decides not to undergo additional treatment, administrative data will not distinguish that patient from patients whose cancer never recurred, except by diagnoses near death or entry into hospice.
  • No code exists for the intent of treatment. For example, radiation can be either therapeutic or palliative. Researchers may choose to assume that a patient with distant spread receiving radiation is being treated with an intent to control symptoms (since radiation alone will not control distant spread) but often the specific physical target of the radiation is not noted (e.g. brain, lung, primary tumor site). Likewise, a researcher's classification of the intent of the radiation may not match the patient's belief about the intent of the treatment.
  • Codings will not reflect diagnostic test or error. For example, if surgery is undertaken for suspected gall stones and, in the course of surgery, metastatic cancer is found, the diagnosis will be coded as cancer, as if it were known beforehand.
  • Administrative data can only identify services received (for which a bill was submitted), not services needed. Knowing that a patient received medication for pain control does not mean that the patient had his/her pain controlled. Likewise, patients with no pain medications may or may not have experienced pain. Claims cannot tell the whole 'story' behind the pattern of services. Did a patient not receive care following protocol because his/her physician did not follow guidelines? Could a patient not physically tolerate a treatment or refuse treatment for some other reason? There is no code for 'declined X treatment.' Knowing care received does not provide information about care offered or the reasons why it was or was not accepted." http://symptomresearch.nih.gov/chapter_18/sec7/claimsdata.htm

"Data analysis demonstrated that while both groups experienced improvement in both physical and mental health status, only those with coronary artery disease experienced significantly stronger physical functioning...Had CorSolutions relied solely on static claims data to analyze member outcomes, the distinguishing factors around burden would have been completely overlooked, as would have any related opportunities to modify treatment for the group accordingly." http://www.qualitymetric.com/improving/diseaseMgmt.aspx

"The advantages of claims data are low cost, ease of patient follow-up over long periods, and the absence of reporting bias. The limitations are the adequacy of the data used to control for patient comorbidity and the lack of outcome information on functional status." http://jama.ama-assn.org/cgi/content/abstract/257/7/933

"Administrative data are readily available, are inexpensive to acquire, are computer readable, and typically encompass large populations. They have identified startling practice variations across small geographic areas and supported research about outcomes of care. Many hospital report cards (which compare patient mortality rates) and physician profiles (which compare resource consumption) are derived from administrative data. However, gaps in clinical information and the billing context compromise the ability to derive valid quality appraisals from administrative data. With some exceptions, administrative data allow limited insight into the quality of processes of care, errors of omission or commission, and the appropriateness of care. In addition, questions about the accuracy and completeness of administrative data abound. Current administrative data are probably most useful as screening tools that highlight areas in which quality should be investigated in greater depth. The growing availability of electronic clinical information will change the nature of administrative data in the future, enhancing opportunities for quality measurement." http://www.annals.org/cgi/content/full/127/5_Part_2/666

"...insurance claims data lack important diagnostic and prognostic information when compared with concurrently collected clinical data in the study of ischemic heart disease. Thus, insurance claims data are not as useful as clinical data for identifying clinically relevant patient groups and for adjusting for risk in outcome studies, such as analyses of hospital mortality." http://www.annals.org/cgi/content/abstract/119/8/844

Conclusion


Claims data provide some useful measures of clinicians’ performance, but are grossly inadequate metrics for incentives, transparency of cost & effectiveness, and continuous quality improvement. So, instead of using claims data in isolation, they should be augmented with detailed clinical outcomes data that (a) offer more valid measures of performance, and (b) enable researchers to establish and evolve evidence-based practice guidelines.

external image meter.asp?site=s24wellness