Скачать презентацию Risk-Adjustment Methodologies and Applications in the VA Amy Скачать презентацию Risk-Adjustment Methodologies and Applications in the VA Amy

a468f598823f4756ca9dc776f5eff3cd.ppt

  • Количество слайдов: 60

Risk-Adjustment Methodologies and Applications in the VA Amy K. Rosen, Ph. D. Director, Risk Risk-Adjustment Methodologies and Applications in the VA Amy K. Rosen, Ph. D. Director, Risk Assessment and Patient Safety Center for Health Quality, Outcomes and Economic Research A VA HSR&D Center of Excellence (Bedford, MA) & Professor, Health Policy and Management Boston University School of Public Health

Purpose of Talk l l Introduce concept of risk adjustment Describe two well-known diagnosis-based Purpose of Talk l l Introduce concept of risk adjustment Describe two well-known diagnosis-based risk-adjustment tools: Diagnostic Cost Groups (DCGs) and Adjusted Clinical Groups (ACGs) Discuss applications in the VA Development of psychiatric risk-adjustment measure for the VA

Why is Risk Adjustment Necessary? l l l Health status of population can vary Why is Risk Adjustment Necessary? l l l Health status of population can vary significantly Goal is to provide equitable compensation and make appropriate comparisons Allocations based on efficiency and quality, not selection

Risk Adjustment The process by which the health status of a population is taken Risk Adjustment The process by which the health status of a population is taken into account when evaluating patterns or outcomes of care or setting capitation rates

Applications of Risk. Adjustment Measures l l l Payment management (Prospective) Provider profiling (Concurrent) Applications of Risk. Adjustment Measures l l l Payment management (Prospective) Provider profiling (Concurrent) Disease/Case management (Prospective) Quality and outcomes (Concurrent) Resource allocation (Prospective)

Types of Risk-Adjustment Measures Types of Risk-Adjustment Measures

Evaluation Criteria l l l Predictive validity Subgroup fit Administrative feasibility Incentives for efficiency Evaluation Criteria l l l Predictive validity Subgroup fit Administrative feasibility Incentives for efficiency Resistance to gaming

Diagnosis-Based Risk Adjustment l l Increasing use of risk adjustment based on diagnosis codes Diagnosis-Based Risk Adjustment l l Increasing use of risk adjustment based on diagnosis codes from administrative data Persisting concerns with reliability and validity of diagnosis codes l l l Outpatient data - not much known about reliability Variability in coding practices across providers and facilities Upcoding and diagnostic creep Tentative coding Risk-adjustment measures minimize some of these (e. g. , excluding ill-defined codes)

What are diagnosis-based riskadjustment measures? l Diagnosis-based measures use demographics/diagnostic information from claims/encounters to: What are diagnosis-based riskadjustment measures? l Diagnosis-based measures use demographics/diagnostic information from claims/encounters to: l Classify patients into clinically homogeneous groups based on expected need for resource utilization § § § l Create clinical profile Identify clinical needs Evaluate clinical management programs Predict relative resource use § § § Predict expenditures Same year as diagnosis (Concurrent Models) Subsequent year to diagnoses (Prospective Models)

Data Requirements l l l Defined population of patients Claims/encounter data available for all Data Requirements l l l Defined population of patients Claims/encounter data available for all members of the population (12 months) Unique patient identifiers (i. e. , social security numbers) Age and gender ICD-9 -CM diagnosis codes from face-to-face clinical encounters Optional: payer, DOD, sociodemographics

Family of Diagnostic Cost Group (DCG) Models l Type of population l l Clinical Family of Diagnostic Cost Group (DCG) Models l Type of population l l Clinical data available l l All-encounter, inpatient only, age/sex and pharmacy Year of prediction l l Commercial, Medicare, Medicaid Concurrent or prospective Recognizes cumulative effect of multiple conditions in predicting costs

DCG Model Overview Relative Health Status ICD-9 -CM Codes Relative Risk Scores Clinical Groups DCG Model Overview Relative Health Status ICD-9 -CM Codes Relative Risk Scores Clinical Groups DCG Categories Clinical Profiles Resource Use Predictor

DCG Clinical Classifications ICD-9 -CM codes (n = 15, 000+) Dx. Groups (n =781) DCG Clinical Classifications ICD-9 -CM codes (n = 15, 000+) Dx. Groups (n =781) PIP-DCG Clinical Classification PIP=Principal Inpatient Based on Inpatient Dx only Single-condition model Hierarchies imposed for predictions Condition Categories (CCs) (n= 184) Relative risk score Aggregated Condition Categories (ACCs) (n= 30)

Clinical Vignette: 59 year old woman AMI, COPD, renal insufficiency (Release 5. 0) ICD-9 Clinical Vignette: 59 year old woman AMI, COPD, renal insufficiency (Release 5. 0) ICD-9 -CM 410. 91 AMI of unspecified site, initial episode of care 491. 2 obstructive chronic bronchitis Dx. Group CC 72. 01 AMI, initial episode of care 50 AMI 96. 01 emphysema/ chronic bronchitis 64 COPD 518. 1 Interstitial emphysema 586 renal failure nos 585 chronic renal failure 106. 04 renal failure, unspecified 106. 03 chronic renal failure 78 Renal Failure

Hierarchical Condition Categories (HCCs) l 31 Hierarchies are imposed on the CCs to produce Hierarchical Condition Categories (HCCs) l 31 Hierarchies are imposed on the CCs to produce HCCs. The clinical hierarchies: l l l Identify the most costly manifestation of each distinct disease Decrease the model’s sensitivity to coding idiosyncrasies Examples: Diabetes, Cancer, Heart, Mental Health

Cancer Hierarchy (Release 5. 0) Metastatic Cancer High Cost Cancer Moderate Cost Cancer Low Cancer Hierarchy (Release 5. 0) Metastatic Cancer High Cost Cancer Moderate Cost Cancer Low Cost Cancer Carcinoma in Situ Uncertain Neoplasm Skin Cancer Except Melanoma Benign Neoplasm

Current and Prospective Predictions Current and Prospective Predictions

How Do All-Encounter DCG Models Predict? l Linear additive formulas (OLS regressions) combine predictions How Do All-Encounter DCG Models Predict? l Linear additive formulas (OLS regressions) combine predictions based on HCCs and age/sex cells subject to: l l Hierarchical restrictions Exclusions of CCs in prospective models l l that are not useful for predicting costs (minor injuries) vague and discretionary CCs based on concerns about gaming in payment models

DCG Predictions: Relative Risk Score (RRS) l l l Illustrate annual resource use as DCG Predictions: Relative Risk Score (RRS) l l l Illustrate annual resource use as determined from DCG cost weights RRS calculated by adding cost weights of an individual’s HCCs and dividing by benchmark (i. e. , Medicare) mean dollar amount RRS normalized so that population mean = 1. 00

Prospective Relative Risk Score Calculated Health Score for Year 2 0. 45 5. 71 Prospective Relative Risk Score Calculated Health Score for Year 2 0. 45 5. 71 0. 95 1. 84 0. 90 0. 89 0 18. 09 … 0. 46 43. 30 54 year old male HCC Diabetes with renal manifestation Type 1 diabetes Congestive heart failure Acute myocardial infarction Vascular disease with complication Vascular disease Dialysis status …. . Diabetes & congestive heart failure Relative Risk Score

Which Providers are “More Efficient”? Which Providers are “More Efficient”?

Adjusted Clinical Groups (ACGs) l l Clustering of morbidity is a better predictor of Adjusted Clinical Groups (ACGs) l l Clustering of morbidity is a better predictor of health care resource use than presence of specific diseases Level of resources necessary for delivering health care services is correlated with the morbidity of that population

Generating ACG Output (Version 4. 5) 15, 000 ICD-9 -CM Diagnosis Codes Step. 1: Generating ACG Output (Version 4. 5) 15, 000 ICD-9 -CM Diagnosis Codes Step. 1: Adjusted Diagnosis Groups Step 2: Collapsed ADGs (32 ADGs) (12 CADGs) Step 3: CADGs combined into Major Adjusted Categories (MACs) (26 MACs) AGE, GENDER Step 4: Adjusted Clinical Groups (106 ACGs)

Examples of ADGs and Their Common ICD-9 -CM Codes ADG Common Diagnosis (ICD-9 -CM Examples of ADGs and Their Common ICD-9 -CM Codes ADG Common Diagnosis (ICD-9 -CM Code) 1 3 9 Time Limited: Minor Time Limited: Major Likely to Recur: Progressive 10 Chronic Medical: Stable 11 Chronic Medical: Unstable 23 Psychosocial: Time Limited, Minor Psychosocial: Recurrent or Persistent, Stable Psychosocial: Recurrent or Persistent, Unstable Noninfectious Gastroenteritis (558. 9) Phlebitis of Lower Extremities(451. 2) mpaction of Intestine (560. 3) Malignant Hypertensive Renal Disease With Renal Failure (403. 01) Cerebral Thrombosis (434. 0) Adult Onset Type II Diabetes w/ Ketoacidosis 250. 10) Essential Hypertension (401. 9) Adult-Onset Type I Diabetes (250. 00) Malignant Hypertensive Heart Disease (402. 0) Sickle-Cell Anemia (282. 6) Diabetes Mellitus Without Complication 250. 03) Cannabis Abuse, Unspecified (305. 20) 24 25 Panic Disorder (300. 01) Bulimia (307. 51) Catatonic Schizophrenia (295. 2) Alcohol Withdrawal Delirium Tremens (291. 0)

Clinical Vignette: 40 year old woman: diabetes, hypertension (Release 4. 5) ICD-9 -CM V Clinical Vignette: 40 year old woman: diabetes, hypertension (Release 4. 5) ICD-9 -CM V 70. 0, Adult Routine Exam 250. 00, Adult Onset Diabetes, without complications 401. 9, Essential Hypertension 250. 41, Diabetes with renal manifestations ADG CADG MAC ACG 31: Preventative Administrative 10: Chronic Medical: Stable 4100: 2 -3 other ADG combinations Age >34 6: Chronic Medical: Stable 24: Multiple ADG Categories 9: Likely to recur: Progressive 5: Chronic Medical: Unstable

Applying DCGs/ACGs in VA l l Explore the feasibility of adapting diagnosisbased measures to Applying DCGs/ACGs in VA l l Explore the feasibility of adapting diagnosisbased measures to the VA population Examine how well each measure explains concurrent resource utilization and predicts future resource utilization in the VA Evaluate their performance in clinically meaningful groups Profile networks on their efficiency after adjustment for case-mix

ADG Categories in the VA and a Fee for Service Managed Care Population ADG Categories in the VA and a Fee for Service Managed Care Population

ACC Categories in the VA and Medicare ACC Categories in the VA and Medicare

Predictive Ratios for Patients with MH/SA Disorders DCG/HCC model + dummy markers Predictive Ratios for Patients with MH/SA Disorders DCG/HCC model + dummy markers

Predictive Ratios For Subgroups of Veterans: Concurrent Models Predictive Ratios For Subgroups of Veterans: Concurrent Models

Actual and Predicted Ambulatory Provider Encounters: Concurrent Models Actual and Predicted Ambulatory Provider Encounters: Concurrent Models

ACG, DCG, and Unadjusted Efficiency Indices By Network ACG, DCG, and Unadjusted Efficiency Indices By Network

Improved Special Population Data *Note: A value greater than 1 means that the actual Improved Special Population Data *Note: A value greater than 1 means that the actual cost exceeds the predicted cost (or price).

What Weaknesses Remained? l l l Did not predict mental health costs well Did What Weaknesses Remained? l l l Did not predict mental health costs well Did not explain long-term care costs Did not predict special population costs

Patient Safety Indicators (PSIs) l l l Developed by Agency for Healthcare Research and Patient Safety Indicators (PSIs) l l l Developed by Agency for Healthcare Research and Quality (AHRQ) Screen for potential safety events in the inpatient setting Risk adjustment based on age, sex, age/sex interactions, DRGs, 27 comorbidities (AHRQ comorbidity software) Examine observed and risk-adjusted PSI rates in VA 16 medical/surgical PSIs relevant to VA

AHRQ Comorbidities for “Decubitus Ulcer” Congestive heart failure AIDS: Acquired immune deficiency syndrome I AHRQ Comorbidities for “Decubitus Ulcer” Congestive heart failure AIDS: Acquired immune deficiency syndrome I Valvular disease Lymphoma Pulmonary circulation disorders Metastatic cancer Peripheral vascular disorders Solid tumor without metastasis Hypertension (combine uncomplicated and Rheumatoid arthritis/collagen vascular complicated) diseases Other neurological disorders Obesity Chronic pulmonary disease Weight loss Diabetes, uncomplicated Blood loss anemia Diabetes, complicated Deficiency anemias Hypothyroidism Alcohol abuse Renal failure Drug abuse Peptic ulcer disease excluding bleeding Depression Additional VA comorbidities Paralysis Liver disease Psychoses

Characteristics of VA and NIS Samples: Discharges and Patients Characteristics of VA and NIS Samples: Discharges and Patients

“Decubitus Ulcer” l l VA does well in non-VA comparison Within VA comparison changes “Decubitus Ulcer” l l VA does well in non-VA comparison Within VA comparison changes direction PSI 3 Facility VA AHRQ VA VA Obs / Observed Expected AHRQ Expt VA Expt A 19. 45 22. 92 17. 60 0. 85 1. 11 B 18. 16 24. 10 17. 99 0. 75 1. 01 C 19. 42 24. 21 20. 50 0. 80 0. 95

Conclusions l l l Despite different ways of evaluating model performance, model-based resource allocation Conclusions l l l Despite different ways of evaluating model performance, model-based resource allocation for subgroups of veterans would not be adequate Existing methods (ACGs/DCGs) generally underestimate health care costs of individuals with mental health/substance abuse (MH/SA) disorders Non-VA based risk adjustment can be misleading in VA facility comparisons

Adequate Risk Adjustment: Important for Veterans with MH/ SA Disorders l l l The Adequate Risk Adjustment: Important for Veterans with MH/ SA Disorders l l l The VA is the largest mental health service delivery system in the United States Prevalence of mental disorders in VA: 30% Goal: develop and validate a psychiatric diagnosis-based risk-adjustment measure (the “Psy. CMS”) for veterans with MH/SA disorders

Guiding Principles l l l Incorporate all 526 adult MH/SA codes Develop clinically homogeneous Guiding Principles l l l Incorporate all 526 adult MH/SA codes Develop clinically homogeneous categories based on resource utilization Demonstrate face validity Include “manageable” # of categories Minimize “gaming” Predict concurrent/prospective utilization and costs

Methods l Sample l All veterans who received any health care in the VA Methods l Sample l All veterans who received any health care in the VA during Fiscal Year 1999 (October 1, 1998 through September 1, 1999) and had a MH or SA diagnosis (ICD-9 -CM codes 290 -312. 9 or 316) (n=914, 225)

Methods l Data l l Diagnostic and utilization data from VA inpatient and outpatient Methods l Data l l Diagnostic and utilization data from VA inpatient and outpatient administrative data Costs obtained from VA Health Economics and Resource Center (HERC) FY 99 data used for concurrent modeling; data split into 60% development sample (n=548, 535) and 40% validation sample (n=365, 690) FY 00 data used for prospective modeling

Variables l Dependent Variables l l Total MH/SA costs: sum of costs associated with Variables l Dependent Variables l l Total MH/SA costs: sum of costs associated with all outpatient and inpatient MH/SA utilization Outpatient MH/SA encounters: sum of all visits associated with any MH/SA diagnosis code, plus all visits in MH/SA specialty clinics Inpatient MH/SA utilization: number of days a patient resided in any inpatient setting for MH or SA treatment Independent Variables l Age, gender, diagnostic information (all MH/SA primary and secondary diagnoses)

Methods Data Analysis (Four major steps): l 1. 2. 3. 4. Classification and categorization Methods Data Analysis (Four major steps): l 1. 2. 3. 4. Classification and categorization of all MH/SA codes into diagnostic classification system Examined distribution of MH/SA disorders using Psy. CMS Assessed predictive validity of the Psy. CMS using concurrent and prospective modeling Compared performance of Psy. CMS with ACGs and DCGs

Psy. CMS Mood/Psychosis Hierarchy Psy. CMS Mood/Psychosis Hierarchy

Psy. CMS Anxiety Hierarchy Psy. CMS Anxiety Hierarchy

Psy. CMS Alcohol Hierarchy Psy. CMS Alcohol Hierarchy

Psy. CMS Drug Hierarchy Psy. CMS Drug Hierarchy

Diagnostic Cost Group (DCG) Mental Health Groupings Dx. Groups 53. 01 alcoholic psychoses 53. Diagnostic Cost Group (DCG) Mental Health Groupings Dx. Groups 53. 01 alcoholic psychoses 53. 02 drug psychoses 59. 01 alcohol dependence 59. 02 drug dependence 54. 01 delirium/delusions/hallucinations 54. 02 hallucinations, symptomatic 55. 01 schizophrenic disorders 56. 01 manic & depressive (bipolar) disorder 56. 02 major depressive disorders 57. 01 paranoid states 57. 02 other nonorganic psychoses 60. 01 personality disorders, including dissociative identity disorder 134. 04 attempted suicide/self-inflicted injury 60. 06 nonpsychotic organic brain syndrome 60. 07 depression, excluding depressive psychosis 60. 11 autism, other childhood psychoses 60. 12 anorexia/bulimia nervosa 60. 19 prolonged posttraumatic stress disorder 58. 01 panic disorders/attacks 58. 02 generalized anxiety disorder 58. 04 somatoform/dissociative disorders 58. 05 phobic disorders 58. 06 obsessive-compulsive disorders 58. 03 other & unspecified anxiety states 58. 07 other & unspecified neurotic disorders 59. 03 non-dependent abuse of alcohol 59. 04 tobacco use disorder 59. 05 other nondependent drug abuse 60. 02 sexual deviations & disorders 60. 03 psychosomatic illness 60. 04 acute reaction to stress 60. 05 adjustment reaction, excluding prolonged depressive 60. 08 behavior disorder 60. 09 emotional disorders of childhood/adolescence 60. 10 other mental disorders 60. 13 attention deficit disorder, other hyperkinetic syndrome 60. 14 learning/development learning disorder Hierarchical Condition Categories (HCCs) Condition Categories Drug/Alcohol Dependence/ Psychoses Yes HCC 31 No Psychosis & Other Higher Cost mental Disorders Yes HCC 32 No Depression & Other Moderate Cost Mental Disorders No Anxiety Disorders Yes HCC 33 HCC 34 No Lower Cost Mental Disorders Yes HCC 35

Adjusted Clinical Group (ACG) Psycho-social Groupings ICD-9 -CM Psychiatric Codes ADG 23 ADG 24 Adjusted Clinical Group (ACG) Psycho-social Groupings ICD-9 -CM Psychiatric Codes ADG 23 ADG 24 ADG 25 Psycho-social: Time Limited, Minor Psycho-social: Recurrent or Persistent, Stable Psycho-social: Recurrent or Persistent, Unstable CADG 10 • CADG 10 Only • CADG 10 & All Other Remaining CADG Combinations Psycho-Social MAC 10 CADG 1 Psychosocial ADG 25? • CADG 10 & CADG 1 & CADG 2 & CADG 3 • CADG 10 & CADG 1 & CADG 3 No MAC 24 • CADG 10 & MAC 17 • CADG 10 & CADG 12 & Anything Else Acute: Minor and Psychosocial ACG 1300 Refer to MAC 24 Decision Tree Yes ADG 24? ADG 25? No ACG 2500 MAC 23 ACG 2700 Acute: Minor & Acute: Major & Likely to Recur & Psychosocial MAC 21 Yes No ACG 1500 ACG 1400 All Other Combinations Not Listed Above Acute: Minor & Likely to Recur & Psychosocial ACG 3500 Yes ADG 24? Yes No ACG 2600 ACG 3700 MAC 12 Pregnancy Refer to MAC 12 Decision Tree

Results: Prevalence of Selected Psy. CMS Categories Results: Prevalence of Selected Psy. CMS Categories

Table 1: Total MH/SA Costs for Selected Categories Table 1: Total MH/SA Costs for Selected Categories

Table 2: Model Goodness of Fit for Concurrent (FY 99) Validation Samples Table 2: Model Goodness of Fit for Concurrent (FY 99) Validation Samples

Table 3: Model Goodness of Fit for Prospective (FY 00) Validation Samples Table 3: Model Goodness of Fit for Prospective (FY 00) Validation Samples

Conclusions l l l Psy. CMS appears to be valid and reliable measure for Conclusions l l l Psy. CMS appears to be valid and reliable measure for MH/SA risk adjustment Psy. CMS performs better than other systems in predicting concurrent and prospective MH/SA costs/utilization It can serve as risk-adjustment system for describing MH/SA populations, profiling MH/SA services, and budgeting future MH/SA resources

Any Questions? Any Questions?

Amy Rosen, Ph. D. E-mail akrosen@bu. edu CHQOER Bedford VAMC 200 Springs Road (152) Amy Rosen, Ph. D. E-mail [email protected] edu CHQOER Bedford VAMC 200 Springs Road (152) Bedford, MA 01730 U. S. A Phone (781) 687 -2960 Fax (781) 687 -3106 http: //www. va. gov/chqoer/RAPS. htm

References l Rosen AK, Loveland S, Anderson J, Rothendler J, Hankin C, Moskowitz M, References l Rosen AK, Loveland S, Anderson J, Rothendler J, Hankin C, Moskowitz M, Berlowitz DR. Evaluating diagnosis-based case-mix measures: how well do they apply to the VA population? Medical Care 2001; 39(7): 692 -704. l Rosen AK, Loveland S, Anderson J. Applying DCGs to Examine the Disease Burden of VA Facilities: Comparing the Six “Evaluating VA Costs” Study Sites to Other VA Sites and Medicare. Medical Care, June 2003: 41(6 suppl): II-91 -II-102. l Rosen AK, Loveland S, Anderson J, Hankin C, Breckenridge J, Berlowitz DR. Diagnostic Cost Groups (DCGs) and concurrent utilization among patients with substance abuse disorders. Health Services Research, 2002: 37(4): 1079 -1102. l Rakovski C, Rosen AK, Loveland S, Anderson JJ, Berlowitz DR, Ash A. Evaluation of diagnosisbased risk adjustment measures among specific subgroups: can existing measures be improved by simple modifications? " Health Services and Outcomes Research Methodology, 2002: 3(1): 5774. l Rosen AK, Rakovski C, Loveland S, Anderson JJ, Berlowitz DR. Profiling resource use across providers: do different outcomes affect assessments of provider efficiency after case-mix adjustment? American Journal of Managed Care, 2002: 8(12): 1105 -1115. l Rosen AK, Loveland S, Rakovski C, Christiansen C, Berlowitz DR. Do different case-mix measures affect assessments of provider efficiency? Lessons from the VA. The Journal of Ambulatory Care Management 2003: 26(3): 229 -242. l Rosen AK, Reid R, Broemeling AM, Rakovski C. Applying a risk adjustment framework to primary care: can we improve on existing measures? Annals of Family Medicine, 2003: 1(1): 44 -51.

References (cont’d) l Rosen AK, Trivedi P, Amuan M, & Montez M. The John References (cont’d) l Rosen AK, Trivedi P, Amuan M, & Montez M. The John Hopkins Adjusted Clinical Groups ( ACGs) case-mix system: A risk-adjustment methodology currently available at the VA Austin Automation Center. VIRe. C Insights Vol. 4, No. 1. Hines, IL: VA Information Resource Center, 2003. Available at http: //virec. research. med. va. gov. l Liu CF, Sales AE, Sharp ND, Fishman P, Sloan KL, Todd-Stenberg J, Nichol WP, Rosen AK, Loveland S. Casemix adjusting performance measures in a VA population: pharmacy- and diagnosis- based approaches. Health Services Research, 2003: 38 (5): 1319 -1338. l Warner G, Hoenig, H, Montez M, Wang F, Rosen AK. Evaluating diagnosis-based risk-adjustment methods in the spinal cord dysfunction population. Archives of Physical Medicine and Rehabilitation, 2004: 85(2): 218 -226. l Rosen AK, Christiansen CL, Montez ME, Loveland S, Shokeen P, Sloan KL, and Ettner SL. Evaluating riskadjustment methodologies for patients with mental health and substance abuse disorders in the Veterans Health Administration. International Journal of Healthcare Technology and Management, 2006: 7 (1/2): 43 -81. l Sloan KL, Montez ME, Spiro A III, Christiansen CL, Loveland S, Shokeen P, Herz L, Eisen S, Breckenridge, JN, Rosen AK. Development and validation of a psychiatric case-mix system. Medical Care 2006: 44: 568 -580. l Montez ME, Christiansen CL, Ettner SL, Loveland S, Shokeen P, and Rosen AK. Performance of statistical models to predict mental health and substance abuse cost. BMC Medical Research Methodology, October 2006: 6: 53. l Rosen AK, Zhaos S, Rivard P, Loveland S, Montez M, Elixhauser A, and Romano P. Tracking Rates of Patient Safety Indicators over Time: Lessons from the VA. Medical Care 2006: 44(9): 850 -861. l www. dxcg. com l www. acg. jhsph. edu l Risk Adjustment for Measuring Health Care Outcomes, edited by Lisa Iezzoni, Health Administration Press, 3 rd edition, 2003.