Скачать презентацию Data Mining in VAERS to Enhance Vaccine Safety Скачать презентацию Data Mining in VAERS to Enhance Vaccine Safety

ac8a9a98b304681ac8ce0e8d918db7df.ppt

  • Количество слайдов: 26

Data Mining in VAERS to Enhance Vaccine Safety Monitoring at the FDA Robert Ball, Data Mining in VAERS to Enhance Vaccine Safety Monitoring at the FDA Robert Ball, MD, MPH, Sc. M Dale Burwen MD, MPH M. Miles Braun, MD, MPH Division of Epidemiology Office of Biostatistics and Epidemiology DIMACS, October 18, 2002

What is the Vaccine Adverse Event Reporting System (VAERS)? – National system for surveillance What is the Vaccine Adverse Event Reporting System (VAERS)? – National system for surveillance of adverse events after vaccination initiated by National Childhood Vaccine Injury Act 1986 and established 1990 – Jointly managed by FDA and CDC – Reports received from health professionals, vaccine manufacturers, and the public

Post-licensure Safety Monitoring • How we do it – VAERS • Potentially rapid detection Post-licensure Safety Monitoring • How we do it – VAERS • Potentially rapid detection of signal of new safety concern • Rarely allows determination of causality – Enhanced surveillance • Obtain standardized information on reports – Controlled studies of hypothesized causal relationships raised in surveillance – Communicate results

Uses of VAERS • • Detecting unrecognized adverse events Monitoring known reactions Identifying possible Uses of VAERS • • Detecting unrecognized adverse events Monitoring known reactions Identifying possible risk factors Vaccine lot surveillance

Limitations of VAERS • • Reported diagnoses are not verified Lack of consistent diagnostic Limitations of VAERS • • Reported diagnoses are not verified Lack of consistent diagnostic criteria Wide range in data quality Underreporting Inadequate denominator data No unvaccinated control group Usually not possible to assess whether a vaccine caused the reported adverse event

Analysis of VAERS Data • Describe characteristics and look for patterns to detect “signals” Analysis of VAERS Data • Describe characteristics and look for patterns to detect “signals” of adverse events plausibly linked to a vaccine • Signals detected through analysis of VAERS data almost always require confirmation through a controlled study

Fundamental Problem in Assessing Spontaneous Reports • VAERS ~10 -15 K reports / year Fundamental Problem in Assessing Spontaneous Reports • VAERS ~10 -15 K reports / year • AERS ~20 K reports / year (CBER) • How can a sensitive system to detect potential product problems not be overloaded and overwhelmed by information to which we have to respond?

“Data Mining” • Identify events reported more commonly for one product than others – “Data Mining” • Identify events reported more commonly for one product than others – Proportional Reporting Ratios (PRR) – Empirical Bayesian Geometric Mean (EBGM) – Don’t account for medical knowledge or biases in reporting • EBGM algorithm implemented by Lincoln Technologies and PPD Informatics – VAERS Data Mining Environment (VDME) • PRR algorithm implemented in standard packages (e. g. SAS, STATA) on an ad hoc basis

Proportional Reporting Ratio • Compares the adverse event profile of one vaccine to other Proportional Reporting Ratio • Compares the adverse event profile of one vaccine to other vaccines Number of reports with Adverse Event Y Vaccine X Other vaccines a c PRR = [a/(a+b)] / [c/(c+d)] Other Adverse Events b d Total (a+b) (c+d)

Proportional Reporting Ratio • Compares the adverse event profile of one vaccine to other Proportional Reporting Ratio • Compares the adverse event profile of one vaccine to other vaccines • Evans has proposed using PRR 2, n 3, and chi square 4 as criteria for selecting pairs for further evaluation

Background: Empirical Bayesian Data Mining • Similar to PRR in comparing one vaccine to Background: Empirical Bayesian Data Mining • Similar to PRR in comparing one vaccine to others • Calculates observed and expected frequencies – Observed: # of reported events/vaccine – Expected: Based on overall frequency of the event for all vaccines, and the total # of reports of the vaccine of interest • Identifies cells with very small expected counts – accounts for the instability of the small number

Empirical Bayesian Data Mining • Ranks vaccine-event combinations by Empirical Bayesian Geometric Mean (EBGM) Empirical Bayesian Data Mining • Ranks vaccine-event combinations by Empirical Bayesian Geometric Mean (EBGM) • Dumouchel has proposed EBGM 2 as criterion to select pairs for further evaluation • Multi-item Gamma Poisson Shrinkage (MGPS) algorithm detects multi-way combinations – V=vaccine; S=symptom • VSSS

Rotavirus Vaccine. Intussusception • • Clinical Trials Signal Wild type RV & intussusception study Rotavirus Vaccine. Intussusception • • Clinical Trials Signal Wild type RV & intussusception study FDA - licensure CDC - recommendations for use Post-marketing Surveillance (VAERS) Background rates Population-based incidence rates Withdrawal

Rotavirus Vaccine and Intussusception: Signal Emergence Rotavirus Vaccine and Intussusception: Signal Emergence

Vaccine Profiles Vaccine Profiles

Anthrax Vaccine: 3 -Dimensional Assessment (V-S-S) Anthrax Vaccine: 3 -Dimensional Assessment (V-S-S)

Effect of Stratification on EBGM: EBGM Anthrax Vaccine and Selected COSTARTS Effect of Stratification on EBGM: EBGM Anthrax Vaccine and Selected COSTARTS

Selection of “Item Sets” for Empirical Bayesian Data Mining • The choice of “Item Selection of “Item Sets” for Empirical Bayesian Data Mining • The choice of “Item Sets” influences the Multi-item Gamma Poisson Shrinkage (MGPS) algorithm • Currently all combinations (e. g. 2 D v-v, s-s, v-s where v=vaccine; s=symptom) • If input is restricted to only v-s combinations the magnitude of the EBGM and rank for pairs with small numbers are affected • Appropriate selection of Item Sets needs systematic evaluation

Effect of Item Set Selection on EBGM Effect of Item Set Selection on EBGM

Challenges • What is the best method? – Bayesian vs. PRR vs. other? – Challenges • What is the best method? – Bayesian vs. PRR vs. other? – What are criteria for making this decision? • How should each method be applied and interpreted? – What level of PRR/EBGM? – How should statistics be interpreted?

Challenges • Should data mining methods be used for automated screening or as analytic Challenges • Should data mining methods be used for automated screening or as analytic tools? – Importance of stratification suggests need for intermediate level epi/stat sophistication in users – Users need training to properly interpret results • Computing resources – Substantial effort required for data preparation – Software needs user-friendly features to enhance end-user control over: • • Defining data subsets of interest Stratification Combining adverse event terms Selecting item sets prior to data mining

Challenges • Usual method of monitoring for signals: • • Physician review of individual Challenges • Usual method of monitoring for signals: • • Physician review of individual reports as they arrive Physician review of serious reports Committee review of serious reports at weekly meeting Physician review of monthly numerical summaries of selected vaccines • Periodic vaccine or disease-specific surveillance summaries • Where does data mining best fit in this process? • How can data mining results be best communicated to decision makers, health care providers, and the public?

Next Steps and Future Challenges • Continue using PRR and Empirical Bayesian methods in Next Steps and Future Challenges • Continue using PRR and Empirical Bayesian methods in routine practice • Systematic comparison of methods • Simulation study in collaboration with CDC • Large size of AERS database, especially with 2 way and 3 way interactions – Is simpler better? e. g. PRR with chi-square • Drug dictionary in AERS

Summary • Automated summary of a large amount of data • Potential for improving Summary • Automated summary of a large amount of data • Potential for improving usual methods of monitoring for signals – Other methods should also be considered • Further understanding and experience is needed

Acknowledgments • FDA – Manette Niu, Phil Perucci, other CBER staff, Ana Szarfman and Acknowledgments • FDA – Manette Niu, Phil Perucci, other CBER staff, Ana Szarfman and other CDER staff • CDC – Henry Rolka, Vitali Poole, Penina Haber, John Iskander, and other CDC staff • Others – Lincoln Technologies, Inc. – PPD Informatics – William Du. Mouchel

Selected References • Dumouchel W. Bayesian data mining in large frequency tables, with an Selected References • Dumouchel W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. American Statistician 1999; 53: 177 -190. • Evans SW, Waller PC, Davis S. Use of proportional reporting ratios for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf 2001; 10: 483 -486. • Niu MT, Erwin DE, Braun MM. Data mining in the US Vaccine Adverse Event Reporting System (VAERS): early detection of intussusception and other events after rotavirus vaccination. Vaccine 2001; 19: 4627 -4634.