Details of FDA s Pilot Risk Ranking Model for

Details of FDA’s Pilot Risk. Ranking Model for GMP Inspections Nga Tran, Dr. P. H. , Exponent Brian J. Hasselbalch, FDA July 2004 1

Application of Risk Ranking • Regulatory programs: – EPA, Cal EPA – USDA • Risk management – Do. D – Industry 2

Examples of Risk Ranking Models 3

EPA Waste Minimization Prioritization Tool (WMPT) • A regulatory decision tool – Foundation for the EPA’s Resource Conservation and Recovery Act Persistent, Bioaccumulative, and Toxic (PBT) List of chemicals. • Framework of expert judgment • Identify chemicals or emissions of potential concern – Using key physical-chemical properties and associated cutoff criteria. 4

EPA Waste Minimization Prioritization Tool (WMPT) -- Scoring system 5

The TRI Relative Risk-Based Environmental Indicator (Bowves et al, 1997) 6

A Hierarchical Framework of Scoring and Ranking Methodology, Pennington and Yue (2001) Comp lexity Group Technique Description Fate and exposure measures Effect measure Low High 1 Direct summation of emission data None 2 Comparison in terms of effect None Selected toxicological measures and/or benchmarks 3 Scoring and Ranking Selected parameters such as degradation half-life and bioaccumulation measures Selected toxicological measures and/or benchmarks 4 Model-based approaches Integrated model predictions of fate and exposure (intake fractions) Selected toxicological benchmarks and/or dose-response measures 5 Site-specific risk assessment Site-specific fate and exposure estimations using models and/or measurements Generic or site-specific toxicological benchmarks and/or dose-response 7 measures

Do. D • Comparative risk pre-deployment – Rank locations based on potential chemical risks – Risk score is a function of volume and inherent chemical toxicity • Comparative risk during deployment – Ranking and comparing disparate risks – Quick/simple tools 8

Risk Assessment Matrix U. S. Army Field Manual 3 -100. 12. Probability Severity Frequent Likely Occasional Seldom Unlikely Catastrophic E E H H M Critical E H H M L Marginal H M M L L Negligible M L L E – Extremely high risk H – High risk M – Moderate risk L – Low risk 9

Risk Level Definitions (US CHPPM, Technical Guidance-248) Risk Level Defined Consequence (FM 100 -14) Extremely High Expected loss of ability to accomplish the mission. Black (Unit Requires Reconstitution). Unit below 50% strength. Expected significant degradation of mission capabilities in terms of the required mission standard, inability to accomplish all parts of the mission, or inability to complete the mission to standard if hazards occur during the mission. Red (Combat Ineffective). Unit at 50 -69% strength. Moderate Expected degraded mission capabilities in terms of the required mission standard will have a reduced mission capability if hazards occur during mission. Amber (Mission Capable, with minor deficiencies). Unit at 70 -84% strength. Low Expected losses have little or no impact on accomplishing the mission. Green (Mission Capable). Unit at 85% strength or better. High Unit Status (FM 101 -5 -1) 10

Hazard Severity Ranking Chart for Military Deployments (US CHPPM, Technical Guidance-248) Nature of Individual's Health Effect(s) Associated with Exposures Near the Guideline Percent of Exposed Symptoms occurring after the mission Persons to Exhibit Symptoms (ATTACK RATE) Chronic/permanent injury or disease (e. g. , cancer) Symptoms occurring during the mission Mild illness or Injury or illness temporary irritation that impairs (reversible, shortfunctional abilities term, nuisance) Incapacitation or Death >50 % Marginal Critical Catastrophic 31 - 50 % Negligible Marginal Critical Catastrophic 10 - 30 % Negligible Marginal Critical < 10 % Negligible Marginal Critical 11

Chemical Hazard Probability Ranking Chart for Military Deployments (US CHPPM, Technical Guidance 230) Percent of Personnel that will Experience Exposures to Concentrations Equal to or Greater than the MEG* <10% 10< 25 % 25 <50 % 50 <75 % >75 % Unlikely Seldom Occasional Likely Frequent *Determination of the percent of personnel exposed to a chemical or mixture specifically above a guideline level can be based on modeling, gridding, or generalized assumptions. 12

Consumer Product Example • Ranking to target products for which additional toxicity testing of High Production Volume chemical is needed. – Laundry detergent, consumer care products, etc. . – Many products, limited resources • Ranking based on exposure to consumer products – “High end” product use frequency (e. g. , number of times use per day) as surrogate for exposure 13

Example Consumer Product Exposure Model for Ranking Dermal Direct Contact FQ x A x PR x CF x DA BW FQ: Frequency of use (use/day) A: Amount used (g/use) PR: Percent retained (%) CF: Conversion factor (1000 mg/g) DA: Dermal absorption (100%) BW: Body weight 14

Ranking of Consumer Product by Exposures to High Production Volume Chemical A Product Category Exposure Rank Aftershave Hair Care 2 Laundry Detergent – Liquid 3 Body moisturizer 4 Bar Soap 5 Cleansing products 6 Hard Surface Cleaner – Liquid 7 Dish Detergent – Liquid 1 8 15

Food Microbiological Hazard Example Van Gerwen et al (2000) • SIEFE: Stepwise and Interactive Evaluation of Food Safety by an Expert System. • A decision support system for microbiological risk assessment for food products and their production processes • Risks are first assessed broadly, using order of magnitude estimates, i. e. level 1 risk assessment. • Characteristic numbers are used to quantitatively characterize microbial behavior during the production process to highlight the major risk-determining phenomena 16

Categorization of the probability of having problems, growth and inactivation by characteristic numbers PC (probability characterization); SC (step characterization); OC is the characteristic number for occurrence characteristic; HC for health problem characterization. 17

Food risk ranking model (Ross and Sumner, 2002) • Tool to compare food-borne risks for ranking and prioritizing risks from diverse sources. • 11 questions relating to 3 main factors: – severity of hazard – likelihood of disease causing dose of hazard being present in a meal – probability of exposure to the hazard 18

Weighting values used in the Ross and Sumner’s model Weights Comments SEVERE hazard - causes death to most victims 1 arbitrary weighting factors MODERATE hazard - requires medical intervention in most cases 0. 1 MILD hazard - sometimes requires medical attention 0. 01 MINOR hazard - patient rarely seeks medical attention 0. 001 1. Hazard Severity 19

2. How susceptible is the consumer ? Weights GENERAL - all members of the population Comments 1 100% of population SLIGHT - e. g. , infants, aged 5 20% of population VERY - e. g. , neonates, very young, diabetes, cancer, alcoholic etc 30 3% of population 200 0. 1% of population EXTREME - e. g. , AIDS, transplants recipients, etc. 20

7. Effect of Process Weights The process RELIABLY ELIMINATES hazards 0 The process USUALLY (99% of cases) ELIMINATES hazards 0. 01 The process SLIGHTLY (50% of cases) REDUCES hazards 0. 5 Comments arbitrary weighting factors The process has NO EFFECT on the hazards 1 The process INCREASES (10 x) the hazards 10 The process GREATLY INCREASES (1000 x ) the hazards 1000 user input OTHER 21

USDA- Food Safety and Inspection Service • "Inspector Optimization System" (IOS) – Basis of assigning the processing inspector workforce to provide for greater flexibility in targeting inspection resources to areas of greater public health risk and concern – Report to Congress on Risk-Based Inspection to the United States House and Senate Appropriations Committees, March 2, 2001 • Each plant receives a Food Safety Hazard Coefficient (HC) • Inherent hazards • Expert Elicitation to rank inherent hazards 22

Food Safety Hazard Coefficient (Species Variable + Process Variable) x Volume Variable = Food Safety Hazard Coefficient Species variable designed to reflect the inherent biological, chemical, and physical hazards in meat and poultry arriving at inspected establishments Process variable captures the inherent hazards (biological, chemical, and physical) of the establishment’s operations Take into account how a process normally works, the likelihood of fluctuations or deviations from the norm, the effect these fluctuations or deviations may have on hazards in the product, and the potential resulting implications for public health as the product leaves the establishment. Volume variable – facility size correlated with the number of persons who consume food from the establishment (exposure). 23

Expert elicitation • To determine the species and process hazards • Two groups of experts (species and processing) • Government, academia, and industry. Species Categories Used in Hazard Elicitation Steers/Heifers Lambs/Sheep/Goats Veal/Calves Young Chickens Cows/Bulls Young Turkeys Market Hogs Older Poultry Sows/Boars Ducks/Geese 24

Process Categories Used in Hazard Elicitation Raw product, ground Heat-treated, shelf-stable product Raw product, not ground Heat-treated, but not fully cooked, not shelf-stable Slaughter Fully cooked, not shelf-stable product Thermally processed, Not heat-treated, shelf-stable product commercially-sterile Secondary inhibitors, not shelf-stable product 25

“…. FSIS is exploring the development of a Hazards Control Coefficient (HCC). Imagine if we could divide the universe of plants into categories based on the risk inherent in their product (ground beef vs. beef jerky), and on the compliance history of the plant, we could determine which plants have the lowest vs. the highest probability of producing safe product. Such a scheme would help the Agency make resource allocation decisions across this country's more than 6, 000 meat and poultry establishments based on risk, and thus maximize food safety and public health protection. ” Remarks prepared for delivery by Under Secretary for Food Safety Dr. Elsa A. Murano, at the National Food Processor's Association's Annual Conference, Washington, DC, May 20, 2004. 26

The CDER-OC process • Began one year ago • Internal experts from CDER, CVM, CBER, and ORA (Risk Mgmt & Workplan WG) – Generated a list of risk factors that are descriptors of site risks for consideration in development of site selection model – Qualitatively valued: High, Medium, Low 27

Examples of risk factors Factor Risk Administered through membranes Medium Aseptically produced finished dosage forms High Dermatological OTC liquids/creams/ointments/powders with no daily dose limit Low Excipients Low Aseptic fill using isolator (self-contained isolators and form-fill-seal), nonterminal sterilized Medium Aseptic sterile filtration and filling (non-isolator, non-terminal sterilization) High Assembly/fabrication processing (as opposed to batch processing) High Cartoning / packaging Low Large number of approvals/time Low Complaints received suggesting Class I or Class II health hazard situations High Contract micronizers Medium First-time applicants/filers Medium 28

Develop Risk Ranking Framework • High level risk management begins with organization • Categorized list of risk factors into 3 components – Product – Process – Facility 29

Categorization of risk factors, e. g. Component Factor Product Administered through membranes Aseptically produced finished dosage forms Dermatological OTC liquids/creams/ointments/powders with no daily dose limit Excipients Process Aseptic fill using isolator (self-contained isolators and formfill-seal), non-terminal sterilized Aseptic sterile filtration and filling (non-isolator, non-terminal sterilization) Assembly/fabrication processing (as opposed to batch processing) Cartoning / packaging Facility Large number of approvals/time Complaints received suggesting Class I or Class II health hazard situations Contract micronizers First-time applicants/filers 30

A RISK-BASED FRAMEWORK FOR PRIORITIZING SITES FOR c. GMP INSPECTION SITE RISK POTENTIAL PRODUCT PROCESS FACILITY Site risk potential (SRP): A function of the risk potentials for the PRODUCT, PROCESS and FACILITY component risk factors. The SRP is a weighted combination of the PRODUCT, PROCESS and FACILITY risk potentials Semi-quantitative risk potentials represented as “high, ” “medium, ” or “low, ” and are numerically discrete variables in calculations. 31

Drilling Down to Component Factors • Selected factors for product, process and facility components • Assigned weights – Empirically derived – Expert judgment • Developed logical algorithm to combine factors to calculate a site score – The Model 32

Drilling Down and Populating the Product, Facility, and Process Components 33

Product Component Factors SRP (Site Risk Potential) Site-Product Score Site-Process Score Site-Facility Score Product Scores Intrinsic Factors 1997 – 2003 Recall STERILITY Frequency MEDICAL GAS Severity RX/OTC HHE Class III, I 34

Product Component Data • Data source: FACTS (Field Accomplishments and Compliance Tracking System) – Site information/identifier – Product codes • CDER Recall Database – Product codes – Rx/OTC and Classification of Hazard • Years: – Recall: 1997 -2004 – Product: 2000 -2004 35

Recall Weight Matrix 36

Product Categorization • Need to link recall data to sites • Indirect linkage -- correlate product codes in recall data with product codes in site database • Recall weights assigned to Recall-FACTs harmonized product codes 37

Examples of Product Code Correlation FACTS Process Indicator Code letter Recall System Product Codes Recall Code Description General Category Q OAER Aerosol, Inhalation/Systemic inhaled S BULK Bulk Pharmaceutical API/Excipient R OPWRR Oral Solid For Reconstitution oral, solid, immediate release capsules/powders A OTABE Tablet, Effervescent oral, solid, immediate release tablets A OTABS Tablet, Sublingual/Buccal oral, solid, immediate release tablets C TEMR Tablet, Extended/Modified Rel oral, extended-release B OTABX Tablet, Extended Release oral, extended-release E OCAP Oral Capsule oral, solid, immediate release capsules/powders E OCAPH Capsule, Hard Gelatin oral, solid, immediate release capsules/powders F OCAPX Oral Capsule Extended Release oral, extended-release L TSPY Topical Spray topical, dermal L TSUS Topical Suspension, Mixture topical, dermal L TTCT Topical Tincture topical, dermal 38

Facility Component Factors SRP (Site Risk Potential Site-Product Score Site-Process Score History of inspection Site-Facility Score History of Compliance and Violation Volume (est. ) 39

Facility Component Data • Data source: FACTS (Field Accomplishments and Compliance Tracking System) • Years: 2000 -2004 • All sites, foreign and domestic 40

Process Component Factors SRP (Site Risk Potential) Site-Product Score Site-Process Score Relevant inherent process risk factors Site-Facility Score Relevant process controls and risk mitigating factors Producti, facilityj 41

Process Component Data • No data, but do have staff with experience and knowledge expert elicitation survey • Elicitation survey drafting began Nov. 2003 • Inter-Center/ORA workgroup effort • ORA; CVM; CBER; CDER; OC 42

Expert Elicitation Survey Challenges – What are the relevant process-related risk factors? (i. e. , sources of variability and poor quality) – What, if any, unit operations are more liable to a loss of control or at risk to contamination? – What products? Answers are product dependent • Large number of products • Identify “mutually exclusive” category of products -- tradeoffs in number of product categories and responder burden 43

Starting point methodology. . . ISPE Baseline Pharmaceutical Engineering Guide, vol. 2, Oral Solid Dosage Forms 44

Defining product types and unit operations Ø Product types Ø Several CDER/agency coding schemes; too much Ø Types selected are representative of most product types in the market Ø Codes grouped into product types with similar unit ops Ø Distinguish high/low actives Ø Unit operations Ø WG member knowledge + • • Remington: Science and Practice of Pharmacy, 20 th edition Modern Pharmaceutics, 3 rd edition Pharmaceutical Process Validation, 3 rd edition ISPE Baseline Pharmaceutical Engineering Guide, vol. 2, Oral Solid Dosage Forms (1 st edition, 1998) 45

DRLS / EES / FACTS profile class codes Profile class description CHG Capsules, Prompt Release CSG Capsules, Soft Gelatin TCM Tablets, Prompt Release TTR Tablets, Extended Release CTR Capsules, Modified Release TCT Tablets, Delayed Response POW Powders (Includes Oral and Topical) OIN Ointment, Non-sterile (Includes Cream, Jelly, Paste) SUP Suppositories LIQ Liquids (Solutions, Suspensions, Elixers and Tinctures) SVL Small Volume Parenterals (Lypholized) SVS Sterile-Filled Small Volume Parenterals (Drugs) SVT Terminally Sterilized Small Parenterals LVP Large Volume Parenterals SNI Sterile Non-Injectable CBI Biotechnology Crude Drug TRP Therapeutic Recombinant Products Transdermal TDP Transdermal Patches MDI ADM Aerosol Dispensed Medication CEX Plant/Animal Extraction Crude Drug CRU Crude Bulk Drugs - Non-Synthesized CSN Non-Sterile Bulk by Chemical Synthesis CSS Sterile Bulk by Chemical Synthesis CFN Non-Sterile Bulk by Fermentation Crude Drug CFS Sterile Bulk by Fermentation Crude Drug Product group Solid oral immediate release Solid oral modified release Powders Semisolids Liquids, nonsterile (solution / suspension) Liquids, sterile Biotech API synthesis API fermentation 46

47

PROCESS RISK FACTORS A) Process control 1) To what degree does this unit of operation contribute to variability in quality of the final product? 2) How difficult is it to maintain this unit of operation in a state of control? 3) If a problem does occur, how reliable are the current detection methods? B) Contamination 4) Is this unit of operation more or less vulnerable to contamination from previous product? 5) Is this unit of operation more or less vulnerable to contamination from the environment? 48

Solid oral drugs, immediate release (i. e. , tablets, capsules), "High Active" Variables Process control 3) If a problem does occur, how reliable are the current detection methods? 4) Is this unit of operation more or less vulnerable to contamination from previous product? 5) Is this unit of operation more or less vulnerable to contamination from the environment? 1 = minimal 2 = minimal to moderate 3 = moderate to high 4 = high to very high 5 = very high 1 = slightly difficult 2 = slightly to moderately difficult 3 = moderately difficult 4 = moderately to very difficult 5 = very difficult 1 = very reliable 2 = very to moderately reliable 3 = moderately reliable 4 =moderately to slightly reliable 5 = slightly reliable 1 = slightly vulnerable 2 = slightly to moderately vulnerable 3 = moderately vulnerable 4 = moderately to very vulnerable 5 = very vulnerable Measuring Mixing/Blending Wet Granulation Dry Granulation Milling Drying Compression (Tablet) Encapsulation (Hard Gel) Coating Pelleting Primary packaging Questions Answering Choices Units of Operation 1) To what degree does this unit of operation contribute to variability in quality of the final product? 2) How difficult is it to maintain this unit of operation in a state of control? Contamination 49

The survey also asks experts to rank products alone (not by unit of operation) by the general areas of concern: 50

Survey Status • Delivery By e-mail • 50 FDA experts participated – reviewers from CDER – senior CDER compliance staff – senior ORA field staff • 90% response rate • Results being analyzed 51

Data analysis • Exploratory Data Analysis (EDA) • Product rank consistent with unit of operation drill down data – correlation • Developing process weights based on unit of operation drill down survey data • K-mean cluster analysis – Average rank per product category per question a) Process controls: questions 1, 2 and 3 b) Contamination potential: questions 4 and 5 – Coefficient of Variance (CV) Weighted average rank per product category per question • Principal component and fuzzy arithmetic for expert categorical data analysis 52

Process Component Factors SRP (Site Risk Potential) Site-Product Score Site-Process Score Process Controls Variability in quality Survey question 1 Maintain state of control Survey question 2 Site-Facility Score Contamination Previous product Survey question 4 Environmental Survey question 5 Reliable detection Survey question 3 53

Plain language summary A site will tend to be less frequently selected for inspection if it: 1. has been inspected recently and/or relatively few previous violations of GMPs and/or a smaller volume of product (facility weight); 1. makes non-sterile, OTC drugs, and/or other product types that are not associated with a high frequency of serious recalls (product weight); 1. makes products estimated to be relatively straightforward to manufacture and not vulnerable to contamination (process weight) 54

Current and Future Risk Factors in Site Score Modules Factors to Update SPR Score (including mitigating factors) Factors in Baseline SPR Score Field Inputs to Update SPR Score Rx x x Other Defects future Not medical gas x Complaint future “Must work right” future Therapeutic range future Higher intrinsic toxicity, e. g. , enhanced risk mgt controls, require periodic patient monitoring future x Process Recalls Sterile Product x Never covered? Enhanced controls (e. g. , PAT, process capability metrics) future New construc. ? future Corp. changes? future Expert Elicitation Compliance history x History of Inspection x Volume (est. ) x future Facility Successful practices study 55 future

Questions to the Subcommittee 1. Can you identify alternative approaches that would systematically prioritize manufacturing sites for GMP inspections? 2. In what areas would additional data provide the most value added in prioritizing manufacturing sites for GMP inspections? 3. Are there other metrics that should be incorporated, e. g. , measuring process control? 56