Verbal Autopsy overview of applications worldwide Professor Alan

Verbal Autopsy: overview of applications worldwide Professor Alan Lopez Melbourne School of Population and Global Health The University of Melbourne

The importance of cause of death information • Cause of death information at a population level helps to identify health priorities for planning – If we know what most people are dying from we can develop public health policy and direct resources towards these diseases – This information also helps us to evaluate health interventions and whether they have succeeded in reducing deaths due to specific causes • Provides trend data on overall mortality patterns – Supports ongoing population health surveillance, including emergence of new epidemics, etc. – Helps identify health research priorities

How do we obtain useful cause of death information for populations? • Optimal method is physician certification of sequence of morbid events leading to death, using WHO International Form of Medical Certificate of Cause of Death • This certificate is coded by trained coders to select the underlying cause of death using ICD rules (currently ICD-10) • However, limited availability of physicians in many LMIC, plus poor training, means this is only done for about one-third (15 -20 million) of 55 million annual deaths worldwide. • For remainder, we need a simple, cost-effective method to diagnose Co. D reliably. Some form of “Verbal autopsy” is required, based on symptom reporting by families.

What is Verbal Autopsy? • Process for deriving a cause of death from responses provided by the family to a structured set of questions on signs/symptoms experienced by the deceased prior to death • 3 components a) Questionnaire to seek information from family about symptoms, durations, etc. b) Method to diagnose cause of death from questionnaire (typically, physician review) c)Target cause of death list that a) & b) above are compatible with (typically, 50 -60 causes)

When to use VA? • VA is useful for deaths that occur outside of a hospital or health facility (e. g. home death, Do. A, death on public roads, etc. ) • VA is intended to provide essential, hopefully reliable information on causes of death in populations with limited access to health care (i. e. Cause-Specific Mortality Fractions or CSMF) • Other goal of VA is to correctly diagnose cause of individual deaths • Principle is to mirror the clinical process in terms of symptom assessment/medical history i. e. verbal autopsy ≡ pathology/autopsy

How has VA been applied? (1) • Most commonly, VA has been applied in research settings where it is important to know the true cause of death of individuals (i. e. HDSS, longitudinal follow-up studies of individuals with known baseline exposures, etc. ) • Widely applied in HDSS in the INDEPTH network ( 30, 000100, 000 population) primarily in Africa, but without any validation data to assess diagnostic accuracy. • In principle, this validation step provides information on performance characteristics of your VA questionnaire (e. g. for which causes does the questionnaire not work? ). Need reliable ‘gold standards’ to validate VA performance.

How has VA been applied? (2) • Best current practice is to have physicians review the VA responses, including the Open Narrative, and to complete a death certificate according to rules and procedures of the ICD. This death certificate should then be coded to determine Underlying Cause of Death (UCOD) • Common practice is to have 2 physicians review each VA. If they disagree, ask a 3 rd physician. If still disagreement (rare), case is usually dropped. How much confidence can we have in this procedure?

Challenges with Physician Coded VA (PCVA) • Diagnostic variability (skill? ) • Replicability (intra and inter-physician diagnostic consistency) • Time • Cost • Accuracy? ?

SOLUTIONS • If VA is to be routinely applied in VR systems, we need to solve these problems. Need to replace physicians, but by what? • Can we use computer logic that recognizes patterns of responses in the data and associates these patterns with the true cause of death • Collectively, these methods are known as Automated Diagnostic Methods (ADM)

Typology of cause of death diagnosis methods Based on Expert Judgment T Individual Cause of Death Assignment Cause-Specific Mortality Fractions Only Medical Certification of Causes of Death Physician Certified Verbal Autopsy (PCVA) Inter. VA/In. Silico. VA Symptom Pattern(Simplified) Based on Empirical Analysis of Gold Standard Co. D Data (Automated Methods) Tariff Random Forest Machine Learning Ensemble Methods King-Lu Direct CSMF Estimation

Are computers more accurate than physicians? At the individual level, Tariff is 20% more accurate than Physicians in diagnosing VA’s; but medically certified death certificates in hospitals are still the most accurate source of cause of death information +20% Inter. VA Physician certified VA Tariff Hospital Death Certificates

Can computers accurately diagnose the cause of death? Accuracy of computer methods versus doctors in diagnosing causes of death, i. e. ensuring the cause of death distribution is correct at the population level +9% +13% Inter VA Physician certified VA KL Tariff Hospital Death Certificates

What is the Tariff method to diagnose VA’s? What is Tariff? How does Tariff work? Tariff is based on the “signal to noise” concept. Each questionnaire item gets a score, according to the number of times it was endorsed by the respondent (i. e. , respondents answered “yes” to a question about the presence of a symptom during a VA interview; e. g. “did the decedent have yellowish eyes? ”). Any symptom reported in the VA that is known to be highly predictive of a particular cause of death (“signal”) will have a significantly higher score (“tariff”) assigned to it for that particular cause of death, compared with other, less strongly associated symptoms (“noise”). The cause with the highest score (“tariff”) for a particular death is assigned as the cause of death for that individual. Tariff scores can be + or - depending on the strength of the signal.

How to calculate Tariff scores for various causes of death The table below gives the tariff values for a sample of cause-symptom pairs that have been calculated from previous research. Suppose the family member said that their deceased had fever, and was diagnosed with TB (i. e. these symptoms were endorsed and get a score of 1). Symptoms We multiply the endorsement score for each symptom by the tariff value and sum them to get a set of tariff scores for various causes. In this example, AIDS has the highest Tariff score (6), and is therefore the Co. D assigned for this case. Tariff Matrix AIDS Malaria Fall … Suffered a fall 0 0 32 … Diagnosed w TB 5 0 0 … Fever 1 1. 4 -1 … Diagnosed w Dementia 0 -1 0 …

Which VA questionnaire should I use? Choosing the right VA questionnaire What are my options? Choosing the right VA questionnaire is just as important as selecting the best method of analysing the questionnaire responses. The choice should be based on: • Time to administer the interview (the less time required, the better) • Possibility for automated diagnostic methods to be run on the questionnaire (such as the Tariff method) • Ability to diagnose an adequate number of leading causes of death (Tariff allows identification of 34 causes for adults, 21 for children and 10 for neonates) The following four VA questionnaire options, all based on the same questions, are the most commonly used: 1. WHO – Full-version 2. WHO – Reduced version (based on expert opinion) 3. Population Health Metrics Research Consortium (PHMRC) – Full version 4. PHMRC – Item reduced version

Short form VA questionnaire + Tariff method = Smart. VA Using the short form VA questionnaire Smart. VA can be run on any android smartphone or tablet, and can include both picture and language capabilities The Bloomberg Data for Health (D 4 H) Initiative, in collaboration with the University of Melbourne and IHME in Seattle, have developed a software application called Smart VA which incorporates the short questionnaire and the Tariff method.

Using Smart VA to generate policy-relevant data: application in Vietnam

Myanmar: application of Smart. VA to nationally representative sample of 6, 000 deaths, 2017: Comparison with broad cause categories of GBD Broad group Smart. VA GBD Group 1: Communicable, reproductive, nutritional 15% 17% TB & Pneumonia 8% 8% Other Group 1 7% 8% 74% 76% All cardiovascular 31% 27% All cancers 11% 19% Other NCDs 33% 30% 11% 7% Group 2: Non-communicable Group 3: Injuries

Advantages of using tablets for collecting and processing VAs Strategic • Speed – diagnoses available the same day • Accuracy – minimises errors in data entry; diagnoses Co. D more accurately ( Smart. VA) than physicians reading VAs • Cost – largely costless except for cost of tablet • Standardisation/comparability – much higher inter-rater reliability than doctors Operational • Automated data quality checks (i. e. , pregnancy can’t last for more than 10 months) • Automated data entry from VA questionnaire via Smart VA software • Interviewers obliged to follow survey skip patterns • Questionnaire is easily translatable into other languages • Picture and audio capability

• • Global applications of VA in national CRVS/SRS systems Smart. VA (used as input into physician certification): China (Shanghai, plus field trial in 14 DSP sites), Philippines, Colombia (? ), Brazil, Mexico (child deaths) Smart. VA (primarily without physician input): PNG, Solomon Islands, Myanmar, Bangladesh, Rwanda, Peru(? ), Sri Lanka (? ) Smart. VA/Inter. VA/In. Silico. VA from WHO 2016: Kenya, Tanzania, Ghana, Morocco, Colombia (? ) SRS: physician review of VA in India, Bangladesh, Indonesia (Smart. VA ? )

Conclusions • Verbal autopsy only real alternative to improve information on Co. D patterns outside of hospitals in most countries • Recent advances in IT and research on methods, VA questionnaire design, etc. now mean automated VA methods ready for widespread application in VR systems (pending further trials in D 4 H countries) • Automated VA being used in many forms: optimal is as input to physician certification (e. g. Philippines), but also for Do. A, home deaths, deaths with ill-defined causes etc). • Automated VA: most important advance in public health measurement in last 50 years