Automatic Strategies for Decision Support in Telephone Triage

Automatic Strategies for Decision Support in Telephone Triage Framework and Testbed in Smalltalk Carlos E. Ferro ceferro@ciudad. com. ar Director: Dan Rozenfarb

Agenda n n n Introduction Software Application: Expert. Care Overview of the Framework: n n n Concept representation Session, automation and simulation Strategies Some statistics Examples and results

Telephone Triage n n n Phone call from a patient Initial data gathered Questions and answers Presumptive diagnosis Ambulance dispatch or treatment indications

Expert. Care Initial identification data on a standard form

Expert. Care Selection of all initial symptoms reported by the caller

Expert. Care Question in plain Spanish Session information n List of symptoms List of scored diagnoses New questions are suggested. Answers are recorded. Diagnoses are re -evaluated.

Expert. Care As new information is gathered, some diagnoses are separated according to their score

Expert. Care composition Ontology Inference Engine Scorer Interrogator Knowledge base: Symptoms, Syndromes with attributes Interrogatory Rules Modules of Expert. Care system

Expert. Care ontology n n n Symptoms (e. g. fever, headache, dyspnea) Syndromes (e. g. appendicitis, osteomyelitis, asthma, schizofrenia) Systems (e. g. circulatory, digestive) Severities (red, yellow, green) Frequencies (high, medium, low)

Expert. Care syndrome definition Syndrome definitions are logical expressions in terms of symptoms. Examples: Definition of Appendicitis : “right iliac fossa pain” AND “abdominal pain” AND NOT “appendix operation” Definition of Massive Obesity : “intense weight increase” OR “intense body fat increase”

Expert. Care size in numbers Rules 3209 Symptoms 2383 Syndromes 673 Other 157 Rules account for 50% of size, but 80% of complexity and 90% of costs. They also hinder software evolution.

Target Our main metrics is the amount of questions: n n n Red (Emergency): 3 or 4 questions Yellow (Urgency): 4 or 5 questions Green: around 6, but may reach 12

Solution approach n n n Automated strategy Dynamic interrogatory Navigation and gathering of information from the knowledge base Adaptation to session status Framework for session and strategies Virtual lab as testbed

Concept representation

Logical expressions for definitions

Session, Diagnoses and other

Automation n n Answer. Provider simulates a patient/caller Strategy guides the interrogatory, suggesting #next. Question. For: a. Call. Session SUnit tests run through all syndromes using different strategies Statistical. Collector gathers and caches information from the knowledge base

Grouping and statistics Syndromes by System 90 80 # Syndromes 70 60 50 40 30 20 10 0

Grouping and statistics

Grouping and statistics Systems per symptom 25 12 1 2 3 4 5 6 7 8 9 10 11 13 19 31 55 193 956

Step-by-step example Diabetic Ketoacidosis System: Metabolic Frequency: Low Severity: Red Definition: (Diabetes n n OR History of diabetes) AND (Unconsciousness OR Confusion OR Ketonic breath OR Dyspnea) n This is a typical red syndrome. According to the definition, Answer. Provider can choose among 8 pairs of symptoms (2 x 4). Each one is called a subsyndrome

Step-by-step example Diabetic Ketoacidosis_2 Definition: Diabetes AND Confusion Choosing clues: n Diabetes Systems pregnancy and metabolic n Confusion Associated to 9 different systems n Let’s choose Diabetes as a clue, and try to establish the presence of Confusion, in order to make a Diabetic Ketoacidosis diagnosis.

Example - Step 1 772 syndromes 51 systems positive evidence 2 systems 18 syndromes Diabetes 6 syndromes 2 systems Choosing symptoms to ask: We should try to discern the system from among these two

Example - Step 1 System pregnancy System metabolic 6 syndromes 31 symptoms 13 syndromes 48 symptoms Diabetes System pregnancy 1 syndrome 1 symptom System metabolic 5 syndromes 8 symptoms Choosing symptoms to ask: Using information from the knowledge base and some abductive reasoning, we have 9 candidates left. We choose symptom pregnancy, in order to confirm or discard system pregnancy.

Example - Step 2 772 syndromes 51 systems positive evidence Diabetes Not pregnancy 1 system 5 syndromes Now we “know” that only one system has chances left.

Example - Step 2 System pregnancy 6 syndromes 31 symptoms System metabolic 1 syndrome 13 syndromes 48 symptoms 3 syndromes System metabolic 1 syndrome 9 syndromes Diabetes Not pregnancy System pregnancy 0 syndrome 0 symptom 5 syndromes 8 symptoms 2 syndromes Now we try to discern severity, first trying to decide whether it is red or yellow. Using information from the knowledge base and some abductive reasoning, we have 8 symptom candidates left. Here is where we need some tool for comparing or choosing among them. For instance, we could ask for symptom dyspnea.

Example - Step 3 772 syndromes 51 systems positive evidence Diabetes Not pregnancy Not dyspnea 1 system 5 syndromes The new information did not reduce syndromes

Example - Step 3 System metabolic 1 syndrome 13 syndromes 48 symptoms 3 syndromes System metabolic 1 syndrome Diabetes Not pregnancy Not dyspnea 5 syndromes 8 symptoms 9 syndromes 2 syndromes We still try to discern severity, because “not dyspnea” only rejected some branches of some syndromes, but did not reduce the total number. Now we have 7 symptom candidates left. This way, we could use up to 7 more questions to “hit” the symptom that the simulated patient has and make a diagnosis.

Strategies n One family of first attempts, using none or little information: Sequential. Strategy, Random. Strategy, More. Satisfiers. Strategy, Less. Satisfiers. Strategy, Middle. Satisfiers. Strategy, More. Critic. Separation. Strategy n Second family, attempting to guess the system by different indicators: Guess. System. By. Frequency. Strategy, More. Correlation. Strategy, Less. Negation. Strategy, Guess. System. Using. Pairs. Strategy, Less. Negation. Pair. Strategy

Results of preliminary strategies Strategy Average Red Diag Error Sev Error Median % % Sequential 739. 04 745 19. 00 5. 58 Random 746. 58 732. 5 18. 05 5. 46 Less. Satisfiers (*) 726. 28 932 18. 84 0. 00 Middle. Satisfiers 579. 28 794 18. 05 2. 44 85. 29 52 14. 23 2. 51 More. Critic. Separation 174. 44 35 18. 29 7. 13 Guess. System. By. Frequency 362. 37 219 18. 18 1. 46 92. 12 134 75. 77 66. 15 9. 27 2 40. 48 23. 81 More. Satisfiers Guess. System. By. Severity Guess. System. Using. Pairs (**)

Strategies - Support We coined the notion of support n n Intuitively, it is a numeric representation of the degree of likelihood of a given set of syndromes in the current session. Calculation is straigthforward. v v Syndromes with full diagnoses add a large positive value. Syndromes with disproved diagnoses add a large negative value. For the rest, symptoms confirmed add positive value and symptoms negated add negative value. Finally we perform a normalization.

Support. Separation. Strategy The third family of strategies is based on support. n Most promising results n 15 different strategies n Hierarchy 7 levels deep n Every level evolving from the previous one n SUCCESS according to target

Results of support strategies Strategy Red Average Median Diag Error % Sev Error % Support. Separation. With. Implication Support. Separation. Implication. Tracking. Closing Support. Main. Syndrome. Scoring Support. Only. Positive. Closing Support. Only. Positive. Candidates. Closing Support. Only. Positive. Closing. Dif. Sev Support. Only. Positive. Strict. Closing Support. Less. Missing. Score Support. More. Coincidences. Pass. Thru_1 Support. More. Coincidences. Pass. Thru_2 Support. More. Coincidences. Pass. Thru_3 6. 09 4. 62 3. 98 2. 95 3. 96 5. 41 3. 34 2. 40 1. 85 2. 12 2. 13 0. 22 1. 65 1. 96 3 2 2 1 2 3 2 1 1 0 1 1 15. 32 18. 29 15. 32 16. 86 15. 32 15. 20 16. 86 16. 50 20. 55 16. 86 57. 84 16. 50 15. 32 5. 58 7. 13 5. 58 5. 70 5. 11 6. 77 6. 41 6. 29 6. 77 39. 55 6. 41 5. 23 Support. More. Coincidences. Pass. Thru_4 2. 12 1 15. 20 5. 10

Results of support strategies Strategy Support. Separation. With. Implication Support. Separation. Implication. Tracking. Closing Support. Main. Syndrome. Scoring Support. Only. Positive. Closing Support. Only. Positive. Candidates. Closing Support. Only. Positive. Closing. Dif. Sev Support. Only. Positive. Strict. Closing Support. Less. Missing. Score Support. More. Coincidences. Pass. Thru_1 Support. More. Coincidences. Pass. Thru_2 Support. More. Coincidences. Pass. Thru_3 Support. More. Coincidences. Pass. Thru_4 Average 11. 75 8. 53 7. 31 5. 94 7. 18 7. 29 5. 11 4. 24 3. 15 4. 65 3. 61 0. 48 2. 63 2. 62 3. 04 3. 07 Yellow Median Diag Error % Sev Error % 6 9. 61 3. 91 6 16. 76 9. 72 5 10. 33 4. 16 4 16. 25 10. 39 5 10. 95 4. 47 6 10. 80 4. 88 4 10. 69 4. 27 3 12. 08 5. 91 2 11. 21 5. 19 3 12. 90 6. 17 2 11. 52 5. 40 0 55. 89 35. 63 2 11. 47 5. 45 2 11. 10 5. 09 2 10. 00 4. 99

Results of support strategies Strategy Support. Separation. With. Implication Support. Separation. Implication. Tracking. Closing Support. Main. Syndrome. Scoring Support. Only. Positive. Closing Support. Only. Positive. Candidates. Closing Support. Only. Positive. Closing. Dif. Sev Support. Only. Positive. Strict. Closing Support. Less. Missing. Score Support. More. Coincidences. Pass. Thru_1 Support. More. Coincidences. Pass. Thru_2 Support. More. Coincidences. Pass. Thru_3 Support. More. Coincidences. Pass. Thru_4 Average 25. 43 17. 67 16. 54 5. 79 16. 46 11. 92 13. 44 5. 18 2. 82 5. 23 3. 39 0. 43 2. 12 2. 11 2. 24 2. 27 Green Diag Error Median % Sev Error % 8 18. 50 9. 97 8 23. 55 10. 66 6 18. 11 9. 97 1 19. 01 16. 43 6 18. 20 9. 97 6 19. 65 10. 00 5 18. 08 10. 00 1 18. 34 10. 48 1 18. 41 14. 48 1 18. 11 9. 70 1 18. 14 13. 82 0 38. 97 13. 91 1 18. 29 14. 12 1 18. 29 14. 06 1 17. 66 10. 00 1 17. 39 9. 19

Conclusions and remarks It was great doing this work because: n n Enhancing the Expert. Care application could have a direct impact on the population’s health. Automated strategies allow Expert. Care architecture to be used in other domains. We applied a scientific research approach and techniques to this “real world” software problem. We learned from Artificial Intelligence, Object. Oriented Programming and Medicine in an interdisciplinary work.

Conclusions and remarks n n n Smalltalk proved to be an adequate tool because: Representation of the knowledge base was almost trivial. Building a virtual lab for essays and benchmarks was very easy. Additional tools for exploring the knowledge base and studying it were easy to implement. There were no barriers for implementing and testing several strategies with diverse heuristics. It was easy to get feedback and to debug troublesome cases, in order to enhance and refine strategies

Future work (technical) n n A visual tool for representing the session. It should be some navigational metaphor. The tool could be enhanced for tracing during simulation runs. More tools for developers to understand interact with strategy/session. More tools for better comparative benchmarking.

Future work (domain model) n n Integrate with Expert. Care Incorporate exceptions and special rules Test with real samples Try some adaptation to other knowledge bases

The End Questions?