PERSIVAL a System for Personalized Search and Summarization

PERSIVAL a System for Personalized Search and Summarization over Multimedia Information 1

PERSIVAL team members ä Medical Informatics ä ä Medical School – cardiac anesthesiology ä ä Shih-Fu Chang Center for Research on Information Access, Health Sciences Library ä ä Steven Feiner, Luis Gravano, Vasileios Hatzivassiloglou, Kathleen Mc. Keown Electrical Engineering ä ä Desmond Jordan Computer Science ä ä James Cimino, Carol Friedman, Steven Johnson Judith Klavans, Pat Molholt, Elizabeth La. Rue, David Millman Cognitive Science ä Andre Kushniruk (York), Vimla Patel (Medical Informatics) 2

Students ä Computer Science ä ä ä ä ä Min-Yen Kan Simon Lok Smaranda Muresan Sergey Sigelman (programmer) Medical Informatics ä ä Eugene Agichtein Michel Galley Noemie Elhadad Panos Ipeirotis Michael Charney (programmer) Eneida Mendonca Lyudmila Shagina (programmer) ä ä Yoon –Ho Seol Di Wang Electrical Engineering ä Shahram Ebadollah 3

Goals ä Personalized access to distributed, multimedia resources ä ä information access information fusion information understanding Provision of patient-specific information ä ä interaction within context for clinicians, at the point of patient care for patients, in terms that can be understood online patient record serves as a user model 4

Rounds ä ä ä Patient-centric Current: Access to clinical data Missing: Access to literature that fits patient profile 5

Unique Contributions System focus: querying, search, presentation ä Questions are asked within the context of patient information ä A uniform, personalized view of distributed resources on the internet through querying and browsing ä Concise, patient specific presentation of relevant information through summarization ä Access to textual documents linked with access to multimedia video: library of echocardiogram ä Dynamic layout of heterogeneous information 6

Where are we now? ä ä Prototypes of each system component Local library of journal articles and consumer health sites ä ä ä Facilities for distributed online search Scenarios for development and testing with three patients Initial system integration ä ä 20 highly ranked journals 30, 000 articles Restricted to a limited set of examples Formative evaluation of system components 7

Overall Integrated Demo ä What is the prognosis for atrial fibrillation and myocardial infarction? ä ä ä Clinician as user On viewing patient discharge summary Journal articles: controlled clinical trials Re-ranking of search results using patient record What is the treatment for endocarditis? ä ä ä Patient as user On viewing lab results Consumer health information 8

User Interface Focus ä Asking questions within context of patient record ä Evidence based medicine to suggest questions ä Selection of relevant information from the patient record ä Demo of Medlee 11

Distributed Search ä Meta-searcher for automated interaction with heterogeneous, distributed sources ä Use of machine learning and query probes to automatically determine topics of distributed sources ä Information extraction from web pages 13

Re-ranking search results ä Re-rank articles which better match the patient record -> more relevant articles ä Use natural language techniques to analyze article and patient records ä Articles with many terms and values matching the patient record score higher 15

Presentation Focus ä Multimedia summarization ä ä ä Journal articles, consumer health, video Highlight retrieved results to help user in finding relevant information Personalize summary for patient Define unknown terminology Methods for summarizing and search echocardiograms Dynamic layout and organization of results ä Explicitly control level of detail 17

Milestones ä Where we said we would be vs. where we are: ä ä Year 2: skeletal end-to-end system prototype with minimal personalization, interactivity, and limited coverage of structured documents Year 3: Extend to full prototype, with increased personalization, interactivity, limited coordination of multimedia, full range of structured documents, and restricted coverage of consumer documents ä ä ä Use of evidence-based medicine, machine learning to categorize sources by topic, provision of definitions, thin-client computing to allow PERSIVAL on mobile, hand-held devices Year 4: Scale prototype with increased robustness, personalization, coverage to full range of documents and fully integrated multimedia. Coordinate with end-to-end evaluation Year 5: Refine components based on Year 4 evaluation. Transition PERSIVAL to deployment in cooperation with Health Sciences Library 18

Plans for next year ä Increase robustness ä ä ä Extend question asking to different patient contexts, different question types Allow summarization and re-ranking of online articles Extend journal summarization to new genres Extend layout to dynamically incorporate different types of summary input Multimedia integration ä ä ä Implement scenarios for integration Increase interaction with video summary in layout Enhanced multimedia prototype 19