Automated Detection of Change in Serial MR Brain Studies Julia Patriarche, Ph. D. November 18, 2008
Brain Tumor Suspected
Brain Tumor Suspected Neurologic exam, Imaging, Biopsy
Brain Tumor Suspected Neurologic exam, Imaging, Biopsy Resection, radiation, chemotherapy
Brain Tumor Suspected Neurologic exam, Imaging, Biopsy Resection, radiation, chemotherapy Serial Imaging
Baseline Followup
Baseline Followup Change or No Change?
Baseline Followup
Scanner has changed Scanner Software has changed Pulse Sequences have changed Acquisition parameters have changed Gradient Coils have changed RF Inhomogeneities have changed Registration has changed …
Change Detection is not so simple 1. Separation of acquisition changes from disease related changes 2. Information overload 3. Change Blindness / Inattentional Blindness
Change Blindness Our visual systems have evolved to control our sensitivity to change What we see is in the moment is richly detailed, but what is held in memory from one moment to the next is not the detailed visual information – it is abstracted.
Mirror Example
Ronald A. Rensink, University of British Columbia http: //www. psych. ubc. ca/~rensink/flicker/download/
Changing case of letters Mc. Conkie, G. W. and Zola, D. “Is visual information integrated across successive fixations in reading? ” Percept. Psychophysiol. 1979; 25: 221 -224. AITe. Rn. At. Ed Ca. Se
Changing case of letters Mc. Conkie, G. W. and Zola, D. “Is visual information integrated across successive fixations in reading? ” Percept. Psychophysiol. 1979; 25: 221 -224. AITe. Rn. At. Ed Ca. Se alt. Er. Na. Te. D c. As. E
Change Blindness There really isn’t a mechanism for changes in this abstracted information to grab your attention.
Change Blindness There really isn’t a mechanism for changes in this abstracted information to grab your attention. If you don’t specifically attend to some feature, or some aspect of that feature at both time-points and then consciously compare them, there is a very good chance you’ll miss changes.
Change Blindness There really isn’t a mechanism for changes in this abstracted information to grab your attention. If you don’t specifically attend to some feature, or some aspect of that feature at both time-points and then consciously compare them, there is a very good chance you’ll miss changes. Trying to find subtle changes, in extent, or character is very difficult.
Change Blindness There is a whole area of study devoted to change blindness
Change Blindness There is a whole area of study devoted to change blindness People tend not to be aware of the extent of change blindness (i. e. change blindness is a topic of study)
Change Blindness There is a whole area of study devoted to change blindness People tend not to be aware of the extent of change blindness (i. e. change blindness is a topic of study) Visual change detection is so difficult that there are even games where you’re supposed to find the differences.
Change Blindness The point: side-by-side presentation is very poorly matched to the human visual system. Unregistered images, inhomogeneity effects, information overload make this problem even more challenging.
Baseline Follow-up Change or No Change?
Other reasons radiologists might miss changes: Satisfaction of Search Change occurs in an unexpected location Change is in a sub-part of a complicated lesion Etc.
Not surprisingly… Change detection is thought to be not very reproducible
What is desired: 1. Systems which reduce the quantity of data presented to the radiologist, 2. Systems which help to separate acquisition related change from disease related change, 3. Systems which present data in a way matched to the human visual system 4. Systems which produce objective, reproducible, and accurate metrics
Something which draws the radiologist’s attention to those changes. Something which presents what is of interest – changes – to the radiologist. It’s easier to understand the nature of the changes as a whole, if you can view an image of the changes.
Change Detector System Flow
Acquisition Considerations To the Greatest Degree Possible: Reduce slice thickness Eliminate inter-slice gaps 3 D acquisition Equivalent scanner software Equivalent pulse sequences Equivalent acquisition parameters Correct for decay of gradient coils Correct for inhomogeneties Identical administration of contrast
As with an experiment, Control the variables, to isolate the changes of interest
Change Detector Algorithm Organization
The algorithm is designed to detect changes which are: Subtle in extent Subtle in degree The algorithm produces both: Quantitative summaries Qualitative color change maps
Baseline Followup
Baseline Followup Color Change Map Original Clinical Interpretation: Stable / Time to Progression Diagnosis = 7 months
Baseline Followup
Baseline Followup Color Change Map Original Clinical Interpretation: Stable / Time to Progression Diagnosis = 3. 25 months
Baseline Followup
Baseline Followup Color Change Map Original Clinical Interpretation: Stable / Time to Progression Diagnosis = 4. 75 months
Baseline Follow-up
Baseline Follow-up Color Change Map Original Clinical Interpretation: Stable / Time to Progression Diagnosis > 18 months
Determining that something hasn’t changed, can be as tricky as determining that it has
Study #1: Assisted Serial MRI Examination
Sensitivity: Fraction of the actually positive cases which were correctly identified Specificity: Fraction of the actually negative cases which were correctly identified
Study #1: Assisted Serial MRI Examination • 6 neuroradiologists examined a series of 50 serial MR pairs from 28 patients • Asked to rate each case as stable or progressing • Double cross over study with one month separation
Study #1 Results • All but one neuroradiologist had improved sensitivity • Two NRs achieved perfect sensitivity with the change detector, none without • Mean accuracy improved from 0. 859 to 0. 891 • Mean sensitivity was largely unchanged, from 0. 892 to 0. 891 • Mean specificity improved moderately from 0. 848 to 0. 874
Study #1 Results: Interobserver Agreement • Without the change detector, there were 17 cases in which all the neuroradiologists agreed with the gold standard • With the change detector, there were 23
Study #2: Automated Serial MRI Examination • Automated system examined 88 serial comparisons • Divided cases into: stable or progressing • Automated diagnosis was compared to original clinical diagnosis
Suggests that the change detector may be able to detect changes earlier than expert neuroradiologists. Patriarche JW, Erickson BJ. "Part 2. Automated Change Detection and Characterization Applied to Serial MR of Brain Tumors May Detect Progression Earlier Than Human Experts". Journal of Digital Imaging 2007; 20(4): 321 -328.
Reasons for good outcome: • No objective definition of progression.
Reasons for good outcome: • No objective definition of progression. • Subtle changes difficult to see • Especially when images displayed side-by-side • Especially when images not registered
Reasons for good outcome: • No objective definition of progression. • Subtle changes difficult to see • Especially when images displayed side-by-side • Especially when images not registered • Changes may not be obvious looking at only one pulse sequence, but might present across pulse sequences
Reasons for good outcome: • No objective definition of progression. • Subtle changes difficult to see • Especially when images displayed side-by-side • Especially when images not registered • Changes may not be obvious looking at only one pulse sequence, but might present across pulse sequences • Changes unexpected in location, etc.
Reasons for good outcome: • No objective definition of progression. • Subtle changes difficult to see • Especially when images displayed side-by-side • Especially when images not registered • Changes may not be obvious looking at only one pulse sequence, but might present across pulse sequences • Changes unexpected in location, etc. • Satisfaction of Search
Reasons for good outcome: • No objective definition of progression. • Subtle changes difficult to see • Especially when images displayed side-by-side • Especially when images not registered • Changes may not be obvious looking at only one pulse sequence, but might present across pulse sequences • Changes unexpected in location, etc. • Satisfaction of Search • Information overload - impractical or impossible to wade through every possible comparison manually
The Point: • Computers are great at methodically wading through large amounts of data, looking for needles in hay-stacks, integrating large amounts of data at once, applying serial processing steps, analyzing data mathematically, and bringing noteworthy observations to the attention of the neuroradiologist.
Current Work Eric Kischell has implemented a very streamlined workflow • Provides full automation, end-to-end Liqin Wang has implemented a web-based interface • Neuroradiologists can review change detector output from any computer with a web-browser • Allows sophisticated interaction with output, e. g. via two color map modes, linked cursors, and ‘flicker mode’ viewing of serial images.
Clinical Deployment: • Facilitate studies with more patients and neuroradiologists • Facilitate more varied study designs • Directly aid patients
Other Developments: • Full automation / full reproducibility - removes barriers to widespread deployment.
Other Developments: • Full automation / full reproducibility - removes barriers to widespread deployment. • Greatly improved sensitivity and specificity
Layered AI Architecture: Change detector possesses multiple layers of automated reasoning / information gathering. Starts with simplistic but highly reliable knowledge, and uses this to acquire case-specific information. Uses this case-specific information to acquire more complex information, etc. up the hierarchy.
Components are modular and interchangeable.
Obviously: If everything is built on the lower levels – if the reasoning is based on the knowledge gleaned by those lower levels -- those levels must produce very reliable information.
Information Overload: • The phenomenon is extremely wide-spread. • The problem / opportunity will almost definitely continue to grow.
Quantitative & Automated Analyses: • Makes information overload, information affluence
Change Detector: • May help to train new practitioners • May serve as a model of how computer can liberate neuroradiologist from doing mundane things sub-optimally, and instead have computer wade through mountains of analysis and present the clinician with refined and concise data tailored to their purpose. • May serve as a model of how computer can perform a mountain of peripheral analyses and bring to clinician’s attention things they weren’t looking for (but which are important). • May serve as a model of ‘layered analyses’ which are impractical or impossible without computer assistance.
All of this: Designed to translate what is acquired, into what is desired – data tailored to the task at hand, by using processing and domain specific knowledge.
All of this: Designed to translate what is acquired, into what is desired – data tailored to the task at hand, by using processing and domain specific knowledge. Don’t ask the radiologist to do the routine / protocol-driven work – define the procedure, and build a machine to perform that procedure.
All of this: Designed to translate what is acquired, into what is desired – data tailored to the task at hand, by using processing and domain specific knowledge. Don’t ask the radiologist to do the routine / protocol-driven work – define the procedure, and build a machine to perform that procedure. Use the computer’s ability to do routine pre-defined tasks precisely, every time, and to handle complex quantitative and layered analyses with ease; to do massive numbers of procedure-driven analyses on each case, to present highly processed information tailored to the task at hand, and to direct attention to identified features which require it.
All of this: Designed to translate what is acquired, into what is desired – data tailored to the task at hand, by using processing and domain specific knowledge. Don’t ask the radiologist to do the routine / protocol-driven work – define the procedure, and build a machine to perform that procedure. Use the computer’s ability to do routine pre-defined tasks precisely, every time, and to handle complex quantitative and layered analyses with ease; to do massive numbers of procedure-driven analyses on each case, to present highly processed information tailored to the task at hand, and to direct attention to identified features which require it. Use the computer as a smart helper, who goes away and does tons of analysis, and who brings to the attention of the radiologist things which should be attended to.
Some clinical reasons the change detector is of value: Create a quantitative definition of progression Possible improved outcomes through earlier interventions Potentially facilitate screening Potentially facilitate personalized therapy Compare novel therapies, through automatic, reproducible, and quantitative measures of change.
Future Directions: Many possibilities.
Future Directions: Many possibilities. Add additional pulse sequences (e. g. diffusion, perfusion, etc. ).
Future Directions: Many possibilities. Add additional pulse sequences (e. g. diffusion, perfusion, etc. ). Add more complex anatomical knowledge (e. g. so change detector knows what is white matter, gray matter, CSF, and can act appropriately.
Future Directions: Many possibilities. Add additional pulse sequences (e. g. diffusion, perfusion, etc. ). Add more complex anatomical knowledge (e. g. so change detector knows what is white matter, gray matter, CSF, and can act appropriately. Identify and quantify different kinds of changes, ex. attempt to differentiate between character change and mass effect; identify and quantify atrophy.
Future Directions: Many possibilities. Add additional pulse sequences (e. g. diffusion, perfusion, etc. ). Add more complex anatomical knowledge (e. g. so change detector knows what is white matter, gray matter, CSF, and can act appropriately. Identify and quantify different kinds of changes, ex. attempt to differentiate between character change and mass effect; identify and quantify atrophy. Apply technique to different anatomical regions (e. g. breast, etc. )
Future Directions: Many possibilities. Add additional pulse sequences (e. g. diffusion, perfusion, etc. ). Add more complex anatomical knowledge (e. g. so change detector knows what is white matter, gray matter, CSF, and can act appropriately. Identify and quantify different kinds of changes, ex. attempt to differentiate between character change and mass effect; identify and quantify atrophy. Apply technique to different anatomical regions (e. g. breast, etc. ) A variety of clinical studies.
Acknowledgements Christopher Wood, M. D. Norbert Campeau, M. D. Edward Lindell, M. D. Vladimir Savcenko, M. D. Norman Arslanlar, D. O. Bradley Erickson M. D. , Ph. D.
Acknowledgements Bradley Erickson, M. D. , Ph. D. – Director, Mayo Clinic Radiology Informatics Lab Brian O’Neill, M. D. – Director, Mayo Neurology Fellowship Program Christopher Chute, M. D. , Dr. P. H. – Director, Mayo Informatics Fellowship Program Eric Kischell – Programmer, implemented change detector integrated system Liqin Wang – Programmer, implemented registration algorithm, web interface Mayo Brain Computation Workgroup University of Hawaii – Department of Information and Computer Sciences NIH T 32 NS 07494 -03 NIH T 15 LM 07041 -23
Скачать презентацию Automated Detection of Change in Serial MR Brain
af652a951a695fe76fb6a097f2ee2c3c.ppt