Towards Clinically-relevant Standardization of Image Quality Ehsan

Towards Clinically-relevant Standardization of Image Quality • Ehsan Samei, Duke University • Alan Rowberg, University of Washington • Ellie Avraham, Eastman Kodak Company • Craig Cornelius, Eastman Kodak Company 22 -Sept-2003 1

Objectives • Describe current medical image quality and consistency performance efforts • Identify limitations in existing standards • Outline 3 specific proposals: – Add new image quality factors to standards – Update DICOM IQ performance services – Research technical-clinical connection 22 -Sept-2003 2

DICOM Image Consistency Efforts • Three DICOM initiatives (1998 -99): – Grayscale Standard Display Function (GSDF) – Presentation LUT (P-LUT) – Grayscale Softcopy Presentation State (GSPS) • And in actual practice… – Are widely & effectively implemented – Via IHE Consistent Presentation of Images solution: • Promoted conformance testing • Demonstrated inter-vendor consistency 22 -Sept-2003 3

DICOM GSDF: Barten Curve ~150 cd/m 2 L 2 Absolute luminance increment L 1 << L 2 L 1 ~3 cd/m 2 J 1 J 2 Number of perceptual levels J 1 = J 2 22 -Sept-2003 4

AAPM Task Group 18 Efforts • Team of academic, clinical, and industry contributors • “Assessment of Display Performance for Medical Imaging Systems” (2002): – Practical guidelines for qualitative & quantitative display assessment – Includes all key aspects of display performance – Defines IQ test patterns and procedures – Recommends specific IQ acceptance criteria 22 -Sept-2003 5

Standardization gaps • DICOM GSDF: – Pros: • Mathematical definition based on Human Visual System model – Limitations: • Tonescale consistency only: no other IQ factors • No acceptance criteria / conformance procedures • Only for grayscale images • No Display Device Services (Capabilities) 22 -Sept-2003 • AAPM TG 18 report: – Pros: • Provides Professional recommendations • Covers all key display performance aspects • Acceptance Criteria • Quantitative measures – Limitations: • Only guidelines • Not a “standard” 6

The technical-clinical gap: • The connection between quantifiable IQ metrics & clinical performance is unknown: – Luminance: deviations from GSDF, number of gray levels displayed – Spatial: resolution, noise, geometric distortion – Chromaticity variations – Environmental: ambient light, glare, reflection, –… • Amount of acceptable variation is unknown 22 -Sept-2003 7

3 proposals to bridge the gaps: 1. Extend DICOM standard beyond luminance response 2. Add and update DICOM Service Classes for image quality / performance 3. Promote research on clinical technical image quality relationship 22 -Sept-2003 8

1. Display Image Quality (DIQ) Initiative • Add measurable and quantifiable elements of AAPM display performance procedures • Include testing methodologies & defined limits for clinical / diagnostic performance • Quantify visual performance using – Simple test images – Specific observer protocols – Relative acceptance indicators • For both softcopy and hardcopy presentation 22 -Sept-2003 9

DIQ Softcopy Examples • • Quantify % deviation from GSDF curve Define criteria for min & max luminance Define visual luminance evaluation Evaluate specific image quality factors: – Ambient light limits: specular and diffuse – Spatial resolution with TG 18 -QC/CX – Check geometric distortion with TG 18 -QC 22 -Sept-2003 10

Contrast response comparison GSDF contrast +/- 10% Non-standardized display contrast 22 -Sept-2003 11

AAPM TG 18 Patterns TG 18 -CT: contrast / luminance response 22 -Sept-2003 TG 18 -MP: bit-depth / continuous grayscale 12

Comprehensive TG 18 -QC The comprehensive TG 18 -QC test pattern for evaluation of key display characteristics: • Resolution • Luminance • Geometric distortion 22 -Sept-2003 13

Other DIQ Extensions • Hardcopy quality metrics: – Media & printer quality (e. g. , visible coating variations, distortions, artifacts) – GSDF compliance, # of JNDs theory vs. actual – Printable matrix size – Spatial frequency response fidelity • Color extensions: – Standardize for grayscale areas of color images – Add descriptions for color image characterization – Color display and print device calibration 22 -Sept-2003 14

2. DICOM Service Extensions • Add Display Performance Service Class – Query image quality / performance information – Control / configure manageable settings – Include new IQ factors, e. g. , MTF, ambient, … • Extend Printer Configuration Retrieval Service Class – Include access to additional IQ factors Note: Measurables include both human-evaluated and automatically-measured values 22 -Sept-2003 15

Use cases: Standardizing Output • Printing application – Retrieves matrix size and MTF of film printer – Determines type of magnification, if any, to be applied to the image for smallest artifact • Display Performance Service Class User – Requests the luminance characteristic curve from its workstation’s display system – Determines if the display is standardized – If needed, computes an internal image tonescale correction, producing GSDF standardized result 22 -Sept-2003 16

Real device standardized performance 300. 00 250. 00 Target GSDF Luminance 200. 00 Actual device 150. 00 100. 00 Desired output Modified Input 345 50. 00 Original Input 443 0. 00 0 22 -Sept-2003 100 200 300 400 Digital Value 500 600 700 17

Use Case: Quality Control Management • Centralized management application: – Queries devices for calibration date, luminance characteristics, ambient light settings, etc. – Records the results in a central database – Creates maintenance lists for displays and printers • Reporting application uses database for: – Regulatory and management reports – Stability and lifetime statistics on displays to support replacement schedules and budgets 22 -Sept-2003 18

Network-wide Quality Control Diagnostic display Summary data Characteristic curve, MTF, other performance measures 22 -Sept-2003 Clinical display Response ce an e m m or ti s rf ife tic Pe & L tis a St Web-based displays Request Modality Capture console Film Printer Characteristic curve, MTF, other performance measures Re g Re ula po to rt ry s Administrator’s Reflective Hardcopy 19

More use cases… • Consultation: – Maximize perceptual similarity to ensure “What I see is what you will get!” – Display performance information gives confidence • Capture Consoles: – “The technologist sees what the doctor will get. ” – Reduce errors, retakes, miscommunication – Extend quality control program to consoles 22 -Sept-2003 20

3. Clinical Significant of IQ Measures We assume there is a connection… Hmm… What deviations matter, and how much? Physical metrics 22 -Sept-2003 Clinical performance 22

Clinical impact: unknown Rendered with DICOM GSDF 22 -Sept-2003 Not rendered with DICOM GSDF 23

Goals of proposed research: • Determine the clinical consequence of variations in image quality metrics (e. g. , GSDF conformance, MTF, noise, …) • Define what constitutes image quality from a diagnostic perspective • Incorporate results into new joint standards that will utilize standardized test patterns, procedures, and clinical use cases 22 -Sept-2003 24

Suggested research approach • Form inter-society committee to: 1. Design specific research projects 2. Obtains & review data sets 3. Solicit & encourage active participation by researchers: radiologists, scientists, … 4. Arrange reporting of results 5. Define recommendations to standards and professional groups 22 -Sept-2003 • Sample research: 1. Obtain images from 3 radiographic modalities 2. Present images, simulating nonstandard display behavior 3. Run observer performance experiments at major professional meetings and events 4. Analyze by ROC methods 25

Conclusions • Existing standards are insufficient to assure consistent, high quality medical image output. • Steps are proposed to further the reach and impact of DICOM toward quality medicine. • New directions will provide benefits for PACS users, administrators, vendors, and patients. 22 -Sept-2003 26

Contact Information • Ehsan Samei, Duke University samei@duke. edu • Alan Rowberg, M. D. , University of Washington, arowberg@earthlink. com • Ellie Avraham, Eastman Kodak Company, ellie. avraham@kodak. com • Craig Cornelius, Eastman Kodak Company, craig. cornelius@kodak. com • AAPM Task Group 18 web site: http: //deckard. duhs. duke. edu/tg 18 22 -Sept-2003 27