Analuse Globalisée des Données d ’Imagerie Radiologique Grid Analysis of Radiological Data Cécile Germain-Renaud Laboratoire de Recherche en Informatique Laboratoire de l’Accélérateur Linéaire CNRS and Université Paris-Sud For the AGIR team
Contexts Analyse Globalisée des Données d’Imagerie Radiologique • A multidisciplinary research network • Funded by programme Masses de Données of the french ministry of research - 3 years from Sept. 2004 • Medical image processing and computational/storage grids LRI – coll LAL LIMSI – coll Tenon, FMP, Sainte Anne CNRS-STIC CNRS-IN 2 P 3 INRIA INSERM Hospitals www. aci-agir. org Al. Gorille CRAN LPC CHRU Clermont CREATIS Rainbow Epidaure Centre Antoine Lacassagne Journée GDR Stic Santé - 09/06/2005 2
Partners Analyse Globalisée des Données d’Imagerie Radiologique • Parallel Architecture - LRI – U. Paris-Sud & CNRS STIC Grid models & middleware – Cécile Germain-Renaud, Romain Texier • LIMSI – CNRS STIC Medical Image processing & software – Clinical research – Angel Osorio, Julien Nauroy and team, Emmanuelle Frenoux • Al Gorille – LORIA - U. Nancy & INRIA Lorraine Grid models & algorithms – Emmanuel Jeannot • CRAN – U. Nancy & CNRS STIC Image processing – compression – Jean-Marie Moureaux, Yann Gaudeau • CREATIS – CNRS STIC, INSERM Medical Image processing – Isabelle Magnin, Patrick Clarysse and team • LPC – U. Clermont & CNRS IN 2 P 3 EGEE & Medical grids – Vincent Breton, Yannick Legré, Antoine Llorens • EPIDAURE - INRIA Sophia Image processing – Xavier Penned, Radu Stefanescu • RAINBOW-I 3 S CNRS STIC & U. Nice Software engineering – distributed components – Johan Montagnat, Tristan Glatard • Centre Antoine Lacassagne Clinical Research – Pierre-Yves Bondiau • Collaboration EGEE-LAL www. aci-agir. org – Charles Loomis, Daniel Jouvenot Journée GDR Stic Santé - 09/06/2005 3
The initial grid concept Analyse Globalisée des Données d’Imagerie Radiologique • 1998 « A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to highend computational capabilities. » • Metacomputing(92), Iway(96) • Key ideas – Electrical power grid – Resource level /Qo. S /Ease of use www. aci-agir. org Journée GDR Stic Santé - 09/06/2005 4
Evolution of the Grid concept Analyse Globalisée des Données d’Imagerie Radiologique • “Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations” [I. Foster and C. Kesselman and S. Tuecke. The Anatomy of the Grid: Enabling Scalable Virtual Organizations. Intl Jal Supercomputer Applications, 2002] • From computing to data – Institutional grids: computation on very large datasets § Particle physics production, Biomedical & Earth science deployed testbeds § Not much intersection with the semantic grid context so far • Resource sharing – Static: authentication, authorization, accounting www. aci-agir. org Journée GDR Stic Santé - 09/06/2005 5
Medical images and grids Analyse Globalisée des Données d’Imagerie Radiologique LHC – PB/year: final decision for grid computing ONE radiology department – 10 TB/year ONE CT-scan dataset 500 MB • Standards for Integrated Data and Computational Grids are emerging: Global Grid Forum • Institutional support for operational deployement – EU FP 6 project EGEE: 30 M€ only for software development – US Grid 3: 26 universities – Strong Computer Science research implication: § Structured grids: Globus § Unused cycles: Condor § Core. Grid www. aci-agir. org Journée GDR Stic Santé - 09/06/2005 6
Objectives Analyse Globalisée des Données d’Imagerie Radiologique • Leverage medical image algorithms and their usage – Integrated computational and storage grids as a unified resource provider for compute/data/network intensive needs § Image processing: research and clinical practice situations § Sharing data and algorithms • Specific grid services – Identification and definition: technology and research – Advanced middleware demonstrators – Impact grid projects § Strong collaboration with EU FP 6 EGEE • Applications testbeds www. aci-agir. org Journée GDR Stic Santé - 09/06/2005 7
Use cases Analyse Globalisée des Données d’Imagerie Radiologique Medical collections Epidemiology Health care policies Ad Hoc Collections Algorithmic & Clinical research e-Learning Individual analysis e-medicine Augmented reality www. aci-agir. org Journée GDR Stic Santé - 09/06/2005 8
Work packages Analyse Globalisée des Données d’Imagerie Radiologique Medical applications evaluation P-Y Bondiau Cardiological images Segmentation I. Magnin Humanitarian Medical Development V. Breton Image registration in oncology X. Pennec Workflow Management J. Montagnat Services for Interactivity C. Germain Medical data Management J. Montagnat Medical data access protocols J-M. Moureaux Middleware evaluation E. Jeannot www. aci-agir. org Dissemination C. Germain Interactive volume reconstuction A. Osorio Medical Apps. Algorithm Gridification Core Grid Medical Services Journée GDR Stic Santé - 09/06/2005 9
Research areas (1) Analyse Globalisée des Données d’Imagerie Radiologique • Scaling critical components of distributed systems – Scheduling: where and when ? § Agent-based scheduling and Qo. S: sharing resources across users § Workflow management: application performance § Contexts: (multi)processor scheduling, parallel scheduling – Medical data and metadata management (coll. LAL and Medi. Grid) § Interoperability between medical service/format standards and grid storage services § Security/privacy and medical requirements eg patient benefit • Adaptive storage and transmission § Compression methods § Network-adaptive compression (coll Network team CRAN) www. aci-agir. org Journée GDR Stic Santé - 09/06/2005 10
Research areas (2) Analyse Globalisée des Données d’Imagerie Radiologique • Medical imaging algorithms – Parallel processing: 3 D + time segmentation, non-linear registration – Bronze standard evaluation • Interaction between compression and image processing – Impact on intrinsic performance § Experiments: deployement and statistical analysis § Rationale ? (semantic) grid users – Integration of multi-scale/multi-level methods and compression www. aci-agir. org Journée GDR Stic Santé - 09/06/2005 11
Compression: QVAZM Analyse Globalisée des Données d’Imagerie Radiologique • A Lattice Vector Quantization method • 3 D correlations: better detection of constant regions and contour preservation Brain RMI 64: 1 SPIHT 3 D QVAZM www. aci-agir. org Journée GDR Stic Santé - 09/06/2005 12
Collection processing testbed Analyse Globalisée des Données d’Imagerie Radiologique PTM 3 D measurements Gold standard Consensus Bronze standard Evaluation PTM 3 D Volume Reconstruction Automatic Nodules CAD Grid-enabled Workflow Registration Algorithms Network emulation QVAZM 3 D Compression SPIHT Compression www. aci-agir. org ADOC Partially reliable transport protocol Journée GDR Stic Santé - 09/06/2005 13
The image database Analyse Globalisée des Données d’Imagerie Radiologique • Goal: Provide an image dataset to the partners • Core: the personal collection of Angel Osorio - mainly CT • Centralised very simple SQL BD • Open to collaboration … www. aci-agir. org Journée GDR Stic Santé - 09/06/2005 14
Clinical situation testbeds (1) Analyse Globalisée des Données d’Imagerie Radiologique • g. PTM 3 D: grid-enabling the PTM 3 D software – Focus: services for interactive grids - visualization pipeline - EGEE – Agent-based scheduling and Qo. S-oriented scheduling – Adaptive compression – Data and metadata management Interaction Acquire www. aci-agir. org Explore Analyse Interpret Render Journée GDR Stic Santé - 09/06/2005 15
Clinical situation testbeds (2) Analyse Globalisée des Données d’Imagerie Radiologique • CHINA – Collaboration between Hospitals for International Neurosurgery Applications – A web-based application – Goals § Enable medical data exchange : • Text-based data • Medical images § Second remote diagnosis 1 – YU Zhu Ping www. aci-agir. org Journée GDR Stic Santé - 09/06/2005 16
Services for Interactive Grids Analyse Globalisée des Données d’Imagerie Radiologique • Basic tools for seamless integration of the grid resources into general workflows • Convergence with non-biomedical applications • Depart from the computing centre model: – Local data or computations must be included – On-demand interactive Id Owner Submitted On f 01 n 01. 10873. 0 qzha 5/19 07: 34 f 01 n 03. 6292. 0 agma 5/22 14: 50 f 01 n 03. 6293. 0 publ 5/22 16: 16 f 01 n 03. 6304. 0 agma 5/22 22: 46 f 01 n 03. 6309. 0 agma 5/23 12: 41 f 01 n 01. 10914. 0 ying 5/23 14: 17 f 01 n 02. 4596. 0 dpan 5/23 15: 33 f 01 n 03. 6310. 0 divi 5/23 16: 03 f 01 n 03. 6312. 0 publ 5/23 17: 03 f 01 n 03. 6313. 0 publ 5/23 17: 41 f 01 n 01. 10921. 0 dani 5/24 08: 42 www. aci-agir. org ST PRI Class Running R R R I I I 50 50 50 fewcpu f 11 n 07 standard f 12 n 02 standard f 03 n 09 standard f 11 n 05 standard f 01 n 11 fewcpu f 06 n 03 standard standard Journée GDR Stic Santé - 09/06/2005 17
From PTM 3 D to g. PTM 3 D Analyse Globalisée des Données d’Imagerie Radiologique • Volume and organs reconstruction help the surgeon to plan the optimal path • g. PTM 3 D first result interactive response time for volume www. aci-agir. org Journée GDR Stic Santé - 09/06/2005 reconstruction 18
Figures Analyse Globalisée des Données d’Imagerie Radiologique Dataset Input data Output data Tasks 87 MB 3 MB 6 MB 18 KB/slice 106 KB/slice 169 5 mn 15 s 1 mn 54 s 37 s 18 s Medium 210 MB body 9. 6 MB 57 MB 25 KB/slice 151 KB/slice 378 33 mn 11 mn 5 s 2 mn 30 s 1 mn 15 s Large body 346 MB 15 MB 86 MB 22 KB/sclice 131 KB/slice 676 18 mn 2 mn 03 Lungs 87 MB 410 KB 4 KB/slice 36 s 24 s Small body www. aci-agir. org 2. 3 MB 24 KB/slice 95 Standalone EGEE Execution. Journée GDR Stic Santé - 09/06/2005 19
Services for interactive grids Analyse Globalisée des Données d’Imagerie Radiologique Front-end • Computational steering – Streams from/to a non-grid machine EGEE User Interface to/from running applications • Coping with the submission penalty – N tasks each with small latency T Interaction Bridge CE Scheduling Agent § Potential completion bandwidth T-1 § Impaired by the submission protocol Worker – Application-level scheduling Demonstrated at EGEE 1 st and 2 nd Conference - Selected as EGEE demo for the first EU review [Germain, Osorio Texier. Interactive Volume reconstruction and measurement MIM 44(2)] www. aci-agir. org Journée GDR Stic Santé - 09/06/2005 20
Scheduling for interactivity Analyse Globalisée des Données d’Imagerie Radiologique • Grid level scheduling cannot be centralized – Broker systems: matchmaking based on asynchronous information • Node schedulers can implement complex policies – Entities such as individual users, Virtual Organizations, applications, date, … – Past and current resource usage – Queue based: batch systems • Short Deadline Jobs – Immediate execution subject to access control without user reservation – No preemption/no utilization degradation – Context: Usage Policy Scheduling – Grid 3 www. aci-agir. org Journée GDR Stic Santé - 09/06/2005 – Fair share across SDJ: Soft real time scheduling with variable 21
Interactive jobs on a grid: a scheduling problem Analyse Globalisée des Données d’Imagerie Radiologique • Short Deadline Job – A moldable application – individual tasks are very fine-grained – Soft deadline – No reservation: should be executed immediately or rejected • Sharing contract – – Bounded slowdown for regular jobs Do not degrade resource utilization No stong preemption Fair share across SDJ • Contexts – (multi) Processor soft real-time scheduling – Network routing Differentiated Services www. aci-agir. org Journée GDR Stic Santé - 09/06/2005 22
Scheduling SDJ Analyse Globalisée des Données d’Imagerie Radiologique Interaction Bridge TP User Interface Interaction Bridge JSS www. aci-agir. org Cluster Scheduler CE User Interface Broker Node Task prioritization Matchmaki ng Permanent reservation on virtual processors Transparent when unused Journée GDR Stic Santé - 09/06/2005 23
Скачать презентацию Analuse Globalisée des Données d Imagerie Radiologique Grid
aceacf87f9c4f532b32d17cb21b9aa62.ppt