SEE-GRID-2 Bringing Interactivity to 3 D Visualisation on

SEE-GRID-2 Bringing Interactivity to 3 D Visualisation on the Grid www. see-grid. eu Branko Marović Branislav Čukanović, Dragan Okiljević, Ivica Barišić, Jovana Vuleta, Danijel Bašić, Vladimir Čakarević et al. University of Belgrade University Computing Center - RCUB http: //www. rcub. bg. ac. yu The SEE-GRID-2 initiative is co-funded by the European Commission under the FP 6 Research Infrastructures contract no. 031775

Introduction Objective Complete interactive analysis of 3 D datasets from medical imaging devices (CT, MR, PET, SPECT…) Easy remote access over the Internet Generalized 3 D tools for diagnosis, surgical planning, and therapy evaluation Virtual distant examination based on 3 D datasets Basic idea Split the system into a client and server part Dedicate the server part to access and processing of volumetric data Produce a lightweight client for user interaction and 3 D rendering Exploit grid resources and paradigms to overcome implementation issues CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 2

CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 3

Screnshots - From Atoms to Galaxies CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 4

VIVE Gridification General benefits § § Distribution of processing and data over several nodes Data transfer and replication Security – high sensitivity of medical data Application-specific monitoring New grid-specific components § TCP binder (Client-server Three-tier) q q Mediating between clients and server jobs Pool of server jobs Persistence Collecting monitoring and performance data § 3 D image management (LFC web front end, spatially segmented file format) § Measurement archive and web view of collected data Adaptations § Server (lcg-Manage. Software, file I/O to both nearby classic and remote DPM/SRM SEs) § Glue web user interface § Client – longer delays and protocol changes CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 7

Performance Patterns Start-up delay must be avoided at any cost! Network delay § Must be made acceptable for small transactions § OK for long-running requests CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 9

VIVE Grid Architecture CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 10

CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 11

CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 12

CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 13

TCP Binder Pool of server jobs § Pre-emptive submission prevents delays § Adapt to site responsiveness § Prevent flooding of queues § Job lifetime management § Maintaining the satisfactory number of jobs in the pool § Per-site policy CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 14

Performance measurement CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 15

Stripping: Partitioned Volume Processing Split a complex request into subvolumes Allocate satellite jobs from TCP binder Issue several parallel requests as client Retrieve partial results Integrate and return the final result Site A Site B Site C CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 16

Developer Resources Grid environment is constantly evolving, but § § § Useful features persist New are constantly being added Bugs are being fixed Gained knowledge remains relevant, must be updated Applications can be easily migrated to new/updated APIs LCG-2 User Guide § https: //edms. cern. ch/file/454439//LCG-2 -User. Guide. html SEEGRID Wiki § http: //wiki. egee-see. org/index. php/SEE-GRID_Wiki Waiting for g. Lite User Guide CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 17

Software/Job Management Software installation management using ESM tools Experiment Software Installation in LCG-2 § http: //goc. grid. sinica. edu. tw/gocwiki/Experiments_Software_Installation SEEGRID Wiki “SEE-GRID ESM Software Installation Guide” q http: //wiki. egee-see. org/index. php/SEE-GRID_ESM_Software_Installation_Guide § Site configuration § Using ESM tools § Example JDL and shell scripts for installation, validation, and removal Job management EDG WMS API (LCG-2) – C++ & Java § http: //www. to. infn. it/grid/workload_management/api. Doc/edg-wms-apiindex. html Exec/spawn of shell scripts that create and submit JDL – simple, but no direct feedback on jobs g. Lite § http: //egee-jra 1 -wm. mi. infn. it/egee-jra 1 -wm/glite-wms-api-index. shtml CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 18

Data Management LCG-2 (LCG-2 User Guide, “man” pages) § LCG-UTILS API – C/C++ § LFC API – C/C++, Python § GFAL API – C/C++, Python q http: //grid-deployment. web. cern. ch/grid-deployment/gis/GFALindex. html SEEGRID Wiki “SG Using file replicas and RFIO: UI configuration, rfiod, usage in apps, limitations and workarounds” q § § http: //wiki. egeesee. org/index. php/SG_Using_file_replicas_and_RFIO: _UI_configuration%2 C_rfiod% 2 C_usage_in_apps%2 C_limitations_and_workarounds Configuring UI, SE, RB Site testing of RFIO/GFAL Typical problems and solutions Java access to LFC and LCG-UTILS Java LFC/GFAL wrapper § http: //grid 02. rcub. bg. ac. yu/LFCJava. API/index. html Customizable LFC web front end (upload, list, replicate, delete) § http: //grid 02. rcub. bg. ac. yu/repmngr/ g. Lite § http: //grid-deployment. web. cern. ch/griddeployment/documentation/Data. Management/R 3. 0/ CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 19

What VIVE Can Offer to Developers? Ready Experience in ESM software installation management Java LFC/GFAL wrapper Customizable LFC web front end In development Performance measurement facility § § § Service implementing measurement archive Event recording API used by applications, sensors, and monitoring points or agents to put events into measurement archive Basic archive querying API suitable for creation of reports HTML-based web application for querying of the archive and display of reports Advanced features, like generation and display of aggregations, a Web service, and MRTG graphs support could also be added. 3 D Java environment framework § § Net. Beans/Swing based GUI: view and tool management Integration of tool GUI and 3 D appearance Scene, lighting, navigation Java 3 D sensors, 3 D and GUI events § Visualisation of other volumetric data sources: Earth sciences (atmospheric science, geology, VIVE oceanography), aerodynamics, astrophysics, thermodynamics, and dynamics of fluids. App-ware for interactive applications? Based on TCP binder experience CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 20

Implementation Technology Evolution Java Web Start Net. Beans Java 3 D EJB 3/Hibernate My. SQL Servlets/JSP Java JNI Unix shell scripts Grid APIs (GRAM, MDS, WMS, GFAL, LCG-UTILS, LFC, …) DICOM JPEG Applets Sockets Java CGI-Bin VRML C++ CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 t 21

Thank you. CE EGEE and SEE-GRID-2 Summer School on Grid Application Support – Budapest 6 th July 2006 22