E-x-t-e-n-d-i-n-g Rocks The Creation and Management of Grid

E-x-t-e-n-d-i-n-g Rocks: The Creation and Management of Grid Systems for Biomedical Research OSGC Conference - May 14, 2008 Vicky Rowley Solution Architect BIRN Coordinating Center - University of California San Diego

BIRN is Data Storage (SRB) and Processing, but… UMN MGH BWH Yale Stanford UI Cluster UCLA UCI UCSD Duke UNM Cluster

Rocks Standard vs. Rocks for BIRN Cluster building focus Data processing focus Lots of big clusters Collaboration focus Data storage/sharing focus A few relatively small clusters Data Grid was needed before clusters processing was needed

So what does BIRN _do_? Large Deformation Diffeomorphic Metric Mapping using the Tera. Grid 4 3 MGH Segmentation JHU Shape Analysis of Segmented Structures Large Scale Distributed Computing 5 BWH Visualization 1 Data Donor Site (Wash. U) N=45 BIRN Data Grid De-identification And upload 2 Preliminary Study: Scientific Goal: • 46 hippocampus data sets classify CPU hours, 4 TB • 30, 000 patient status from morphometric results data

SASHA: Shape Analysis Pipeline Results 6 semantic dementia subjects 21 control subjects 18 Alzheimer subjects Shape-derived metrics can be used to detect class-specific information

The BIRN Collaboratory Today Enabling collaborative research at 28 research institutions comprised of 37 research groups.

How does Rocks make it do that? Installs operating system software Turns individual servers into a “Grid” • • Portals & web servers Data grid for access & management Compute clusters Database servers Distributes, installs and updates 3 rd party, domainspecific scientific software packages Updates system software

What would be better? Add/Improve security & performance monitoring Detect and capture configuration changes Track versions Ideally, reduce, reuse, recycle…

Desired System Qualities Basic System Software Operating System BIRN/Rocks Software Stack Security… Server Definition Software Apache/Tomcat Globus… Application Software Gridsphere HID Mediator Scientific Applications… Custom BIRN Server Agile • Fast response with updates • Self-help for developers Repeatable • Tracking of versions • Tracking of deployments Modular/Flexible • Handles unique site requirements • Handles unique project requirements Customizable Scalable • Highly automated • Supports addition of several sites per year, plus additional projects over 5 years

What’s involved in a single grid? A BIRN Grid Rocks Central & YUM 14 Rocks Rolls (-2 for OS) 17 Custom Rolls: * RHEL 4 * area 51 * base * birnafs * birncondor * birnportal * birnsrb * Cent. OS * condor * cvsserver * freesurfer CVS, SVN & SRB Repos * gama-naregi 1. 0 * gama-naregi 4. 1 * ganglia * gridsphere * hardwareutils * hid * hpc * java * kernel Testbeds * mediator * nagios * oracle * postgres * sciapps * sge * srb 34 * tomcat * updates-Cent. OS * webserver Portal/Web HID Mediator BIRN Rack GPOP GComp Nettools NAS DB Server Registry DB UMLS HID DB MCAT GAMA Server My. Proxy Globus CAS

Software Development & Integration

Software Deployment Development Area Staging Area Production Area -Integrate software for many diverse sources -Verify interoperation of latest code - Stable -Version control at system and sub-component levels -Support demonstration of latest development efforts without disruption to production - Facilitates research -Rolling baseline -Integration and Functional Testing -Functional system/Beta Testing Rocks Development Server Rocks Staging Server - Reliable CVS SRB Rocks Production Server

What we love? Repeatability • All the web servers are the same • All the database servers are the same Flexibility • Mix & Match rolls Level of automation • Experienced person can “kick” a server in 5 minutes • IPs, hostnames, software configuration done Open Source Result: Not one grid - Many! Not one project - Many!

What drives us nuts? Turn around time for updates Steep learning curve RPM building not standard Build time large Software developers are not colocated with integrators Reinstalling to get updates is not an option Lack of advanced roll development training

More info? See the project website: http: //www. nbirn. net Email vrowley_at_ucsd_dot_edu