Integrating Universities and Laboratories In National Cyberinfrastructure Paul Avery University of Florida avery@phys. ufl. edu PASI Lecture Mendoza, Argentina May 17, 2005 PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 1
Outline of Talk Ø Cyberinfrastructure u Data Ø The intensive disciplines and Data Grids Trillium Grid collaboration u Gri. Phy. N, Ø The i. VDGL, PPDG LHC and its computing challenges Ø Grid 3 ØA and Grids and the Open Science Grid bit on networks Ø Education and Outreach Ø Challenges for the future Ø Summary Presented from a physicist’s perspective! PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 2
Cyberinfrastructure (cont) Ø Software programs, services, instruments, data, information, knowledge, applicable to specific projects, disciplines, and communities. Ø Cyberinfrastructure layer of enabling hardware, algorithms, software, communications, institutions, and personnel. A platform that empowers researchers to innovate and eventually revolutionize what they do, how they do it, and who participates. Ø Base technologies: Computation, storage, and communication components that continue to advance in raw capacity at exponential rates. [Paraphrased from NSF Blue Ribbon Panel report, 2003] Challenge: Creating and operating advanced cyberinfrastructure and integrating Argentinascience and engineering applications. PASI: Mendoza, it in (May 17, 2005) Paul Avery 3
Cyberinfrastructure and Grids Ø Grid: Geographically distributed computing resources configured for coordinated use u Fabric: Physical resources & networks provide raw capability u Ownership: Resources controlled by owners and shared w/ others u Middleware: Software ties it all together: tools, services, etc. Ø Enhancing collaboration via transparent resource sharing US-CMS “Virtual Organization” PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 4
Data Grids & Collaborative Research Ø Team-based 21 st century scientific discovery u Strongly dependent on advanced information technology u People and resources distributed internationally Ø Dominant u 2000 u 2005 u 2010 u 2015 -7 Ø Drives factor: data growth (1 Petabyte = 1000 TB) ~0. 5 Petabyte ~10 Petabytes ~1000 Petabytes? How to collect, manage, access and interpret this quantity of data? need for powerful linked resources: “Data Grids” u Computation u Data storage and access u Data movement Ø Collaborative u Data Massive, distributed CPU Distributed hi-speed disk and tape International optical networks research and Data Grids discovery, resource sharing, distributed analysis, etc. PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 5
Examples of Data Intensive Disciplines Ø High energy & nuclear physics u Belle, Ba. Bar, Tevatron, RHIC, JLAB u Large Hadron Collider (LHC) Primary driver Ø Astronomy u Digital sky surveys, “Virtual” Observatories u VLBI arrays: multiple- Gb/s data streams Ø Gravity wave searches u LIGO, Ø Earth GEO, VIRGO, TAMA, ACIGA, … and climate systems u Earth Ø Biology, Observation, climate modeling, oceanography, … medicine, imaging u Genome databases u Proteomics (protein structure & interactions, drug delivery, …) u High-resolution brain scans (1 -10 m, time dependent) PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 6
Our Vision & Goals Ø Develop the technologies & tools needed to exploit a Grid-based cyberinfrastructure End-to-end Ø Apply and evaluate those technologies & tools in challenging scientific problems Ø Develop the technologies & procedures to support a permanent Grid-based cyberinfrastructure Ø Create and operate a persistent Grid-based cyberinfrastructure in support of discipline-specific research goals PASI: Mendoza, Argentina + 17, 2005) Paul Avery 7 Gri. Phy. N + i. VDGL(May DOE Particle Physics Data Grid (PPDG) = Trillium
Our Science Drivers at Large Hadron Collider u New Ø High Energy & Nuclear Physics expts u Top quark, nuclear matter at extreme density u ~1 Petabyte (1000 TB) 1997 – present Ø LIGO (gravity wave search) u Search for gravitational waves u 100 s of Terabytes 2002 – present Ø Sloan Digital Sky Survey 2007 2005 2003 2001 Data growth fundamental particles and forces u 100 s of Petabytes 2007 - ? 2009 Community growth Ø Experiments u Systematic survey of astronomical objects u 10 s of Terabytes 2001 – present PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 8
Grid Middleware: Virtual Data Toolkit VDT NMI Sources (CVS) Build & Test Condor pool 22+ Op. Systems Build Test Pacman cache Package Patching GPT src bundles Binaries RPMs Build Binaries Test Build Binaries Many Contributors A unique laboratory for testing, supporting, deploying, packaging, upgrading, & PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 9 troubleshooting complex sets of software!
VDT Growth Over 3 Years # of components www. griphyn. org/vdt/ VDT 1. 1. 8 First real use by LCG VDT 1. 0 Globus 2. 0 b Condor 6. 3. 1 VDT 1. 1. 11 Grid 3 VDT 1. 1. 7 Switch to Globus 2. 2 PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 10
Components of VDT 1. 3. 5 u Globus 3. 2. 1 u Condor 6. 7. 6 u RLS 3. 0 u Class. Ads 0. 9. 7 u Replica 2. 2. 4 u DOE/EDG CA certs u ftsh 2. 0. 5 u EDG mkgridmap u EDG CRL Update u GLUE Schema 1. 0 u VDS 1. 3. 5 b u Java u Netlogger 3. 2. 4 u Gatekeeper-Authz u My. Proxy 1. 11 u KX 509 PASI: Mendoza, Argentina (May 17, 2005) u System Profiler u GSI Open. SSH 3. 4 u Monalisa 1. 2. 32 u Py. Globus 1. 0. 6 u My. SQL u Uber. FTP 1. 11 u DRM 1. 2. 6 a u VOMS 1. 4. 0 u VOMS Admin 0. 7. 5 u Tomcat u PRIMA 0. 2 u Certificate Scripts u Apache u j. Clarens 0. 5. 3 u New Grid. FTP Server u GUMS 1. 0. 1 Paul Avery 11
Collaborative Relationships: A CS + VDT Perspective Partner science projects Partner networking projects Partner outreach projects Requirements Prototyping & experiments Other linkages Ø Work force Ø CS researchers Ø Industry U. S. Grids Int’l Outreach Production Deployment Computer Virtual Larger Techniques Tech Science Data Science & software Research Toolkit Transfer Community Globus, Condor, NMI, i. VDGL, PPDG EU Data. Grid, LHC Experiments, Quark. Net, CHEPREO, Dig. Divide PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 12
U. S. “Trillium” Grid Partnership Ø Trillium = PPDG + Gri. Phy. N + i. VDGL u Particle Physics Data Grid: $12 M (DOE) (1999 – 2006) u Gri. Phy. N: $12 M (NSF) (2000 – 2005) u i. VDGL: $14 M (NSF) (2001 – 2006) Ø Basic composition (~150 people) u PPDG: 4 universities, 6 labs u Gri. Phy. N: 12 universities, SDSC, 3 labs u i. VDGL: 18 universities, SDSC, 4 labs, foreign partners u Expts: Ba. Bar, D 0, STAR, Jlab, CMS, ATLAS, LIGO, SDSS/NVO Ø Coordinated u Gri. Phy. N: internally to meet broad goals CS research, Virtual Data Toolkit (VDT) development u i. VDGL: Grid laboratory deployment using VDT, applications u PPDG: “End to end” Grid services, monitoring, analysis u Common use of VDT for underlying Grid middleware PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 13 u Unified entity when collaborating internationally
Goal: Peta-scale Data Grids for Global Science Production Team Single Researcher Workgroups Interactive User Tools Virtual Data Tools Request Planning & Scheduling Tools Resource Management Services Security and Policy Services Peta. Ops u Petabytes u Performance u Other Grid Services Transforms Distributed resources Raw data source PASI: Mendoza, Argentina (May 17, 2005) Request Execution & Management Tools (code, storage, CPUs, networks) Paul Avery 14
Sloan Digital Sky Survey (SDSS) Using Virtual Data in Gri. Phy. N Sloan Data Galaxy cluster size distribution PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 15
The LIGO Scientific Collaboration (LSC) and the LIGO Grid: 6 US sites + 3 EU sites (Cardiff/UK, AEI/Germany) i. VDGL has enabled LSC to establish a persistent production grid Birmingham • §Cardiff AEI/Golm • * LHO, LLO: observatory sites * LSC - LIGO Scientific Collaboration - i. VDGL supported PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 16
Large Hadron Collider & its Frontier Computing Challenges PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 17
Large Hadron Collider (LHC) @ CERN km Tunnel in Switzerland & France 27 TOTEM CMS ALICE LHCb Search for Ø Origin of Mass Ø New fundamental forces Ø Supersymmetry Ø Other new particles PASI: Mendoza, Ø 2007 – ? Argentina (May 17, 2005) ATLAS Paul Avery 18
CMS: “Compact” Muon Solenoid Inconsequential humans PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 19
LHC Data Rates: Detector to Storage 40 MHz Physics filtering ~TBytes/sec Level 1 Trigger: Special Hardware 75 GB/sec 75 KHz Level 2 Trigger: Commodity CPUs 5 GB/sec 5 KHz Level 3 Trigger: Commodity CPUs 0. 15 – 1. 5 GB/sec 100 Hz PASI: Mendoza, Argentina (May 17, 2005) Raw Data to storage (+ simulated data) Paul Avery 20
Complexity: Higgs Decay to 4 Muons (+30 minimum bias events) All charged tracks with pt > 2 Ge. V Reconstructed tracks with pt > 25 Ge. V 109 collisions/sec, selectivity: 1 in 1013 PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 21
LHC: Petascale Global Science Ø Complexity: Millions Ø Scale: of individual detector channels Peta. Ops (CPU), 100 s of Petabytes (Data) Ø Distribution: Global distribution of people & resources Ba. Bar/D 0 Example - 2004 700+ Physicists 100+ Institutes 35+ Countries CMS Example- 2007 5000+ Physicists 250+ Institutes 60+ Countries PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 22
LHC Global Data Grid (2007+) Ø 5000 physicists, 60 countries Ø 10 s of Petabytes/yr by 2008 Ø 1000 Petabytes in < 10 yrs? CMS Experiment Online System Tier 0 Tier 1 CERN Computer Center 150 - 1500 MB/s Korea Russia UK 10 -40 Gb/s USA >10 Gb/s U Florida Tier 2 Caltech UCSD 2. 5 -10 Gb/s Tier 3 Tier 4 FIU Physics caches PASI: Mendoza, Argentina (May 17, 2005) Iowa Maryland PCs Paul Avery 23
University Tier 2 Centers Ø Tier 2 facility u Essential university role in extended computing infrastructure u 20 – 25% of Tier 1 national laboratory, supported by NSF u Validated by 3 years of experience (CMS, ATLAS, LIGO) Ø Functions u Perform physics analysis, simulations u Support experiment software u Support smaller institutions Ø Official role in Grid hierarchy (U. S. ) u Sanctioned by MOU with parent organization (ATLAS, CMS, LIGO) u Selection by collaboration via careful process u Local P. I. with reporting responsibilities PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 24
Grids and Globally Distributed Teams Ø Non-hierarchical: Chaotic analyses + productions Ø Superimpose significant random data flows PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 25
Grid 3 and Open Science Grid PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 26
Grid 3: A National Grid Infrastructure Ø 32 sites, 4000 CPUs: Universities + 4 national labs Ø Part of LHC Grid, Running since October 2003 Ø Sites in US, Korea, Brazil, Taiwan Ø Applications in HEP, LIGO, SDSS, Genomics, f. MRI, CS Brazil PASI: Mendoza, Argentina (May 17, 2005) http: //www. ivdgl. org/grid 3 Paul Avery 27
Grid 3 World Map PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 28
Grid 3 Components Ø Computers Ø Uniform & storage at ~30 sites: 4000 CPUs service environment at each site u Globus Toolkit: Provides basic authentication, execution management, data movement u Pacman: Installs numerous other VDT and application services Ø Global & virtual organization services u Certification & registration authorities, VO membership services, monitoring services Ø Client-side tools for data access & analysis u Virtual data, execution planning, DAG management, execution management, monitoring Ø IGOC: Ø Grid i. VDGL Grid Operations Center testbed: Grid 3 dev u Middleware development and testing, new VDT versions, etc. PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 29
Grid 3 Applications CMS experiment p-p collision simulations & analysis ATLAS experiment p-p collision simulations & analysis BTEV experiment p-p collision simulations & analysis LIGO Search for gravitational wave sources SDSS Galaxy cluster finding Bio-molecular analysis Shake n Bake (Sn. B) (Buffalo) Genome analysis GADU/Gnare f. MRI Functional MRI (Dartmouth) CS Demonstrators Job Exerciser, Grid. FTP, Net. Logger www. ivdgl. org/grid 3/applications PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 30
Usage: CPUs Grid 3 Shared Use Over 6 months ATLAS DC 2 CMS DC 04 Sep 10 PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 31
Grid 3 Production Over 13 Months PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 32
U. S. CMS 2003 Production Ø 10 M p-p collisions; largest ever u 2 x simulation sample u ½ manpower ØMulti-VO sharing PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 33
Grid 3 as CS Research Lab: E. g. , Adaptive Scheduling Ø Adaptive data placement in a realistic environment (K. Ranganathan) Ø Enables comparisons with simulations PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 34
Grid 3 Lessons Learned Ø How to operate a Grid as a facility u Tools, services, error recovery, procedures, docs, organization u Delegation of responsibilities (Project, VO, service, site, …) u Crucial role of Grid Operations Center (GOC) Ø How to support people relations u Face-face Ø How to test and validate Grid tools and applications u Vital Ø How role of testbeds to scale algorithms, software, process u Some Ø How meetings, phone cons, 1 -1 interactions, mail lists, etc. successes, but “interesting” failure modes still occur to apply distributed cyberinfrastructure u Successful production runs for several applications PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 35
Grid 3 Open Science Grid Ø Iteratively build & extend Grid 3 OSG-0 OSG-1 OSG-2 … u Shared resources, benefiting broad set of disciplines u Grid middleware based on Virtual Data Toolkit (VDT) u Emphasis on “end to end” services for applications u Grid 3 Ø OSG collaboration u Computer and application scientists u Facility, technology and resource providers (labs, universities) Ø Further develop OSG u Partnerships and contributions from other sciences, universities u Incorporation of advanced networking u Focus on general services, operations, end-to-end performance Ø Aim for Summer 2005 deployment PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 36
http: //www. opensciencegrid. org PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 37
OSG Organization Advisory Committee Universities, Labs Service Providers Technical Groups Executive Board (8 -15 representatives Chair, Officers) Sites Researchers VOs activity 1 1 Activities Research Grid Projects Enterprise PASI: Mendoza, Argentina (May 17, 2005) Core OSG Staff (few FTEs, manager) Paul Avery OSG Council (all members above a certain threshold, Chair, officers) 38
OSG Technical Groups & Activities Ø Technical Groups address and coordinate technical areas u Propose and carry out activities related to their given areas u Liaise & collaborate with other peer projects (U. S. & international) u Participate in relevant standards organizations. u Chairs participate in Blueprint, Integration and Deployment activities Ø Activities are well-defined, scoped tasks contributing to OSG u Each Activity has deliverables and a plan u … is self-organized and operated u … is overseen & sponsored by one or more Technical Groups TGs and Activities are where the real work gets done PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 39
OSG Technical Groups Governance Charter, organization, by-laws, agreements, formal processes Policy VO & site policy, authorization, priorities, privilege & access rights Security Common security principles, security infrastructure Monitoring and Information Services Resource monitoring, information services, auditing, troubleshooting Storage services at remote sites, interfaces, interoperability Infrastructure and services for user support, helpdesk, trouble ticket Training, interface with various E/O projects Support Centers Education / Outreach Networks (new) Including interfacing with various networking projects PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 40
OSG Activities Blueprint Defining principles and best practices for OSG Deployment of resources & services Provisioning Connected to deployment Incidence response Plans and procedures for responding to security incidents Integration Testing & validating & integrating new services and technologies Data Resource Management (DRM) Deployment of specific Storage Resource Management technology Documentation Organizing the documentation infrastructure Accounting and auditing use of OSG resources Interoperability Primarily interoperability between Operations Operating Grid-wide services PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 41
Connections to European Projects: LCG and EGEE PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 42
The Path to the OSG Operating Grid Readiness plan adopted Readiness plan Effort Resources VO Application Software Installation Software & packaging Service deployment Middleware Interoperability Functionality & Scalability Tests feedback Release Description PASI: Mendoza, Argentina (May 17, 2005) Paul Avery Application validation Metrics & Certification Release Candidate OSG Operations-Provisioning Activity OSG Deployment Activity OSG Integration Activity 43
OSG Integration Testbed Brazil PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 44
Status of OSG Deployment Ø OSG infrastructure release accepted for deployment. US CMS MOP “flood testing” successful u D 0 simulation & reprocessing jobs running on selected OSG sites u Others in various stages of readying applications & infrastructure (ATLAS, CMS, STAR, CDF, Ba. Bar, f. MRI) u Ø Deployment process underway: End of July? Open OSG and transition resources from Grid 3 u Applications will use growing ITB & OSG resources during transition u http: //osg. ivdgl. org/twiki/bin/view/Integration/Web. Home PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 45
Interoperability & Federation Ø Transparent use of Federated Grid infrastructures a goal Ø There are sites that appear as part of “LCG” as well as part of OSG/Grid 3 u D 0 bringing reprocessing to LCG sites through adaptor node u CMS and ATLAS can run their jobs on both LCG and OSG Ø Increasing interaction with Tera. Grid u CMS and ATLAS sample simulation jobs are running on Tera. Grid u Plans for Tera. Grid allocation for jobs running in Grid 3 model: with group accounts, binary distributions, external data management, etc PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 46
Networks PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 47
Evolving Science Requirements for Networks (DOE High Perf. Network Workshop) End 2 End Throughput 5 years End 2 End Throughput High Energy Physics Climate (Data & Computation) SNS Nano. Science 0. 5 Gb/s 100 Gb/s 5 -10 Years End 2 End Throughput 1000 Gb/s 0. 5 Gb/s 160 -200 Gb/s N x 1000 Gb/s Not yet started 1 Gb/s 1000 Gb/s + Qo. S for Control Channel Fusion Energy 0. 066 Gb/s (500 MB/s burst) 0. 013 Gb/s (1 TB/week) 0. 2 Gb/s (500 MB/ 20 sec. burst) N*N multicast N x 1000 Gb/s Time critical throughput 1000 Gb/s 0. 091 Gb/s (1 TB/day) 100 s of users 1000 Gb/s + Qo. S for Control Channel Computational steering and collaborations High throughput and steering Science Areas Astrophysics Genomics Data & Computation Today Remarks High bulk throughput Remote control and time critical throughput See http: //www. doecollaboratory. org/meetings/hpnpw PASI: Mendoza, Argentina (May 17, 2005) Paul Avery / 48
Ultra. Light: Advanced Networking in Applications Funded by ITR 2004 PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 10 Gb/s+ network • Caltech, UF, FIU, UM, MIT • SLAC, FNAL • Int’l partners 49 • Level(3), Cisco, NLR
Ultra. Light: New Information System ØA new class of integrated information systems u Includes networking as a managed resource for the first time u Uses “Hybrid” packet-switched and circuit-switched optical network infrastructure u Monitor, manage & optimize network and Grid Systems in realtime Ø Flagship applications: HEP, e. VLBI, “burst” imaging u “Terabyte-scale” data transactions in minutes u Extend Real-Time e. VLBI to the 10 – 100 Gb/s Range Ø Powerful testbed u Significant Ø Strong storage, optical networks for testing new Grid services vendor partnerships u Cisco, Calient, NLR, CENIC, Internet 2/Abilene PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 50
Education and Outreach PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 51
i. VDGL, Gri. Phy. N Education/Outreach Basics Ø Ø $200 K/yr Led by UT Brownsville Workshops, portals, tutorials New partnerships with Quark. Net, CHEPREO, LIGO E/O, … PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 52
June 2004 Grid Summer School Ø First of its kind in the U. S. (South Padre Island, Texas) u 36 students, diverse origins and types (M, F, MSIs, etc) Ø Marks new direction for U. S. Grid efforts u First attempt to systematically train people in Grid technologies u First attempt to gather relevant materials in one place u Today: Students in CS and Physics u Next: Students, postdocs, junior & senior scientists Ø Reaching a wider audience u Put lectures, exercises, video, on the web u More tutorials, perhaps 2 -3/year u Dedicated resources for remote tutorials u Create “Grid Cookbook”, e. g. Georgia Tech Ø Second u South workshop: July 11– 15, 2005 Padre Island again PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 53
Quark. Net/Gri. Phy. N e-Lab Project http: //quarknet. uchicago. edu/elab/cosmic/home. jsp PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 54
Student Muon Lifetime Analysis in Gri. Phy. N/Quark. Net PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 55
CHEPREO: Center for High Energy Physics Research and Educational Outreach Florida International University § § Physics Learning Center CMS Research i. VDGL Grid Activities AMPATH network (S. America) Ø Funded September 2003 Ø $4 M initially (3 years) Ø MPS, CISE, EHR, INT
Grids and the Digital Divide Rio de Janeiro, Feb. 16 -20, 2004 NEWS: Bulletin: ONE TWO WELCOME BULLETIN General Information Registration Travel Information Hotel Registration Participant List How to Get UERJ/Hotel Computer Accounts Useful Phone Numbers Program Contact us: Secretariat Chairmen Background Ø World Summit on Information Society Ø HEP Standing Committee on Inter-regional Connectivity (SCIC) Themes Ø Global collaborations, Grids and addressing the Digital Divide Ø Focus on poorly connected regions Next meeting: Daegu, Korea Ø May 23 -27, 2005 PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 57
Partnerships Drive Success Ø Integrating Grids in scientific research u “Lab-centric”: u “Team-centric”: u “Knowledge-centric”: community Ø Strengthening Activities center around large facility Resources shared by distributed teams Knowledge generated/used by a the role of universities in frontier research u Couples universities to frontier data intensive research u Brings front-line research and resources to students u Exploits intellectual resources at minority or remote institutions Ø Driving advances in IT/science/engineering u Domain sciences u Universities u Scientists u NSF projects u NSF u Research (May 17, 2005) PASI: Mendoza, Argentinacommunities Computer Science Laboratories Students NSF projects DOE Avery IT industry Paul 58
Fulfilling the Promise of Next Generation Science Ø Supporting permanent, national-scale Grid infrastructure u Large CPU, storage and network capability crucial for science u Support personnel, equipment maintenance, replacement, upgrade u Tier 1 and Tier 2 resources a vital part of infrastructure u Open Science Grid a unique national infrastructure for science Ø Supporting the maintenance, testing and dissemination of advanced middleware u Long-term support of the Virtual Data Toolkit u Vital for reaching new disciplines & for supporting large international collaborations Ø Continuing support for HEP as a frontier challenge driver u Huge challenges posed by LHC global interactive analysis u New challenges posed by remote operation of Global Accelerator Network PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 59
Fulfilling the Promise (2) Ø Creating even more advanced cyberinfrastructure u Integrating databases in large-scale Grid environments u Interactive analysis with distributed teams u Partnerships involving CS research with application drivers Ø Supporting the emerging role of advanced networks u Reliable, high performance LANs and WANs necessary for advanced Grid applications Ø Partnering to enable stronger, more diverse programs u Programs supported by multiple Directorates, a la CHEPREO u NSF-DOE joint initiatives u Strengthen ability of universities and labs to work together Ø Providing opportunities for cyberinfrastructure training, education & outreach u Grid tutorials, Grid Cookbook u Collaborative tools for student-led projects & research PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 60
Summary Ø Grids enable 21 st century collaborative science u Linking research communities and resources for scientific discovery u Needed by global collaborations pursuing “petascale” science Ø Grid 3 was an important first step in developing US Grids u Value of planning, coordination, testbeds, rapid feedback u Value of learning how to operate a Grid as a facility u Value of building & sustaining community relationships Ø Grids drive need for advanced optical networks Ø Grids impact education and outreach u Providing technologies & resources for training, education, outreach u Addressing the Digital Divide Ø OSG: a scalable computing infrastructure for science? u Strategies needed to cope with increasingly large scale PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 61
Grid Project References ØOpen Science Grid ØUltra. Light u www. opensciencegrid. org ØGrid 3 u ultralight. cacr. caltech. edu ØGlobus u www. ivdgl. org/grid 3 ØVirtual Data Toolkit u www. griphyn. org/vdt ØGri. Phy. N u www. griphyn. org Øi. VDGL u www. ivdgl. org ØPPDG u www. ppdg. net u www. globus. org ØCondor u www. cs. wisc. edu/condor ØLCG u www. cern. ch/lcg ØEU Data. Grid u www. eu-datagrid. org ØEGEE u www. eu-egee. org ØCHEPREO u www. chepreo. org PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 62
Extra Slides PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 63
Gri. Phy. N Goals Ø Conduct CS research to achieve vision u Virtual Data as unifying principle u Planning, execution, performance monitoring Ø Disseminate u. A through Virtual Data Toolkit “concrete” deliverable Ø Integrate into Gri. Phy. N science experiments u Common Ø Educate, Grid tools, services involve, train students in IT research u Undergrads, postdocs, u Underrepresented groups PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 64
i. VDGL Goals Ø Deploy a Grid laboratory u Support research mission of data intensive experiments u Provide computing and personnel resources at university sites u Provide platform for computer science technology development u Prototype and deploy a Grid Operations Center (i. GOC) Ø Integrate u Into Grid software tools computing infrastructures of the experiments Ø Support delivery of Grid technologies u Hardening of the Virtual Data Toolkit (VDT) and other middleware technologies developed by Gri. Phy. N and other Grid projects Ø Education and Outreach u Lead and collaborate with Education and Outreach efforts u Provide tools and mechanisms for underrepresented groups and remote regions to participate in international science projects PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 65
Analysis Client • Discovery • ACL management • Cert. based access • ROOT (analysis tool) • Python • Cojac (detector viz)/ IGUANA (cms viz) HTTP, SOAP, XMLRPC Clarens CMS: Grid Enabled Analysis Architecture Analysis Client u Clients talk standard protocols to “Grid Services Web Server” Grid Services Web Server Scheduler Sphinx MCRunjob Catalogs Fully. Abstract Planner Metadata Ref. DB Partially. Abstract Planner Chimera Fully. Concrete Planner u Simple Web service API allows simple or complex analysis clients Data Management Virtual Data Mon. ALISA MOPDB Monitoring Replica BOSS u Typical clients: ROOT, Web Browser, …. ORCA Applications ROOT FAMOS POOL Execution Priority Manager u Key features: Global Scheduler, Catalogs, Monitoring, Grid-wide Execution service VDT-Server Grid Wide Execution PASI: Mendoza, Argentina (May 17, 2005) Service u Clarens portal hides complexity Paul Avery 66
“Virtual Data”: Derivation & Provenance Ø Most scientific data are not simple “measurements” u They are computationally corrected/reconstructed u They can be produced by numerical simulation Ø Science & eng. projects are more CPU and data intensive u Programs are significant community resources (transformations) u So are the executions of those programs (derivations) Ø Management u Derivation: u Provenance: of dataset dependencies critical! Instantiation of a potential data product Complete history of any existing data product ØPreviously: Manual methods ØGri. Phy. N: Automated, robust tools PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 67
Virtual Data Example: HEP Analysis decay = bb decay = WW WW leptons decay = ZZ mass = 160 decay = WW Other cuts PASI: Mendoza, Argentina (May 17, 2005) decay = WW WW e Other cuts Paul Avery Scientist adds a new derived data branch & continues analysis decay = WW WW e Pt > 20 Other cuts 68
Packaging of Grid Software: Pacman Ø Language: define software environments Ø Interpreter: create, install, configure, update, verify environments Ø Version 3. 0. 2 released Jan. 2005 Ø LCG/Scram Ø ATLAS/CMT Ø CMS DPE/tar/make Ø LIGO/tar/make Ø Open. Source/tar/make Ø Globus/GPT Ø NPACI/Tera. Grid/tar/make Ø D 0/UPS-UPD Ø Commercial/tar/make Combine and manage software from arbitrary sources. “ 1 button install”: Reduce burden on administrators % pacman –get i. VDGL: Grid 3 LIGO VDTVDT UCHEP i. VDGL % pacman. Argentina (May 17, 2005) PASI: Mendoza, D-Zero ATLAS CMS/DPE Paul Avery NPAC I Remote experts define installation/ config/updating for everyone at once 69
Virtual Data Motivations “I’ve found some interesting data, but I need to know exactly what corrections were applied before I can trust it. ” “I’ve detected a muon calibration error and want to know which derived data products need to be recomputed. ” Describe Discover VDC Reuse Validate “I want to search a database for 3 muon events. If a program that does this analysis exists, I won’t have to write one from scratch. ” PASI: Mendoza, Argentina (May 17, 2005) Paul Avery “I want to apply a forward jet analysis to 100 M events. If the results already exist, I’ll save weeks of computation. ” 70
Background: Data Grid Projects Driven primarily by HEP applications ØU. S. Funded Projects ØEU, u Gri. Phy. N (NSF) u i. VDGL (NSF) u Particle Physics Data Grid (DOE) u Ultra. Light u Tera. Grid (NSF) u DOE Science Grid (DOE) u NEESgrid (NSF) u NSF Middleware Initiative (NSF) Asia projects u EGEE (EU) u LCG (CERN) u Data. Grid u EU national Projects u Data. TAG (EU) u Cross. Grid (EU) u Grid. Lab (EU) u Japanese, Korea Projects Many projects driven/led by HEP + CS Ø Many 10 s x $M brought into the field Ø Large impact on other sciences, education Ø PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 71
“Virtual Data”: Derivation & Provenance Ø Most scientific data are not simple “measurements” u They are computationally corrected/reconstructed u They can be produced by numerical simulation Ø Science & eng. projects are more CPU and data intensive u Programs are significant community resources (transformations) u So are the executions of those programs (derivations) Ø Management u Derivation: u Provenance: of dataset dependencies critical! Instantiation of a potential data product Complete history of any existing data product ØPreviously: Manual methods ØGri. Phy. N: Automated, robust tools PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 72
Muon Lifetime Analysis Workflow PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 73
(Early) Virtual Data Language pythia_input pythia. exe cmsim_input cmsim. exe write. Hits write. Digis CMS “Pipeline”Avery Paul PASI: Mendoza, Argentina (May 17, 2005) begin v /usr/local/demo/scripts/cmkin_input. csh file i ntpl_file_path file i template_file i num_events stdout cmkin_param_file end begin v /usr/local/demo/binaries/kine_make_ntpl_pyt_cms 121. exe pre cms_env_var stdin cmkin_param_file stdout cmkin_log file o ntpl_file end begin v /usr/local/demo/scripts/cmsim_input. csh file i ntpl_file i fz_file_path file i hbook_file_path file i num_trigs stdout cmsim_param_file end begin v /usr/local/demo/binaries/cms 121. exe condor copy_to_spool=false condor getenv=true stdin cmsim_param_file stdout cmsim_log file o fz_file o hbook_file end begin v /usr/local/demo/binaries/write. Hits. sh condor getenv=true pre orca_hits file i fz_file i detinput file i condor_write. Hits_log file i oo_fd_boot file i datasetname stdout write. Hits_log file o hits_db end begin v /usr/local/demo/binaries/write. Digis. sh pre orca_digis file i hits_db file i oo_fd_boot file i carf_input_dataset_name file i carf_output_dataset_name file i carf_input_owner file i carf_output_owner file i condor_write. Digis_log stdout write. Digis_log file o digis_db end 74
Quark. Net Portal Architecture Ø Simpler interface for non-experts Ø Builds on Chiron portal PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 75
Integration of Gri. Phy. N and IVDGL Ø Both funded by NSF large ITRs, overlapping periods u Gri. Phy. N: 9/2005) u i. VDGL: Ø Basic Grid Laboratory, applications (9/2000– (9/2001– 9/2006) composition u Gri. Phy. N: u i. VDGL: u Expts: Ø Gri. Phy. N 12 universities, SDSC, 4 labs (~80 people) 18 institutions, SDSC, 4 labs (~100 people) CMS, ATLAS, LIGO, SDSS/NVO (Grid research) vs i. VDGL (Grid deployment) u Gri. Phy. N: u i. VDGL: Ø Many CS Research, Virtual Data Toolkit 2/3 “CS” + 1/3 “physics” 1/3 “CS” + 2/3 “physics” ( 0% H/W) (20% H/W) common elements u Common Directors, Advisory Committee, linked management Virtual Data Toolkit (VDT) Grid testbeds Outreach effort PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 76
Science Review sharing Analysis exec. discovery data Researcher composition Applications instrument Chimera virtual data system Planning planning Production Manager data Gri. Phy. N Overview Virtual Production Data params discovery storage element Services storage element Grid Fabric PASI: Mendoza, Argentina (May 17, 2005) Pegasus planner DAGman Globus Toolkit Condor Ganglia, etc. Virtual Data Paul Avery Toolkit Execution 77
Chiron/Quark. Net Architecture PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 78
Cyberinfrastructure “A new age has dawned in scientific & engineering research, pushed by continuing progress in computing, information, and communication technology, & pulled by the expanding complexity, scope, and scale of today’s challenges. The capacity of this technology has crossed thresholds that now make possible a comprehensive “cyberinfrastructure” on which to build new types of scientific & engineering knowledge environments & organizations and to pursue research in new ways & with increased efficacy. ” [NSF Blue Ribbon Panel report, 2003] PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 79
Fulfilling the Promise of Next Generation Science Our multidisciplinary partnership of physicists, computer scientists, engineers, networking specialists and education experts, from universities and laboratories, has achieved tremendous success in creating and maintaining general purpose cyberinfrastructure supporting leading-edge science. But these achievements have occurred in the context of overlapping short-term projects. How can we ensure the survival of valuable existing cyberinfrastructure while continuing to address new challenges posed by frontier scientific and engineering endeavors? PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 80
Production Simulations on Grid 3 US-CMS Monte Carlo Simulation Used = 1. 5 US-CMS resources Non-USCMS PASI: Mendoza, Argentina (May 17, 2005) Paul Avery 81
Скачать презентацию Integrating Universities and Laboratories In National Cyberinfrastructure Paul
94d74561510cf5fb1f99417613284270.ppt