Скачать презентацию Tera Grid Software Deployment JP Navarro Tera Grid
c95c163a746a8bcfb634587ab7725d32.ppt
- Количество слайдов: 18
Tera. Grid Software Deployment JP Navarro Tera. Grid GIG - Grid Infrastructure Group Software Integration JP Navarro
Team Participants GIG SI staff currently working on software deployment Mike Showerman (SI Area Director) Charles Bacon, Jason Brechin, Lee Liming, JP Navarro GIG SI Partners NMI/Grids: Peter Couvares, Becky Gietzel, Anatoly Karp Pacman: Saul Youssef JP Navarro
Outline Tera. Grid build & deploy • History: software deployment during construction • Scope: platforms, software components • Future: goals motivating a new approach • New build & deploy • • • Approach overview Building software Software packaging Software configuration Software testing – Inca – Build time tests Initial Implementation Plan • Globus 4. 0. 1 components, tgcp JP Navarro
Tera. Grid Build & Deploy History Thru September 2004 • 6 build scripts: configure, build, package, install • Source -> installed software • Hand configuration • No uniform site customization method • No consistent pre-req, environment setup • Written by a volunteer network of s/w builders, admins • No quality standards • Not tested on every platform • Evolved JP Navarro
Scope: Software 1 Component Tera. Grid CTSS NMI-R 7 APST 2. 3. 1 STORM+Data. Cutter 3. 2. 1 Grid. Port 3. 1 Grid. Solve 2. 0 Network Weather Service 2. 1 Py. Globus 1. 0. 12 JP Navarro
Scope: Software 2 Component Tera. Grid CTSS NMI-R 7 Globus 4. 0. 1 planned 4. 0. 0 Condor NMI 3. 2 6. 7. 7 GSI Open. SSH 3. 7. 1 p 2 3. 5 ? 0. 10. 3 Yes 1. 2. 5. 2 1. 2. 6 My. Proxy 0. 6. 2 0. 6. 4 Uber. FTP 1. 10 1. 13 Inca KX. 509/KCA 20031111 MPICH-G 2 • Tera. Grid planning or likely to upgrade • Convergence possible JP Navarro
Scope: Software 3 Component Tera. Grid CTSS Gx-map >= 0. 3 Soft. Env 1. 4. 2 Tg-policy, tg-accounting Yes SRB client NMI-R 7 3. 2. 1 Mpich-vmi >= 1. 2. 5. 2 >= 1. 1. 2 Python XML Yes Atlas 3. 4. 1 HDF 4 4. 2 HDF 5, PHDF 5 (mpi) JP Navarro
Scope: Software 4 Component Tera. Grid CTSS DB 2 client (where available) Oracle client (where available) PVFS (selected sites) GPFS (most platforms) Vendor MPI NMI-R 7 8. 1 >= 9 i >= 1. 6. 2 >= 2. 2 (GM, Quadrics) Goto (selected platforms) Yes Intel Compilers (selected platforms) 8. 0 Globus (vendor) mpi flavor (selected platforms) MPICH-G 2 (for above) JP Navarro
Scope: Platforms Platform Tera. Grid NMI-R 7 X 86 Red. Hat 9. 0 Yes X 86 Fedore Core 3 Yes X 86 Redhat AS 3 Yes X 86 64 Red. Hat AS 3 Yes Mac OS X Yes Sparc Solaris 9 Yes IA 64 SLES 8 Yes AIX 5. 2 Yes X 86 SLES 8 Yes X 86 SLES 9. 1 Yes X 86 Debian Sarge Yes X 86 Redhat 7. 3 Yes AIX 5. 1 Yes Tru 64 5. 1 (A & B) Yes SGI Pro. Pack 3. 4 Altix IA 64 Yes JP Navarro
Motivation for Change 1. Significantly lower the software deployment bar (effort, complexity) • GIG produced build methods that work on all TG platforms • GIG produced binary packages for all TG platforms • GIG supported installation and configuration methods 2. Timely stack evolution to meet project goals 3. Leverage NMI software build infrastructure to achieve: • Automated, consistent and reproducible software builds on all TG platforms • Tracked build inputs (source, dependencies, build methods) and outputs (binaries, logs) • Improve software quality and reliability 4. Improve consistency and interoperability between TG sites 5. Converge or interoperate with other Grids and Grid software • 6. NMI, OSG, … Retain the ability for Resource Providers “RP”, or anyone else, to build TG software from vendor source on their platforms JP Navarro
New Approach GIG SI Team (3+ FTEs) • Responsible for TG wide build & deploy • Delivers platforms specific binaries to RPs • Provides installation and configuration methods Leverage NMI software build infrastructure • On TG platforms • Launched from TG platforms • Separate build tools, and software source and binary repositories Leverage NMI software build scripts • Only possible for a subset of TG software The actual build scripts have to stand on their own • Usable without the build system to build by hand New Pacman based installation and configuration JP Navarro
The Build System and Tera. Grid Build Facility for the Tera. Grid • • • On demand build infrastructure, launched w/ Grid tools (just another Grid application) Launching and monitoring builds from Tera. Grid (just more Grid jobs) Authenticated source extract and binary store happens on build machines (grid. FTP) Credential propagation with build jobs Dependencies encoded in Soft. Env terms • • Build time software dependencies Run time software dependencies Condor Hawkeye Soft. Env integration Environment setup JP Navarro
Tera. Grid/NMI Build Grid grandcentral TG submit hosts TG submit hosts JP Navarro
Tera. Grid/NMI Automated Build Architecture TG submit host grandcentral TG build host TG Build Tools NMI Build/Test Framework Condor Startd Condor / Condor-G / DAGMan NMI Framework Wrapper TG Software Build Scripts Tera. Grid CVS Build Scripts Build Specs NMI Results DB Build Job Info Sources Binaries Existing NMI component Evolved TG component New TG component JP Navarro
Tera. Grid Software Deployment Architecture TG install host TG Install, Configure Tools Tera. Grid CVS Tera. Grid Software Pacman Caches Config Tools Binaries JP Navarro
TG Install & Configure Tools Pacman 3. x • Common and simpler installation method • Dependency tracking • Verification • Soft. Env integration • Deinstallation Configuration Tools • Looking for good software configuration tools JP Navarro
Software Verification • Inca • Availability verification • Version verification • User environment (Soft. Env) verification • Functionality checking (standalone & distributed) • Historical tracking • Grid Center build time testing (? ) JP Navarro
Initial Implementation Plan • Globus 4. 0. 1 striped grid. FTP and RFT, tgcp 1. 0 • Software in testing for about a month • Production in ~July with as many new methods as possible • Including dependent software • Globus 4. 0. 1 Pre-WS GRAM, RLS, MDS 4 • July and beyond • WS GRAM • The rest of the evolving stack JP Navarro