LIGO LSC Data Grid Workshop March 24 -26

LIGO LSC Data. Grid Workshop March 24 -26, 2005 Livingston Observatory

Part One: Introduction • • • A: Workshop Agenda and Pragmatics B: Defining “the Grid” C: Who’s Who in the Grid World D: Overview of the LSG Data. Grid E: Lab 1: Getting Started

A: Workshop Agenda and Pragmatics

Workshop Agenda • Thursday, March 24 • Introduction • Grid Security • Data Management • Friday, March 25 • Job Management • Workflow Management • My. Proxy (Coming Attractions!) • Saturday, March 26 • Local Presentations

Preparation for the Labs • We assume a Red. Hat 9 installation— • Although it’s not impossible that other platforms may work just as well. • We’ll assume you’ve installed LSC Data. Grid Client Toolkit. • We assume your security credentials are already in place.

Bio-Imperatives • Food • Lunches • Dinner • Plumbing

Temporal Disclaimer • The state of the art is: the art is always changing. • Grid infrastructure standards are, however, firming up. • For the most part, we’re going to be talking about how things work at the moment. • We’ll warn you when we go into Coming Attractions mode.

Who Are Those Guys? • GRIDS Center • David Gehrig, NCSA-UIUC • Mike Freemon, NCSA-UIUC • Jaime Frey, University of Wisconson—Madison

Now, everybody—

B: Defining “the Grid”

“Grid” • Buzzword of the year(s). • In enterprise computing, different meanings at different times. • It often simply means “cluster computing. ” • In research, it usually means…

Definition: 1998 “A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities. ” Ian Foster and Carl Kesselman: The Grid: Blueprint for a New Computing Infrastructure

Definition: 2002 “A Grid is a system that • coordinates resources that are not subject to centralized control • using standard, open, general-purpose protocols and interfaces • to deliver nontrivial qualities of service. ” Ian Foster, ANL: What is the Grid? A Three-Point Checklist

A Working Definition • A distributed computing environment that coordinates • Computational jobs • Data placement • Information management • Scales from one computer to thousands • Capable of working across many administrative domains

C: Who’s Who in the Grid World

National Middleware Initiative • Middleware: an evolving layer of services that resides between the network and more traditional applications for managing security, access, and information exchange • www. nsf-middleware. org • Funds GRIDS Center • Funds Open Grid Computing Environment

GRIDS Center • www. grids-center. org • Grid Research Integration, Deployment, and Support Center • Mission: making grid technology deployable and useful outside the development labs • Packaging • Education

The Globus Alliance • www. globus. org • Creates core infrastructure services • Sponsors include: • DARPA, Do. E, NSF, NASA • e-Science (UK), Vetenskapsrådet (Sweden), KTH (Royal Institute of Technology, Stockholm) • IBM, Microsoft Research, Cisco Systems

Globus: Participating Institutions • • • Argonne National Laboratories Information Sciences Institute/USC University of Chicago University of Edinburgh (UK) Center for Parallel Computers (Sweden) “Globus Academic Affiliates”

Globus Toolkit: GT 3 • Software services and libraries • Resource monitoring, discovery, and management • Security • File management • Note! GT 4: Expected release sixth quarter of 2004

Py. Globus • www-itg. lbl. gov/gtg/projects/py. Globus/ • Lawrence Berkeley National Laboratory • An interface to the Globus toolkit using the Python scripting language

Condor • A serial/parallel job management system for a pool of compute nodes: • job queueing mechanism, scheduling policy, priority scheme, resource monitoring, and resource management. • Can be used with Globus Toolkit • www. cs. wisc. edu/condor/ • We’ll use “local Condor” and Condor-G

i. VDGL: International Virtual Data Grid Laboratory • www. ivdgl. org • Goals • • Deploy a Grid laboratory Use Grid software tools in experiments Support delivery of Grid technologies Education and outreach • i. VDGL pacman and VDT • LSC is an active participant

Gri. Phy. N: Grid Physics Network • www. griphyn. org • Coalesced around four experiments • Compact Muon Solenoid and ATLAS (“A Toroidal LHC Apparatu. S”) at LHC/CERN • Laser Interferometer Gravitational-wave Observatory • Sloal Digital Sky Survey • Petabytes of data annually

VDT: Virtual Data Toolkit • www. cs. wisc. edu/vdt/ • Goal: to make it as easy as possible for users to deploy, maintain and use grid middleware • Initially developed by Gri. Phy. N and i. VDGL • Now includes LHC Computing Grid (LCG) and Physics Particle Data Grid (PPDG).

VDT: Components • Basic Grid Services • Condor, Globus • Virtual Data Tools • Virtual Data System • Utilities • Such as GSI-Open. SSH

D: Overview of the LSG Data. Grid

What is the LSC Data. Grid? • A collection of LSC computational and storage resources… • … linked through Grid middleware… • … into a uniform LSC data analysis environment.

LSC Data. Grid Sites • Tier 1: Cal. Tech • Tier 2: UWM and PSU • Tier 3: UT-Brownsville and Salish Kootenai College (SKC) • Linux clusters at GEO sites Birmingham, Cardiff and the Albert Einstein Institute (AEI) • LDAS instances at Caltech, MIT, PSU, and UWM

For this Workshop • LSC Data. Grid Sites • ldas-grid. ligo. caltech. edu • ldas-grid. ligo-wa. caltech. edu • ldas-grid. ligo-la. caltech. edu • We’ll use ldas-grid. ligo-la. caltech. edu as our head node • Full list of LSC Data. Grid resources at www. lscgroup. phys. uwm. edu/lscdatagrid/resources • More discussion of LSC Data. Grid later

E: Lab 1 — Getting Started

Lab 1 — Getting Started This lab will verify: • Your software is installed correctly • Your sacrifices have pleased the webgod Ping • Your security credential (i. e. proxy certificate) is okay • Your environment variables won’t suddenly go away

Credits • Some slides in this presentation were adapted from presentations from • Gry. Phy. N Grid Summer Workshop 2004 • The Globus Consortium