Globus status perspective Olle Mulmo

Globus – status & perspective Olle Mulmo <? ? ? @pdc. kth. se> PDC Enabling Science

What this talk will cover • Background: Why we are Where we are today • Where we will go tomorrow [Insert proper acknowledgement to everyone I have stolen slides from here] PDC Enabling Science

Metacomputing. . . • A concept conceived of in the (late? ) ’ 80 s • 1990’s: Dramatic increase in network bandwitdh • • • Opens up for entire new problem areas Solving ”larger-than-a-supercomputer” problems Remote visualization Remote instrumentation Collaboration (VR) • 1995: • I-WAY, proof-of-concept demo/project • 1996: • The Globus Project sees the day PDC Enabling Science

The Globus approach • Middleware approach • A set of orthogonal APIs that applications can use • Command-line tools that use the APIs (for scripting) • IP hourglass model Applications Diverse global services Core services • Portability requirements • No perl on the Cray at the time • Poor C++ support • -> Implementation using bourne shell (sh) and ANSI C only Local OS PDC Enabling Science

Hot technologies half a decade ago • Network communication • • • Raw ATM vs TCP/IP? Use Nexus to avoid technology lock-in Already ported to several UNIX flavors Automatic data conversion Optimization for local communication (AAL 5, TCP, MPI, SHMEM) • Information Discovery • • Queryable, Extensible (”give me a computer with 128 M RAM”) Use LDAP (Lightweight Directory Access Protocol) Add TTL on all published data Add heart beats to know the test bed status PDC Enabling Science

Hot technologies half a decade ago (cont. ) • Security • • Hot potato: Local control a must -> mapfiles (what users running on what accounts) PKI evolving, big push from SSL -> let’s use it • Resource Allocation • No standard: myriads of different local schedulers • -> RSL (Resource Specification Language) + wrappers PDC Enabling Science

First round: the GUSTO testbed • 1997: The Globus Metacomputing Toolkit • MDS (Metacomputing Directory Service) • GRAM (Globus Resource Allocation Manager) • DUROC (multi-site coordinated startup) • No security (plain-text passwords) • No file transfers PDC Enabling Science

The GUSTO testbed (cont. ) • 1998: Globus release 1. 0 • Security added: GSI (Globus Security Infrastructure) • • PKI (or Kerberos) below GSS-API SSL handshakes Proxy certificate delegation No encryption due to export laws • File transfer added: • GASS (fetch files over https: // or file: //) PDC Enabling Science

The GUSTO testbed (cont. ) • . . . not really that heavily used, though. . . PDC Enabling Science

Lessons learned from Globus v 1. x • Severe performance problems with MDS • A centralized LDAP server didn’t scale very well • -> MDS-2 framework (GRIS and GIIS) • Complaints on configuration and build times • You got everything and all the ”flavors” • debug, nothreads, pthreads, native threads, with MPI, without MPI, . . . • -> Packaging technology • One of the multi-site jobs often got stuck in a queue / poor network performance • -> GARA (Advanced reservation) PDC Enabling Science

Lessons learned from Globus v 1. x • Users don’t want to hack their favorite applications • Easy access • They have NT on their desktops • Mass storage accessibility is important • -> Data and replica management work begins PDC Enabling Science

1999: The year of the Grid • The Concept of a Grid evolves • Resource sharing & coordinated problem solving in dynamic, multi -institutional virtual organizations PDC Enabling Science

1999 -now: The Grid boom. . . • The Grid book covers a lot of ground in the community • Lots of grid projects spring to life • Increase in funding • Commercial interests (Microsoft, Sun, IBM, . . . ) • Grid Forum is founded • ”The place to standardize Grid-related stuff” • Many of the protocols and approaches tried out by Globus are adopted as a first shot PDC Enabling Science

Some renaming is required. . . • GSI -> Grid Security Infrastructure • GRAM -> Grid Resource Allocation Management • MDS -> Monitoring and Discovery Service • Never say ”Metacomputing” again. . . PDC Enabling Science

Back to the drawing board: Globus v 2. 0 • • • New packaging technology (not RPM) Precompiled binaries Grid. FTP MDS-2: decentralized LDAP framework Security on all communications channels • Repackaged with add-ons by several Data. Grid projects • Gri. Phy. N, EDG, NMI, . . . PDC Enabling Science

Where we are today • The Globus Toolkit™ centers around four key protocols • Connectivity layer: • Security: Grid Security Infrastructure (GSI) • Resource layer: • Resource Management: Grid Resource Allocation Management (GRAM) • Information Services: Grid Resource Information Protocol (GRIP) and Index Information Protocol (GIIP) • Data Transfer: Grid File Transfer Protocol (Grid. FTP) PDC Enabling Science

A second look • Many different protocols and methods are used • LDAP, HTTP, FTP, . . . • Missing or lack of support for • Databases • Notification systems • Error propagation • The security part needs more work • Dynamic, Multi-VO Authorization schemes • Audit • Proxy delegation (IETF draft not likely to become an RFC) PDC Enabling Science

Let’s go shopping: Hot technologies of today • XML & Web services • Generic and extensible • Many tools available • Backed up by industry • Attribute Certificates • Give a certificate holder certain privilegies (access to files, resources, . . . ) • NMI is pursuing this path • -> Globus ships CAS (similar concept) PDC Enabling Science

Web services means generic encapsulation • ”Corba rewritten in XML” (and then some) • WSDL: Web Services Definition Language • Defines a service and what methods that can be invoked • Stub compilers exist for several programming languages • -> Easy to wrap your old application • SOAP: Simple Object Access Protocol • Invocation protocol, operating on an instance of a WSDL target • WS-Inspection and UDDI • Conventions (naming etc. ) for querying and locating WSDL documents with certain properties PDC Enabling Science

Open Grid Services Architecture (OGSA) • Web Services for the Grid • WSDLs for some key components • Factory, Registry, Grid Service • Dynamic properties (lifetime, version compatibility) are added • Open-source implementation in Java, . NET & C • Guidelines, conventions and best practices • • Global naming Lifetime managment Notification framework. . . PDC Enabling Science

OGSA • Still a draft (continously revised) • Alpha release of Core implementation (Java) • The Factory and generic Grid Service part • Base implementation next • Current Globus functionality refactored as Grid Services • Alpha at the end of this summer • Globus 3. 0 (Q 1 ’ 03? ) • Old APIs will still work (or so they say) • . . . but the wire protocols are new GT 3 Data Services Other Grid Services GT 3 Base Services GT 3 Core PDC Enabling Science

Next steps: your guess is as good as mine. . . • Experiences learned with v 2 and v 3 • Access to tertiary storage and databases • How much can be gotten ”for free” from other WS initiatives? • Other forms of security infrastructures? • Passport / Liberty Alliance technologies • ”My. Proxy” effort • Continued work on authorization & delegation PDC Enabling Science