Globus Toolkit 4 Ian Foster Argonne National Laboratory

Globus Toolkit® 4 Ian Foster Argonne National Laboratory University of Chicago Univa Corporation

2 Credits l Globus Toolkit v 4 is the work of many talented Globus Alliance members, at u u USC Information Sciences Corporation u National Center for Supercomputing Applns u U. Edinburgh u Swedish PDC u Univa Corporation u l Argonne Natl. Lab & U. Chicago Other contributors at other institutions Supported by DOE, NSF, UK EPSRC, and other sources

3 On April 29, 2005 the Globus Alliance released the finest version of the Globus Toolkit to date! Don’t take our word for it! Read the UK e. Science Evaluation of GT 4 www. nesc. ac. uk/technical_papers/UKe. S-2005 -03. pdf (Reachable from www. globus. org, under “News”)

4 Overview l Background and Globus approach l Globus Toolkit: current capabilities l Future directions l Related tools

5 “… A new age has dawned in scientific and engineering research, pushed by continuing progress in computing, information, and communication technology, and pulled by the expanding complexity, scope and scale of today’s challenges. The capacity of this technology has crossed thresholds that now make possible a comprehensive cyberinfrastructure on which to build new types of scientific and engineering knowledge environments and organizations, and to pursue research in new ways and with increased efficacy…” National Science Foundation Blue Ribbon Advisory Panel, 2003

8 History In the early 90 s, I (Foster) and others (e. g. , Carl Kesselman, USC-ISI) enjoyed helping scientists apply distributed computing u u Opportunities seemed ripe for the picking Application of technology always uncovers new and interesting requirements u Science is cool u Big/innovative science is even cooler

9 History (continued) While helping to build/integrate a diverse range of applications, the same problems kept showing up over and over again u u Too many different security systems Too many different scheduling/execution mechanisms Too many different storage systems Too many different monitoring/status/event systems

10 What Kinds of Applications? l Computation intensive u u u l Interactive simulation (climate modeling) Large-scale simulation and analysis (galaxy formation, gravity waves, event simulation) Engineering (parameter studies, linked models) Data intensive u u l Experimental data analysis (e. g. , physics) Image & sensor analysis (astronomy, climate) Distributed collaboration u u Online instrumentation (microscopes, x-ray) Remote visualization (climate studies, biology) Engineering (large-scale structural testing)

11 Key Common Feature The size and/or complexity of the problem requires that people in several organizations collaborate and share computing resources, data, instruments

12 An Example Problem l The Large Hadron Collider (LHC) l Largest machine ever built by humans! l Located at CERN, Geneva Switzerland l Particle accelerator and collider with a circumference of 16. 8 miles l Scheduled to go into production in 2007

13 An Example Problem (continued) l Will generate 10 Petabytes (107 Gigabytes) of information per year l This information must be processed and stored somewhere l It is beyond the scope of a single institution to manage this problem

14 Virtual Organizations • Distributed resources and people • Linked by networks, crossing admin domains • Sharing resources, common goals • Dynamic R R R VO-A R VO-B

15 Virtual Organizations • Distributed resources and people • Linked by networks, crossing admin domains • Sharing resources, common goals • Dynamic • Fault tolerant R R R VO-A R VO-B

16 The Globus Approach

17 The Role of the Globus Toolkit l l A collection of solutions to problems that come up frequently when building collaborative distributed applications Heterogeneity u l A focus, in particular, on overcoming heterogeneity for application developers Standards We capitalize on and encourage use of existing standards (IETF, W 3 C, OASIS, GGF) u GT also includes reference implementations of new/proposed standards in these organizations u

18 Layers in the Grid

19 A Typical e. Science Use of Globus: Network for Earthquake Eng. Simulation Links instruments, data, computers, people

20 Without the Globus Toolkit Web Browser Application Developer 0 Grid Community 0 Data Viewer Tool Chat Tool Credential Repository Application services organize VOs & enable access to other services Camera Telepresence Monitor Camera C Collective services aggregate &/or virtualize resources Database service D Database service E Data Catalog Certificate authority Users work with client applications Compute Server Registration Service 12 Globus Toolkit Web Portal Compute Server B Simulation Tool 10 Off the Shelf A Database service Resources implement standard access & management interfaces

21 With the Globus Toolkit Globus Web Browser GRAM Simulation Tool Globus GRAM Globus Index Service CHEF Compute Server Camera Application Developer 2 Off the Shelf Data Viewer Tool 9 Globus Toolkit Grid Community 4 4 CHEF Chat Teamlet My. Proxy Telepresence Monitor Globus DAI Globus MCS/RLS Application services organize VOs & enable access to other services Globus DAI Globus Certificate Authority Users work with client applications Camera DAI Collective services aggregate &/or virtualize resources Database service Resources implement standard access & management interfaces

22 The Globus Toolkit: “Standard Plumbing” for the Grid l Not turnkey solutions, but building blocks & tools for application developers & system integrators u l Easier to reuse than to reinvent u l Some components (e. g. , file transfer) go farther than others (e. g. , remote job submission) toward enduser relevance Compatibility with other Grid systems comes for free Today the majority of the GT public interfaces are usable by application developers and system integrators u u Relatively few end-user interfaces In general, not intended for direct use by end users (scientists, engineers, marketing specialists)

23 The Application-Infrastructure Gap Dynamic and/or Distributed Applications Shared Distributed Infrastructure B A 1 1 9 9

24 Bridging the Gap: Grid Infrastructure Users l u u l Composition Service-oriented applications Wrap applications as services Compose applications into workflows Service-oriented Grid infrastructure u Provision physical resources to support application workloads Workflows Invocation Appln Service Provisioning

25 Grid Infrastructure l Distributed management u u Of software services u l Of physical resources Of communities and their policies Unified treatment u u u Build on Web services framework Use WS-RF, WS-Notification (or WS-Transfer/Man) to represent/access state Common management abstractions & interfaces

Globus is Open Source Grid Infrastructure l Implement key Web services standards u l Software for Grid infrastructure u u u l Service-enable new & existing resources E. g. , GRAM on computer, Grid. FTP on storage system, custom application services Uniform abstractions & mechanisms Tools to build applications that exploit Grid infrastructure u l State, notification, security, … Registries, security, data management, … Enabler of a rich tool & service ecosystem 26

27 An e. Business Use of Globus: SAP Demonstration @ Globus. World l 3 Globus-enabled applns: u u u l CRM: Internet Pricing Configurator (IPC) CRM: Workforce Management (WFM) Web Browsers / Batch Processes SCM: Advanced Planner & Optimizer (APO) Applications modified to: u u (typically several thousand requests) Adjust to varying demand & resources Use Globus to discover & provision resources Request: Price Query 1 2 IPC Server IPC Delegation of Dispatcher Request 2 IPC Response: Pricelist. Server Depending on: - Time - Discount - Number of Items -… 3 SAP AG R/3 Internet Pricing & Configurator (IPC)

28 Overview l Background and Globus approach l Globus Toolkit: current capabilities l Future directions l Related tools

29 The Globus Toolkit is a Collection of Components l A set of loosely-coupled components, with: u u Libraries u l Services and clients Development tools GT components are used to build Gridbased applications and services u l GT can be viewed as a Grid SDK GT components can be categorized across two different dimensions u By broad domain area u By protocol support

30 GT Domain Areas l Core runtime u l Security u l Provision, deploy, & manage services Data management u l Apply uniform policy across distinct systems Execution management u l Infrastructure for building new services Discover, transfer, & access large data Monitoring u Discover & monitor dynamic services

31 GT Protocols l l Web service protocols u WSDL, SOAP u WS Addressing, WSRF, WSN u WS Security, SAML, XACML u WS-Interoperability profile Non Web service protocols u Standards-based, such as Grid. FTP u Custom

32 “Stateless” vs. “Stateful” Services File. Transfer Service move l Client Without state, how does client: u u l move (A to B) Determine what happened (success/failure)? Find out how many files completed? Receive updates when interesting events arise? Terminate a request? Few useful services are truly “stateless”, but WS interfaces alone do not provide built-in support for state

33 File. Transfer. Service (without WSRF) File. Transfer Service move (A to B) : transfer. ID Client what. Happen state tell. Me. When cancel l Developer reinvents wheel for each new service u u u Custom management and identification of state: transfer. ID Custom operations to inspect state synchronously (what. Happen) and asynchronously (tell. Me. When) Custom lifetime operation (cancel)

34 WSRF in a Nutshell l l Service EPR EPR u Get. RP Get. Mult. RPs Resource Set. RP Query. RPs Service State representation u l State identification u l u l Set. Termination. Time Immediate. Destruction Notification Interfaces u u l Get. RP, Query. RPs, Get. Multiple. RPs, Set. RP Lifetime Interfaces u Destroy Endpoint Reference State Interfaces Subscribe Set. Term. Time Resource Property Subscribe Notify Service. Groups

35 File. Transfer. Service (w/ WSRF) File. Transfer. Service create. Resource Transfer Client get. RP RPs create. Resource (A to B) : EPR query. RPs destroy l Developer specifies custom method to create. Resource and leaves the rest to WSRF standards: u u u State exposed as Resource + Resource Properties and identified by Endpoint Reference (EPR) State inspected by standard interfaces (Get. RP, Query. RPs) Lifetime management by standard interfaces (Destroy)

Globus Toolkit version 2 (GT 2) 36 Web Services Components Pre-WS Authentication Authorization Grid. FTP Security Data Mgmt Grid Resource Monitoring Alloc. Mgmt & Discovery (GRAM) (MDS) Execution Mgmt Info Services C Common Libraries Common Runtime Non-WS Components

Globus Toolkit version 3 (GT 3) Community Data Access Authorization & Integration WS Authentication Authorization Pre-WS Authentication Authorization Reliable File Transfer Grid Resource Alloc. Mgmt (WS GRAM) Grid. FTP Grid Resource Monitoring Alloc. Mgmt & Discovery (GRAM) MDS 3 (MDS) Replica Location Security Data Mgmt Java WS Core C Common Libraries e. Xtensible IO (XIO) Execution Mgmt Info Services Common Runtime 37 Web Services Components Non-WS Components

Globus Toolkit version 4 (GT 4) Grid Telecontrol Protocol Community Scheduling Framework 38 Contrib/ Preview Deprecated Web. MDS Python WS Core Community Data Access Workspace Authorization & Integration Management Trigger C WS Core Reliable File Transfer Grid Resource Allocation & Management Index Java WS Core Pre-WS Authentication Authorization Grid. FTP Pre-WS Grid Resource Monitoring Alloc. & Mgmt & Discovery C Common Libraries Credential Mgmt Replica Location www. globus. org e. Xtensible IO (XIO) Security Data Mgmt Delegation Authentication Authorization Data Replication Core Execution Mgmt Info Services Common Runtime Web Services Components Non-WS Components

39 Globus Toolkit: Open Source Grid Infrastructure Data Replication Globus Toolkit v 4 www. globus. org Credential Mgmt Replica Location Grid Telecontrol Protocol Delegation Data Access & Integration Community Scheduling Framework Web. MDS Python Runtime Community Authorization Reliable File Transfer Workspace Management Trigger C Runtime Authentication Authorization Grid. FTP Grid Resource Allocation & Management Index Java Runtime Security Data Mgmt Execution Mgmt Info Services Common Runtime

40 4. 0 is not a typical “. 0” release, but the culmination of months of testing 3. 0. 2 3. 0. 1 3. 0. 0 3. 2. 1 3. 2. 0 3. 9. 0 3. 3. 0 4. 0. 1 3. 9. 2 3. 9. 1 3. 9. 3 CVS trunk Stable release branch Development release Stable release 3. 9. 4 4. 0. 0 3. 9. 5

41 GT 4 Components SERVER Your Python Client Your C Service py. Globus WS Core C WS Core Java Services in Apache Axis Python hosting, Plus GT Libraries and Handlers GT Libraries Pre-WS MDS Your Python Service Pre-WS GRAM X. 509 credentials = common authentication RLS Your Java Service GRAM RFT Delegation Index Trigger Archiver CAS OGSA-DAI GTCP Interoperable WS-I-compliant SOAP messaging Your C C Client Your Java Client Your Python Client My. Proxy Your C C Client Simple. CA Your Java Client Grid. FTP CLIENT C Services using GT Libraries and Handlers

42 Our Goals for GT 4 l Usability, reliability, scalability, … u u l Web service components have quality equal or superior to pre-WS components Documentation at acceptable quality level Consistency with latest standards (WS-*, WSRF, WS-N, etc. ) and Apache platform u u l WS-I Basic Profile compliant WS-I Basic Security Profile compliant New components, platforms, languages u And links to larger Globus ecosystem

43 Globus Toolkit: Open Source Grid Infrastructure Data Replication Globus Toolkit v 4 www. globus. org Credential Mgmt Replica Location Grid Telecontrol Protocol Delegation Data Access & Integration Community Scheduling Framework Web. MDS Python Runtime Community Authorization Reliable File Transfer Workspace Management Trigger C Runtime Authentication Authorization Grid. FTP Grid Resource Allocation & Management Index Java Runtime Security Data Mgmt Execution Mgmt Info Services Common Runtime

44 GT 4 Web Services Runtime l Supports both GT (GRAM, RFT, Delegation, etc. ) & user-developed services l Redesign to enhance scalability, modularity, performance, usability l Leverages existing WS standards u u l WS-I Basic Profile: WSDL, SOAP, etc. WS-Security, WS-Addressing Adds support for emerging WS standards u l WS-Resource Framework, WS-Notification Java, Python, & C hosting environments u Java is standard Apache

45 GT 4 WS Core in a Nutshell Service EPR EPR Get. RP Get. Mult. RPs Resource Set. RP Query. RPs Subscribe Set. Term. Time Destroy Implementation of WSRF: Resources, Endpoint. References, Resource. Properties Operation Providers: pre-build implementations of WSRF operations Notification implementation: Topics, Topic. Set, Embedded Notification Consumer service Implementations of Resources (Reflection. Resource, Persistent. Reflection. Resource) and Resource. Properties (Simple. Resource. Property, Reflection. Resource. Property)

47 GT 4 WS Core in a Nutshell Service Container Service Get. RP Get. Mult. RPs EPR Get. Mult. RPs Set. RP EPRResource EPR Set. RP EPRResource Query. RPs RPs Query. RPs Subscribe Set. Term. Time Resource. Home Destroy Service Container: host multiple services in container; one JVM process …more details: based on AXIS service container, processes SOAP messages, Resource. Context extension.

48 GT 4 WS Core in a Nutshell Service Container Service Get. RP Get. Mult. RPs EPR Get. Mult. RPs Set. RP EPRResource EPR Set. RP EPRResource Query. RPs RPs Query. RPs Subscribe Set. Term. Time Resource. Home Destroy PIP PDP Secure Communication: Transport, Message, Conversation (Transport demonstrates best performance) Configurable Security Policies: Policy Information Points (PIPs), Policy Decision Points (PDP) -- chained Example authorization PDPs: Grid. Map, SAML implementations, XACML policies

49 GT 4 WS Core in a Nutshell Service Container Service PIP Get. RP Get. Mult. RPs EPR Get. Mult. RPs Set. RP EPRResource EPR Set. RP EPRResource Query. RPs RPs Query. RPs Subscribe Set. Term. Time Resource. Home Destroy Work. Manager DB Conn Pool PDP JNDI Directory Work. Manager: “thread pool”, site independent “work” manager Apache Database Connection Pool library (JDBC “Data. Source” implementation) JNDI Directory: manages internal, shared objects (Resource. Homes, Work. Manager, Configuration objects, …)

50 GT 4 WS Core in a Nutshell Apache Tomcat Service Container Service PIP Get. RP Get. Mult. RPs EPR Get. Mult. RPs Set. RP EPRResource EPR Set. RP EPRResource Query. RPs RPs Query. RPs Subscribe Set. Term. Time Resource. Home Destroy Work. Manager DB Conn Pool PDP JNDI Directory Deploy Service Container “standalone” or within Apache Tomcat

51 GT 4 Web Services Runtime Custom Web Services Custom GT 4 WSRF Web Services WS-Addressing, WSRF, WS-Notification WSDL, SOAP, WS-Security Registry Administration GT 4 Container User Applications

52 Modeling State in Web Services Resource allocation Authentication & Authorization are applied to all requests Entity eate Stateful Cr Factory service s State Addres State inspection Lifetime mgmt Notifications Service requestor (e. g. , user application) Discovery Stateful Entities Register Stateful Entity Interactions standardized using WSDL and SOAP Registry

53 WSRF & WS-Notification l Naming and bindings (basis for virtualization) u l Every resource can be uniquely referenced, and has one or more associated services for interacting with it Lifecycle (basis for fault resilient state mgmt) u u l Resources created by services following factory pattern Resources destroyed immediately or scheduled Information model (basis for monitoring, discovery) u u Operations for querying and setting this info u l Resource properties associated with resources Asynchronous notification of changes to properties Service groups (basis for registries, collective svcs) u l Group membership rules & membership management Base Fault type

54 GT 4 -C py. Grid. Ware WSRF: : Lite WSRF. NET Languages supported WSRF/WSNs Compared (HPDC 2005) GT 4 -Java C Python Perl C#/C++/VBasic, etc. WS-Security password profile Yes No In progress Yes WS-Security X. 509 profile Yes In progress Yes WS-Secure. Conversation Yes No Yes TLS/SSL Yes Yes Yes Multiple Callout None Yes Not default Yes Yes Memory Footprint JVM + 10 M 22 KB 12 MB Depends Memory size per WS-Resource Depends on resource state 70 B Depends on resource state 0 (file/DB) or 10 B (process) Depends on resource state Unmodified hosting environment Yes No Yes (Apache) Yes Compliance with WS-I Basic Profile Yes Yes In progress Yes Compliance with WS-I Basic Security Profile Yes Yes No Yes Log 4 J Yes Yes WSE diagnostics WS-Resource. Lifetime Yes Yes Yes WS-Resource. Properties Yes Yes Yes WS-Service. Group Yes Yes Yes WS-Base. Faults Yes Yes Yes WS-Base. Notification Yes Consumer Yes No Yes WS-Brokered. Notification Partial No No No Yes WS-Topics Partial No Partial Authorization Persistence of WS-Resources Logging

55 Get. RP Test Distributed client and service on same LAN (times in milliseconds) 149. 67 No Security 25. 57 X 509 Signing HTTPS 181. 96 17. 1 140. 5 55. 6 81. 39 10. 05 8. 23 2. 34 GT GT py W W 4 4 Gri SR SR - J - C d. W F: F. av ar : Lite NET a e N/A 14. 8 11. 46 2. 85 12. 91 GT GT py W W Gr SR SR 4 4 Gri SR SR 4 4 - J - C d. W F: F. - J - C id. W F: F. : L NE av av ar ite T ar : Lite NET a a e e

GT 4 WS Core Performance 56 (1) Message-level security (times in milliseconds) GT 4 Java GT 4 C GT 4 Python WSRF. NET Get. RP 181. 96 14. 77 140. 50 81. 39 Set. RP 182. 04 14. 99 142. 21 82. 48 Create. R 188. 46 14. 98 132. 26 96. 22 Destroy. R 182. 03 15. 76 136. 12 86. 89 Notify 219. 51 N/A 244. 93 101. 57 (2) Transport-level security (times in milliseconds) GT 4 Java GT 4 C GT 4 Python WSRF. NET get. RP 11. 46 2. 85 149. 67 12. 91 set. RP 11. 47 2. 86 150. 79 12. 3 create. R 18. 00 2. 82 132. 60 20. 84 destroy. R 14. 92 2. 71 149. 21 16. 05 Notify 29. 26 9. 67 169. 07 45. 0 “WSRF/WSNs Compared, ” HPDC 2005.

57 Globus Toolkit: Open Source Grid Infrastructure Data Replication Globus Toolkit v 4 www. globus. org Credential Mgmt Replica Location Grid Telecontrol Protocol Delegation Data Access & Integration Community Scheduling Framework Web. MDS Python Runtime Community Authorization Reliable File Transfer Workspace Management Trigger C Runtime Authentication Authorization Grid. FTP Grid Resource Allocation & Management Index Java Runtime Security Data Mgmt Execution Mgmt Info Services Common Runtime

58 Globus Security l Control access to shared services u l Address autonomous management, e. g. , different policy in different work-groups Support multi-user collaborations u u l Federate through mutually trusted services Local policy authorities rule Allow users and application communities to set up dynamic trust domains u Personal/VO collection of resources working together based on trust of user/VO

59 Virtual Organization (VO) Concept l VO for each application or workload l Carve out and configure resources for a particular use and set of users

60 GT 4 Security Authz Callout: SAML, XACML SSL/WS-Security with Proxy Services (running Certificates on user’s behalf) Access Compute Center Rights CAS or VOMS issuing SAML or X. 509 ACs Users Rights Local policy on VO identity or attribute authority My. Proxy VO Rights’ KCA

61 GT 4 Security l Public-key-based authentication l Extensible authorization framework based on Web services standards u SAML-based authorization callout l u Integrated policy decision engine l l As specified in GGF OGSA-Authz WG XACML policy language, per-operation policies, pluggable Credential management service u My. Proxy (One time password support) l Community Authorization Service l Standalone delegation service

62 GT 4’s Use of Security Standards Supported, Fastest, but slow but insecure so default

63 GT-XACML Integration l e. Xtensible Access Control Markup Language u OASIS standard, open source implementations l XACML: sophisticated policy language l Globus Toolkit ships with XACML runtime u Included in every client and server built on GT u Turned-on through configuration l … that can be called transparently from runtime and/or explicitly from application … l … and we use the XACML-”model” for our Authz Processing Framework

64 GT Authorization Framework

65 Other Security Services Include … l My. Proxy u u Web portal integration u l Simplified credential management Single-sign-on support KCA & kx. 509 u l Simple. CA u l Bridging into/out-of Kerberos domains Online credential generation PERMIS u Authorization service callout

66 Globus Toolkit: Open Source Grid Infrastructure Data Replication Globus Toolkit v 4 www. globus. org Credential Mgmt Replica Location Grid Telecontrol Protocol Delegation Data Access & Integration Community Scheduling Framework Web. MDS Python Runtime Community Authorization Reliable File Transfer Workspace Management Trigger C Runtime Authentication Authorization Grid. FTP Grid Resource Allocation & Management Index Java Runtime Security Data Mgmt Execution Mgmt Info Services Common Runtime

67 GT 4 Data Management l Stage/move large data to/from nodes u u l Grid. FTP, Reliable File Transfer (RFT) Alone, and integrated with GRAM Locate data of interest u l Replicate data for performance/reliability u l Replica Location Service (RLS) Distributed Replication Service (DRS) Provide access to diverse data sources u u File systems, parallel file systems, hierarchical storage: Grid. FTP Databases: OGSA DAI

Grid. FTP in GT 4 l Disk-to-disk on Tera. Grid 100% Globus code u No licensing issues u Stable, extensible l IPv 6 Support l XIO for different transports l Striping multi-Gb/sec wide area transport u l 27 Gbit/s on 30 Gbit/s link Pluggable u Front-end: e. g. , future WS control channel u Back-end: e. g. , HPSS, cluster file systems u Transfer: e. g. , UDP, Net. BLT transport 68

69 Reliable File Transfer: Third Party Transfer l Fire-and-forget transfer l Web services interface l Many files & directories l Integrated failure recovery l Has transferred 900 K files RFT Client SOAP Messages RFT Service Grid. FTP Server Master DSI Protocol Interpreter Grid. FTP Server Data Channel IPC Link IPC Receiver Notifications (Optional) Protocol Interpreter Master DSI IPC Link Slave DSI Data Channel Slave DSI IPC Receiver

70 Replica Location Service l Identify location of files via logical to physical name map l Distributed indexing of names, fault tolerant update protocols l l GT 4 version scalable & stable Managing ~40 million files across ~10 sites Index Local Update Bloom DB send filter (secs) (bits) 10 K <1 2 1 M 2 24 10 M 5 M 7 175 50 M

Reliable Wide Area Data Replication 71 LIGO Gravitational Wave Observatory Birmingham • §Cardiff AEI/Golm Replicating >1 Terabyte/day to 8 sites >30 million replicas so far MTBF = 1 month www. globus. org/solutions

72 OGSA-DAI l Provide service-based access to structured data resources as part of Globus l Specify a selection of interfaces tailored to various styles of data access—starting with relational and XML

73 The OGSA-DAI Framework Application Client Toolkit OGSA-DAI service Engine SQLQuery read. File XPath XSLT GZip Grid. FTP Activities JDBC XMLDB File Data Resources SQL My. SQL DB 2 Server XIndice SWISS PROT Databases

74 Extensibility Example OGSA-DAI service Engine SQLQuery Multiple JDBC SQL GDS SQL JDBC My. SQL SQL JDBC

OGSA-DAI: A Framework for Building Applications l Supports data access, insert and update u u u l Supports data delivery u u l SOAP over HTTP FTP; Grid. FTP E-mail Inter-service Supports data transformation u u l Relational: My. SQL, Oracle, DB 2, SQL Server, Postgres XML: Xindice, e. Xist Files – CSV, Bin. X, EMBL, OMIM, SWISSPROT, … XSLT ZIP; GZIP Supports security u X. 509 certificate based security 75

76 OGSA-DAI: Other Features l A framework for building data clients u l A framework for developing functionality u u l Client toolkit library for application developers Extend existing activities, or implement your own Mix and match activities to provide functionality you need Highly extensible u u Customise our out-of-the-box product Provide your own services, client-side support, and data-related functionality

77 Globus Toolkit: Open Source Grid Infrastructure Data Replication Globus Toolkit v 4 www. globus. org Credential Mgmt Replica Location Grid Telecontrol Protocol Delegation Data Access & Integration Community Scheduling Framework Web. MDS Python Runtime Community Authorization Reliable File Transfer Workspace Management Trigger C Runtime Authentication Authorization Grid. FTP Grid Resource Allocation & Management Index Java Runtime Security Data Mgmt Execution Mgmt Info Services Common Runtime

78 Execution Management (GRAM) l Common WS interface to schedulers u l Unix, Condor, LSF, PBS, SGE, … More generally: interface for process execution management u u Stage data u Monitor & manage lifecycle u l Lay down execution environment Kill it, clean up A basis for application-driven provisioning

79 GT 4 WS GRAM l 2 nd-generation WS implementation optimized for performance, flexibility, stability, scalability l Streamlined critical path u l Flexible credential management u l Use only what you need Credential cache & delegation service Grid. FTP & RFT used for data operations u Data staging & streaming output u Eliminates redundant GASS code

80 GT 4 WS GRAM Architecture Service host(s) and compute element(s) Job tions func Delegate GT 4 Java Container GRAM services Delegation Transfer request RFT File Transfer SEG Compute element Local job control Deleg ate sudo Client Job events GRAM adapter Grid. FTP control Local scheduler User job FTP data Grid. FTP Remote storage element(s)

81 GT 4 WS GRAM Architecture Service host(s) and compute element(s) Job tions func Delegate GT 4 Java Container GRAM services Delegation Transfer request RFT File Transfer SEG Compute element Local job control Deleg ate sudo Client Job events GRAM adapter Grid. FTP control Local scheduler User job FTP data Delegated credential can be: Made available to the application Grid. FTP Remote storage element(s)

82 GT 4 WS GRAM Architecture Service host(s) and compute element(s) Job tions func Delegate GT 4 Java Container GRAM services Delegation Transfer request RFT File Transfer SEG Compute element Local job control Deleg ate sudo Client Job events GRAM adapter Grid. FTP control Local scheduler User job FTP data Delegated credential can be: Used to authenticate with RFT Grid. FTP Remote storage element(s)

83 GT 4 WS GRAM Architecture Service host(s) and compute element(s) Job tions func Delegate GT 4 Java Container GRAM services Delegation Transfer request RFT File Transfer SEG Compute element Local job control Deleg ate sudo Client Job events GRAM adapter Grid. FTP control Local scheduler User job FTP data Delegated credential can be: Used to authenticate with Grid. FTP Remote storage element(s)

84 WS GRAM Performance l Time to submit a basic GRAM job Pre-WS GRAM: < 1 second u WS GRAM: 2 seconds u l Concurrent jobs Pre-WS GRAM: 300 jobs u WS GRAM: 32, 000 jobs u l Various studies are underway to test latest software

85 GT 4 WS GRAM Performance Number of Client Threads (M) 2 4 8 16 32 64 128 1 7 15 29 57 80 69 69 70 2 15 29 58 79 74 70 70 64 4 29 58 78 77 68 69 52 69 8 59 77 77 72 65 27 69 16 77 77 75 64 27 50 32 76 75 68 64 67 64 75 73 70 66 65 128 Sustained Job Load Per Client Thread (N) 1 80 72 64 63 71 All numbers are simple jobs/minute, no delegation or staging

86 Workspace Service: The Hosted Activity Policy Client Allocate/provision Configure Initiate activity Monitor activity Control activity Interface Activity Environment Resource provider

87 Activities Can Be Nested Client Policy Client Environment Interface Resource provider

88 For Example … Deploy service Deploy container Deploy virtual machine Deploy hypervisor/OS Procure hardware JVM VM VM Hypervisor/OS Physical machine Provisioning, management, and monitoring at all levels

89 Dynamic Service Deployment Community A • Community scheduling logic • Data distribution • Community management • Science services • . . . … Community Z Requirements: • Community control • Persistence • Resource guarantees • Non- interference

90 Virtual Machine Costs Job in booted VM GRAM job in paused VM GRAM job

91 Virtual OSG Clusters OSG cluster Xen hypervisors Tera. Grid cluster

92 Globus Toolkit: Open Source Grid Infrastructure Data Replication Globus Toolkit v 4 www. globus. org Credential Mgmt Replica Location Grid Telecontrol Protocol Delegation Data Access & Integration Community Scheduling Framework Web. MDS Python Runtime Community Authorization Reliable File Transfer Workspace Management Trigger C Runtime Authentication Authorization Grid. FTP Grid Resource Allocation & Management Index Java Runtime Security Data Mgmt Execution Mgmt Info Services Common Runtime

93 Monitoring and Discovery l “Every service should be monitorable and discoverable using common mechanisms” u l WSRF/WSN provides those mechanisms A common aggregator framework for collecting information from services, thus: u u MDS-Trigger: perform action on condition u l MDS-Index: Xpath queries, with caching (MDS-Archiver: Xpath on historical data) Deep integration with Globus containers & services: every GT 4 service is discoverable u GRAM, RFT, Grid. FTP, CAS, …

GT 4 Monitoring & Discovery WS-Service. Group Clients (e. g. , Web. MDS) GT 4 Container Registration & WSRF/WSN Access GT 4 Container MDSIndex Automated registration in container GRAM 94 MDSIndex adapter GT 4 Cont. Custom protocols for non-WSRF entities MDSIndex Grid. FTP User RFT

95 Index Server Performance l As the MDS 4 Index grows, query rate and response time both slow, although sublinearly l Response time slows due to increasing data transfer size u u l Full Index is being returned Response is re-built for every query Real question – how much over simple WSN performance?

96 Information Providers l GT 4 information providers collect information from some system and make it accessible as WSRF resource properties l Growing number of information providers u u l Ganglia, Clu. Mon, Nagios SGE, LSF, Open. PBS, PBSPro, Torque Many opportunities to build additional ones u E. g. , network monitoring, storage systems, various sensors

97 GT 4 Summary SERVER Your Python Client Your C Service py. Globus WS Core C WS Core Java Services in Apache Axis Python hosting, Plus GT Libraries and Handlers GT Libraries Pre-WS MDS Your Python Service Pre-WS GRAM X. 509 credentials = common authentication RLS Your Java Service GRAM RFT Delegation Index Trigger Archiver CAS OGSA-DAI GTCP Interoperable WS-I-compliant SOAP messaging Your C C Client Your Java Client Your Python Client My. Proxy Your C C Client Simple. CA Your Java Client Grid. FTP CLIENT C Services using GT Libraries and Handlers

98 GT 4 Documentation is Much Improved!

99 Overview l Background and Globus approach l Globus Toolkit: current capabilities l Future directions l Related tools

100 The Globus Commitment to Open Source l Globus was first established as an open source project in 1996 l The Globus Toolkit is open source to: u allow for inspection l u encourage adoption l u in pursuit of ubiquity and interoperability encourage contributions l l for consideration in standardization processes harness the expertise of the community The Globus Toolkit is distributed under the (BSD-style) Apache License version 2

101 The Future: Structure l NSF Community Driven Improvement of Globus Software (CDIGS) project u u l 5 years of funding for GT enhancement Regular Globus roadmaps outlining plans Glob. Dev http: //dev. globus. org u Apache-like community development site u Community governance of components u “Globus Toolkit” & other related software u Open for business early 2006 u “Globus Alliance” = “Glob. Dev committers”

102 Glob. Dev l The current set of Globus components will be organized into several “Globus Projects” u l Each project will have its own group of “Committers” u l Projects release products committers are responsible for governance on matters relating to their products The “Globus Management Committee” will u u provide overall guidance and conflict resolution approve the creation of new Globus Projects

103 The Future: Content l We now have a solid and extremely powerful Web services base l Next, we will build an expanded open source Grid infrastructure u u Virtualization New services for provisioning, data management, security, VO management u u l End-user tools for application development Etc. , etc. And of course responding to user requests for other short-term needs

104 The Future l We now have a solid and extremely powerful Web services base l Next, we will build an expanded open source Grid infrastructure u u Virtualization New services for provisioning, data management, security, VO management u u l End-user tools for application development Etc. , etc. And of course responding to user requests for other short-term needs

105 Short-Term Priorities: Security l Improve GSI error reporting & diagnostics l Secure password, one-time password, Kerberos support for initial log on l Trust roots, use of Grid. Logon l Identity/attribute assertions in GT auth. callouts (e. g. , Shib, PERMIS, VOMS, SAML) l Extend CAS admin & policy support l Security logging with management control for audit purposes

106 Short-Term Priorities: Data Management l Space & bandwidth management in Grid. FTP l Concurrency in globus-url-copy l Priorities in RFT l Data replication service l Enhance policy support in data services l Physical file name creation service l Scalable & distributed metadata manager

107 Short-Term Priorities: Execution Management l Implement GGF JSDL once finalized l Advance reservation support l Policy-driven restart of “persistent” jobs l Improved information collection for jobs l Improved management of job collections l Credential refresh l Development of workspace service l Integration of virtual machines (Xen, VMware) and associated services l Windows port of WS GRAM

108 Short-Term Priorities: Information Services l Many more information sources, including gateways to other systems l Automated configuration of monitoring l Specialized monitoring displays l Performance optimization of registry l Archiver service l Helper tools to streamline integration of new information sources

109 Short-Term Priorities: WS Core l Streamlined container configuration l Remote management interface l Dynamic service deployment l Service isolation: multiple service instances l WS-Notification, subscription performance l Full functionality in C WS Core l Optimized WS-Service. Group support l WS-Secure. Conversation support

110 What to Expect from the Globus Alliance in the Coming Months l Support for users of GT 4 u Working to make sure the toolkit meets user needs u Answering questions on the mailing lists u Further improving documentation l Normal evolution of performance, scalability and feature enhancements l Further development of tools and services in support of VOs l Expanding contributions to Globus

111 Overview l Background and Globus approach l Globus Toolkit: current capabilities l Future directions l Related tools

112 The Globus Ecosystem l Globus components address core issues relating to resource access, monitoring, discovery, security, data movement, etc. u l A larger Globus ecosystem of open source and proprietary components provide complementary components u l GT 4 being the latest version A growing list of components These components can be combined to produce solutions to Grid problems u We’re building a list of such solutions

113 Many Tools Build on, or Can Contribute to, GT 4 -Based Grids l l l l l Condor-G, DAGman MPICH-G 2 GRMS Nimrod-G Ninf-G Open Grid Computing Env. Commodity Grid Toolkit Gri. Phy. N Virtual Data System Virtual Data Toolkit Grid. Xpert Synergy l l l Platform Globus Toolkit VOMS PERMIS GT 4 IDE Sun Grid Engine PBS scheduler LSF scheduler Grid. Bus Tera. Grid CTSS NEES IBM Grid Toolbox …

114 Documenting The Grid Ecosystem: Software Components for Grid Systems And Applications www. grids-center. org

115 Example Solutions l Portal-based User Reg. System (PURSE) l VO Management Registration Service l Service Monitoring Service l Tera. Grid TGCP Tool l Lightweight Data Replicator l Gri. Phy. N Virtual Data System

116 Condor-G l l The Condor Project @ U Wisconsin Madison develops software for high-throughput computing on collections of distributed compute resources Condor-G is an interface to GRAM created by the Condor team that allows users to submit jobs to GRAM servers

117 Grid. Shib l Allows the use of Shibboleth-transported attributes for authorization in GT 4 deployments u And, more generally, SAML support l 2 year project started December 1, 2004 l Participants u u Kate Keahey, UChicago/Argonne (PI) u Frank Siebenlist, Argonne u l Von Welch, UIUC/NCSA (PI) Tom Barton, UChicago Beta software released September 16, 2005

118 Handle System l The Handle System from CNRI (http: //www. handle. net) is a generalpurpose global name service enabling secure name resolution over the internet l The Handle System-GT Integration Project leverages the Handle System for identifier and resolution services through tight integration with GT 4’s Web services protocols

119 MPICH-G 2 l MPICH-G 2, developed at Northern Illinois University and Argonne National Lab, is a grid-enabled implementation of the MPI v 1. 1 standard l MPICH-G 2 is implemented using the pre-WS GRAM component in GT 4; integration with GT 4 WS GRAM is expected in the near future

120 Nimrod/G l Nimrod is a specialized parametric modeling system from Monash University l Nimrod/G uses a simple declarative parametric modeling language to express parameter sweep experiments. Based on GT 4 WS services, Nimrod/G enables the formulation, execution and monitoring of multiple individual parametric experiments

121 Ninf-G 4 l Ninf-G 4, from AIST, is a reference implementation of the GGF standard Grid. RPC API l Ninf-G 4 is provides higher-level programming APIs for the development and execution of parallel applications on the Grid

122 PERMIS l PERMIS is an EU-funded Privilege Management service that implements Role. Based Access Control l Thanks to the work of the UK Grid Engineering Task Force, services running in a Java WS Core container can use PERMIS via GT 4’s SAML authorization callouts

123 SRB l SRB is a package from SDSC providing a uniform interface for connecting to network -based heterogeneous data resources l GT 4’s Grid. FTP includes an interface to SRB data sources, and vice versa

124 Sun Grid Engine l Sun Grid Engine is an open source distributed resource management system from Sun Microsystems l In a collaboration between the London e. Science Centre, Gridwise and MCNC, the Sun Grid Engine has been integrated with GT 4

125 Tells Us About Your Grid Tools & Solutions l We list links to related projects on the “Related Software” of the Globus Toolkit web www. globus. org/toolkit/tools/ l “Solutions” are documented on the Globus web www. globus. org/solutions/ l If we’ve got details wrong or you have a GT 4 -related tool to list on our website, please send mail to [email protected]. org

126 Questions?