JRA 1 Meeting 28 -30 Jun 2004 www

JRA 1 Meeting, 28 -30 Jun 2004 www. eu-egee. org Grid Access Service Predrag Buncic EGEE is a project funded by the European Union under contract IST-2003 -508833

Introduction • JRA 1 Clusters § Integration § Testing § Information and Monitoring § Data Management § Workload Management § Security § Etc… JRA 1 Meeting, 28 -30 June 2004 - 2

Current g. Lite Prototype • A Prototype Middleware on a testbed consists of § § Ali. En “shell” Job submission: • • § Data Management • • • § VOMS for certificate handling/SE gridmap files (NIKHEF) My. Proxy for certificate delegation in GAS (Grid Access Service) • § Ali. En File & Metadata catalog Ali. En SE with Castor & D-Cache SE with SRM grid. FTP for transfers Ali. En FTD Aiod/GFal RLS (EDG) Security • • § Alien CE->Condor-G->blaph->PBS/Condor Globus Gatekeeper Prototype with a few file cataloging functions R-GMA & EDG WMS (not integrated yet) Extra Terrestrial Cluster [looking for help!] JRA 1 Meeting, 28 -30 June 2004 - 3

Talk Outline 1) API and Grid Access Service (GAS) § § Why and how? Service Controller (or Controller Service)? 2) GAS & Prototype (=>Pablo) 3) Package Manager § Why and how? JRA 1 Meeting, 28 -30 June 2004 - 4

API and GAS JRA 1 Meeting, 28 -30 June 2004 - 5

Starting from Ali. En… JRA 1 Meeting, 28 -30 June 2004 - 6

Stepping stone: ARDA… JRA 1 Meeting, 28 -30 June 2004 - 7

Design team working document. . JRA 1 Meeting, 28 -30 June 2004 - 8

Design team working document. . JRA 1 Meeting, 28 -30 June 2004 - 9

DJRA 1. 1 JRA 1 Meeting, 28 -30 June 2004 - 10

Client (application) side: API An API would be a library of functions used for building client applications, graphical user interfaces or even Grid Web portals (e. g. Ali. En, Genius or Clarens). The API is used also to authenticate user to the Grid, let them submit jobs inquire job status and manage jobs access the files available on the Grid and put users files onto the Grid The application should be able to gain access files on the Grid by issuing requests to copy files to local temporary storage or via POSIX like interface to a near Storage Element. JRA 1 Meeting, 28 -30 June 2004 - 11

Server side: GAS • The Grid Access Service (GAS) is the counterpart of the user API on service side and represents the user entry point to a set of core services. Grid Applications Grid Service Components Interface specification § Many of the User Interface API functions are simply delegated to the methods of the GAS. In turn many of the GAS functions are delegated to the appropriate service. Messages Operations Construction specification Grid Service Component Library Composition Logic § GAS service will be constructed out of Service Components that will in turn present a uniform public interface to underlying service. § The components from which the interface is constructed can be defined by the VO preferences at run time Composition Type Message Dependency The Service Components are realized as a pluggable library with each component providing an interface to the specific middleware service. The intention was to define end user interface that allows them to interface their application to the Grid and keep this interface reasonably stable and protected from inevitable changes in the middleware. JRA 1 Meeting, 28 -30 June 2004 - 12

Grid Login Use Case As a first step in connecting to the Grid, the user application uses the API to connect to a Before attempting to connect to the Grid, a user configurable list his or her temporary is expected to registerof Configuration Services. These are the public independent credentials with VO services that can exist per VO or serve multiple Many They inquire. Interface API functions are VOs. of the User VO credential wallet (like the NERSC secure configuration and Information Servicesmethodsas the GAS. My. Proxy). simply delegated to the as well of use DNS information to deliverthe GAS functions are In turn many of initial configuration back to the appropriate service. delegated user API GAS service will be constructed out of Service Components that will in turn present a uniform public interface to underlying The application then connects to the GAS service. During its creation the user is authenticated Factory and various over operations are user and his rights for passes Grid the secure line name and password needed Service. The checked against the Authorization to get delegation GAS of user the user credentials and keeps credentials from the credential wallet. If authorization information. this operation is successful, the GAS Factory will start a GAS service for the user and return the service endpoint. The application then connects to its endpoint and gains access to other Grid service. JRA 1 Meeting, 28 -30 June 2004 - 13

Controller Service An instance of GAS should be created in a service environment in the proximity of the user (local site) where proper container has been located The GAS factory will ask Controller Service for appropriate service endpoint The Controller can decide to create local service or can contact another Controller GAS lifetime will be restricted to the lifetime of delegated proxy credentials and will be managed by the Controller Service and user who will be able to destroy his own GAS instance. JRA 1 Meeting, 28 -30 June 2004 - 14

GAS: Summary • The GAS model of accessing the Grid is in many ways (authentication, proxy service) similar to various Grid Portals but it is meant to be distributed (the GAS Factory can start GAS in a service environment close to the user in network terms) and is therefore more scalable and resilient. • As opposed to a traditional Web Portal, the GAS interface is more dynamics and reflects the role of the user in the system. • The GAS is intended to be used by the application and not by the interactive user therefore it offers no presentation layer. • The traditional Grid Portal can actually be easily constructed by specializing GAS into a Portal Service that will provide necessary content to a presentation layer provided by the Web Server. • Similarly, the specialized application services can be constructed by extending GAS or providing appropriate Service Components. • Grid services also have to be accessible directly using their respective mechanisms (i. e. not via the GAS). Package Manager Service JRA 1 Meeting, 28 -30 June 2004 - 15

Package Manager Service • A Package Management Service is a helper service that automates the process of installing, upgrading, configuring, and removing software packages from a shared area (software cache) on a grid site. • Explicitly requested by the users of a prototype • This service represents an extension of a traditional package managment systems to distributed computational environment and it should use one of the estabilished package management systems as a backend. • Some well-known examples of such systems include § § § RPM, Red Hat's package manager, used not only by Red Hat Linux but by several other Linux distributions. dpkg/APT (used originally by Debian GNU/Linux, now ported to other systems). Portage, used by Gentoo Linux and inspired by the BSDc ports system. The ``ports tree'' system used by Free. BS Net. BSD, Open. BSD and the like. Pacman, package manager developed at Boston Univerity and used by several Grid projects (International Virtual Data Toolkit - i. VDGL, Grid 3) JRA 1 Meeting, 28 -30 June 2004 - 16

Basic assumptions • The software is distributed in packages, usually encapsulated into a single file that contains metadata that describes the package's details, including its name, checksums, and dependencies on any other packages that it needs to work. • It may also include information on how to configure the package for use and how to remove the package cleanly when it is no longer required. • The package manager then uses this information to install, configure, and remove packages as requested by the user. • The PM Service operates in the context of a VO and understands and resolves possible dependencies between the package versions provided by the V. O. administrator. • This service is not responsible for the maintenance and deployment of middleware or system software components, unless the VO takes the responsibility to provide and maintain the middleware and/or system software as a part of the VO contributed software. JRA 1 Meeting, 28 -30 June 2004 - 17

Use case scenario • In a typical scenario, the VO package administrator creates the binary package caches for one or more computing platforms, verifies and possibly digitally signs them. These caches are then published and made available for download via the PM Service. • On the execution site, a local instance of PM Service will, on request from CE or JW, fetch and install binary packages into the local package cache. This local package cache should reside on a file system managed by the PM Service assuring that unused old packages are removed if disk space is needed to install newer versions. • The existence of binaries can be advertised, thus minimizing download of packages from multiple locations. In this way, the PM Service could maintain the hierarchy of package caches to assure scalability and provide a fail-over capability. • Access to VO packages should be controlled and possibly restricted and audited. • The easiest way to achieve that is to treat the packages as any other File Catalogue entry and to apply common Authentication, Authorization and Auditing mechanisms. The integrity of individual packages should be verified by appropriate checksums. • The package metadata information (including checksum information) should be retrieved from a trusted and certified VO site, independently from the package itself. JRA 1 Meeting, 28 -30 June 2004 - 18

Package Manager: Summary • Service urgently required by users • The software components needed for realization of such service exist • Possible implementation scenario § Reworked Ali. En Packman component exposed as a service using one (or more) of the package managers as a backend § Try to extract minimal package manager interface to allow alternative package manager backends § Personal preference: start with Portage • Some prototyping needed JRA 1 Meeting, 28 -30 June 2004 - 19

Issues JRA 1 Meeting, 28 -30 June 2004 - 20

Issues § Configuration Service • • § Discover VO services Bootstrap client application Alternative transport and messaging protocols • • SOAP over IM protocol (Jabber) No need for incoming IP connectivity Service presence information as bonus Scalable asynchronous system logging (syslog) JRA 1 Meeting, 28 -30 June 2004 - 21