Review resources access policy procedures rules and challenges

Review resources access policy, procedures, rules and challenges: The Italian experience and future challenges Antonia Ghiselli INFN-CNAF Workshop on e. Infrastructures (Internet and Grids) The new foundation for knowledge-base Societies Roma, Accademia Nazionale dei Lincei 9 December 2003 – n° 1

Outline u Introduction: n u INFN-Grid and the national research grid n u INFN resource sharing experience in the past Goals and Results Italian-Grid present status n n u Resource access mechanism and management tools production service : Management, operations and support organization International Grid scenario: LCG and EGEE n n u Challenges: Multi-grids for multi-VOs Multi–grids : definitions and issues Conclusions – n° 2

INFN Computing Resource sharing in the past u 80 th user TRENTO MILANO u RJE to INFN resources by INFN users UDINE PADOVA TORINO PAVIA GENOVA TRIESTE PARMA BOLOGNA u Resource sharing within a single distributed community (agreement between sites based on common convenience ) user LNL FERRARA Network CNAF PISA FIRENZE S. Piero user PERUGIA LNGS ROMA 2 ROMA L’AQUILA user LNF u Access policy agreement: SASSARI n n NAPOLI low priority queues during the night Proxy logins mechanism user BARI SALERNO LECCE CAGLIARI COSENZA PALERMO CATANIA VAX/VMS cluster LNS – n° 3

INFN Computing Resource sharing in the past u 90 th : Condor – INFN collaboration user TRENTO MILANO UDINE PADOVA TORINO PAVIA u Condor submit to INFN desktops and workstations GENOVA TRIESTE PARMA BOLOGNA FERRARA Condor on WAN CNAF PISA u Users users FIRENZE S. Piero Resource sharing by INFN user PERUGIA LNGS policy agreement: transparent access through CPU cycle stealing ROMA 2 u Access ROMA L’AQUILA user LNF SASSARI NAPOLI BARI SALERNO u ~300 user LNL LECCE machines, still up. CAGLIARI COSENZA PALERMO CATANIA LNS – n° 4

INFN Computing Resource sharing in the past u 1999 MILANO u Globus user TRENTO evaluation on WAN u Preliminary grid tests to the INFN-Grid project. UDINE PADOVA TORINO PAVIA GENOVA user LNL TRIESTE PARMA BOLOGNA FERRARA user CNAF Globus test PISA FIRENZE S. Piero user PERUGIA LNGS ROMA 2 ROMA L’AQUILA user LNF SASSARI NAPOLI BARI SALERNO LECCE CAGLIARI COSENZA PALERMO CATANIA LNS – n° 5

INFN-Grid – goals (started at 2000) 1. To promote computational grid technologies research & development: Middleware 1. Through european and international projects 1. 2. FIRB: Grid. it To participate to the implementation of the global Grid infrastructure for the LHC community 1. 5. National layout: 20 sites To set up the national Grid Infrastructure for the national research community 1. 4. Internal R&D activities To implement the INFN grid infrastructure 1. 3. Data. Grid, Data. TAG, GLUE LCG: Tier 1 and n*Tier 2 To set up the e. Infrastructure for the European Research Area 1. EU FP 6: EGEE, IG-BIGEST – n° 6

INFN-Grid – collaborations and results u EU n - Datagrid : middleware development WMS = job submission to the Grid, s s n CE and SE selection on the basis of job requirements specification, CPU load, CE-SE network conditions…. . Support for interactive jobs Job checkpointing Support for parallel jobs Virtual Organization authentication and authorization service: VOMS (VO Membership Service, EDG/EDT) u EU – Data. TAG : inter-grid Interoperability; EU-US collaboration within the GLUE framework n Grid Resources Information modeling: GLUE schema for Computing and Storage Element n n First World. Grid demo by nov. 2002 within IST 2002 and SC 2002 events n u Authorization/authentication service : VOMS-VOX integration (EDT-Fnal/CMS coll. ) Grid monitoring system based on GLUE schemas extension Italian Grid. it : Grid management and support infrastructure n First tools in production n R&D on Resource Utilization Policies – n° 7

Italian – Grid now (Site/resource map) INFN TRENTO MILANO UDINE PADOVA TORINO LNL PAVIA GENOVA PARMA BOLOGNA TRIESTE FERRARA National Grid (Internet) CNAF PISA FIRENZE S. Piero PERUGIA LNGS ROMA 2 ROMA L’AQUILA LNF SASSARI NAPOLI BARI SALERNO CAGLIARI COSENZA PALERMO CATANIA LECCE CMS T 2/3 Atlas T 2/3 Alice T 2/3 LHCb T 2/3 Babar VIRGO T 2 (50 -80 nodes) T 3 (10 -15 nodes) T 1 Cnaf (~200) grid. it resources INFN (15 -25 nodes) INAF (5 -10 nodes) INGV (NEC computers), BIO (tbd) general purpose resources (8 -15 nodes) LNS Tot. ~ 600 nodes , next year ~ 1000 – n° 8

Resource access policies: Basic grid Authorization, authentication mechanisms Security characteristics: u Login u via X. 509 certificates from PKI/Certificate Authorities (CA) Single sign-on. n The user is not required to repeat login procedures on the grid more than once. u Delegation. n Once a user has successfully identified himself with the Grid, it is possible for grid services to act on the behalf of the user as if they were the user himself. u User-based n All trust mechanism have the user’s credential at their core. s If a user wants to access farms A and B, there should be no need for farms A and B to trust each other. u Integrated n trust relationship. with local systems. The grid security mechanism does not supplant the local authorization mechanism, but instead work on top of it. u New membership concept: user belongs to a Virtual Organization – n° 9

User: CA, VO and Resource Providers u CAs: Policies u CERN Grant authorization at the VO level. n u Each VO has its own VOMS server. n and procedures mutual thrust CA’s Certificates are issued by a set of well-defined Certification Authorities (CAs). Contains (group / role / capabilities) triples for each member of the VO. RP’s evaluate authorization granted by VO to a user and map into local credentials to access resources CESNET CNRS German. Grid-Ireland INFN cert-request Request NIKHEF Nordu. Grid LIP Authentication cert signing Russian Data. Grid VOMS pseudo -cert US–DOE Root CA agreement (map into Local credential) cert/crl update C=IT/O=INFN VOMS pseudo /L=CNAF -cert /CN=Pinco Palla /CN=proxy US-DOE Sub CA Cross. Grid (administer user membership, roles and Capabilities) Resource provider DATAGRID-ES Grid. PP VO-Manager Service – n° 10

Resource access policies u Authentication/ authorization: coded and tested procedures and tools u New n n issue : resource sharing according to Service Level Agreement first trials based on “grid level priority queues” ongoing research on more sophisticated mechanisms based on accounting + resource utilization Policies management VO-users (Requirements Support) Resource providers / AA/SLA Grid release VO-managers (VOMS and SLA Control) Grid management organization Grid operations / support Certificate Authorities Grid deployment planning – n° 11

Italian Grid organization : integrates all the actors to provide flexible and efficient grid computing service Experiments (VOs) GRID resources Projects/owners Grid Resource Coordination Committee Service level Agreement Resource availability Shared resources Management coordination VO representatives, Grid technical coord. , Operations resp. grid experts • Deployment Planning • resource Policy application • ……. Grid Technical coordination release Configuration management Central management Team Operations coordination Grid. Service support Experimemt or research org. support VO User Support support for New VO-users VO admin Site-man Resource admin VO admin New VO admin & support User Application Release distribution, documentation and porting – n° 12

Tools for Operations u Software repository : release maintenance and distribution u Installation n Configuration and automatic installation tools for the production infrastructure sites u Release n validation: Integration/customization of middleware release with application specific software u GRID n and configuration: Site and GRID service validation Testing programs to verify and validate site and services installation u Site manager support u Grid services, VO services support and User support u Monitoring: s s s Grid. ICE Based on automatic resource discovery from Grid Information System Dynamic monitoring of Grid services, Grid resources and Jobs Customized view for Grid Operation Center operators, and site managers VO-managers and Grid Users n n – n° 13

0 perations Portal u u u User documentation site managers documentation Software repository Monitoring Trouble tickets system Knowledge base http: //grid-it. cnaf. infn. it – n° 14

Get your personal certificate – n° 15

How to register to a VO – n° 16

Monitoring tool – n° 17

Grid services User Interface Grid Monitoring (Grid. ICE) VO server ingv Resource Broker BDII Information Index INFN-Padova INGV-Bologna GIIS GRIS RLS VO server atlas GRAM Computing Element GRIS 1 Storage Element GIIS GRAM Computing Element GRIS 1 Storage Element Worker. Node . . . – n° 18

Grid Service monitoring – n° 19

Outline u Introduction: n u INFN-Grid and the national research grid n u INFN resource sharing experience in the past Goals and Results Italian-Grid present status n n u Resource access mechanism and management tools production service : Management, operations and support organization International Grid scenario: LCG and EGEE n n u Challenges: Multi-grids for multi-VOs Multi-grids: definitions and issues Conclusions – n° 20

International Grids scenario u LCG : First international experience on sharing resources between national grids n Grid Resource sharing issues : s how to guarantee the committed CPU power and satisfy local needs s How to guarantee priorities on VO-owned resources n Different needs for different VOs (HEP experiments plans) n Management coordination n Support coordination u EGEE : project based on national grids interconnection for an increased number of VOs n n Not only middleware but mainly policies, service level agreement and management coordination issues Need to find a model …. . – n° 21

Grid access challenge: Grid and Virtual Organisations u. The real problem at the basis of the grid idea is how to implement a coordinated resource sharing on a large scale for a multi-institutional and dynamic virtual organisation. - u. From computer sharing to grid sharing u. From multiple users to multiple VOs (INFN experiments + others research organizations) – n° 22

Challenges: Capability to provide multi-Grid computing service to Multi-VO General scenario VO services and private resources Shared Resources and Services VO services and private resources VO services – n° 23

VO-Virtual Grid on top of Multi-Grids u International community VO is a multi-institutional distributed user u Etherogeneous grid environment n Dedicated VO services n Dedicated resources n Shared resources with different policies VO-User VO - Virtual Grid VO-User RB RB VO-User VO-monitoring VOMS Vo-RLS Coordinated Vo-support National and International Grids Italian-Grid EGEE same middleware shared resources US-Grid same core services – n° 24

multi - grids : definitions and issues u National n grid identity and authority boundaries A coordinated set of shared resources and services providing defined SLAs. n A single management and operations organization n Specific authorization, accounting and monitoring tools n A collection of user communities (VOs) u Federation n Cooperating grids to provide services to the common VOs? s n of grids, what does’t mean? Which level of transparency to VO-users? Which Interoperability Requirements: s s common or interoperable collective services? (level of service interoperability) s n common core services? Common Resource sharing policies? What level of management/operations/support coordinations? – n° 25

Conclusions u Production also: n grid does not mean only efficient, stable services but A topology/organizational model capable to provide the most flexible and efficient computing service to VO-users across multiple grids n Sufficient level of service quality (SLA) n Operations and support coordination n the minimum level of interoperability in order to allow VO virtual grid configuration across multiple grids – n° 26