EGI-In SPIRE Migration of g Lite services to

EGI-In. SPIRE Migration of g. Lite services to the GLUE 2. 0 schema Stephen Burke (RAL) EGI CF 2012, Munich EGI-In. SPIRE RI-261323 www. egi. eu

Overview • Schema migration plan • Implementation and deployment – BDII infrastructure – Service publication – Clients • Outlook • NB Only discussing g. Lite services here • No comments on storage services – due to lack of knowledge! GLUE 2. 0 migration - EGI CF Munich EGI-In. SPIRE RI-261323 2 www. egi. eu

Migration plan Schema migration is a complex process: 1) Define the abstract schema 2) Define the LDAP rendering 3) Implement the schema in the BDII and roll out 4) Write and deploy information providers • You are here! 5) Update client tools to understand GLUE 2 6) ((Retire GLUE 1)) • The schema interacts with everything, so the rollout must be a gradual process without breaking anything GLUE 2. 0 migration - EGI CF Munich EGI-In. SPIRE RI-261323 3 www. egi. eu

BDII implementation • Merged LDAP schema, GLUE 1. 3 + GLUE 2 • Single LDAP server, on port 2170 as usual • Separate root DNs – o=glue vs o=grid – Should be no crosstalk other than data volume • Resource BDII: GLUE 2 Group. ID=resource, o=glue • Site BDII: GLUE 2 Domain. ID=, o=glue • Top BDII: GLUE 2 Domain. ID=, GLUE 2 Group. ID=grid, o=glue • Rolled out as part of g. Lite 3. 2 – Resource BDII in production since September 2009 – Site BDII in production since August 2010 – Top-level BDII in production since October 2010 • EMI BDII is basically the same • Some sites are still on g. Lite 3. 1! GLUE 2. 0 migration - EGI CF Munich EGI-In. SPIRE RI-261323 4 www. egi. eu

Information providers • Generic service publisher to publish the GLUE 1 Glue. Service object in production for several years – FTS and LFC have their own dedicated providers • Upgraded to publish the GLUE 2 Service, Endpoint and Access. Policy – Backward compatible with the GLUE 1 publisher – Supports all relevant GLUE 2 attributes, and Services with multiple Endpoints • Progressively rolled out as new versions of services are released – Already have WMS, LB, My. Proxy, bdii_site, bdii_top, msg. broker. *, VOMS and VOBOX in production – In EMI 2 for AMGA, Argus – In work for Hydra, Nagios, Frontier/squid, … • Easy to add publication for any service (APEL? ) • FTS and LFC have upgraded their own providers GLUE 2. 0 migration - EGI CF Munich EGI-In. SPIRE RI-261323 5 www. egi. eu

CREAM • For GLUE 1 we introduced the glite-CLUSTER node type to allow the Glue. Cluster and Glue. Subcluster objects to be published from a different node – Supports sites with multiple CEs submitting to the same cluster – Also publishes the Glue. Service object for the “RTEPublisher” – a Grid. FTP server used to allow VOs to publish Run. Time. Environment tags in the Sub. Cluster – No-CLUSTER mode continues to publish everything from the CREAM node for small sites • For GLUE 2, use the CLUSTER node to publish all objects except the CREAM and CEMon Endpoints (and associated Access. Policy) – Objects are merged in the site BDII – The detailed plan is described in a wiki page: https: //wiki. italiangrid. it/twiki/bin/view/CREAM/Cream. Glue 2 – EMI 2 will have complete publication – Batch system integration for PBS, LSF and SGE is done GLUE 2. 0 migration - EGI CF Munich EGI-In. SPIRE RI-261323 6 www. egi. eu

Deployment status • 378 sites published in GLUE 1 in the CERN top BDII (as of 25/3/12) • 218 sites publishing a GLUE 2 Admin. Domain • Missing sites mainly still have a g. Lite 3. 1 site BDII – EGI now asking all sites to upgrade • Case-sensitivity in site name in the GOC DB – Sites we noticed were ticketed and fixed • May be other problems at some sites? • No explicit steps needed to configure for GLUE 2, just clones the GLUE 1 configuration – Sites may not have realised that they’re publishing! GLUE 2. 0 migration - EGI CF Munich EGI-In. SPIRE RI-261323 7 www. egi. eu

Published Endpoints 314 126 56 32 66 103 3 5 6 4 17 352 256 bdii_site bdii_top dcap emi. storm gsidcap gsiftp https lcg-file-catalog lcg-local-file-catalog My. Proxy org. glite. ce. CREAM org. glite. ce. Monitor CERN services currently missing - GGUS ticket submitted 3 3 3 74 9 31 3 76 1 1 66 34 53 org. glite. Channel. Management org. glite. Delegation org. glite. File. Transfer org. glite. lb. Server org. glite. RTEPublisher org. glite. voms-admin org. glite. wms. WMProxy org. nordugridftpjob org. nordugrid. ldapglue 1 org. nordugrid. ldapglue 2 org. nordugrid. ldapng SRM VOBOX xroot GLUE 2. 0 migration - EGI CF Munich EGI-In. SPIRE RI-261323 8 www. egi. eu

Clients • All clients need to become GLUE 2 -aware – Must be backward-compatible – Can happen gradually • WMS: JDL – in progress, first version in EMI 2 • Service discovery: lcg-info(sites), glite-sd-query – First version of OGF/SAGA service discovery tool available – No work on lcg-infosites? • Monitoring, resource accounting: gstat – in progress (Taiwan) • User tools - ? ? ? GLUE 2. 0 migration - EGI CF Munich EGI-In. SPIRE RI-261323 9 www. egi. eu

Summary • Define LDAP schema and deploy in BDIIs – 1. 3 and 2. 0 together in parallel – Now deployed in production • But sites are slow to upgrade! • Write and deploy information providers to populate the new objects – Generic Service publisher available • Being rolled out progressively – Computing. Service publication (for CREAM) developed incrementally • Full version with EMI 2 • Including support for main batch systems • Update clients to look at the new information – Workload management, data management, service discovery, monitoring, accounting, user, … – Upgrades need to be backward-compatible • Need to start validating the published information • Eventually want to make GLUE 2 the default – Maybe start in 2013? ? ? – GLUE 1 still available as a fallback GLUE 2. 0 migration - EGI CF Munich EGI-In. SPIRE RI-261323 10 www. egi. eu

References • OGF GLUE working group home page – http: //forge. ogf. org/sf/projects/glue-wg • GLUE 2. 0 specification – http: //www. ogf. org/documents/GFD. 147. pdf • LDAP rendering specification (draft) – http: //forge. ogf. org/sf/go/doc 15518? nav=1 GLUE 2. 0 migration - EGI CF Munich EGI-In. SPIRE RI-261323 11 www. egi. eu