Enabling Grids for E-scienc. E Accounting Portal Pablo Rey, Javier Lopez (CESGA) Cristina Del Cano, John Gordon (RAL) ARM-11 Lyon www. eu-egee. org EGEE-II INFSO-RI-031688 EGEE and g. Lite are registered trademarks
Overview Enabling Grids for E-scienc. E • APEL recent developments • Tier 2 Accounting and Reporting • Status of sites • Accounting by User. DN and FQAN Views – – User View Site Admin View VO Manager View VO Member View • APEL SAM Tests • Future Work • Demo EGEE-II INFSO-RI-031688 ARM-11 Lyon 2
APEL recent developments Enabling Grids for E-scienc. E • User. DN encryption • FQAN Publishing • Using blah log – should be compatible with CREAM • Multi-CE support • SAM Tests • YAIM to handle User. DN publishing EGEE-II INFSO-RI-031688 ARM-11 Lyon 3
User Level Accounting Enabling Grids for E-scienc. E • User Level Accounting Delivered – User. DN captured from CE log files (grid-jobmap logs) – APEL uses the data to build accounting records – Data published to GOC with on-the-fly encryption using APEL public key (1024 bit RSA) – At the GOC data are extracted from RGMA and stored in a Central Accounting Repository. – Data decrypted using APEL private key User Level summary table created On-the-fly encryption using EGEE Portal certificate – Encrypted table pushed to CESGA portal – Portal decrypts data and provides SSL based access to the summaries. EGEE-II INFSO-RI-031688 ARM-11 Lyon 4
VOMS Groups and Roles Enabling Grids for E-scienc. E • User. FQAN – Capture User. FQAN from grid-jobmap log on CE – FQAN chain processed at the GOC to derive Group and Role from the primary part of the chain. – If User. FQAN present, we can use the Group to derive the VO of the user submitted job (otherwise we use the local unix group). EGEE-II INFSO-RI-031688 ARM-11 Lyon 5
Status of sites (I) Enabling Grids for E-scienc. E CERTIFIED sites NOT publishing accounting data to GOC in the last 3 months ROC Asia. Pacific Sites INDIACMS-TIFR, KR-KISTIGCRT-01 Central. Europe PEARL-AMU* CERN MCGILL-LCG 2*, SDU-LCG 2*, UFRJ-IF*, Yer. Ph. I* Northern. Europe IMCSUL, LSG-LUMC*, Ui. B*, VDU-IF-LCG 2 * Sites not publishing at all EGEE-II INFSO-RI-031688 * ARM-11 Lyon 6
Status of sites (II) Enabling Grids for E-scienc. E UNREGISTERED sitenames Sitename CE ROC EGEE-SCIER ctb 31. gridctb. uoa. gr SEE GR-03 -HEPNTUA-TEST 1 ce 05. hep. ntua. gr SEE gri 47 xl. to. infn. it Italy HEPHY-UIBK-TESTBED test-lcg. CE. uibk. ac. at CE ICEPP_TESTBED 1 gridtb 02. icepp. jp Asia. Pacific IL-BGU-TEST cd-grid 1. bgu. ac. il SEE INFN-PADOVA-SLC 4 prod-ce-01. pd. infn. it Italy INFN-TORINO-DEV ce-test-20. to. infn. it Italy ITEP-SL 4 ceglite. itep. ru Russia IU_ATLAS_Tier 2 aviss. avidd. iu. edu OSG JSTest. Bed bf 32. hep. man. ac. uk UKI KTU-ELEN-TEST pupa. elen. ktu. lt NE PHI-AGRID deimos. ehv. campus. philips. com ? ? PIC-SA 3 vce 01. pic. es SWE TUD-ZIH gridopt 1. zih. tu-dresden. de CE TW-TEST ece Asia. Pacific UTA_SWT 2 Unknown OSG Could be deleted the data of these sitenames or could you provide us a GOC sitename? EGEE-II INFSO-RI-031688 ARM-11 Lyon 7
Status of sites (III) Enabling Grids for E-scienc. E EGEE-II INFSO-RI-031688 ARM-11 Lyon 8
Status of sites (IV) Enabling Grids for E-scienc. E Italian Sites start to publish User. DN EGEE-II INFSO-RI-031688 ARM-11 Lyon 9
Status of sites (V) Enabling Grids for E-scienc. E In SWE federation we have published almost all the records with the User. DN information since the beginning of EGEE-II INFSO-RI-031688 ARM-11 Lyon 10
Status of sites (VI) Enabling Grids for E-scienc. E LCG-CE with Patch 898? ? EGEE-II INFSO-RI-031688 ARM-11 Lyon 11
Status of sites (VII) Enabling Grids for E-scienc. E EGEE-II INFSO-RI-031688 ARM-11 Lyon 12
Status of sites (VIII): Special cases Enabling Grids for E-scienc. E • OSG and Nordu. Grid sites don’t use APEL to publish the accounting data so they don’t publish User. DN and FQAN information. • NIKHEF publishing their own encrypted User. DN strings – Example LCGUser. ID: HPfh 56 sbc 3 AYKDn 1 Yusxgg – Can only attribute usage to the VO • INFN use the DGAS sensor and then publish into the APEL Portal – Others could do the same. • Will share current plans with OSG and Nordu. Grid. EGEE-II INFSO-RI-031688 ARM-11 Lyon 13
Why sites don’t publish User. DN? Enabling Grids for E-scienc. E • To publish the User. DN, sites have to set the publish. Global. User. Name option to “yes”. • Some sites may wish to suppress the DN for reasons of personal privacy. Once they have a Policy, WLCG may mandate its sites to publish • By default, the DN is suppressed from publication. • Addition of new variable in YAIM (testing in PPS): APEL_PUBLISH_USER_DN. If it is set to “yes”, it will enable User. DN encryption. The default is “no”. • Normally, sites don’t change the default values, so if we want to obtain the User. DN information the default value should be set to “yes”. EGEE-II INFSO-RI-031688 ARM-11 Lyon 14
Why sites don’t publish FQAN? Enabling Grids for E-scienc. E • APEL misconfiguration: Sites use the deprecated option Gk. Log. Processor instead of the Blahd. Log. Processor option. – LCG-CEs that implement the Accounting Log File (Savannah Patch #898) no longer need to process the Gate. Keeper and Messages Logs. – The GK/Msgs log functionality is kept in order to maintain backwards compatibility. • There are sites that don’t have the Accounting Log Files (/opt/edg/var/gatekeeper/grid-jobmap_YYYYMMDD) in the CE. WHY? EGEE-II INFSO-RI-031688 ARM-11 Lyon 15
Accounting Portal: Views Enabling Grids for E-scienc. E • Apart of the Global View and the Reports tab, 4 news views using the User. DN and FQAN information are in development: – User View. – Site Admin View. – VO Manager View. – VO Member View. EGEE-II INFSO-RI-031688 ARM-11 Lyon 16
User View Enabling Grids for E-scienc. E • Statistics of usage for all jobs belonging to the User. DN (CPU, WCT, distribution of usage between ROCs and sites, . . . ) • What happens if the User changes their User. DN? How does the User access their data if they no longer have the old certificate? Do we need a mechanism to track the User. DN history? • We could associate the old certificates with the new certificate. Example: – New certificate: /DC=es/DC=irisgrid/O=cesga/CN=Pablo-Rey – Old certificates: /C=ES/O=DATAGRID-ES/O=CESGA/CN=Pablo Rey Mayo cert 001 EGEE-II INFSO-RI-031688 ARM-11 Lyon 17
Site Admin View Enabling Grids for E-scienc. E • This view let a Site Administrator to access to statistics of usage in its sites: Usage for Top 10 Users (Anonomised User. DN), area of pie shows the Total Usage by the SITE and the contribution of each of the Top 10 Users and Others, average Wall Clock Time (WCT) for all jobs belonging to each User, . . . • The list of site administrators is taken from the GOCDB Portal. EGEE-II INFSO-RI-031688 ARM-11 Lyon 18
VO Manager View Enabling Grids for E-scienc. E • Each VO Manager could access to statistics of usage in its VO: Usage for Top 10 Users (Anonomised User. DN), area of pie shows the Total Usage by the VO and the contribution of each of the Top 10 Users and Others, average Wall Clock Time (WCT) for all jobs belonging to each User, . . . • The list of VO managers is taken from the VO Identity Cards of the CIC Portal. The Managers and Deputies are taken. • APG requested CIC a new field for resource manager in the VO Card. EGEE-II INFSO-RI-031688 ARM-11 Lyon 19
VO Member View Enabling Grids for E-scienc. E • This View will allow to have an overview of the status of the VO at a lower level of detail than the VO Manager View (grouping by VOMS roles and groups). • The list of VO members is taken from the VOMS Servers registered in the VO Identity Cards of the CIC Portal. • We use a script (voms 2 users) based on the voms 2 gacl script wrote by Steve Traylen. EGEE-II INFSO-RI-031688 ARM-11 Lyon 20
Accounting Enforcement Task Enabling Grids for E-scienc. E • COD (or at least some ROC Managers) have opened GGUS tickets (30257, 31272, 31279, 31280, …) to sites that was not publishing accounting data, so: – This task has been stopped: Tickets not opened since the end of last year. – Should the tickets that are still opened (4) be closed? . • APEL SAM tests available EGEE-II INFSO-RI-031688 ARM-11 Lyon 21
Future work Enabling Grids for E-scienc. E • APEL – Accounting of local work (non-grid) – MPI jobs – Alternative transport layer • Portal – VO-based structures – eg ATLAS Tier 2 Cloud EGEE-II INFSO-RI-031688 ARM-11 Lyon 22
Thanks Enabling Grids for E-scienc. E APEL-SUPPORT@JISCMAIL. AC. UK egee-admin@cesga. es EGEE-II INFSO-RI-031688 ARM-11 Lyon 23
Скачать презентацию Enabling Grids for E-scienc E Accounting Portal Pablo
79a85209759d0c901014555df9781b0a.ppt