Enabling Grids for E-scienc E Simply monitor a

Enabling Grids for E-scienc. E Simply monitor a grid site with Nagios J. Casey, CERN E. Imamagic, SRCE ISGC 2008 www. eu-egee. org EGEE-II INFSO-RI-031688 EGEE and g. Lite are registered trademarks

Overview Enabling Grids for E-scienc. E • • Nagios-based grid monitoring Site monitoring prototype Demo Current status Future work Conclusions EGEE-II INFSO-RI-031688 ISGC 2008 / Simply monitor a grid site with Nagios 2

Nagios Enabling Grids for E-scienc. E • Open source monitoring framework – widely used & actively developed • Host and service problems detection and recovery • Provides wide set of basic sensors – easy to develop custom sensors • Centralized vs. distributed deployment • High configurability – service dependencies, fine-grained notification options • Web interface – status view, administration EGEE-II INFSO-RI-031688 ISGC 2008 / Simply monitor a grid site with Nagios 3

Nagios-based Grid Monitoring Enabling Grids for E-scienc. E • Monitoring CRO-GRID Infrastructure (2004 -2006) – Globus Toolkit Pre-WS & WS, UNICORE, other services – active recovery of services – http: //www. cro-ngi. hr • Monitoring EGEE resources in Central Europe (CE) – core services since mid 2006 – all CE sites for 1 st line support since September 2006 – http: //nagios. ce-egee. org • Grid Services Monitoring (GSM) WG – site monitoring prototype, mid 2007 – http: //crnjak. srce. hr/nagios (egee. srce. hr) – https: //pps-monitoring. cern. ch/nagios (CERN-PPS) EGEE-II INFSO-RI-031688 ISGC 2008 / Simply monitor a grid site with Nagios 4

Site Monitoring Prototype Enabling Grids for E-scienc. E Site admins Get remote results Get site’s & nodes information Issue alarms Get Nagios results Get site status Get VOMS proxy Refresh proxy Probe descriptions Monitoring server My. Proxy Live node checks … Get nodes information Service checks Site nodes … Site BDII EGEE-II INFSO-RI-031688 CE SE LFC ISGC 2008 / Simply monitor a grid site with Nagios 5

Grid Probes Enabling Grids for E-scienc. E • Provided by SRCE, CERN, OSG • Security facilities & services – CA distribution, Certificate lifetime, My. Proxy • Monitoring & information services – R-GMA, BDII, MDS, Grid. ICE • Job management services – Globus Gatekeeper, RB, WMS, WMProxy, Job matching • File management services – Grid. FTP, SRM, DPNS, LFC, FTS EGEE-II INFSO-RI-031688 ISGC 2008 / Simply monitor a grid site with Nagios 6

Standard Components Enabling Grids for E-scienc. E • Specifications defined by GSM WG • Probe wrapper – enables integration of standardized probes – Grid Monitoring Probes Specification – https: //twiki. cern. ch/twiki/bin/view/LCG/Grid. Monitoring. Probe. Spec ification • Publisher & remote gatherers – integration with other tools – Grid Monitoring Data Exchange Standard – https: //twiki. cern. ch/twiki/bin/view/LCG/Grid. Monitoring. Data. Excha nge. Standard EGEE-II INFSO-RI-031688 ISGC 2008 / Simply monitor a grid site with Nagios 7

Nagios Config Generator Enabling Grids for E-scienc. E • Uses multiple information sources – SAM, BDII, active heuristic checks • Modular approach – plugging in additional information sources – integration with other monitoring systems (e. g. LEMON) • User-defined rules – configuration tuning for non-standard grid sites • Standalone configuration – integration with existing Nagios server EGEE-II INFSO-RI-031688 ISGC 2008 / Simply monitor a grid site with Nagios 8

Remote g. Lite UI Enabling Grids for E-scienc. E • Avoid installation of grid middleware on Nagios server – execute grid probes on existing g. Lite UI – use Nagios Remote Plugin Executor (NRPE) Service checks Site nodes … Site BDII EGEE-II INFSO-RI-031688 CE SE LFC ISGC 2008 / Simply monitor a grid site with Nagios 9

Enabling Grids for E-scienc. E EGEE-II INFSO-RI-031688 ISGC 2008 / Simply monitor a grid site with Nagios 10

Enabling Grids for E-scienc. E EGEE-II INFSO-RI-031688 ISGC 2008 / Simply monitor a grid site with Nagios 11

Enabling Grids for E-scienc. E EGEE-II INFSO-RI-031688 ISGC 2008 / Simply monitor a grid site with Nagios 12

Enabling Grids for E-scienc. E SAM Standard probes NPM EGEE-II INFSO-RI-031688 ISGC 2008 / Simply monitor a grid site with Nagios 13

Current Status Enabling Grids for E-scienc. E • Three sets of standard probes integrated – SRCE, CERN, OSG • Two external monitoring systems – SAM, ENOC Down. Collector • Several deployments – CERN-PPS, SRCE, NIKHEF, PIC, IN 2 P 3, Scot. Grid • RPMs in apt and yum repository • Installation and configuration manual • More info https: //twiki. cern. ch/twiki/bin/view/LCG/Grid. Service. Monitoring. Info EGEE-II INFSO-RI-031688 ISGC 2008 / Simply monitor a grid site with Nagios 14

Future Work Enabling Grids for E-scienc. E • NCG development – providing configuration for multiple sites (regional monitoring) – providing configuration for multiple VOs • Integration with global monitoring systems – Active. MQ messaging system – Operations Automation Team mandate • Enabling “on-host” check via NRPE – process, logs, ports, files, etc • Probe description & site topology databases definition EGEE-II INFSO-RI-031688 ISGC 2008 / Simply monitor a grid site with Nagios 15

Conclusions Enabling Grids for E-scienc. E • Nagios – highly configurable monitoring framework with notifications, service dependencies, … – widely used by site admins • Grid extensions – integration with existing infrastructure (user certificates, VOMS, GOCDB, SAM) – probes for key grid services • Implementation of GSM WG specifications – probe wrapper, publisher & remote gatherers – easy integration with existing probes and monitoring systems EGEE-II INFSO-RI-031688 ISGC 2008 / Simply monitor a grid site with Nagios 16

Enabling Grids for E-scienc. E Thank You! Questions? EGEE-II INFSO-RI-031688 ISGC 2008 / Simply monitor a grid site with Nagios 17