Cross Grid Task 3 3 Grid Monitoring Trinity

Cross. Grid Task 3. 3 Grid Monitoring Trinity College Dublin (TCD, AC 14 – CR 11) Brian Coghlan, Stuart Kenny, David O’Callaghan CYFRONET Academic Computer Center, Krakow (CYFRO, C 01) Bartosz Balis, Slawomir Zielinski, Kazimieriz Balos ICM, University of Warsaw (ICM, AC 2 – C 01) Krysztof Nawrocki, Adam Padee Poznan July-2003 Cross. Grid Task 3. 3

Cross. Grid Task 3. 3 Grid Monitoring Provides monitoring information from four main sources: - Applications (OCM – G) - gathers performance data from an executing application - used by application developers in order to understand an application’s behavior and improve its performance. - Infrastructure (JIMS) - gather and expose information concerning the state of devices used to build a grid environment - notify the user not only about simple events, but derived ones as well, - take managerial actions in cases of failures. - Instruments(/Networks) (SANTA-G) - allow information captured by external monitoring instruments to be introduced into the Grid information system. - used in validation and calibration of both intrusive monitoring systems and systemic models, and also for performance analysis. Poznan July-2003 Cross. Grid Task 3. 3

Cross. Grid Task 3. 3 Grid Monitoring - Derived Results - gathering information from other monitoring tools and creation of one consistent user interface. - generation of forecasts of future grid state using Kalman Filters and neural networks. Poznan July-2003 Cross. Grid Task 3. 3

Grid Monitoring System Infrastructure monitoring R-GMA/OGSA info Application monitoring Poznan July-2003 Cross. Grid Task 3. 3

Task 3. 3. 1 OCM-G, Current State § § OCM-G integrated with GT. Secure communication based on globus_io between components (authentication, possibly encryption). Service Managers run on a "well known” port (3331, configurable). Configuration via local config files (user home dir or /opt/cg/etc) § Still one central Service Manager § Registration of application processes improved § § Support for user-defined events (probes) added. CVS status: • code up to date. • building with autobuild, on RH 6. 2. • need to make changes to comply with developers guide. • No longer need for shared fs ! • can handle multi-site applications unless firewalls block communication • Locks to get rid of race condition while forking LMs Poznan July-2003 Cross. Grid Task 3. 3

Task 3. 3. 1 OCM-G, Task Contacts § § Task 2. 4 - G-PM fully integrated with OCM-G in its current functionality. G-PM now needs user certificate to connect to the OCM-G. Poznan July-2003 Cross. Grid Task 3. 3

Task 3. 3. 1 OCM-G, Integration § § § Smooth integration with G-PM. Communication based on globus_io. No dependencies to other Globus / EDG components. Poznan July-2003 Cross. Grid Task 3. 3

Task 3. 3. 1 OCM-G, Problems and Issues § § Building under RH 7. 3 – problems with globus_io development package. Interface to Grid Benchmarks should be defined. Poznan July-2003 Cross. Grid Task 3. 3

Task 3. 3. 2 SANTA-G, Current State § § § Improve the schema of information available: - Done, still more to do Add more SQL parsing support: - Done, added more WHERE predicates - Supports =, > , < queries Add on-line data acquistion: - Sensor now starts/stops TCPdump at startup/shutdown - Allows querying of dynamically generated network traffic Integrate Sensor and Query. Engine components: - Sensor now contacts Query. Engine at startup - informs it when a new log file is generated, informs QE of shutdown Enhance Viewer functionality - Improved Viewer GUI. - Graphical packet display, displays timestamps in correct format, automatically resolves IP addresses… - Query Builder added to allow user to construct complex queries Poznan July-2003 Cross. Grid Task 3. 3

Task 3. 3. 2 SANTA-G, Task Contacts § § EDG WP 3 - SANTA-G makes use of the EDG R-GMA. - has also contributed to it, Canonical. Producer was an extension to the EDG R-GMA developed as part of Task 3. 3. 2. Task 3. 3. 3 JIMS - integration with this task has begun - work should be completed by the end of the summer (see next slide). Poznan July-2003 Cross. Grid Task 3. 3

Task 3. 3. 2 SANTA-G, Integration JMX Request JMX Client R-GMA SQL SANTA-G (R-GMA Producer) Poznan July-2003 Result. Set R-GMA Consumer API Cross. Grid Task 3. 3 JMX

Task 3. 3. 2 SANTA-G, Integration JMX Request JIMS (MBean Server) R-GMA Producer API R-GMA Producer Code JMX Poznan July-2003 Cross. Grid Task 3. 3 R-GMA SQL Result. Set

Task 3. 3. 2 SANTA-G, Problems and Issues § Need the most recent EDG R-GMA RPMS - Canonical Producer not in earlier release!! § R-GMA RPMs Redhat 7. 3 only! § Still To Do: - Expand schema of available information - Improve SQL support - Complete SANTA-G/JMX integration - Testing - Investigate security Poznan July-2003 Cross. Grid Task 3. 3

Task 3. 3. 3 JIMS, Current State § § JIRO-based Infrastructure Monitoring System – JIMS - ported from JDMK to pure JMX reference implementation • host monitoring module, ready. • SNMP is in progress SOAP Gateway for integration with other CG tasks • exposes Web Services based interface • makes integration with OGSA (Open Grid Services Architecture) easier: ØWeb Services Gateway module Øsimple SOAP client for testing purposes Poznan July-2003 Cross. Grid Task 3. 3

JIMS, SOAP Gateway architecture Poznan July-2003 Cross. Grid Task 3. 3

JIMS, SOAP Gateway Facilities § § Web Services Gateway serves as a mediator between MBean Servers in monitored stations and external applications Place for registering active monitored stations and removing non -existent ones Poznan July-2003 Cross. Grid Task 3. 3

JIMS, Test SOAP Client Interface Poznan July-2003 Cross. Grid Task 3. 3

Task 3. 3. 3 JIMS, Problems and Issues What is done: • Host monitoring system - JIMS - ready • SOAP Gateway - before deadline • Open (not commercial) implementation of discovery services - before deadline To do: • integration with CVS and autobuild process, by the end of this week • Simplifying installation process • Adding functionality: Øother mechanisms for monitored stations unregistering Øsecurity when connecting modules via Web Services (SOAP/XML) Poznan July-2003 Cross. Grid Task 3. 3

Task 3. 3. 4 Post. Processing, Current State § § § Forecaster based on linear Kalman filter implemented and available as RPM. More work needed to put it in CVS, will be done during the meeting. Current solution for real monitoring data from clusters is VOCentric Ganglia Poznan July-2003 Cross. Grid Task 3. 3

Task 3. 3. 4 Post. Processing, Integration § For integration meeting will provide 2 RPM’s: • ganglia-monitor-core-mcastmin-2. 4. 1 -1. i 386. rpm Øserves as monitoring daemon on worker nodes • gmmetad-2. 2 -1. i 386. rpm located on cluster CE. Gathers information from monitoring daemons and passes it to central monitoring host ØRPM slightly altered wrt to original Ø § Would like to install these on X# clusters for testing during integration meeting. Poznan July-2003 Cross. Grid Task 3. 3

Task 3. 3. 4 Post. Processing, Problems and Issues § § 3 rd part which binds forecaster and data sources under development Not ready for integration meeting. Poznan July-2003 Cross. Grid Task 3. 3