Collaborative environment and workflow decomposition for remote instrumentation

Collaborative environment and workflow decomposition for remote instrumentation Roberto Pugliese ELETTRA - Sincrotrone Trieste SCp. A On Behalf of the GRIDCC Collaboration Bo. F - Remote Instrumentation Services in Grid Environment (RISGE) OGF 20 - Manchester 7 -9 May 2007

Outline n The GRIDCC project introduction ¨ pilot applications ¨ n GRIDCC technologies IE ¨ VCR ¨ Workflows ¨ n Final considerations Roberto. Pugliese@elettra. trieste. it Bo. F RISGE - OGF 20

The GRIDCC Project n n n The GRIDCC - Grid Enabled Remote Instrumentation with Distributed Control and Computation (www. gridcc. org) It is a 3 -years EU FP 6 project started in September 2004 Define and implement the “Instrument Element” to allow a standard remote access to any type of sensors and instruments Tight integration between instruments and the e. Infrastructure Complex workflows integrating instruments, computational and storage resources Human Computer interaction via Virtual Control Room groupware tools that support distributed and heterogeneous teams of people in the collaborative access to the extended e. Infrastructure Roberto. Pugliese@elettra. trieste. it Bo. F RISGE - OGF 20

The GRIDCC partners Participant name Country Istituto Nazionale di Fisica Nucleare Italy Institute Of Accelerating Systems and Applications Greece Brunel University UK Consorzio Interuniversitario per Telecomunicazioni Italy Sincrotrone Trieste S. C. P. A Italy IBM (Haifa Research Lab) Israel Imperial College of Science, Technology & Medicine UK Istituto di Metodologie per l’Analisi ambientale – Consiglio Nazionale delle Ricerche Italy Universita degli Studi di Udine Italy Greek Research and Technology Network S. A. Greece Roberto. Pugliese@elettra. trieste. it Bo. F RISGE - OGF 20

GRIDCC Architecture Security Services Virtual Control Room Information and Monitoring Services (IMS) Problem Solver Collaborative Services (CS) Instrument elements Instrument (IE) Element (IE) Storage Element (SE) Compute element Computing element (CE) Element (CE) Web Service Interface WMS Workload Management System Wf. MS Work. Flow Management System AS Agreement Service Roberto. Pugliese@elettra. trieste. it Execution Wf. MS Services WMS AS Information System (BDII) Bo. F RISGE - OGF 20

GRIDCC Architecture Computational Grid Instruments Grid CE IE IE DATA Instrument Element Instrument Manager CE Information & Monitoring System SE Problem Solver VCR Workflow ES VCR Collaborative Environment Roberto. Pugliese@elettra. trieste. it Execution Services Bo. F RISGE - OGF 20

GRIDCC main target areas Remote process control Accelerator control (Tele-) Biomedicine Robotics Automotive Electronic microscopes (Large-scale) scientific experiments High energy particle physics (Radio-) Telescopes GRIDCC Middleware Widely Sparse Instrumentation Power Grids Monitoring of the territory Monitoring of the sea Geo-hazard prediction Distributed laboratories Transportation monitoring Sensor network Roberto. Pugliese@elettra. trieste. it Bo. F RISGE - OGF 20

Remote Operations of ELETTRA: a feedback via Grid IM Accelerator IM Position Monitors bi Remote Experts O ee S on ti th oo. E b I 2 D EM at he t xi e IM Correctors SE CE Operator Roberto. Pugliese@elettra. trieste. it Bo. F RISGE - OGF 20

I Instrument Element Requirements n n n Interactive access to allow control and monitoring Standard interface to the physical devices Fast publishing of the data acquired by instruments Fast information (logs, errors, etc. ) publishing to track the behaviour of instruments and possibly solve problems Quality of Service and Advanced Reservation Computing Grid integration Move data from instruments to the Grid storage resources (Storage Element) and viceversa ¨ Process data produced by instruments using Grid computing resources (Computing Element) ¨ Roberto. Pugliese@elettra. trieste. it Bo. F RISGE - OGF 20

The Instrument Element (IE) Qo. S Roberto. Pugliese@elettra. trieste. it I’m here Instrument Element SRM/GRIDFTP Discovery Grid Access Instrument Control Web Service Custom (JMS, . . . ) Data/Info Pubblishing Access to Instruments Custom + Plugins Advance Reservation Bo. F RISGE - OGF 20

Instrument Element: multichannel I/O n SOA accessible operations to control and monitor the instruments (via VIGS) execute a command ¨ get / set attributes / parameters ¨ Multi channel data production Data S ubsc s Status Parameters Grid Interaction VIGS Commands VIGS IMS SRM riber IMS VCR ES L St ogs ate , E s, rro Mo rs, nit ors IE SRM n Instruments Storage Element (SE) Virtual Instrument Grid Service Information and Monitor Service Storage Resource Management Roberto. Pugliese@elettra. trieste. it Bo. F RISGE - OGF 20

Instrument Element Architecture Problem Solver Instrument Element Instrument Manager SRM/SE Grid. FTP Data Publisher SE IMS Resource Service The term Instrument Element describes a set of services that enable the remote control and monitoring of instruments IMS Access Control Manager VIGS IMS Data Flow State Flow Error Flow Monitor Flow Control Flow Data Collector IMS Proxy Control Manager Instrument Protocols Control Manager Event Processor Input Manager FSM Engine Resource Proxy Real Instruments Roberto. Pugliese@elettra. trieste. it Bo. F RISGE - OGF 20

Instrument Discovery n n The discovery of the instruments or of the IEs is an issue when the number of elements is high We can have two use cases: ¨ Quasi static case n The number of IEs is well defined n In this case a register based discovery mechanism can be used. 2 th n GRIDCC tesbed is using the LCG BDII (Berkely Database oo Information Index ) based on LDAP b n GLUE schema n The information collected in the BDII follow a tio i ¨ Dynamic case ib x n The number of IEs can change verye quickly, they are very simple devices, often with poor hardware e th support n The discovery is just use to know which are the online IEs at n A new approach has been O developed based on Peer to Peer (P 2 P) M protocols E ee D S Roberto. Pugliese@elettra. trieste. it Bo. F RISGE - OGF 20

Instrument Element: Scalable on embedded systems Custom Board IE Xilink Virtex IV PPC 405 Standalone Axis FPGA Linux Custom Logic Java. VM Grid. CC IM Web Service Instrument Manager Custom Electronics 1 Gbps Ethernet Montavista www. montavista. com USER INTERFACE Jam. VM http: //jamvm. sourceforge. net Roberto. Pugliese@elettra. trieste. it Bo. F RISGE - OGF 20

The Virtual Control Room (VCR) n The VCR provides a collaborative environment where users: Meet and collaborate by means of groupware tools (e. g. chat, shared calendar, logbook, video conference tools) ¨ Search, discover and browse resources such as people, instruments and other e. Infrastructure resources (e. g. computing and storage) ¨ Transparently operate with remote instruments using the GRIDCC middleware Infrastructure ¨ Roberto. Pugliese@elettra. trieste. it Bo. F RISGE - OGF 20

Architecture of the VCR User interface Layer providing General and Application Specific tools Services Layer providing low-level functionalities needed by the VCR Web-based portal providing some basic functionalities and support for aggregating components Roberto. Pugliese@elettra. trieste. it Bo. F RISGE - OGF 20

VCR in action Roberto. Pugliese@elettra. trieste. it Bo. F RISGE - OGF 20

Grid in the palm of your hand Roberto. Pugliese@elettra. trieste. it Bo. F RISGE - OGF 20

Wf. MS Architecture Roberto. Pugliese@elettra. trieste. it Bo. F RISGE - OGF 20

Remote Operations Workflows: Orbit. FB Roberto. Pugliese@elettra. trieste. it Bo. F RISGE - OGF 20

Final considerations n n n n The GRIDCC project is integrating instruments into the “classic” computational/storage Grids. The Instrument Element allows: ¨ virtualisation of the real instruments ¨ support of a variety of instruments from high end to embedded devices The Virtual Control Room is a groupware tools that support distributed and heterogeneous teams of people in the collaborative access to the extended e. Infrastructure The Complex workflows integrating instruments, computational and storage resources can be executed Sustainability Production installations Support new user communities Roberto. Pugliese@elettra. trieste. it Bo. F RISGE - OGF 20