Скачать презентацию GRIDCC A real-time interactive GRID to integrate instruments Скачать презентацию GRIDCC A real-time interactive GRID to integrate instruments

83a6113648acbcebec0d9dc28523dc3c.ppt

  • Количество слайдов: 29

GRIDCC A real-time interactive GRID to integrate instruments, computational and information resources widely spread GRIDCC A real-time interactive GRID to integrate instruments, computational and information resources widely spread on a fast WAN Francesco Lelli, Eric Frizziero Istituto Nazionale di Fisica Nucleare Laboratori Nazionali di Legnaro Italy F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

Overview • The Grid. CC Project: Introduction • Grid Extension: the Instrument Element • Overview • The Grid. CC Project: Introduction • Grid Extension: the Instrument Element • Instrument Element prototype status • The Grid. CC Test-bed: Pilot application F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

The Grid Technologies to extend the limit of a single computer (center) Storage Element The Grid Technologies to extend the limit of a single computer (center) Storage Element Computing Element Grid Gateway Grid Technologie s Computing Element User Interface Computing Element F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

Extending the Grid Concepts Grid Gateway Terrestrial probes to monitor The volcano activities Grid Extending the Grid Concepts Grid Gateway Terrestrial probes to monitor The volcano activities Grid Technologie s Satellite views to monitor the volcano Control and Monitor Room F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005 To model calculations and disaster predictions

The Grid. CC Project ulations lc odel Ca ta for M Da ns Predictio The Grid. CC Project ulations lc odel Ca ta for M Da ns Predictio Instruments Grid + Computational Grid. CC Project F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

The GRIDCC project(II) Use of the Grid technology, as extension of the Web Service The GRIDCC project(II) Use of the Grid technology, as extension of the Web Service Technologies, to develop a a widley distributed control system with access to grid enabled computing and data storage facilities Virtual Cntr. Room Instrument 1 Supporting Services Instrument 2 Diagnostics Virtual Cntr. Room F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005 Instrument 3 Computing Element Storage Element

General on the Grid. CC Project • Funded by EU in the Frame Program General on the Grid. CC Project • Funded by EU in the Frame Program 6 • 10 Partners from 3 EU Countries + (Israel) • About 40 people engagged Country Istituto Nazionale di Fisica Nucleare Italy Institute Of Accelerating Systems and Applications Greece Brunel University UK Consorzio Interuniversitario per Telecomunicazioni Italy Sincrotrone Trieste S. C. P. A • It is a 3 years project. Started the 1 st September 04 Participant name Italy IBM (Haifa Research Lab) Israel Imperial College of Science, Technology & Medicine UK Istituto di Metodologie per l’Analisi ambientale – Consiglio Nazionale delle Ricerche Italy Universita degli Studi di Udine • www. gridcc. org F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005 Italy Greek Research and Technology Network S. A. Greece

Grid. CC Main Architecture Security. Aut. S Service Instrument Element TGS Pol. R Computing Grid. CC Main Architecture Security. Aut. S Service Instrument Element TGS Pol. R Computing Element Virtual Control Room Web Service Interface Storage Elements Element Exec. Wf. MS Service WMS Agr. S User direct Action Indirect Action F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005 Global Problem Solver Existing Grid Infrastructures

Instrument Element Architecture create() destroy() execute() get. State() Access Control Manager Virtual Instrument Grid Instrument Element Architecture create() destroy() execute() get. State() Access Control Manager Virtual Instrument Grid Service (VIGS) Instrument Element Inf & Mon Service Resource Service Instrument Manager Problem Solver • The term Instrument Element describes a set of services that provide the needed interface and implementation that enables the remote control and monitoring of physical instruments. Data Mover Data Flow State Flow Error Flow Monitor Flow Control Flow Data Collector IMS Proxy Control Manager Event Processor Input Manager Real Instruments F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005 FSM Engine Resource Proxy

Interacting with Instrument Elements y rit cu ce Se ervi S 1) Full Grid. Interacting with Instrument Elements y rit cu ce Se ervi S 1) Full Grid. CC Environment nt ent umeet rumenntt t ns e Instrumment Instr llemen E m I Ele E IMS Comp u CCmmuutiting o o p ptinng g E Eleme n Elelemnnt t me et GPS Sto rr El ltooar ge EEeem aage lem ee e m gn en tss nt t y rit cu ce Se ervi S 2) Partial Grid. CC Environment Ex Se ecuti rvi on ce VCR This mode of operation can be used when the application does not need to access CEs and SEs. It coud for instance exploit the workflow manager of the execution service to do unattended cycles of operations and control the system via VCR Ex Se ecuti rvi on ce Security Service R VC t en n umentt tr m e t Insrulm en t t u em Insr. Eem en t l Ins E m e El -I S C l. C ++ Pe su a Vi su al Ba sic ET W . N Vi F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005 JSP rl Ja va ++ t ent men umentntt tr e ns ue Instrumment Instr llemen E m I Ele E Comp u CCmmuutiting o o p ptinng g E Eleme n Elelemnnt t me et VCR 3) Standalone Environment IE is web service based, any web service compliant clients can reach it. This mode of working is very useful for small systems and for prototyping and debug large systems IMS GPS Sto rr El ltooar ge EEeem aage lem ee e m gn en tss nt t

Access Control Manager Instrument Element Implementations The IE components are typically implemented into a Access Control Manager Instrument Element Implementations The IE components are typically implemented into a fully equipped Machines (e. g. dual core cpus, large memory, large disks, etc). Instrument Element Resource Service Inf & Mon Service This is true for RS, IMS and PS. For IM (and DM) there are 2 possibilities, according to the application type: • IM implemented in a fully equipped machine • IM embedded into the instrument that should be controlled Problem Solver Data Mover Instrument Manager IMS RS IM Embedded Web Service IM IM IM F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005 A FPGA (Full Programmable Gate Array) Xlink Virtex II used to implement embedded IM. Status • PPC core used as reference • Linux porting on the core • JVM porting • Axis standalone porting • first preliminary tests on the remote invocation of the methods implemented into FPGA.

Partition/Configuration retrieve methods Partition and Lock setting methods Discovery Manager Subscribe Manager Configuration setting Partition/Configuration retrieve methods Partition and Lock setting methods Discovery Manager Subscribe Manager Configuration setting methods Discovery methods • • Partition&Lock Configuration Manager Available Resources Partition Definitions Manager Configuration Definitions RS Data Bases Resource Service Architecture The Resource Service (RS) handles all the resources of an IE and manages their partition (if any). A resource can be any hardware or software component involved in the IE (instruments, Instrument Managers, IMS components) RS stores the configuration data of the resources and download them to resource target when necessary Resources can be discovered, allocated and queried. It is the responsibility of the RS to check resource availability and contention with other active partitions when a resource is allocated for use. A periodic scan of the registered resources keeps the configuration database up to date. RS is interfaced to the WMS F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

 • Instrument Manager • Instruments Instrument Manager SUBSCRIBERS Errors Log info Monitor State • Instrument Manager • Instruments Instrument Manager SUBSCRIBERS Errors Log info Monitor State PUBLISHERS (Instruments nodes) Instruments Instrument Manager Information and Monitor System (IMS) The Information and Monitor Service (IMS) collects messages and monitor data coming from GRID resources and supporting services and stores them in a database. There are several types of messages collected from the sub-systems. The messages are catalogued according to their type, severity level and timestamp. Data can be provided in numeric formats, histograms, tables and other forms. The IMS collects and organizes the incoming information in a database and publishes it to subscribers. These subscribers can register for specific messages categorized by a number of selection criteria, such as timestamp, information source and severity level. F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

Instrument Manager Data Collector IMS Proxy Control Manager Event Processor FSM Engine Input Manager Instrument Manager Data Collector IMS Proxy Control Manager Event Processor FSM Engine Input Manager Resource Proxy GUI or other IMs Data Flow State Flow Error Flow Monitor Flow Control Flow Instrument Resources • IM is composed by 3 main components: • Data Collector. It get data from the controlled instruments and make them available to the data mover. A local storage of the data is even foreseen. • IMS Proxy. It receives error/state/log/monitor information from the controlled resources and forward them to IMS • Control Manager: • Input Manager. It handles all the input events of the IM. These inculdes commands from GUIs or other IMs, errors/state/log/monitor messages. • Event Processor. It handles all the incoming message and decide where to send them. It has processing capability • FSM. A finite state machine is implemented • Resource Proxy. It handles all the outgoing connections with the resources. F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

Data Mover Instrument Resources Web Service Interface: get_data() Data Mover SE interface (e. g. Data Mover Instrument Resources Web Service Interface: get_data() Data Mover SE interface (e. g. Grid. FTP) Http Server and TCP/IP raw socket Data Collector IM • • • IE The task of this element is to get data from the “data collector” of the IM and send them to: – Web service interface for generic data dump (e. g. slow storage, spy stream, etc. ) – a grid storage element (SE) for further data processing using available grid CEs. – Http server – Tcp/ip socket for fast data dump Data Mover exposes its methods to the IE web service interface Standard protocols and, possibly, standard tools will be used to implement it F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

DB IMS Performances Pub/Sub (JMS) TCP/IP Web Service Interface IMS Proxy Errors/log/states messages (xml DB IMS Performances Pub/Sub (JMS) TCP/IP Web Service Interface IMS Proxy Errors/log/states messages (xml and java objs) F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

Instrument Manager Performances (I) Data Collector IMS Proxy Control Manager Event Processor FSM Engine Instrument Manager Performances (I) Data Collector IMS Proxy Control Manager Event Processor FSM Engine Input Manager Resource Proxy Test 1 Test 2 Test 1: Web Service invocation and status switch of FSM Test 2: Soap Server receiving XML message format. DOM based parser F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

Instrument Manager Performances (II) 1+2 1 2 3 IM present prototype 1 3 Optimized Instrument Manager Performances (II) 1+2 1 2 3 IM present prototype 1 3 Optimized enivronment F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

Main Grid. CC Pilot Applications: Power Grid Instrument Manager Instrument Element F. Lelli E. Main Grid. CC Pilot Applications: Power Grid Instrument Manager Instrument Element F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

Main Grid. CC Pilot Applications: Remote Operation of an Accelerator Elettra Synchrotron F. Lelli Main Grid. CC Pilot Applications: Remote Operation of an Accelerator Elettra Synchrotron F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

Main Grid. CC Pilot Applications: Control and Monitor of high energy experiments F. Lelli Main Grid. CC Pilot Applications: Control and Monitor of high energy experiments F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

Main Grid. CC Pilot Applications: Control and Monitor of high energy experiments Elettra Synchrotron Main Grid. CC Pilot Applications: Control and Monitor of high energy experiments Elettra Synchrotron F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

The CMS Data Acquisition 2 107 electronics channels 40 MHz • O(104 ) distributed The CMS Data Acquisition 2 107 electronics channels 40 MHz • O(104 ) distributed Objects to – control – configure – monitor • On-line diagnostics and problem solving capability • Highly interactive system (human reaction time fraction of second) 100 Hz F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005 • World Wide distributed monitor and control

CMS Prototype F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005 CMS Prototype F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

Another Grid. CC applications: Migraine Attacks Treatments EEC 1. Data taking 2. Data Processing Another Grid. CC applications: Migraine Attacks Treatments EEC 1. Data taking 2. Data Processing GRID 3. Result Visualization and control 1 minute loop F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005 4. Action

The other Grid. CC pilot applications • Meteorology (Ensemble Limited Area Forecasting) • Device The other Grid. CC pilot applications • Meteorology (Ensemble Limited Area Forecasting) • Device Farm for the Support of Cooperative Distributed Measurements in Telecommunications and Networking Laboratories • Geo-hazards: Remote Operation of Geophysical Monitoring Network (see first slides) F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

Conclusion • The Grid. CC project is integrating instrument into traditional computational/storage Grids. • Conclusion • The Grid. CC project is integrating instrument into traditional computational/storage Grids. • IEs need an high interaction and interactivity between itself and the users. • Our IE prototype is currently installed in heterogeneous applications F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

Question? • Thx for your time More information: www. gridcc. org Acknowledgement: The Grid. Question? • Thx for your time More information: www. gridcc. org Acknowledgement: The Grid. CC project is supported under EU FP 6 contract 511382. F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005

Problem Solver Step 3 On-line information can be analyzed in order to detect possible Problem Solver Step 3 On-line information can be analyzed in order to detect possible malfunctions Pub/Sub (JMS) On Line Analisys Data Mining Tools Step 1 The control manager can perform an autonomous recovery action where the cost for the determination it is not so heavy. Instrument Manager IMS Proxy DB Algorithms evaluations : Rule Induction, Tree, Functions, Lazy, Clusters and Associative State Flow Error Flow Monitor Flow Instrument Manager IMS Proxy Control Manager Step 2 Persistent information can be analyzed in order to extract knowledge F. Lelli E. Frizziero, Tri. Grid Workshop, march 13, 2005 Instrument Manager IMS Proxy Control Manager