Nin Jo Production System Event Driven Architecture in

Nin. Jo Production System Event Driven Architecture in Nin. Jo June 2008

Outlook: Production System and EDA n SOA – a definition n EDA - Event driven architecture n Nin. Jo Production System n Nin. Jo Event Services and Production Agents

SOA n Service orientation is an architectural concept that refers to the loose coupling of a service (an abstract resource with a defined job) and its provider (the physical asset(s) that perform the job tasks). n A requestor only knows what the service’s job is and how to request it. n The service is the only one that knows its implementation. n SOA is an IT architecture strategy for business solution (and infrastructure solution) delivery based on the concept of service orientation.

SOA n A composite application typically serves one business domain. n Composite applications are often delivered in a portal. n common SOA implementations are e. g. BEA weblogic application servers, combined with J 2 EE (JAVA Enterprises) components n sometimes they call it also “enterprises” architecture and mean “SOA”, implemented with WEB services. n Very often there is a database behind the webservice (“persistent objects”) n Services can as well communicate via CORBA

SOA n Most of the service invocations are synchronous in nature. 4 so the caller waits for completion 4 With a many-to-many communication between several services which we need to control the -> Nin. Jo Production System this would block the data flow n Synchronous service call would be NOT sufficient for Nin. Jo Production System n We need EDA (Event Driven Architecture)

Outlook: Production System and EDA n SOA – a definition n EDA - Event driven architecture n Nin. Jo Production System n Nin. Jo Event Services and Production Agents

Terminology - Event n WHAT IS AN EVENT? 4 An event is a notable thing that happens inside or outside your business. 4 An event (business or system) may signify a problem or impending problem, an opportunity, a threshold, or a deviation. 4 Each event occurrence has an event header and event body 4 Event header 4 contains elements describing the event occurrence, such as the event specification ID, event type, event name, event timestamp, event occurrence number, and event creator. 4 These elements are consistent, across event specifications. 4 Event Body. The event body describes what happened. 4 must be fully described so any interested party can use the information without having to go back to the source system.

Event Driven Architecture : EDA n “Event” = change of a system state, which can be send as a message n Extremely decoupled, event producers do not know consumers 4 event generating processes, 4 event storage and transmission = event channel, 4 event processing processes n Messaging middleware: good scalability and distribution n Event processors subscribe for certain event types n Event generators and processors can be called “agents” 4 It can be services, human, automated processes, sensors, . .

Event Driven Architecture definition n An event (see definition on slide 7) happens n It will be disseminated immediately to all interested parties (human or automated). n The interested parties evaluate the event, and optionally take action. n The event-driven action may include the invocation of a service, the triggering of a business process, and/or further information publication/syndication. n the whole service invocation and data/message flow becomes asynchronous and more complex! n SOA + Event Driven Architecture = SOA 2. 0

EDA + SOA + X (CORBA) + Where goes Nin. Jo? ? n The most viable, agile architectures will be comprised of a blend of architecture strategies, 4 including service-oriented architecture, 4 event-driven architecture, 4 process-based architecture, 4 federated information, 4 enterprise integration and 4 open source adoption. n The best choices are the ones that match your business! n So let us analyse the Nin. Jo Production System use cases…

Outlook: Production System and EDA n SOA – a definition n EDA - Event driven architecture n Nin. Jo Production System n Nin. Jo Event Services and Production Agents

Nin. Jo workstation system n We have a Nin. Jo workstation system, consisting of: 4 The client 4 Data servers and import agents 4 Automon monitoring (and event service) 4 We have Nin. Jo Batch production (is triggered by a web service) 4 We have “Science modules” (producing data and products) 4 We are developing Auto. WARN (automated generation of warn status) n We need more types of science modules n We need intelligent triggering of products

Communication flow n All these processes communicate with each other in certain ways 4 Most 1 dimensional: Import->Server->client 42 way communication: client<->event service<->Automon 4 Import->notification->Science module->data server 4 Import-> notification- >Automon->Event-service->several clients n Problems: 4 Rather static dependencies of processes 4 Restart of processes in certain start order 4 Multiple connections or information chains of multiple science modules not available 4 Data / product availability should be signaled to the client or trigger batch production

Science Service Coupling n What does a Nin. Jo Science Module do? n The Science Module computes new data products given specific input data and stores it on an appropriate data server, example mesocyclon cells, point forecasts n There should be a server side system of communicating (or chained) science modules

Science Service Coupling n Now we started thinking about a “production system”, 4 The science modules should be triggered by incoming data and/ or certain rules 4 The clients, other science modules and also batch should be informed, if a product is available 4 Technically we end up in the definition of a couple of services and processes which communicate with each other – in an asynchronous, decoupled way 4 We also considered a common “Service” interface for all processes – so Nin. Jo goes SOA ? (!) n What we need is: . . n …. The Nin. Jo Production System

Nin. Jo production system n Characterization : n the Nin. Jo production system is not an independent and self-contained component n it is rather a collection of well coordinated components n with the task of automated product creation, in particular 4 graphical products (batch) 4 data products (science modules) 4 Text products (warnings, reports) n coordination of the involved components is based on events sent between them

Information Flow n the extension of the Nin. Jo server-side to a more powerful production system goes hand in hand with a significant increase of Nin. Jo inter-process communication n control is provided by events n The loose coupling via events is the most powerful and flexible way of controlling complete production chains, distributed over several processes, server hardware and via LAN/WAN n Nin. Jo will go EDA !

Outlook: Production System and EDA n SOA – a definition n EDA - Event driven architecture n Nin. Jo Production System n Nin. Jo Event Services and Production Agents

Event Service (or channel) n Event Service The Event Service is the component that centrally collects all new issued events and redistributes them to all interested parties. n The event service stores events in a database and delivers them to all registered clients as soon as they are active (online) n Used already in Auto. Mon, Auto. WARN n Clients or server side processes can register to certain event types n Event types can be e. g. 4 Automon warnings (when a predefined warning rule has fired) 4 incoming data notification 4 production trigger events (from data monitor : „model run available“)

Event Service (or channel) n responsibilities 4 registration and deregistration of Event Consumers 4 reception of new events issued by Event Producers 4 distribution of events to Event Consumer that subscribed to that specific event type 4 persistent storage of processed events 4 management of stored events (e. g. cleanup of outdated events) 4 response to queries regarding event history 4 publication of implemented interfaces in the Nin. Jo Naming Service (NNS) 4 synchronization with optionally running redundant Event Services

Production Agents n Every connected server side component, like import, data or monitoring services, science module, batch, . . 4 Can be an Event Generating Service (EGS) 4 Or an Event Consuming Service (ECS) 4 Or both n We call the involved components, which issue events or consume events: “production agents”

Production Agents n Import System The Import System ingests new data to the data servers and thereby issues 'new data' events to the Event Service.

Production Agents n Data Monitor 4 the Data Monitor filters single 'new data' events to an aggregated event that can be used instead. 4 Can be used to find out, if “enough” data is available for a certain product n Scheduler The Scheduler creates time-based events.

Production Agents n Science Service The Science Service computes new data products given specific input data and stores it on an appropriate data server. 4 It can be triggered by ‘new data’ events and produce ‘new data product’ events n Science Service responsibilities 4 registration and deregistration with Event Service 4 reception of events delivered by the Event Service 4 access of required input data from a data server 4 computation of the science data, e. g. mesocyclon cells 4 storage of the result data on a data server 4 submission of a 'new data' event to the Event Service

Use Case 1: Science Service n all connections shown in the diagrams are performed by means of a middleware system, in our case by means of CORBA. n the Data Monitor and Scheduler are unified in one component

Production Agents n Batch The Batch system creates graphical Nin. Jo products and provides them to the interested clients. 4 Is triggered by scheduler events or by Product. On. Demand - requests via the Batch-web service 4 Can be triggered by “new data” events 4 Can be triggered by “new product” events 4 Can produce “new data/product” events

Use Case 2: Batch Production

Production Agents n Auto. MON monitors weather events by means of current observations and different NWP models. n In case a rule is fulfilled an event is generated.

Use Case 3: Auto. MON

Outlook n New communication framework under development n Event based communication additional to data based communication n Central event service is there, new event types will be defined n Step by step development of Nin. Jo Production System by 4 Adding new communication functionality to existing and upcoming Science/production agents 4 Developing additional infrastructure services like Data. Monitor 4 Coupling (loose, by events) of all concerned services, servers and agents 4 Benefit of OO and modular architecture: reusability of all existing software frameworks and components 4(e. g. rather small changes in client components)

Outlook 2 n Time frame ? n This will be a big architectural extension n We will only start with small steps and need to prove the concepts before we take final decisions n Good news: we can add new components to Nin. Jo easily n Since still research necessary, the complete solution of that automated production will take years before it is operational 4 But partial solutions and infrastructure components might be usable much earlier