Introduction to the new mainframe Chapter 11 b

Introduction to the new mainframe Chapter 11 b objectives Be able to: describe the message broker fulfills the ESB requirements: 1. 2. 3. 4. MATCHES & ROUTES communications between services CONVERTS between different transport protocols TRANSFORMS between different data formats IDENTIFIES & DISTRIBUTES business events © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Key terms in this chapter Message Nodes Pub/sub Message

Introduction to the new mainframe An Example A Fred Smith, Graphics Card, 32, 1. 50, 07/11/06 [Customer, Order, Quantity, Price, Date] B C <order> <name>Mr. Smith</name> <item>Graphics Card</item> <quantity>32</quantity> <price>1. 05</price> <date>11/07/06</date> </order> © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe The Complexity of Application Integration A E B D

Introduction to the new mainframe Concepts Broker = “container” for message flows Message flow = deployable unit of mediation, definition of ‘one-way’ mediation • Input = means of getting message into message flow • Output = means of getting message out of message flow Message node = atomic unit of message processing Message flow Input Message node Output © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Concept: Node Types Protocol-specific Input Transformation Output Logical construct Processing Examples: HTTP Input JMS Output Database Insert MQ Get XML Transform © Copyright IBM Corp. , 2005. All rights reserved. Try Catch

Introduction to the new mainframe Message Node Processing Examples Data. Insert Compute IF Body. Person. height > 183 THEN INSERT INTO Database. Tall. People (Name, Height, Age) VALUES (Body. Person. Name, Body. Person. height, Body. Person. age); ENDIF; Java Compute IF (XML format required) THEN Output. Root. Properties. Message. Format = 'XML'; ELSE IF (custom format) Output. Root. Properties. Message. Format = 'CWF'; ELSE IF (SWIFT format) Output. Root. Properties. Message. Format = 'TDS'; ENDIF; public class jcn extends Mb. Java. Compute. Node { public void evaluate(Mb. Message. Assembly assembly) throws Mb. Exception { … String person. Age = (String)assembly. get. Message(). evaluate. XPath(“/Body/Person/Age”); … } © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Concept: Logical Message Model – Message Sets <Person age=‘ 32’ height=‘ 172’> <name> <first>Fred</first> <last>Smith</last> </name> </Person> struct { int height; int age; char firstname[24]; char lastname[24]; } Person; 172 32 Fred TYPE Person age name height integer Smith first last string PER + 172 + 32 + Fred Smith © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Concept: Message Driven Flow output connectors node input connector Action input message tree input terminal output terminals © Copyright IBM Corp. , 2005. All rights reserved. output message trees

Introduction to the new mainframe Message Broker Architecture Overview Message Sets and flows are built in Eclipse and deployed to the runtime broker. The user name server sends principals to the Config Manager and broker. The z/OS Broker and Config Manager have full family capabilities, as well some z/OS specific features. © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Message Broker ESB Functions - Routing l Ability to change or filter destination of messages q Support for one-way and request-response interaction patterns q Database, queue and in-memory caches for persisting transient state q Comprehensive composition and flow-of control capabilities q Branching, ordering, exception processing, tracing q Support for creation of new messages during processing l Full range of messages can be routed and filtered q XML – for XML payloads (full schema support) q Record based – for COBOL, C payloads q Industry standard – SWIFT, EDI… q User defined – RYO formats l Different routing and filtering rules q Message based q Message Header and Body routing are supported q External reference q Database lookup, PLUS in-memory cache for infrequently changing data q User algorithm l Partial Parsing for efficient routing of large messages © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Message Flows Output target (Failure) Input source Output target § Reusable § Scalable § Transactional Transform Output target © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Message Broker ESB Functions - Protocol Conversion l de facto Standard protocols q MQ, HTTP, JMS, TCP/IP » Native input nodes provides l “Standards” protocols q Web Services via SOAP/HTTP, SOAP/JMS » Native HTTP(S), Any JMS 1. 1 connectivity » SOAP and MIME parsers provided l Technology interactions q Files, VSAM, QSAM, CICS to access traditional data sources l Industry protocols q SCADA, MQe for mobile devices and sensors l User Defined protocols q For User ISV provided extensions, full user defined node interface. © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Web. Sphere Message Broker – Protocols and Applications Industry and Vendor Protocols IBM Protocols Web. Sphere MQ (+ PM 4 Data) Any 3 rd-party JMS (Enterprise applications (+ managed file transfer)) (TIBCO EMS, Sonic MQ, BEA JMS, web. Methods, See Beyond, Vitria) Web. Sphere MQ Everyplace TIBCO Rendezvous (Mobile device applications) (plug-in component) Web. Sphere MQ Telemetry HTTP and HTTP(S) (RFID, sensors & actuators) FTP and File Web. Sphere MQ Real-time (Very low latency over WANs, and the Internet) TCP/IP Sockets Web. Sphere MQ Multicast SMTP (Reliable Multicast Messaging (RMM)) (Very low latency for LANs) SAP CICS Oracle JDEdwards Siebel Custom Peoplesoft Enterprise Applications © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Message Broker ESB Functions - Transformation l Message Broker models all data via “Message Tree” q All data formats (XML and non-XML) understood natively by message broker q Message Tree created by native parser from source input message format q Transformed using appropriate transformation language q Output tree used to generate required target output message format q …leads to high performance q Native formats means data is only transformed when it necessary q Partial parsing only reads required fields l Use transformation languages to match skill set q Graphical mapping for non-programmers q XSLT for XML orientation q SQL for database and declarative orientation q Java for widely available skill set benefits q ESQL q WTX to leverage existing maps – q Web. Sphere Transformation Extender for Message Broker q Transformation languages can be mixed and matched © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Web. Sphere Message Broker - Mediation Heterogeneous service mediation Web. Sphere Service Registry and Repository Web. Sphere (WAS/Portal). NET Client Some Client SOAP/JMS SOAP/HTTP XML/HTTP XML/MQ Client XML/MQ Text/MQ Client Text/MQ Web. Sphere Message Broker XML/HTTP COBOL Copybook/MQ 3 rd-party App Server provider. NET provider Legacy/custom provider CICS IDOC/ALE SAP © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Message Broker ESB Functions – Event Handling l Processing external events q From simple message exchange to message pattern detection q Ability to distribute information to interested parties l Publish Subscribe and more complex event processing (CEP) l Timer based processing q Processing triggered by timed event l Fixed or variable timers q Ability to generate new timer events from current processing l Message Validation q Confirm that messages are well formed as they enter and leave the bus l e. g. SWIFT validation, XML schema validation © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Key Supportpacs Broker Explorer (IS 02) CICS/VSAM Nodes (IA 12/IA 13) SOAP Envelope/SOAPExtract Nodes (IA 9 O) Message Broker Client for WSRR (IA 9 Q) Importer formatted text message models (IA 7 D) Complex Event Processing Nodes (IA 0 S) © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Highlights of Web. Sphere Message Broker 6. 1 Consumability and Productivity • Reduced time to get started • Simplify debug, test, deploy tasks Enhanced SOA Support • WS-Addressing and WS-Security built-in • Data. Power as WS-Security option • WSRR Integration Extended Connectivity • Gigabyte file support, including FTP • SAP, Siebel, Peopl. Soft nodes • SMTP and JDBC Database nodes Administration and Security • Enterprise security with LDAP and TFIM • MB Explorer and real-time performance • Numerous manageability improvements Performance and Platform Coverage • More 64 bit platforms; JDBC XA; Java 5 • Ultra High Performance XML parser • Compacted memory footprint © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Data. Power Appliance Exploit for Web Services security • Single tool for policy description • Security best practices • WS-Security at appropriate point in topology • Built-in XML threat protection; Hardened device • Built-in service level management • Manage traffic using policy; WSDM and WS-Man • Scale as volumes increase • Enhanced performance with SOA appliance • Add capacity when necessary Administration User Experience • Operational reconfiguration only • Applications and Message Flows unchanged • Right click on flow and select “Use Data. Power” • Data. Power performs WS-Security processing • Forwards processed request to MB © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Support for Web Services l Support provider and consumer scenarios q Provider: l SOAP input & SOAP reply q Consumer: l l l Synchronous SOAP request Asynchronous SOAP request and reply Can be combined to provide Web Service intermediary SOAP Extract and SOAP Envelope nodes Simplify processing of SOAP payload and headers l Support WS-Security and WS-Addressing “out of the box” q Support for WS-Addressing Endpoint References and Message addressing properties q Support for WS-Security authentication, encryption and signing l Username password, X 509 certificates for authentication l Comprehensive encryption and signing algorithms (from JSSE/JCE) l Configuration using Policy Sets © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe File Processing Built-in l Local and remote (FTP) files l Advanced file processing q File input and File output nodes q Combine with other MB nodes l (e. g. ) File to MQ, File to database, File record filtering l Large file handling q Allows very large files (gigabyte) to be processed without using excessive storage q Appropriate broker parsers have been enhanced to request data on demand l Comprehensive support for record detection q Simple: LF, EOL, CRLF, Fixed Length, Whole-file, User-defined q Parser: Use an existing message definition to identify record boundaries © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Web. Sphere Service Registry and Repository l Integrated support for WSRR q Registry contains variety of “entities” (documents) such as WSDL, XSD… l Includes entity category, its relationships and its associated user properties q True governance achieved through registry determining MB processing q Development and runtime usage aspects which can be used together l Development Activity q Use WSRR AD plug-in to search registry for particular entity q Entity can “kick start” message flow and message set creation l E. g. Retrieve WSDL and drag-drop to configure external Web Service call l Runtime interactions q Message flows can query and/or select specific registry entities l l Information cached for high performance access Registry changes result in cache refresh via built-in Pub. Sub mechanism q 2 new nodes to support most popular processing scenarios l l Query – retrieve entity details; other nodes can act on this Select – choose a specific service instance via node matching criteria q WSRR interactions can be overridden dynamically based on message content l Expanded expression support to include literals and variables © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe TCP/IP Nodes Typical scenarios • Connect existing sockets based client application with MQ messaging infrastructure • Connect existing applications to server based sockets programs Broker as TCP/IP client or server program • TCPIPClient. Input, TCPIPClient. Request, TCPIPClient. Output nodes • TCPIPServer. Input, TCPIPServer. Request, TCPIPServer. Output nodes Highly customizable • Full protocol supporting including handshakes, request-reply… • Change TCP/IP operational characteristics without flow redeploy • • Modify TCP/IP ports and addresses Modify socket characteristics, e. g. SO_KEEPALIVE, SO_LINGER… Exploits stream based parsing to interpret stream data as messages • Simple: LF, EOL, CRLF, Fixed Length, Whole-file, User-defined • Parser: Use an existing message definition to identify record boundaries Mature, proven technology • Based on IA 98 Support Pac © Copyright IBM Corp. , 2005. • Established technology with many user deployments All rights reserved.

Introduction to the new mainframe Transport Headers and Triggering Process Transport headers without programming (6102) • • Shows most common transport header properties for MQ, JMS and HTTP New users can easily understand modify common transport properties SMTP node for email generation • e. g. “operator” notification of alert conditions TCPIP nodes for legacy integration (6102) • Client and server side sockets support, inbound and outbound Collector node for more advanced triggering scenarios • • • Coordinate message from multiple, disjoint sources Wait for multiple input conditions Process when all satisfied © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Transport Header Nodes Simplify manipulation of transport headers • MQMD, MQDLH • HTTPInput, HTTPResponse, HTTPRequest, HTTPReply • JMS header, JMS application properties, JMS provider properties Header creation and modification • Set header properties with meaningful constant • MQEI_INLIMITED, MQFMT_NONE • Set header properties from message • Set MQ correl. Id to $Root/MQMD/Msg. Id Delete operations with single radio button • Delete header • Clear fields © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Integrated Support for Major EIS Systems SAP for z/OS, SAP and People. Soft for z. Linux Web. Sphere Adapters delivered “out of the box” as built-in nodes • • • Simplifies management and improves performance for key integration scenarios These are the JCA based Web. Sphere adapters Adapter license still required Support for inbound and outbound scenarios • • Message-to-EIS and EIS-to-message scenarios Adapter nodes integrate with all built-in MB nodes Enterprise Metadata Discovery (EMD) • Significant tooling support • • Simplify for key data structure discovery Accelerates generation of message sets High Performance access • Adapters access native message broker tree © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Web 2. 0 Support HTTP REST support • POST, PUT, GET DELETE verbs/methods now supported Inbound REST request • Handled using HTTP Input/HTTP Reply nodes • REST verb captured in Local. Environment • Router. List also populated • enables easy discrimination between REST verbs in flow • URL declared on HTTP input node • • Includes URL list and wildcard support for diverse noun support http: //example. com/noun/object 1, object 2, object 3 simply handled by one flow Outbound REST request • Static: URL and verbs declared on request node • Dynamic: URL and verbs set by in Local. Environment to override node settings Other HTTP request node enhancements • Increased Local. Environment overrides including HTTP request timeout, SSL parms… © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Common Event Infrastructure Generate CEI Events from message flow • Design Time: New CEI node to specifically identify business relevant messages • Administration Time: Operationally activate generic events • Notification Points: Message entry, Message Exit, Message Exception… • Payload options: Full, Digest, Headers… Processing the Events • Enables integration with Web. Sphere Monitor and Modeller • Message Broker Admin Audit Offering provides entry level audit capabilities (IS 02) Monitor Easy to identify events to be reported • Simple conditional expression – e. g. Purchase. Order. Total. Cost > Hi. Value • Simple specification of data elements captured – e. g. PO. Customer. ID, PO. Total. Cost, … • Reason qualifier (success or failure) Multiple reporting options • MQ, File, Database (JDBC) • Supplied MDB converts CEI MQ message • Output timing: Lazy or immediate © Copyright IBM Corp. , 2005. All rights reserved. Analyze

Introduction to the new mainframe Explorer Eclipse Administration Alternative MQ Explorer based administration • • Simplifies administration of MQ, MB networks in single Explorer console Uses MQ Eclipse ‘extension points’ to provide seamless experience Comprehensive administration facilities • • All features in Broker Administration Also includes new features such as multi Execution Group deploy IS 02 Cat 3 support Pac • • Fully supported in production Initial release July 2006 Performance Monitor • • • Easily view CPU, IO and other metrics in Eclipse Available March 2007 © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Powerful, Easy to Use Tooling l Full function Toolkit in smallest ever install l Builds on advances in Toolkit 6. 0. 2 l Wizards q Guides you through solution creation q Novice and expert modes l WSDL Drag Drop q Quickly create Web Services solutions l Drag and Drop Mapping q Now includes calling Java from map l Many new ease of use features q “Message Viewer” visualizes expressions q Discovery wizards for SAP and People. Soft q BAR file rebuild l Integrated Test Facility q q Unit Test License included Test Client to test flows Direct debug using Java Debug Protocol “Component Trace” to follow message path © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Advanced Security Features Message Broker now has powerful runtime security model • • Supports cross domains security processing Identity, Authentication and Authorization are native capabilities MQ, HTTP, JMS, Web Services transports can all provide identity • Attributes on input and output nodes Eclipse editor for security profile administration Policy decision points technologies: LDAP, TFIM LDAP Authorization Yes Authentication Yes Identity mapping Yes No Rich identity context supported • • • Type can be Username/password or X 509 certificates Token from default or user defined message location e. g. {type=USERNAME, token=user, issued. By=org, applies. To=flow} Issued. By can be default or user defined Applies. To is fully qualified flow name resource Broker. Execution. Group. Flow Identity appears in Message Tree ALSO in 6. 1: Simplified Basic Authentication for WS and HTTP request nodes • TFIM can add username/password certificate to request © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Tivoli Composite Application Manager for SOA Monitor service interactions involving MB • Understand service flows and relationships • Measure response times, message counts and size… • Web Services (SOAP), MQ, JMS, HTTP transports supported • Ability to dynamically modify message flow properties Web Services Navigator Service Management Automation Response Time • Supports thresholds and automatic responses • Built-in and extensible alerts, situations, workflows • Integrated console with OMEGAMON for Messaging, CAM for WAS • WSRR integration to view deployed services Delivery • Requires ITCAM for SOA 7. 1 • Provided as MB message tracking exits Message Count © Copyright IBM Corp. , 2005. All rights reserved. Message Size

Introduction to the new mainframe Web. Sphere Transformation Extender Launcher New Message Broker offering approaches full TX Launcher capabilities • Allows TX maps to run natively inside Message Broker • WTX Extender 8. 2 will run on MB 6. 1 GA Integrated development and deployment • Eclipse based Type Designer and Map Designer integrate with MB Toolkit • Integrated deployment of TX assets deployed in Broker Archive New WTX node • Identifies TX map to be used in Message flow which can be configured dynamically • Each map’s card is visible as dynamic node property • Collector node trigger correlates multiple inputs ready for downstream WTX node • If necessary, can use existing TX adapters New WTX parser • Accesses MB tree directly to provide optimum performance Message Broker supports unparalleled range of transformation technologies • ESQL, Java, Graphical Mapping, XSLT and WTX © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Usage Patterns with Message Broker Service Enablement Message Enablement File Processing Service Virtualization Message Brokering Event Processing OR OR OR © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Web. Sphere Message Broker on System z l Universal Connectivity q Simplify application connectivity to provide a flexible and dynamic infrastructure l Routes and transforms messages FROM anywhere, TO anywhere q Supports a wide range of protocols » MQ, JMS 1. 1, HTTP(S), Web Services, File, EIS (SAP, People. Soft…), TCP/IP, User Defined q Supports a broad range of data formats » Binary (C/COBOL), XML, Industry (SWIFT, EDI, HIPAA…), User Defined q Interactions and Operations » Route, Filter, Transform, Enrich, Monitor, Distribute, Decompose, Correlate, Detect… l Simple programming q Message Flows to describe application connectivity comprising… q Message Nodes which encapsulate required integration logic which operate on… q Message Tree which describes the data in a format independent manner q Transformation options include Graphical mapping, Java, ESQL, XSL and WTX l Operational Management and Performance q Extensive Administration and Systems Management facilities for developed solutions q z/OS and z/Linux natively supported q Offers performance of traditional transaction processing environments © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Advantages of Web. Sphere Message Broker for z/OS l. High Availability q Takes full advantage of Parallel Sysplex aware resource managers (e. g. Web. Sphere MQ) q Fully ARM-enabled q Supports Web. Sphere MQ clustering lz/OS-Specific Connectivity q VSAM q QSAM q CICS l. Workload Management q Goal-oriented resource allocation q Workload scaling, workload isolation l. Reporting and Chargeback q SMF q Coordinated reporting (ENF 37) l. Extend transformation options with Web. Sphere TX (Transformation Extender) q Data enhancement q Complex, many-to-many transformation © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe The Enterprise Service Bus An Enterprise Service Bus (ESB) is a flexible connectivity infrastructure for integrating applications and services. An ESB performs the following between requestor and service MATCHES & ROUTES communications between services CONVERTS between different transport protocols TRANSFORMS between different data formats IDENTIFIES & DISTRIBUTES business events Shape = Transport protocol Color = Data format © Copyright IBM Corp. , 2005. All rights reserved.

Introduction to the new mainframe Summary Significant additions in 2008 • Significant updates to all areas of the product to enhance universal connectivity capabilities Continuing to satisfy five key themes satisfying a broad range of customer requirements • • • Consumability and Productivity Enhanced SOA support Administration & Systems Management Extended Connectivity Platform Support and Performance Message Broker is a key IBM connectivity technology • Unparalleled range of connectivity options and capabilities • Supports users’ range of experience and needs • Industry leading performance in a broad range of scenarios © Copyright IBM Corp. , 2005. All rights reserved.