Bringing Visibility and Control to Net Centric Systems

Topics u Net Centric Environments u ESM across the Lifecycle § § § § Visibility Synchronizing with Other Governance Systems Policy Management Service Level Monitoring and Management Exception Management Security Validation u Case Studies u Q&A © 2007 Amber. Point, Inc. 2

Wide Variety of Net Centric Infrastructure Agency Portal Enterprise Service Bus External Agency Service 1 Mission Specific Services Materiel Order Service External Agency Service 2 EJB Applications DBMS © 2007 Amber. Point, Inc. MQ-based Warehouse 3

Wide Variety of Vendors Agency Portal Enterprise Service Bus External Agency Service 1 Mission Specific Services Materiel Order Service External Agency Service 2 EJB Applications DBMS © 2007 Amber. Point, Inc. MQ-based Warehouse 4

New Types of Problems u Full Picture? Map and Monitor? u Everything Seems to be Up & Running (Green Lights), s uethat but the users are calling the Help Desk claiming Iss s on i they aren’t getting service rat e Op Responses t u Non-Responses and Corrupted no es, u u Debugging Distributed Systems Iss n tio ica u How ldo you lock down Security at the Endpoint? pp A u How do you Validate new versions of Services that are Shared across dozens of applications? © 2007 Amber. Point, Inc. 5

The Role of Enterprise Service Management u Traditional Management Tools and Techniques will not solve these new issues. Traditional Management is an Operations Problem. u ESM (SOA Management) is an Application Problem u As a Result, the Enterprise Service Management system will be § Monitored by your Operations Staff but § Used by your Development and Tier II/Tier III Staffs § To solve problems that they would otherwise have to write complex code and dig through logs to fix. © 2007 Amber. Point, Inc. 6

…Throughout the Lifecycle ESM should automate real-time visibility and control at every stage of the SOA lifecycle Development Staging Production Business Logic IDE’s Process Tools Diagnostics Runtime Policies • Performance • Security • Availability • Logging Validation Performance Discovery • Automatically enforce governance More Policies • Performance • Security • Availability • Logging • SLAs • Audit Service Levels Capacity Planning © 2007 Amber. Point, Inc. Discovery • Automatically discover rogue services 7

Enterprise Service Management Real-time visibility, control and validation <soapenv: Envelope xmlns: soapenv="http: //schema s. xmlsoap. org/soap/envelope/"> <soapenv: Body> <po-number> A 234235 </po-number> </soapenv: Body> </soapenv: Envelope> Enterprise Service Bus External Agency Service 1 External Agency Service 2 Visibility, control and validation regardless of SOA infrastructure decisions © 2007 Amber. Point, Inc. 8

Keys to Successful ESM u Visibility – Knowing What’s Out There and What’s Going On… § What services, application flow, and infrastructure are installed and in use § Who’s using it § Is it meeting expectations / obligations u Control – Taking Action to Prevent or Correct Issues… § Define and enforce runtime policies – make sure proper policies are active § Diagnose failures / prevent them u Ensuring Integrity – Ensuring Changes Don’t Impact the Whole Application Environment… § Automatically check for the correctness of the running system § Detect and validate changes before they impact users and partners u Do This across the Entire Infrastructure u And Do It All as Automatically as Possible § Reduces risks and costs § Automation is the single most important thing that makes SOA scaleable © 2007 Amber. Point, Inc. 9

Automatic End-to-End Visibility u Dynamic discovery of service and supporting components “in the wild” § § § Web services Consumers Dependencies Application Flow & Transactions Runtime Policies Runtime Metadata u Automatically track transactions § § Platform Repositories Home-grown Databases Enterprise Service Bus Service Registries Intended Design Non-invasive; no message modifications Feed impact analysis, error detection, etc. u In most environments, no single source of information is always right Running Reality Ensures a complete view of the SOA application environment © 2007 Amber. Point, Inc. 10

Automatic Synchronization with Other Governance Systems Service Network Runtime Governance policies Read Platform Repositories Service Registries • Services • Metadata • Policies Home-grown Databases Discovery service contract Service Levels Enterprise Service Bus Exceptions Write • Services (discovered, changes) • Scorecard Information • Policies (new, changes) Policy Mgmt Runtime Repository u Publish § § § data Reality Design Changes to endpoints and policies Scorecard metrics Dependencies ? u Discover discrepancies between intentions (design/dev) and reality (runtime) Support Federated Information Exchange © 2007 Amber. Point, Inc. ? vs. ? 11

Automatic Policy Provisioning via the WS-Policy Standard u Policies that follow the WS-Policy Standard u Provisioned with a “where clause” u Automatically applies policies based on dynamic attributes and message content. § § § All production services All services in Accounting application All services deployed in Web. Logic containers u User-defined attributes for services, containers & policies u Assignments are reevaluated as attributes change One-at-a-Time Approach p 1 p 1 S 1 p 50 Profile Based Approach Logging all services Security S 2 Encryption where “Blue Force” Load-Bal Weighted where deployed on Web. Logic app servers S 3 5, 000 policy points S 5 S 1 100 svcs x 50 policies S 3 S 2 S 6 S 4 Sn u Manage system on “autopilot” where policies are automatically assigned as appropriate. u Eliminate production mistakes by reducing manual steps. © 2007 Amber. Point, Inc. 12

Service Level Management Service- and Mission-level Visibility Service View User Summary and Objectives Alerts Historical Reporting u Enforce agreements based on Mission-Specific criteria § “Gold” users, Warfighters in Battle, First Responders, etc. u Granular visibility – groups, users, services, operations u Preventative and corrective actions © 2007 Amber. Point, Inc. 13

Real-time Monitoring of Mission Centric Processes Probe Applications for Inconsistencies Expected Delivery Disconnect Alert • Check transaction progress and correctness • Does not need to be synchronous or see every step © 2007 Amber. Point, Inc. 14

Exception Management Stop Digging through Logs! Technical Faults Mission-Specific Exceptions Drill into Exception Content & Context Process Flow - Exception context - Response times Agency Portal Materiel Orders System Mgmt Augments System-level View Red Force Tracking External Agency Warehouse Machines App Servers Network Shipping Partner u Mission-specific visibility using exception content and context § § Ex: External Agency Services failure Alert when “no order confirmation within 3 minutes after completion” u Visibility in operational issues – services, transactions, operations, messages § SOAP faults, database errors, etc. © 2007 Amber. Point, Inc. 15

Security First- and Last-Mile Enforcement First-Mile Security Extensive Integration - Client-side agent - Automatic enforcement of out-bound security - Identity Management Systems - Security Appliances - App Server / ESB / OS Security Client Application Agency Portal Identity Management Systems <? xml version='1. 0'? > <Payment. Info xmlns='http: //example. org/paymentv 2'> <Name>John Smith</Name> <Encrypted. Data Type='http: //www. w 3. org/2001/04/xmlenc#Element' xmlns='http: //www. w 3. org/2001/04/xmlenc#'> <Cipher. Data> <Cipher. Value>A 23 B 45 C 56</Cipher. Value> </Cipher. Data> </Encrypted. Data> </Payment. Info> <Name> <Encrypted Type='http <Cipher. Da <Cipher </Ciphe Blue Force Tracking Service Red Force Tracking Service Logistics and Materiel Last-Mile Security - Plug-ins provide endpoint protection - No ability to circumvent © 2007 Amber. Point, Inc. Firewall Shipping Partner Security Policies - Authentication - Authorization - Credential Mapping - Censorship - Crypto 16

Check SOA Services & Application Integrity for… Type of Changes Code Is the new version backwards compatible? Policy If we institute this new governance policy (security, routing, etc. ), will it break dependent applications? Usage What will happen if our usage doubles? Triples? Quadruples? Will an OS security patch break my system? Infrastructure How will introducing a load-balancer change my capacity? Impact of Change Functionality Does the service produce the same responses as last time? Quality of Service Acceptable performance and throughput? © 2007 Amber. Point, Inc. 17

The SOA Validation Problem System Integrity Always at Risk Agency Portal Blue Force Tracking Materiel Orders Red Force Tracking A Change Here May Impact Everyone Warehouse Shipping Partner u Service reuse creates dependencies u Impact of any changes ripple throughout the system § § Real impact of planned changes is hard to predict Impact of unplanned or unannounced changes can be devastating u Yet, it quickly becomes impossible to setup and replicate all dependent systems for testing elsewhere Need way to continuously check for integrity – both in staging and in production © 2007 Amber. Point, Inc. 18

SOA Validation Safe environment to validate changes before deployment Agency Portal Materiel Orders u Captures requests/responses from all dependent consumers Blue Force Tracking Red Force Tracking Warehouse Changes to: - Code - Policy - Usage - Infrastructure § § USTRANSCOM Message content Context – headers, distribution patterns u Replays looking for differences in response u Capacity planning using realworld message distributions © 2007 Amber. Point, Inc. 19 19

Case Studies Real-World Examples © 2007 Amber. Point, Inc.

Rapidly Becoming the SOA Runtime Governance Standard in the US Federal Government u ESM Product of Choice, DISA Net-Centric Enterprise Services (NCES) § Working with DISA for 3+ years to define ESM Services under NCES Program u NGA Geo. Scout Program u Intelligence Community – Multiple Installed Projects u US Transportation Command – GTN Modernization Effort u Engaged with Multiple Net-Centric Labs, COIs and NCES Pilot Programs § JHU APL, MIT Lincoln Labs § ASAP ACTD – Integrated Broadcast Service program § C 2 SSA COI - C 2 Community u Engaged with all major NCES and Related Industry Organizations § AFEI and NCOIF § SOA Foundation, ISR, Security Working Groups u NIAP Common Criteria Security Certification § Only SOA Management Vendor in the Program © 2007 Amber. Point, Inc. 21

Lehman Brothers Making enterprise-wide SOA governance pain-free u 3, 000 Developers NY Design § Different departments / divisions § Different priorities and “fire drills” § Different locations London Design Dev Design-time Approval Registry QA Unapproved Svcs QA Process Production Registry Approved Svcs Stage Prod u After a year, only 8 registered services § Too busy to update the registry § No value to the developers © 2007 Amber. Point, Inc. 22

Using Automatic Runtime Governance to Achieve Design-time Governance NY Design Runtime Repository Asset Database Discovered Svcs Performance Errors/Exceptions Machines Owners Locations London Design Dev QA QA Stage Prod Design-time Approval Registry Process Unapproved Svcs Production Registry Approved Svcs u Uses Amber. Point’s automatic discovery of running services and dependencies at each stage of their SOA lifecycle § Synchronizes with home-grown Asset DB and Design-time Repository © 2007 Amber. Point, Inc. 24

Service Detail Screen u When service was discovered u How long service has been up Discovered Info u Type of service u Link to WSDL u Metadata from Asset DB (42 fields) u All data can be used in policy definitions Asset DB Info © 2007 Amber. Point, Inc. 25

“What’s in it for me? ” A lot. Comprehensive insight without lifting a finger Dependencies NY Design Performance London Design Dev QA Diagnostics QA Stage Change Analysis Stage Prod u Opt-in for expanded control § Security, load balancing, failover, etc. © 2007 Amber. Point, Inc. 26

Results: Visibility and Cooperation u From only 8 registered services after previous approach to 100 registered services in first couple months u ROI reporting visible throughout the company u Runtime results automatically feed other consoles § JMX-based home grown system § Internal SOA coordination site § HP Open. View u Transformed the environment to one where groups were vying to be the ones that could “cooperate the most” © 2007 Amber. Point, Inc. 27