IBM Research System S High-Performance Stream Computing

IBM Research Outline § System S Overview § System S for Energy Trading §

IBM Research Stream processing will be everywhere 3 © 2008 IBM Corporation – All

IBM Research What is Stream Processing? Minimizing time to react Process data as it is continuously generated Database/data warehouse data Extracting and organizing information and intelligence Data Sources Stream Processing System 4 © 2008 IBM Corporation – All Rights Reserved

IBM Research Without Stream Processing? Minimizing time to react Process data as it is continuously generated Transaction processing data Data Sources 5 Batch processing Database/data warehouse Extracting and organizing information and intelligence © 2008 IBM Corporation – All Rights Reserved

IBM Research What Makes a Stream Processing System? Minimizing time to react Process data as it is continuously generated data packet stream Database/data warehouse data Sensor Network operator Extracting and organizing information and intelligence Stream Processing System 6 © 2008 IBM Corporation – All Rights Reserved

IBM Research What Makes a Stream Processing System? Tooling Developer UI Composition UI Analyst UI Application Interconnection of operators Runtime Environment Job management, resource management, content routing, programming model, object store Hardware Platform Servers, networks, storage, operating system, file system Stream Processing System 7 © 2008 IBM Corporation – All Rights Reserved

IBM Research Continuous Event and Stream Processing Event/Data Volume & Diversity High Volume Complex Analysis Time Sensitive Time Sensitivity Analysis Complexity § Stream Processing enables… – high message/data rates, – low (msec-secs) latency, – advanced analysis § Today’s Complex Event Processing (CEP) solutions target… – 10 K messages/sec, – secs-minutes latency, – rules-based analysis 8 © 2008 IBM Corporation – All Rights Reserved

IBM Research System S Stream Processing § New stream computing paradigm § Pull information from anywhere in real time § Ultra-low latency, ultra-high throughput § Scalable 9 © 2008 IBM Corporation – All Rights Reserved

IBM Research System S: A Closer Look System S continually adapts to new inputs, new modalities System S applications can seamlessly process structured (event) and unstructured data 10 Analytics may be a combination of provided and user-developed/legacy operators This notional System S application… • Calculates VWAP • Calculates P/E, based earnings from Edgar • Refines earnings based on encumbrances identified in newsfeeds © 2008 IBM Corporation – All Rights Reserved

IBM Research SPADE Building Blocks Classifiers, Annotators, Correlators, Filters, Aggregators Correlate Transform Annotator Edge Adapters Segmenter Filter 11 Classifier © 2008 IBM Corporation – All Rights Reserved

IBM Research Application Programming Source Adapters Sink Adapters Operator Repository MARIO: Automated Application Composition SPADE: Stream processing dataflow scripting language § Consumable § Reusable set of operators § Connectors to external static or streaming data sources and sinks Platform Optimized Compilation 12 © 2008 IBM Corporation – All Rights Reserved

IBM Research SPADE § SPADE (Stream Processing Application Declarative Engine) is an intermediate language for streaming applications. – Simplifies design of applications used by System S – Hides complexities of • manipulating data streams (e. g. , contains generic language support for data types and building block operations) • fanning out applications to distributed heterogeneous nodes • transporting data through diverse computer infrastructures (ingesting external data, routing intermediate results, looping in feedback, branching, outputing the results, . . . ) 13 © 2008 IBM Corporation – All Rights Reserved

IBM Research Basic Promises of SPADE § SPADE is easy to use – Programmers provide descriptions of stream-based data processing tasks using SPADE’s intermediate language – SPADE’s query engine comes up with an execution plan, builds it, and hands it off to System S runtime for deployment § SPADE enables rapid application development – Customizable operators – do not require low-level coding – Support for user defined operators and legacy code § SPADE is high performance – Optimized code generation 14 © 2008 IBM Corporation – All Rights Reserved

IBM Research A simple example [Application] Source. Sink trace [Typedefs] typespace sourcesink typedef id_t Integer typedef timestamp_t Long Source Aggregate Functor Sink [Program] // virtual schema declaration vstream Sensor (id : id_t, location : Double, light : Float, temperature : Float, timestamp : timestamp_t) // a source stream is generated by a Source operator – in this case tuples come from an input file stream Sen. Source ( schema. For(Sensor) ) : = Source( ) [ “file: ///Sen. Source. dat” ] {} // this intermediate stream is produced by an Aggregate operator, using the Sen. Source stream as input stream Sen. Aggregator ( schema. For(Sensor) ) : = Aggregate( Sen. Source <count(100), count(1)> ) [ id. location ] { Any(id), Any(location), Max(light), Min(temperature), Avg(timestamp) } // this intermediate stream is produced by a functor operator stream Sen. Functor ( id: Integer, location: Double, message: String ) : = Functor( Sen. Aggregator ) [ log(temperature, 2. 0)>6. 0 ] { id, location, “Node ”+to. String(id)+ “ at location ”+to. String(location) } // result management is done by a sink operator – in this case produced tuples are sent to a socket Null : = Sink( Sen. Functor ) [ “cudp: //192. 168. 0. 144: 5500/” ] {} 15 © 2008 IBM Corporation – All Rights Reserved

IBM Research Performance optimization and scalability § Split/Aggregate/Join architectural § Operator Fusion – Fine-granularity operators – From small parts, make a big one that fits Logical app view pattern – High-ingest rate input stream must be split – Aggregate: model creation – Join: correlation § Code generation – Actual code must match the underlying runtime environment • • • Number of cores Interconnect characteristics Architecture-specific instructions – Driven by automatic profiling – Driven by incremental learning of application characteristics 16 Physical app view § Compiler-based optimization © 2008 IBM Corporation – All Rights Reserved

IBM Research Operator Fusion - Illustration One PE per Operator A truly random partitioning 17 Spade compiler can generate optimized operator grouping schemes Fuse all except sources and sinks © 2008 IBM Corporation – All Rights Reserved

IBM Research System S Runtime Services Optimizing scheduler assigns operators to processing nodes, and continually manages resource allocation Runs on commodity hardware – from single node to blade centers to high performance multi-rack clusters Processing Element Container Processing Element Container System S Data Fabric Transport Operating System X 86 Blade Box 18 X 86 Blade X 86 FPGA Blade X 86 Cell Blade © 2008 IBM Corporation – All Rights Reserved

IBM Research System S Runtime Services Adapts to changes in resources, workload, data rates Processing Element Container Optimizing scheduler assigns operators to processing nodes, and continually manages resource allocation Runs on commodity hardware – from single node to blade centers to high performance multi-rack clusters Processing Element Container System S Data Fabric Transport Operating System X 86 BG Node Blade 19 FPGA X 86 BG node Blade Cell X 86 BG node Blade Capable of X 86 BG node Blade exploiting specialized hardware © 2008 IBM Corporation – All Rights Reserved

IBM Research Distributed operation Site B Site C 20 Site A © 2008 IBM

IBM Research Advantages of Stream Processing as Parallelization Model § Streams as first class entity – Explicit task and data parallelism – More intuitive approach to multicore exploitation § Automated composition – Query optimization over wellknown operators – Inquiry optimization using semantic tagging of operators and data sources § Operator and data source profiling Source Adapters Operator Repository Sink Adapters Automated, Optimized Composition (SPADE, MARIO) Automated, Optimized Deploy and Management (Scheduler) for better resource management § Reuse of operators across stored and live data – Map. Reduce is similar programming model with storage as transport 21 © 2008 IBM Corporation – All Rights Reserved

IBM Research System S Pilots Trading Advantage Identification of and response to opportunities in real-time market data 5 Million events / sec Millisecond latency Manufacturing Improving the quality semi-conductor wafers with dynamic manufacturing tools tuning 2. 5 K events / sec 10 msec latency Semiconductor Solutions Government Detect & respond to phenomena based on large volumes of structured and unstructured information Astrophysics World’s largest and first fully digital radio observatory for astrophysics, space and earth sciences, and radio research 1. 5 Million events / sec 22 © 2008 IBM Corporation – All Rights Reserved

IBM Research Sneak Preview: IBM Info. Sphere Streams Applications Data Warehouse Business Process Management

IBM Research The Energy Trading Scenario using Stream Computing § Sample application showing power of Stream Computing – Only one of many possible applications/services § Weather conditions and events drive pricing of energy futures – natural events interfering with energy supply – announcements, news stories, … § Energy traders today struggle to integrate info from multiple sources – cannot get it in real time, to inform their trade decisions – they see 8 screens, integrate manually via a spreadsheet § IBM Stream Computing assembles, deploys applications – integrates diverse sources of data – provides timely correlations, analyses 25 © 2008 IBM Corporation – All Rights Reserved

IBM Research Illustrative Example: Fear and Opportunity in the Gulf News Flash: Ø Hurricane Dean Upgraded to Category 5 • If you saw it coming, story, Same because you watched for Ø Path Projected through Gulf more primal data… viewed 2 hours later. Ø Oil Stocks Uniformly Down • Like hurricane story’s the same, from NOAA The path predictions • Or even weather satellite and/or sensor data but… • Real-time equities trade data • If you’ve been accumulating intelligence on the location (and value) of company assets that could …the live tickers tell be in the path… a different story • If you could apply such analysis before the news cycle… Oil company stocks • You could take advantage – in both directions… down, based on early fears. More affected companies have been identified Properties with significant assets in path are still down (a bit late, no? ) Others are up, showing recovery even before the storm hits 26 © 2008 IBM Corporation – All Rights Reserved

IBM Research Recommendations Based on Hurricane Forecast Capture market data (high Compute portfolio volume)market indicators (low latency) Correlate combined risk and trade VWAP to determine buy/sell recommendations Capture weather sensor data, analyses hurricane predicted path System S platform Estimate impact on portfolios Make DHTML Result Dynamically updated recommendations rendering risk and notify assessment for assets in projected path Web Zero platform of Real-time projections hurricane path 27 © 2008 IBM Corporation – All Rights Reserved

IBM Research Past Projects § Outlier detection from single tripole § Decomposing combined DOA’s from single tripole – SPADE UDOP’s – Linking against Lapack and Blas libraries – About 50 non-trivial processing elements – Being optimized by SPADE team now § Convolutional resampling (t. Convolve) on System S – Mostly built-in operators (soon built-in operators only) – Fully parametrizable using Perl; e. g. , # of w planes – Does scale very well! 29 © 2008 IBM Corporation – All Rights Reserved

IBM Research Outlier detection from single tripole § Receive 3 D electric field § Demultiplex 3 D electric field – Each UDP packet contains multiplexed electric fields § Compute intensity è I(t)=|Ex(t)|2+|Ey(t)|2+|Ez(t)|2 § Detect outliers è Outlier detected if: m. I(t-N: t-1) + T. I(t-N: t-1) I(t) m. I(t-N: t-1) - T. I(t-N: t-1) § Visualize detected outliers in Matlab in real-time 30 © 2008 IBM Corporation – All Rights Reserved

IBM Research SPADE Flow of Operators Field intensity |Ex|2 Windowed statistics ^2 Aggregate Avg 10 c, 1 c Avg m. I 2 UDP Source Data Demux |Ey|2 Barrier Seq, I Barrier m. I Sqrt(m. I 2 -m. I 2) m. I Aggregate Avg 10 c, 1 c |Ez|2 m. I, I Avg m. I-T. I m. I+T. I U, L Filter out Seq<10 UDP sink File sink 31 Filter out empty lists U I L? I, {U, L} Barrier Outlier detection © 2008 IBM Corporation – All Rights Reserved

IBM Research Field Intensity Field intensity |Ex|2 Source Data Demux |Ey|2 Barrier Seq, I

IBM Research Source Data Demux |Ex|2 |Ey|2 |Ez|2 Barrier I Field intensity Seq, ©

IBM Research SPADE Flow of Operators Field intensity |Ex|2 Windowed statistics ^2 Aggregate Avg 10 c, 1 c Avg m. I 2 UDP Source Data Demux |Ey|2 Barrier Seq, I Barrier m. I Sqrt(m. I 2 -m. I 2) m. I Aggregate Avg 10 c, 1 c |Ez|2 m. I, I Avg m. I-T. I m. I+T. I U, L Filter out Seq<10 UDP sink File sink 38 Filter out empty lists U I L? I, {U, L} Barrier Outlier detection © 2008 IBM Corporation – All Rights Reserved

IBM Research Windowed Statistics Windowed statistics ^2 Aggregate Avg 10 c, 1 c Avg m. I 2 Barrier m. I Aggregate Avg 10 c, 1 c Avg m. I Sqrt(m. I 2 -m. I 2) m. I, I m. I-T. I m. I+T. I U, L 39 © 2008 IBM Corporation – All Rights Reserved

IBM Research ^2 Aggregate Avg 10 c, 1 c Avg m. I 2 m.

IBM Research SPADE Flow of Operators Field intensity |Ex|2 Windowed statistics ^2 Aggregate Avg 10 c, 1 c Avg m. I 2 UDP Source Data Demux |Ey|2 Barrier Seq, I Barrier m. I Sqrt(m. I 2 -m. I 2) m. I Aggregate Avg 10 c, 1 c |Ez|2 m. I, I Avg m. I-T. I m. I+T. I U, L Filter out Seq<10 UDP sink File sink 43 Filter out empty lists U I L? I, {U, L} Barrier Outlier detection © 2008 IBM Corporation – All Rights Reserved

IBM Research Outlier Detection Intensity U, L Filter out Seq<10 UDP sink File sink

45 File sink UDP sink Intensity Filter out Seq<10 Filter out empty lists U

46 File sink UDP sink Intensity Filter out Seq<10 Filter out empty lists U

IBM Research Decomposing combined DOA’s from single tripole 47 © 2008 IBM Corporation –

IBM Research Direction of Arrival (DOA) § § Simple case – Two orthogonal waves

IBM Research Pseudo-Orbital Momentum Two-wave case, same frequencies 49 © 2008 IBM Corporation –

IBM Research Pseudo-Orbital Momentum Two-wave case, same phases 50 © 2008 IBM Corporation –

IBM Research Decomposing the combined DOA § Consider n = 1. . N incident waves at time t § Can we possibly estimate DOA(n) for all N? … from a single 3 D sensor? ? ? 51 © 2008 IBM Corporation – All Rights Reserved

IBM Research Signal Processing to the Rescue § Let’s get back to wave equations n § x-component: (Ex) (t) = (Ax) n ei x n n ei t § Estimate n for all n = 1. . N from x-component – Matrix pencil method! – Requires N timestamps only (minimum) § Plug estimates back in {x, y, z}-components – System of 3 x. N linear equations n è Retrieve estimates of (A{x, y, z})n ei {x, y, z}. Ø Estimate DOA(n) for all n = 1. . N n • E. g. , (Vx) = (Ay) 52 n n ei y (Az *)n n e-i z - (Az) n n ei z (Ay *)n n e-i y © 2008 IBM Corporation – All Rights Reserved

IBM Research Summary Simplified Processing Flow Graph Hankel Construct Pick Top Signals (SVD) Frequency estimates X UDP Source Data Demux Y Z Denoise frequency estimates X Y Least Square Solver Z 53 Least Square Solver Separated waves V vector © 2008 IBM Corporation – All Rights Reserved

IBM Research Sample Spade Code 54 © 2008 IBM Corporation – All Rights Reserved

IBM Research Convolutional resampling (t. Convolve) Goal: Evaluate System S as the Central Processing Platform of the Australian SKA Pathfinder (ASKAP) 45 antennas, 30 beamformers § ASKAP System: – 1% of SKA system – Operational in 2012 – 8 h observation produces 2. 3 TB! 55 © 2008 IBM Corporation – All Rights Reserved

IBM Research SKA Processing Graph 66 PEs on 20 nodes Heavy Convolutional PEs Main

IBM Research Scalability § 10000 samples randomly generated § 5+ times reduction in gridding

IBM Research Current Projects § Software imaging with cleaning – Joint algorithmic/software/hardware optimization – … in collaboration with Tim Cornwell et al. § Astronomical Signature Clustering – … in collaboration with Bo Thide, Jan Bergman, Lars K Daldorff 58 © 2008 IBM Corporation – All Rights Reserved