Querying and Monitoring Distributed Business Processes Daniel Deutch, Tova Milo Tel-Aviv University Customer Logistics HR System Bank ERP e. Comm Querying and Monitoring Distributed BPs D. M. CRM VLDB '08 Querying and Monitoring Distributed BPs Supplier
Data model Design Students Query language Streams. . . Select… From… Where… Optimization XML Indexing SOAP sname Transactions Files organization cid=cid WSDL. . . sid=sid name=“Mary” Students 2 Takes Distribution. . . Courses Querying and Monitoring Distributed BPs D. M. VLDB '08
Outline § Introduction to Business Processes § Querying § Monitoring § Summary & Research Directions 3 Querying and Monitoring Distributed BPs D. M. VLDB '08
Outline § Introduction to Business Processes § Querying § Monitoring § Summary & Research Directions 4 Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs What is a Business Process? § Logically related activities that, when combined in a flow, achieve a business goal. § Activities may either be local or remote § Operates in a cross-organization, distributed environment § Abstract representation, independent of implementation § Standards facilitate design, deployment, and execution 5 Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs Web-Based Travel Agency BP Travel request responses Airlines Websites queries Confirmation Consolidate Results responses Travel Service queries Hotels Websites 6 Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs Programming In the large § BPs are designed by Non-programmers § Specify combination of functionalities and flow thereof, to solve a complex problem Example: process an order § The operations/functions are implemented by programmers (programming in the small) § Example: fetch order document 7 Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs Orchestration vs. Choreography § Orchestration: Executable process, message exchange sequences are controlled by the orchestration designer. § Choreography: Non-executable protocol for interactions. § E. g. , the legal sequences of messages exchanged, guaranteeing interoperability [orchestra with a conductor vs. ballet dancers] 8 Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs 9 Orchestration Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs Web Services Meet BPs Local to company A Web Service 1 Web Service 4 Web Service 2 Web Service 5 Web Service 3 Web Service n At company B Company A business process 10 Querying and Monitoring Distributed BPs D. M. VLDB '08 On the Web
Introduction to BPs BP Management § Planning § Modeling & Design § Development & Deployment § Execution, Interacting & Monitoring § Analysis and Optimization 11 Querying and Monitoring Distributed BPs D. M. VLDB '08 Source: Microsoft BPM Description
Introduction to BPs § § § § 12 Modeling Challenges Complex Systems Distributed Settings Interoperability issues Robustness Scale Web interface Legacy systems Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs 13 BP market (BPTrends survey) Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs BP Management buzz § Over 20 million hits in google for “business process” § Over 5 million hits for "business process management" § Vast interest by analysts (e. g. Gartner) § Rapidly growing interest in industry § New Standards 14 Querying and Monitoring Distributed BPs D. M. VLDB '08
Standards History Introduction to BPs BPML BPSS WSCI WS-Choreography (Intallio et al) (eb. XML) (Sun et al) (W 3 C) 2000/05 2001/03 2001/05 2001/06 2002/03 2002/06 2002/08 XLang WSFL WSCL BPEL 4 WS 1. 0 (Micorsoft) (IBM) (HP) (IBM, Microsoft) 2003/01 2003/04 BPEL 4 PEOPLE (Oracle et. Al) 2007/04 2007/06 BPEL 4 WS 1. 1 WSBPEL 2. 0 (OASIS) BPEL 15 Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs 16 Standards stack Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs BPEL in a nutshell § Language for specifying BP behavior based on Web Services (WS) § Define BPs as coordinated sets of Web service interactions § Define both abstract and executable processes § Specifies Web services Composition 17 Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs § § § <receive> <reply> <invoke> <assign> <throw> <wait> <sequence> <flow> (parallel) <if> <while> <repeat. Until> BPEL constructs Communication-related constructs Flow-related constructs § … 18 Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs BPEL as XML § BPEL is XML-based § In general we could use XML editors for specification design § Infeasible in practice 19 Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs BPEL as XML (cont. ) <process> <!– Definition and roles of process participants --> <partner. Links>. . . </partner. Links> <!- Data/state used within the process --> <variables>. . . </variables> <!- Properties that enable conversations --> <correlation. Sets>. . . </correlation. Sets> <!- Exception handling --> <fault. Handlers>. . . </fault. Handlers> <!- Error recovery – undoing actions --> <compensation. Handlers>. . . </compensation. Handlers> <!- Concurrent events with process itself --> <event. Handlers>. . . </event. Handlers> <!- Business process flow --> (activities)* </process> 20 Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs BPEL as XML (cont. ) <if> <condition> bpel: get. Variable. Property('ship. Request', 'props: ship. Complete') </condition> <sequence> <assign> <copy> <from variable="ship. Request" property="props: ship. Order. ID" /> <to variable="ship. Notice" property="props: ship. Order. ID" /> </copy> <from variable="ship. Request" property="props: items. Count" /> <to variable="ship. Notice" property="props: items. Count" /> </copy> </assign> … 21 If (ship. Request = ship. Complete) { ship. Notice. Order. Id = ship. Request. Order. Id; ship. Notice. items. Cnt = ship. Request. items. Cnt; } Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs 22 Basic activities Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs 23 Flow activities Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs 24 Example Editor (eclipse) Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs 25 Example editor (Oracle) Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs 26 Example editor (IBM) Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs 27 Example editor (Microsoft VS) Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs 28 Travel Agency Process Flow Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs Zoom In 29 Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs Challenges § So far: challenges & solutions for modeling § BPs are hard to analyze, debug, and optimize § Again, due to scale, distributed settings, legacy systems, …. . § Good modeling simplifies process specification § But further analysis tools are required 30 Querying and Monitoring Distributed BPs D. M. VLDB '08
Introduction to BPs Analysis Types § Static Analysis • • • “What kind of credit services are used (in)directly? ” “How can I buy a plane ticket? ” “Can one get a price quote without giving first credit card info? ” § Monitoring • “Notify me when a user hacks the system and get a price quote without giving his credit card info” § Log Analysis • 31 “Find all logs where a user bought a plane ticket” Querying and Monitoring Distributed BPs D. M. VLDB '08
Outline § Introduction to Business Processes § Querying BP Specifications § Monitoring § Summary & Research Directions 32 Querying and Monitoring Distributed BPs D. M. VLDB '08
Querying BPs Querying Business Processes § Statically analyze a Business Process “Find all ways in which a user can buy a plane ticket without relying first her credit card details” § Analysis needs • Control flow analysis (Reachability, Cycle Detection, Temporal properties, …) • Structural analysis • Data analysis • Messages Validation, Pointer analysis, Array bounds analysis, … • Combined (data and flow) analysis § Database approach • Treat BPs as data • Design a query language 33 Querying and Monitoring Distributed BPs D. M. VLDB '08
Why not XQuery? Querying BPs § Recall that BPEL is XML-based, then… Why not use XQuery? ! § Similar arguments to BPs design & XML • Need to handle complex technical constructs • Complex, unintuitive queries (many joins & recursion) • No abstraction • No robustness to changes in BPEL standard 34 Querying and Monitoring Distributed BPs D. M. VLDB '08
Querying BPs An Ideal querying/analysis tool § Uniform § Graphical § Scalable § Flexible § Similar to the specification design § Can handle partial information and uncertainty 35 Querying and Monitoring Distributed BPs D. M. VLDB '08
Models & Query Languages Querying BPs Monadic Second Order Logic Context Free Graph Grammars Recursive State Machine Temporal Logic Mu-calculus CTL* Finite State Machine LTL CTL First Order Logic BP Specification 36 Query languages Querying and Monitoring Distributed BPs D. M. VLDB '08
Querying BPs First Try § Finite State Machines (FSM) for software specification § Temporal Logic (TL) for querying all possible behaviors § Very common in software (and hardware) verification § Typically Linear time evaluation (data complexity) § Exponential time (query complexity) 37 Querying and Monitoring Distributed BPs D. M. VLDB '08
Querying BPs Finite State Machines (FSM) § States and transition function (typically no “accepting” state) Finite State Machines (FSM) § The system configuration is encoded within the states § Interested in properties of possible traversal over the states § Temporal Logics express such properties 38 Querying and Monitoring Distributed BPs D. M. VLDB '08
FSM (example) Querying BPs § “Flat” Login § No functions § No recursion Search Reserve Payment § Cycles allowed Cash § Execution path by traversal 39 Credit Confirm Querying and Monitoring Distributed BPs D. M. VLDB '08 Cancel
Querying BPs Temporal Logic - Ingredients § Predicates • • x=0? Was a reservation made? § Logical operators (and, or, not) § Queries • Can a reservation be made without relaying a credit card number? • Must one eventually login if he makes a trip search? 40 Querying and Monitoring Distributed BPs D. M. VLDB '08
Querying BPs Temporal Operators § Quantifiers over execution paths • • A φ - All: φ holds on all paths starting from the current state. E φ - Exists: φ holds in at least one path. § Path-specific quantifiers • • • 41 X φ - Next: φ holds at the next state. G φ - Globally: φ has to hold on the entire subsequent path. F φ - Finally: φ eventually has to hold (somewhere). φ U ψ - Until: φ has to hold until at some position ψ holds, and ψ must hold eventually. φ W ψ - Weak until: φ has to hold until ψ holds. Querying and Monitoring Distributed BPs D. M. VLDB '08
Temporal Logic Querying BPs § Queries • Can a reservation be made without relaying a credit card number? E(F(Reserve) and not F(Credit)) • Must one eventually login if he makes a trip search? A (F(login) or not F(search)) § Logics • Linear time Logic (LTL) No path quantifiers First Order • CTL* Allows path quantifiers • Mu-calculus Introduces fix-point operators 42 (Monadic) Second Order Querying and Monitoring Distributed BPs D. M. VLDB '08
Querying BPs Features & Limitations 1. Scale 2. Expressive Power (Specification) 3. Expressive Power (Query language) 4. (Un)Intuitive Formulation 43 Querying and Monitoring Distributed BPs D. M. VLDB '08
Features and Limitations 1. Scale Login § Evaluation is linear in FSM size, but… FSM size is huge for real -life specifications § Call stack § Data § Unfeasible Evaluation Search Reserve Payment Cash Credit Confirm 44 Querying and Monitoring Distributed BPs D. M. VLDB '08 Cancel
Features and Limitations 1. Scale Solutions § Bounded Model Checking [Biere et. Al ’ 99, ’ 03] , [Clarke et. Al ’ 04], … § Summarization [Reps et. Al ’ 98], [Sagiv et. Al ‘ 05], … § Compact representation • State. Charts [Harel ’ 87] • BDD [Bryant 86’, Lam et. Al ’ 05] 45 Querying and Monitoring Distributed BPs D. M. VLDB '08
Features and Limitations 1. Scale State. Charts § Graphical tool for designing state machine based specifications § Extend basic FSM concepts (superstates) § Simplifies design § UML standard 46 Querying and Monitoring Distributed BPs D. M. VLDB '08
Features and Limitations 1. Scale § Data structure that allows compact representation of data with high similarities BDD search cash confirm (search ^ cash) V (not (search) ^ confirm) 47 1 0 Querying and Monitoring Distributed BPs D. M. VLDB '08
![Features and Limitations Relational DB approach [Lam ’ 05] 1. Scale § Store commands](https://present5.com/presentation/6aa4e2b604ee79cc4f98acd95d56332a/image-48.jpg "Features and Limitations Relational DB approach [Lam ’ 05] 1. Scale § Store commands")
Features and Limitations Relational DB approach [Lam ’ 05] 1. Scale § Store commands as db relations § command c is written at location l § “Open” all possible contexts § Exploit similarities and represent compactly by Binary Decision Diagrams § Datalog queries 48 Querying and Monitoring Distributed BPs D. M. VLDB '08
2. Expressive Power (Specification) Features and Limitations § FSMs have limited expressive power § May yield inaccurate approximation of real -life specification 49 Querying and Monitoring Distributed BPs D. M. VLDB '08
Features & Limitations 2. EX-power (Spec) Models & Query Languages Monadic Second Order Logic Context Free Graph Grammars Temporal Logic Recursive State Machine First Finite State Machine Order Logic BP Specification 50 Query languages Querying and Monitoring Distributed BPs D. M. VLDB '08
Features & Limitations 2. EX-power (Spec) Recursive State Machines (RSMs) § A collection of FSMs § Each with multiple entries & exits § Some states represent calls to other state machines (or to self) § An expansion is replacing a call state by a possible implementation § An execution is a sequence of expansions 51 Querying and Monitoring Distributed BPs D. M. VLDB '08
Features & Limitations 2. EX-power (Spec) RSM (example) Home Page Payment pay search confirm reserve payment Start credit cash Home. Page 52 Querying and Monitoring Distributed BPs D. M. VLDB '08
Features & Limitations 2. EX-power (Spec) Evaluation (Temporal Logic) § Single Exit RSMs • CTL* : Linear data complexity [Benedikt et. Al ‘ 05] • Mu-calculus: also [Alur et. Al ’ 07] § Multiple Exit RSMs • LTL: PTIME • CTL, CTL*: EXPTIME • Mu-calculus: EXPTIME 53 Querying and Monitoring Distributed BPs D. M. VLDB '08
Features & Limitations Expressive Power 2. EX-power (Spec) (Specification) Models & Query Languages Monadic Second Order Logic Context Free Graph Grammars Temporal Logic Recursive State Machine First Finite State Machine Order Logic BP Specification 54 Query languages Querying and Monitoring Distributed BPs D. M. VLDB '08
Features & Limitations 2. EX-power (Spec) Context Free Graph Grammars § Extensions of string grammars to graphs § Labels over graph nodes (VR) or edges (HR) § Terminal and non-terminal labels § Derivation rules for non-terminal labels § No start and end nodes! § Connection (“gluing”) rules, by labels, for the derived sub -graph 55 Querying and Monitoring Distributed BPs D. M. VLDB '08
Features & Limitations Expressive Power 2. EX-power (Spec) (Specification) Context Free Graph Grammars Example Home Page Payment pay search confirm reserve payment credit Start cash Home. Page Connection relation : Start. Payment connects only to Search Confirm is connected to cash, but not to credit 56 Querying and Monitoring Distributed BPs D. M. VLDB '08
Features & Limitations Expressive Power 2. EX-power (Spec) (Specification) Context Free Graph Grammars Evaluation § Depends greatly on the allowed connection relation § A restricted model defines entries and exits for graphs and is equivalent to RSMs § Typically Monadic Second Order Queries § We’ll revisit it later. . 57 Querying and Monitoring Distributed BPs D. M. VLDB '08
Features & Limitations Expressive Power 2. EX-power (Spec) (Specification) Active XML § AXML [Abiteboul, M et. Al ’ 04 -’ 08] § Extension of XML to include embedded Web-Services calls § Very useful for modeling web-sites § Formally – a restriction of context free graph grammars to trees § XML query languages (XPath, XQuery, . . ) § Practically efficient evaluation (for restricted cases) 58 Querying and Monitoring Distributed BPs D. M. VLDB '08
Features and Limitations 3. Expressive Power (Queries) § Temporal logics have limited expressive power § Basically – consider only executions § Good for behavioral properties § Can’t capture structural properties § Bisimulation-invariant § Example 59 Querying and Monitoring Distributed BPs D. M. VLDB '08
Features & Limitations Expressive Power 3. EX-power (Queries) 60 Behavioral vs. Structural Analysis Querying and Monitoring Distributed BPs D. M. VLDB '08
Features & Limitations Expressive Power 3. EX-power (Queries) Solution? Monadic Second Order Logic Context Free Graph Grammars Temporal Logic Recursive State Machine First Finite State Machine Order Logic BP Specification 61 Query languages Querying and Monitoring Distributed BPs D. M. VLDB '08
Features & Limitations Expressive Power 3. EX-power (Queries) Infeasible! § Studied extensively for Context Free Graph Grammars § Linear in grammar size § But unfortunately… § Non-elementary in the query (formula) size § 2^2^2^…. . 2 (tower size depends on query size) § Infeasible for even the smallest queries 62 Querying and Monitoring Distributed BPs D. M. VLDB '08
Features and Limitations 4. (Un)intuitive Formulation § Very difficult to express properties of interest in FO/MSO 4. (Un) intuitive Formulation § Long and error-prone formulas § Temporal Logic is more intuitive § Still, textual and complex, especially for large-scaled analysis § Existing tools provide inadequate interface 63 Querying and Monitoring Distributed BPs D. M. VLDB '08
 = (c[n](v)](https://present5.com/presentation/6aa4e2b604ee79cc4f98acd95d56332a/image-64.jpg "Features & Limitations 4. (Un)intuitive formulation FO+TC Formulas (TVLA syntax) r[n, z](v) = (c[n](v)")
Features & Limitations 4. (Un)intuitive formulation FO+TC Formulas (TVLA syntax) r[n, z](v) = (c[n](v) & Query: Is there a point of code reachable from v in which n points at z? 64 r[n, x 1](v)? z(v) | E(v_1) z(v_1) & TC (v_1, v) (v_3, v_4) (n(v_3, v_4) & !x 1(v_3)) : r[n, z](v) & ! (E(v_1) r[n, z](v_1) & x 1(v_1) & r[n, x 1](v) & !x 1(v))) Querying and Monitoring Distributed BPs D. M. VLDB '08
Features & Limitations 4. (Un)intuitive formulation Temporal Logic (SMV syntax) § 65 PReq. Fullfilled. Def 2: assert (P_state -> (P_REQ_FULFILLED U P_BUTTON_PRESSED ; (( § Enter. TState. Def: assert ((~T_state & X(T_state)) -> X(ENTER_T_STATE ; (( Enter. PState. Def: assert ((~P_state & X(P_state)) -> X(ENTER_P_STATE ; (( § § § Traffic Light Requirement Specification PReq. Fullfilled. Def 1 : assert (P_BUTTON_PRESSED -> (~P_REQ_FULFILLED U P_state ; (( Move. To. PPrev. Def: assert ((~move_to_p & X(move_to_p)) -> MOVE_TO_P_WAS_SET_TRUE_IN_PREV_SEC; ( § Move. To. Ito. TPrev. Def: assert( (~move_to_i_to_t & X(move_to_i_to_t)) -> X(X(MOVE_TO_I_TO_T_WAS_SET_TRUE_IN_PREV_S EC ; ((( Querying and Monitoring Distributed BPs D. M. VLDB '08
Mid-section Summary Querying BPs § Model problems • expressive power • scale Pretty much solved by models we’ve seen § Query Language problems • expressive power • unintuitive formulation Not solved yet § BPQL to the rescue [VLDB’ 06] 66 Querying and Monitoring Distributed BPs D. M. VLDB '08
Possible Solution Querying BPs Monadic Second Order Logic Context Free Graph Grammars Temporal Logic Recursive State Machine Finite State Machine First BPQL Order Logic BP Specification 67 Query languages Querying and Monitoring Distributed BPs D. M. VLDB '08
BPQL Queries Querying BPs § BP patterns (like tree patterns for XML) § Single/double-headed edges (Xpath’s / and //) • edges • paths of arbitrary length § Single/double-bounded activities: • simple zoom-in • unbounded zoom-in 68 Querying and Monitoring Distributed BPs D. M. VLDB '08
Querying BPs Q 1: used credit card services? local 69 Querying and Monitoring Distributed BPs D. M. VLDB '08
Querying BPs Q 2: search without login? 70 Querying and Monitoring Distributed BPs D. M. VLDB '08
Querying BPs Q 3: data flow 71 Querying and Monitoring Distributed BPs D. M. VLDB '08
Querying BPs Queries and their semantics § Query: A BP pattern with some transitive nodes & edges transitive § An embedding: a mapping • from: query graphs • to: [possible flows defined by the] BP graphs § A result: image of query graph under an embedding § Answer: all results 72 Querying and Monitoring Distributed BPs D. M. VLDB '08
Querying BPs Structural vs. Behavioral Semantics • Sub-graph homomorphism vs. bisimulation • Both are supported in BPQL 73 Querying and Monitoring Distributed BPs D. M. VLDB '08
Querying BPs 74 Query with an infinite answer? Querying and Monitoring Distributed BPs D. M. VLDB '08
Querying BPs 75 Finite representation Querying and Monitoring Distributed BPs D. M. VLDB '08
Query Evaluation Algorithm Querying BPs § Systems and queries are essentially Context Free Graph Grammars (Recursive state machines) § We basically compute their intersection § Bad news: These are not closed in general under intersection § Good news: Our systems and queries are sufficiently simple: PSIZE representation (as a BP) can be computed in PTIME (data complexity!) § Distributed query processing (based on AXML) 76 Querying and Monitoring Distributed BPs D. M. VLDB '08
Querying BPs And what about data? § So far we have mainly considered flow § Data is important as well § Data: variable values, message exchange, … § Especially interesting in context of Web § Some representative works § Web-services analysis § Pointer Analysis 77 Querying and Monitoring Distributed BPs D. M. VLDB '08
![Web-services analysis Querying BPs § [Deutsch et. Al ‘ 06] • • Query language](https://present5.com/presentation/6aa4e2b604ee79cc4f98acd95d56332a/image-78.jpg "Web-services analysis Querying BPs § [Deutsch et. Al ‘ 06] • • Query language")
Web-services analysis Querying BPs § [Deutsch et. Al ‘ 06] • • Query language combines LTL and FO LTL for temporal relationships FO for data snapshots Efficient evaluation (restricted versions) § [Fu et. Al ’ 04] • • • 78 Guarded automaton for flow query XPath “guards” relate to data Reduces problem into “conventional” model checking Undecidability for general case Polynomial Data Complexity for bounded message number Querying and Monitoring Distributed BPs D. M. VLDB '08
![Pointer Analysis Querying BPs § [Lam et. Al ’ 05] • Pointer analysis for](https://present5.com/presentation/6aa4e2b604ee79cc4f98acd95d56332a/image-79.jpg "Pointer Analysis Querying BPs § [Lam et. Al ’ 05] • Pointer analysis for")
Pointer Analysis Querying BPs § [Lam et. Al ’ 05] • Pointer analysis for sql injections, buffer overflow, … • Analyzes all possible call stack contexts • BDD-based optimization techniques capture similarities • Efficient run-time, practical system § [Sagiv et. Al ’ 06] • Shape analysis • Summarization techniques • Efficient run-time, practical system 79 Querying and Monitoring Distributed BPs D. M. VLDB '08
Querying BPs § § § § 80 Some Systems SMV, Nu. SMV [Clarke et. Al, ’ 92] bddbddb [Lam et. Al, ’ 04] TVLA [Sagiv et. Al, ’ 99] SPIN [Bell Labs, ’ 91] SLAM [Ball et. Al. ’ 00] Moped [Schwoon ’ 02] Mops [Chen et. Al ’ 02] BPQL [Beeri, D, M, et. Al ’ 05] Querying and Monitoring Distributed BPs D. M. VLDB '08 Finite State Machine Context Free Processes
Outline § Introduction to Business Processes § Querying § Monitoring § Summary & Research Directions 81 Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs What is Monitoring? § The aggregation, analysis, and presentation of real time information about activities inside organizations and involving customers and partners [Gartner] § Provide real-time information on executions 82 Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs Why Monitoring? § Imagine you run an auction service… • • Guarantee fair play: notify on too many cancels Maintain SLA: monitor response time Promotions: prizes for the x 10, 000 transaction Illegal access: notify on buyers attempt to confirm bids without registering first Monitoring is crucial for enforcing business policies and meeting efficiency & reliability goals 83 Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs 84 BPEL XML events <action. Data> <header> <process. Name> auction. House </process. Name> <instance. Id> 517 </instance. Id> <sensor. Target> notify_winner </sensor. Target> <timestamp> 2006 -05 -31 T 11: 32: 46. 510+00: 00 </> </header> … <activity. Data> <activity. Type>invoke </activity. Type> <eval. Point> completion </eval. Point> … Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs Monitoring Systems Layers 1. Absorb the stream of events coming from the BP execution engine 2. Process and filter events, selects relevant events data and automatically triggers actions 3. A dashboard that allows users to follow the processes progress, view custom reports, perform analysis, … 85 Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs Existing Approaches § XML streams management § Complex Event Processing (CEP) § Commercial tools (BAM) § BP-Mon 86 Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs XML streams management § Many works on xml streaming • Query Optimization [Koch et. Al ’ 04, Viglas et. Al ’ 02, …] • Security [Altinel, Franklin ’ 00, Benedikt et. Al, ’ 08, …] • Updates & Concurrency [Grabs et. Al ’ 02, Nicola et. Al ’ 07, …] Automata-based techniques vastly used § BPEL processes emits XML messages § Use XML streaming engines for monitoring? 87 Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs Why not ? § Each XML element describes an individual event § Fairly complex XQuery queries (lots of joins) Difficult to handle by existing streaming engines… § XML stream engines manage tree-shaped data (vs. DAGs) XML stream engines expect to receive elements in document order (but we have here parallel flow). 88 Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs Complex Event Processing § Processing of multiple events to identify semantically meaningful combinations. § Studied extensively for Active Databases [Widom ’ 96], [Payton ’ 99], [Wolski ’ 98], … § Recent works in context of BPs [Wu ’ 06], [Jobst ‘ 07] § Identify meaningful activities combinations that form a business logic § Still, somewhat low-level 89 Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs Business Activity Monitoring (BAM) § An enterprise solution, intended to provide a summary of business activities to operations managers and upper management. § Example products: • • • … 90 Web. Sphere Business Monitor (IBM) Oracle Business Activity Monitoring SAP Net. Weaver Pro. Activity PA and P-BAM (BEA) Business. Bridge (Systar) Querying and Monitoring Distributed BPs D. M. VLDB '08
Main features Monitoring BPs § Event driven decision making • Analytics (CEP) run on events as they are generated • Actions are taken immediately § Ruled-based monitoring and reporting • Set thresholds according to key performance indicators (KPIs) and other business-specific triggers § Real time integration of event and context data • Make decision based on combination of real time, historical and plan and forecast data (multi-dimensional queries) § Built for business users • Customizable dashboards, reports and alerts 91 Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs BP-Mon § BAM tools provide solid solutions § But do not incorporate static analysis § BP-Mon is an integrated framework [VLDB’ 07] § Uses a query language similar to BPQL § Graphical & Intuitive § Semantics: evaluated over run-time executions (vs. potential executions in BP-QL) 92 Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs 93 Running Example Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs Query Example (1) Unfair play (too many cancellations) § As before § New or rep Report/ Report* Sliding window § time based § 94 Instance based Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs 95 Visual Interface Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs Query Example (2) Illegal bidding (mix of static and run-time analysis) 96 Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs Some Nice BP-Mon Features § Intuitive monitoring (looks like BPEL) § Easy deployment (implemented as BPEL) § Greedy embedding § Automata-based evaluation algorithm § Type-based optimization 97 Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs Evaluation algorithm • Incrementally extends a greedy matching to one of a larger prefix • Automaton with pattern nodes as states – Tries to concurrently match the all concrete patterns of a given pattern – Attempts to match events as early as possible – On failure: backtracks & retries Complexity: polynomial in the size of the execution log (with the exponent determined by the size of the pattern) 98 Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs Easy Deployment § The automaton is compiled into BPEL § May be used on any application server § Guarantees portability 99 Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs Type-Based Optimization § Exploit knowledge of specification § Infer Irrelevancy & inconsistency using BP-QL 100 Querying and Monitoring Distributed BPs D. M. VLDB '08
Shameless Advertisement Type Inference and Type Checking for Queries on Execution Traces VLDB’ 08 § Tuesday 10: 45, Theory Session § Study of type systems for real-life logs § Type systems serve as a basis for optimizations 101 Querying and Monitoring Distributed BPs D. M. VLDB '08
Outline § Introduction to Business Processes § Querying § Monitoring § Summary & Research Directions 102 Querying and Monitoring Distributed BPs D. M. VLDB '08
Data model Design Query language Streams. . . XML Optimization Indexing SOAP Transactions Files organization WSDL Distribution . . . 103 Querying and Monitoring Distributed BPs D. M. VLDB '08
Almost done Summary & Research Directions § Important aspects of Business Processes • Design • Analysis • Monitoring § Plenty of work on each subject, in many different fields § Still missing a real synergy • Programming languages, model checking, DB technology • XML Streaming, CEP/BAM • All together… § BPQL: integrated, high-level, intuitive framework for all. 104 Querying and Monitoring Distributed BPs D. M. VLDB '08
Almost done Summary & Research Directions Topics for research: • • • 105 Missing information (partial specifications, logs, …) Probabilistic Processes Data values Interactions Log mining Enhanced query language features Optimizations Further applications and more… Querying and Monitoring Distributed BPs D. M. VLDB '08
Thank You! • Questions? Querying and Monitoring Distributed BPs D. M. VLDB '08 Querying and Monitoring Distributed BPs
Monitoring BPs Example: Web. Sphere § Monitor models can be transformed into executable code for Web. Sphere. § Steps for creating a monitor model: 1. 2. 3. 4. 5. 107 Generate CEI events for BPEL elements. Generate Monitor events. Generate the Monitor model. Create the respective business measures (metric, KPI, dimensions, alerts). Deploy into Web. Sphere Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs 108 Step 1: Choose activity events and variables Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs 109 Step 2: Generate monitoring events Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs Step 3: Generate the monitor model Create monitor model Select events to be monitored 110 Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs Step 4: Create the respective business measures back 111 Querying and Monitoring Distributed BPs D. M. VLDB '08
Probabilistic BP Probabilistic Ps § In real-life BPs, executions depend on various external events User choices, ariable values, server states, … § The set of all traces conforming to a query may be large (possibly infinite) Some results are more interesting than others § Also, some information on the execution may be missing or unknown 112 Querying and Monitoring Distributed BPs D. M. VLDB '08
Probabilistic BP Example Queries § What is the typical/likely behavior (flow) for users that do not finalize their reservation? § Which hotels are preferred by British Airways fliers? TOP-K answers reflect common usage patterns 113 Querying and Monitoring Distributed BPs D. M. VLDB '08
Probabilistic BP Lots of related work § Probabilistic Relational DBs § Probabilistic XML § (Probabilistic) Recursive State Machines with temporal logic as query language § BP and Web applications mining 114 Querying and Monitoring Distributed BPs D. M. VLDB '08
Probabilistic BP TOP-K likely traces § We define likelihood of traces § Then find the top-k most likely out of these conforming to a user query § Compact representation of output, using a type 115 Querying and Monitoring Distributed BPs D. M. VLDB '08
Probabilistic BP Distribution Classes § We distinct three classes of distributions, according to their level of dependency § Memory-less (markovian): no dependencies between formulas. § Bounded-memory: dependency in(at most) B last values of each formula. § General 116 Querying and Monitoring Distributed BPs D. M. VLDB '08
Probabilistic BP Results § For memory-less distribution, we find the TOP-K matches in PTIME (data complexity) § For bounded-memory distributions, NP-completeness in the data size, but we give powerful heuristics § In all settings, NP-completeness in the query size § For general distributions, we show undecidability back 117 Querying and Monitoring Distributed BPs D. M. VLDB '08
![References • ]Biere et. Al ’ 99] A. Biere, A. Cimatti, E. Clarke, M.](https://present5.com/presentation/6aa4e2b604ee79cc4f98acd95d56332a/image-118.jpg "References • ]Biere et. Al ’ 99] A. Biere, A. Cimatti, E. Clarke, M.")
References • ]Biere et. Al ’ 99] A. Biere, A. Cimatti, E. Clarke, M. Fujita, Y. Zhu. Symbolic Model Checking using SAT procedures instead of BDDs. Design Automation Conf. (DAC)'99, 1999. • [Biere et. Al, ’ 03 [A. Biere, A. Cimatti, E. Clarke, O. Strichman, Y. Zhu. Bounded Model Checking. In Advances in Computers, vol. 58, Academic Press 2003. • ]Clarke et. Al ’ 04] Edmund M. Clarke, Daniel Kroening, Joel Ouaknine, Ofer Strichman, Completeness and complexity of bounded model checking, VMCAI 2004. • ]Reps ’ 98] Reps, T , . Program analysis via graph reachability. Information and Software Technology 40, 11 -12 1998 • ]Harel ’ 87] Harel. D. Statecharts: A Visual Formulation for Complex Systems. Sci. Comput. Program. (SCP) 8(3): 231 -274 (1987) • ]Lam et. Al ’ 05] Monica S. Lam, John Whaley, V. Benjamin Livshits, Michael C. Martin, Dzintars Avots, Michael Carbin, Christopher Unkel. Context-sensitive program analysis as database queries. PODS 2005 • ]Benedikt et. Al ‘ 05] Rajeev Alur, Michael Benedikt, Kousha Etessami, Patrice Godefroid, Thomas W. Reps, Mihalis Yannakakis. Analysis of recursive state machines. ACM Trans. Program. Lang. Syst. 27(4): 786 -818 (2005) 118 Querying and Monitoring Distributed BPs D. M. VLDB '08
![References • ]Alur et. Al ’ 05] Rajeev Alur, Swarat Chaudhuri, Kousha Etessami, P.](https://present5.com/presentation/6aa4e2b604ee79cc4f98acd95d56332a/image-119.jpg "References • ]Alur et. Al ’ 05] Rajeev Alur, Swarat Chaudhuri, Kousha Etessami, P.")
References • ]Alur et. Al ’ 05] Rajeev Alur, Swarat Chaudhuri, Kousha Etessami, P. Madhusudan On-the-fly Reachability and Cycle Detection for Recursive State Machines • ]Abiteboul, M et. Al ’ 04 -’ 08] Serge Abiteboul, Omar Benjelloun, Tova Milo: Positive Active XML, PODS 2004 • ]Abiteboul, M et. Al ’ 04 -’ 08] Serge Abiteboul, Omar Benjelloun, Bogdan Cautis, Ioana Manolescu, Tova Milo, Nicoleta Preda: Lazy Query Evaluation for Active XML, SIGMOD 2004 • ]Abiteboul, M et. Al ’ 04 -’ 08] Serge Abiteboul, Omar Benjelloun, Tova Milo: The Active XML project: an overview • SMV, Nu. SMV ]Clarke et. Al, ’ 92] Jerry R. Burch, Edmund M. Clarke, Kenneth L. Mc. Millan, David L. Dill, L. J. Hwang: Symbolic Model Checking: 10^20 States and Beyond Inf. Comput. 98(2): 142 -170 (1992) • bddbddb [Lam et. Al, ’ 05] Monica S. Lam, John Whaley, V. Benjamin Livshits, Michael C. Martin, Dzintars Avots, Michael Carbin, Christopher Unkel. Context-sensitive program analysis as database queries. PODS 2005 • TVLA [Sagiv et. Al, ’ 99] Shmuel Sagiv, Thomas W. Reps, Reinhard Wilhelm: Parametric Shape Analysis via 3 -Valued Logic. POPL 1999 119 Querying and Monitoring Distributed BPs D. M. VLDB '08
![References • SPIN ]Bell Labs, ’ 91] The Spin Model Checker : Primer and](https://present5.com/presentation/6aa4e2b604ee79cc4f98acd95d56332a/image-120.jpg "References • SPIN ]Bell Labs, ’ 91] The Spin Model Checker : Primer and")
References • SPIN ]Bell Labs, ’ 91] The Spin Model Checker : Primer and Reference Manual Addison-Wesley, ’ 98. • SLAM [Ball et. Al. ’ 00] Thomas Ball, Sriram K. Rajamani: Bebop: A Symbolic Model Checker for Boolean Programs. SPIN 2000 • Mops ]Chen et. Al ’ 02] Hao Chen, David Wagner , MOPS : an Infrastructure for Examining Security Properties of Software CCS ‘ 02 BPQL ]Beeri, M, D et. Al ’ 05[ • • [Koch et. Al ’ 04] Christoph Koch, Stefanie Scherzinger, Nicole Schweikardt, Bernhard Stegmaier: Flu. XQuery: An Optimizing XQuery Processor for Streaming XML Data. VLDB 2004 • Viglas et. Al ’ 02]Stratis Viglas, Jeffrey F. Naughton: Rate-based query optimization for streaming information sources. SIGMOD ’ 02 • [Altinel, Franklin ’ 00]Mehmet Altinel, Michael J. Franklin: Efficient Filtering of XML Documents for Selective Dissemination of Information, VLDB ‘ 02 120 Querying and Monitoring Distributed BPs D. M. VLDB '08
![References • [Benedikt et. Al, ’ 08] Michael Benedikt, Alan Jeffrey, Ruy Ley-Wild: Stream](https://present5.com/presentation/6aa4e2b604ee79cc4f98acd95d56332a/image-121.jpg "References • [Benedikt et. Al, ’ 08] Michael Benedikt, Alan Jeffrey, Ruy Ley-Wild: Stream")
References • [Benedikt et. Al, ’ 08] Michael Benedikt, Alan Jeffrey, Ruy Ley-Wild: Stream firewalling of xml constraints, SIGMOD ‘ 08 • [Grabs et. Al ’ 02]Torsten Grabs, Klemens Böhm, Hans-Jörg Schek: XMLTM: efficient transaction management for XML documents. CIKM 2002 • [Widom ’ 96] Jennifer Widom, Stefano Ceri: Active Database Systems: Triggers and Rules For Advanced Database Processing. Morgan Kaufmann 1996 • [Wolski ’ 98] Antoni Wolski, Tarik Bouaziz: Fuzzy Triggers: Incorporating Imprecise Reasoning into Active Databases. ICDE ’ 98 • [Wu ’ 06] Eugene Wu, Yanlei Diao, Shariq Rizvi, High-Performance Complex Event Processing Over Streams, SIGMOD ‘ 06 • [Jobst ‘ 07] Daniel Jobst, Gerald Preissler, Mapping clouds of SOA- and business-related events for an enterprise cockpit in a Java-based environment. PPPJ 2007 121 Querying and Monitoring Distributed BPs D. M. VLDB '08
Monitoring BPs Query Example (2) Monitor response time of data-store § Sliding window • • § Output • • 122 Time based Instance based Xquery like Group by, having Querying and Monitoring Distributed BPs D. M. VLDB '08
Скачать презентацию Querying and Monitoring Distributed Business Processes Daniel Deutch
6aa4e2b604ee79cc4f98acd95d56332a.ppt