Run Time Monitoring of Reactive Systems Mikhail Auguston

Run Time Monitoring of Reactive Systems Mikhail Auguston Naval Postgraduate School maugusto@nps. edu Mark Trakhtenbrot Holon Academic Institute of Technology ilmarktr@yahoo. com

Outline Problems in development of reactive systems Model-based development and analysis Our approach: q TL-based assertion language q Automatic generation of monitors for run-time verification Outline of translation scheme Further steps

Why development of reactive systems is difficult ? Main problem: complex behavior q Intricate event-driven interaction with the environment q Concurrency q Timing factors Typically: q Critical applications (embedded RT controllers, …) q Must fulfill tough requirements (safety, timeliness, … )

Challenges q q Capture requirements to express the expected behavior Check that system meets the requirements ideal: full verification more realistic: testing * how to ensure a proper coverage? * how we know the test result is / isn’t correct? can this task be automated?

So, what to do? Apply formal methods! Precise spec of behavior: q Variants of TL [Pnueli, …] (future/past modalities; timing) q Patterns for typical properties [Avrunin, …] (response, precedence, existence, absence, …) Model-based development: q formal model of system (behavior, structure, …) q statecharts to capture behavior; a renown standard q executable model; a basis for automated tools q model-level analysis (closer to problem domain)

Expressing system properties Properties expressed using a special language, e. g. Temporal Logic Examples: ALWAYS (cross_empty or barrier_closed) ALWAYS (request EVENTUALLY allocate_resource) Meaning: the property holds in any computation More complicated: property holds on part of execution paths (need quantifiers on path) Time-related properties: ALWAYS (p EVENTUALLY (5) q) Q should happen within 5 time units after P.

Expressing system properties May be difficult to use without a strong logic background Combination of English with C-like expressions: Event A follows event B after less than 10 time units Patterns for typical behavioral properties E. g. Precedence patterns (S precedes P): Before R EVENTULLY P (not P Until (S or R)) After Q Between Q and R After Q until R Pattern defines scope in which the property should hold

Statechart for Early Warning System ON [in(DISCONNECTED)] WORKING [in(CONNECTED)] EXECUTE C IDLE COMPARING RESET DO_ SET_UP OUT_OF_RANGE S dly(15)/ st!(PRINT_ALARM) POWER_ON DONE_ SET_UP OFF /CONNECT_OFF DISPLAYING_ ALARM POWER_OFF SETTING_UP CONNECTED CONNECT_ON DISCONNECTED dly(5) SAMPLING WAIT CONNECT_OFF SAMPLING_DONE

Model-based analysis with Statemate Simulation: check, record and replay test scenarios Verification: based on predefined assertion templates Code synthesis: executable C, for host or target OS Watchdog charts: - not part of the system model, but co-executed alongside it - used to drive and monitor system execution (in simulation or generated code run)

Model-based analysis Simulation: - Analysis of quality attributes (e. g. stability with respect to changes of the data) - Analysis of events occurring simultaneously (to resolve scheduling and priorities issues in the system) - "What if" checks (alternative decisions, "impossible" scenarios) - Check rarely occurring situations - Check behavior of the system under extreme values. - Influence of various uncontrolled and unpredictable factors - Time-related analysis (performance , very fast / very slow processes)

Model-based analysis Verification: Generic properties (e. g. existence of racing, or nondeterminism) Model-specific properties, e. g. reachability of a state in which certain condition is fulfilled (formulated in terms of model elements) If violation found: a script is generated, driving simulation to the problematic state

Monitor charts Monitor (watchdog) chart: - Direct access (visibility) to all model elements - Support for box black-box and white-box testing Sample assertion: Processing of a request must be accomplished within 5 seconds, and before receiving of the next request Wait Request Processing Done Process Request or Timeout(5) Error /Violation Notification Violation: entering into ERROR state

Suggested approach: Statemate-based run-time verification Goal: support effective testing (finding errors rather than validation / showing correctness) Monitoring of code generated from system model Main features: q Assertion language to specify: - allowed and forbidden behavior - actions to be taken (collect and report diagnostic data) q Automatic creation of monitor charts from assertions Used Statemate capabilities: q Co-translation of model and monitor charts to C q Co-execution of generated code components

Advantages of the approach Flexible assertion language vs. predefined patterns Smooth transition: assertions executable monitors Overcome limitations of simulation and verification

Overcome limitations of simulation and verification Analyze models reflecting design decisions (tasks/events mapping, …), in realistic target environment q Real time analysis - rather than SIM time SIM synch time (clock-driven systems): q all reactions assume same duration SIM asynch time (event-driven systems): all reactions assume zero duration q q No "finite state" limitation; may connect to real input sources No limitations on model size

Assertion language q Boolean expressions: system state properties in(S) and (x > 5) ; q entered(S) started(A) ; ch(d) Regular expressions: describe sequences of states [SELECT (Open | Read | Write | Close)] SATISFY Open (Read | Write )* Close q Temporal formulas: order properties ALWAYS (request EVENTUALLY response) ALWAYS (execute SOMETIME_WAS do_set_up) q Timed temporal formulas: real-time restrictions ALWAYS (request EVENTUALLY (10) response) q Actions: to trace interesting events (e. g. property violation) reports, profiling info (based on objects’ attributes)

Example - assertion for EWS ALWAYS (OUT_OF_RANGE EVENTUALLY (15) (RESET or started(PRINT_ALARM))) ON_FAIL [printf(“Violation after occurrence of OUT_OF_RANGE at time %f “, OUT_OF_RANGE. occur_time)]

More on temporal properties Manna & Pnueli Classification (P – past formula) : q Safety: ALWAYS (P) , Guarantee: EVENTUALLY (P) q Obligation: Boolean combination of safety & guarantee q Response: ALWAYS (EVENTUALLY(P)) q Persistence: EVENTUALLY (ALWAYS(P)) q Reactivity: Bool. combination of response & persistence q Any TL formula is equivalent to a Reactivity formula; others used for convenience & flexibility Limitations: q nesting depth <= 2 for unrestricted future operators q unrestricted operator not embedded in a restricted one

Sample monitor construction ALWAYS (OUT_OF_RANGE EVENTUALLY (15) (RESET or started(PRINT_ALARM))) / ERROR_MESSAGE

Translation of restricted formula ALWAYS (N) P Case 1: ------P a is basic formula Case 2: ------P contains only restricted temporal operators /RES: =false P /RES: =false IDLE START_P IDLE dly(1) F not P D Maximum time needed to compute value of formula FRM: D F . . . START_P D F F D Maximum time needed to compute value of formula FRM: t(FRM) = N + t(P) D At least one component is in D / RES: =false All components are in F / RES: =true t(FRM) = N Computation may finish in less than N time units P P START_P F dly(N) dly(2) P dly(N)/ RES: =true IDLE

Translation of unrestricted formula ALWAYS (P) P is a resticted formula: t(P) = N ================== /RES: =false COMP_0 COMP_1 /T 0: = CURR_TIME P COMP_2 COMP_N IDLE dly(2)/ T 2: =CURR_TIME dly(1)/ T 1: =CURR_TIME dly(N)/ TN: =CURR_TIME START_P P D RESTART_P_0/ RES: =true F . . . START_P F D RESTART_P_1/ RES: =true RESTART_P_i = dly(N - mod(CURR_TIME-Ti, N)) P P F D RESTART_P_2/ RES: =true . . . START_P D F RESTART_P_N/ RES: =true At least one component is in D / RES: =false D

Translation of unrestricted formula EVENTUALLY (P)

Translation of unrestricted formula ALWAYS (EVENTUALLY (P) )

Semantic issues Monitoring: deals with finite runs; semantics of TL – based on infinite runs q q [Eisner, et. al. ]: reasoning on truncated executions Assume the run finishes 4 sec after the last p ALWAYS (p EVENTUALLY(10) q) ; false if no q after the last p ALWAYS (p ALWAYS(10) q) ; true if always q after the last p q q In general: user responsible to provide proper actions

Further steps Implementation: q done - experiments with manually created monitors q next step – actual implementation A more friendly assertion language q e. g. combine English with formulas Optimized translation for certain types of assertions Apply to other design paradigms using statecharts q e. g. in OO: monitor systems with dynamically created objects (problematic for model checking)