A Tutorial on Automated Verification Tevfik Bultan
Who are these people and what do they have in common? 2007 Clarke, Edmund M 2007 Emerson, E Allen 2007 Sifakis, Joseph 1996 Pnueli, Amir 1991 Milner, Robin 1980 Hoare, C. Antony R. 1978 Floyd, Robert W 1972 Dijkstra, E. W.
Outline • • • Software’s Chronic Crisis Temporal Logics and Model Checking Problem Symbolic Model Checking Automata Theoretic Model Checking Software Verification Using Explicit State Model Checking with Java Path Finder • Bounded Model Checking • Symbolic Software Model Checking with Predicate Abstraction and Counter-Example Guided Abstraction Refinement
Software’s Chronic Crisis Large software systems often: • Do not provide the desired functionality • Take too long to build • Cost too much to build • Require too much resources (time, space) to run • Cannot evolve to meet changing needs – For every 6 large software projects that become operational, 2 of them are canceled – On the average software development projects overshoot their schedule by half – 3 quarters of the large systems do not provide required functionality
Software Failures • There is a long list of failed software projects and software failures • You can find a list of famous software bugs at: http: //www 5. in. tum. de/~huckle/bugse. html • I will talk about two famous and interesting software bugs
Ariane 5 Failure • A software bug caused European Space Agency’s Ariane 5 rocket to crash 40 seconds into its first flight in 1996 (cost: half billion dollars) • The bug was caused because of a software component that was being reused from Ariane 4 • A software exception occurred during execution of a data conversion from 64 -bit floating point to 16 -bit signed integer value – The value was larger than 32, 767, the largest integer storable in a 16 bit signed integer, and thus the conversion failed an exception was raised by the program • When the primary computer system failed due to this problem, the secondary system started running. – The secondary system was running the same software, so it failed too!
Ariane 5 Failure • The programmers for Ariane 4 had decided that this particular velocity figure would never be large enough to raise this exception. – Ariane 5 was a faster rocket than Ariane 4! • The calculation containing the bug actually served no purpose once the rocket was in the air. – Engineers chose long ago, in an earlier version of the Ariane rocket, to leave this function running for the first 40 seconds of flight to make it easy to restart the system in the event of a brief hold in the countdown. • You can read the report of Ariane 5 failure at: http: //www. ima. umn. edu/~arnold/disasters/ariane 5 rep. html
Mars Pathfinder • A few days into its mission, NASA’s Mars Pathfinder computer system started rebooting itself – Cause: Priority inversion during preemptive priority scheduling of threads • Priority inversion occurs when – a thread that has higher priority is waiting for a resource held by thread with a lower priority • Pathfinder contained a data bus shared among multiple threads and protected by a mutex lock • Two threads that accessed the data bus were: a high-priority bus management thread and a low-priority meteorological data gathering thread • Yet another thread with medium-priority was a long running communications thread (which did not access the data bus)
Mars Pathfinder • The scenario that caused the reboot was: – The meteorological data gathering thread accesses the bus and obtains the mutex lock – While the meteorological data gathering thread is accessing the bus, an interrupt causes the high-priority bus management thread to be scheduled – Bus management thread tries to access the bus and blocks on the mutex lock – Scheduler starts running the meteorological thread again – Before the meteorological thread finishes its task yet another interrupt occurs and the medium-priority (and long running) communications thread gets scheduled – At this point high-priority bus management thread is waiting for the lowpriority meteorological data gathering thread, and the low-priority meteorological data gathering thread is waiting for the medium-priority communications thread – Since communications thread had long-running tasks, after a while a watchdog timer would go off and notice that the high-priority bus management thread has not been executed for some time and conclude that something was wrong and reboot the system
Software’s Chronic Crisis • Software product size is increasing exponentially – faster, smaller, cheaper hardware • Software is everywhere: from TV sets to cell-phones • Software is in safety-critical systems – cars, airplanes, nuclear-power plants • We are seeing more of – distributed systems – embedded systems – real-time systems • These kinds of systems are harder to build • Software requirements change – software evolves rather than being built
Summary • Software’s chronic crisis: Development of large software systems is a challenging task – Large software systems often: Do not provide the desired functionality; Take too long to build; Cost too much to build Require too much resources (time, space) to run; Cannot evolve to meet changing needs
What is this?
First Computer Bug • In 1947, Grace Murray Hopper was working on the Harvard University Mark II Aiken Relay Calculator (a primitive computer). • On the 9 th of September, 1947, when the machine was experiencing problems, an investigation showed that there was a moth trapped between the points of Relay #70, in Panel F. • The operators removed the moth and affixed it to the log. The entry reads: "First actual case of bug being found. " • The word went out that they had "debugged" the machine and the term "debugging a computer program" was born.
Can Model Checking Help • The question is: Can the automated verification techniques we have been discussing be used in finding bugs in software systems? • Today I will discuss some automated verification techniques that have been successful in identifying bugs
State of the art in automated verification: Model Checking • What is model checking? – Automated verification technique – Focuses on bug finding rather than proving correctness – The basic idea is to exhaustively search for bugs in software – Has many flavors • Explicit-state model checking • Symbolic model checking • Bounded model checking
Model Checking Evolution • Earlier model checkers had their own input specification languages – For example Spin, SMV • This requires translation of the system to be verified to the input langauge of the model checker – Most of the time these translations are not automated and use ad-hoc simplifications and abstractions • More recently several researchers developed tools for model checking programs – These model checkers work directly on programs, i. e. , their input language is a programming language – These model checkers use well-defined techniques for restricting the state space or use automated abstraction techniques
Explicit-State Model Checking Programs • Verisoft from Bell Labs – C programs, handles concurrency, bounded search, bounded recursion. – Uses stateless search and partial order reduction. • Java Path Finder (JPF) at NASA Ames – Explicit state model checking for Java programs, bounded search, bounded recursion, handles concurrency. – Uses techniques similar to the techniques used in Spin. • CMC from Stanford for checking systems code written in C
Symbolic Model Checking of Programs • CBMC – This is the bounded model checker we discussed earlier, bounds the loop iterations and recursion depth. – Uses a SAT solver. • SLAM project at Microsoft Research – Symbolic model checking for C programs. Can handle unbounded recursion but does not handle concurrency. – Uses predicate abstraction and BDDs.
Beyond Model Checking • Promising results obtained in the model checking area created a new interest in automated verification • Nowadays, there is a wide spectrum of verification/analysis/testing techniques with varying levels of power and scalability – Bounded verification using SAT solvers – Symbolic execution using combinations of decision procedures – Dynamic symbolic execution (aka concolic execution) – Various types of symbolic analysis: shape analysis, string analysis, size analysis, etc.
What to Verify? • Before we start talking about automated verification techniques, we need to identify what we want to verify • It turns out that this is not a very simple question • First we will discuss issues related to this question
Temporal Logics and Model Checking Problem
A Mutual Exclusion Protocol Two concurrently executing processes are trying to enter a critical section without violating mutual exclusion Process 1: while (true) { out: a : = true; turn : = true; wait: await (!b or !turn); cs: a : = false; } || Process 2: while (true) { out: b : = true; turn : = false; wait: await (!a or turn); cs: b : = false; }
Reactive Systems: A Very Simple Model • We will use a very simple model for reactive systems • A reactive system generates a set of execution paths • An execution path is a concatenation of the states (configurations) of the system, starting from some initial state • There is a transition relation which specifies the next-state relation, i. e. , given a state what are the states that can follow that state
State Space • The state space of a program can be captured by the valuations of the variables and the program counters • For our example, we have – two program counters: pc 1, pc 2 domains of the program counters: {out, wait, cs} – three boolean variables: turn, a, b boolean domain: {True, False} • Each state of the program is a valuation of all the variables
State Space • Each state can be written as a tuple (pc 1, pc 2, turn, a, b) • Initial states: {(o, o, F, F, F), (o, o, F, F, T), (o, o, F, T, F), (o, o, F, T, T), (o, o, T, F, F), (o, o, T, F, T), (o, o, T, T, F), (o, o, T, T, T)} – initially: pc 1=o and pc 2=o • How many states total? 3 * 2 * 2 = 72 exponential in the number of variables and the number of concurrent components
Transition Relation • Transition Relation specifies the next-state relation, i. e. , given a state what are the states that can come immediately after that state • For example, given the initial state (o, o, F, F, F) Process 1 can execute: out: a : = true; turn : = true; or Process 2 can execute: out: b : = true; turn : = false; • If process 1 executes, the next state is (w, o, T, T, F) • If process 2 executes, the next state is (o, w, F, F, T) • So the state pairs ((o, o, F, F, F), (w, o, T, T, F)) and ((o, o, F, F, F), (o, w, F, F, T)) are included in the transition relation
Transition Relation The transition relation is like a graph, edges represent the next -state relation (o, o, F, F, F) (o, w, F, F, T) (o, c, F, F, T) (w, o, T, T, F) (w, w, T, T, T)
Transition System • A transition system T = (S, I, R) consists of – a set of states S – a set of initial states I S – and a transition relation R S S • A common assumption in model checking – R is total, i. e. , for all s S, there exists s’ such that (s, s’) R
Execution Paths • An execution path is an infinite sequence of states x = s 0, s 1, s 2, . . . such that s 0 I and for all i 0, (si, si+1) R Notation: For any path x xi denotes the i’th state on the path (i. e. , si) xi denotes the i’th suffix of the path (i. e. , si+1, si+2, . . . )
Execution Paths A possible execution path: ((o, o, F, F, F), (o, w, F, F, T), (o, c, F, F, T)) ( means repeat the above three states infinitely many times) (o, o, F, F, F) (o, w, F, F, T) (o, c, F, F, T) (w, o, T, T, F) (w, w, T, T, T)
Temporal Logics • Pnueli proposed using temporal logics for reasoning about the properties of reactive systems • Temporal logics are a type of modal logics – Modal logics were developed to express modalities such as “necessity” or “possibility” – Temporal logics focus on the modality of temporal progression • Temporal logics can be used to express, for example, that: – an assertion is an invariant (i. e. , it is true all the time) – an assertion eventually becomes true (i. e. , it will become true sometime in the future)
Temporal Logics • We will assume that there is a set of basic (atomic) properties called AP – These are used to write the basic (non-temporal) assertions about the program – Examples: a=true, pc 0=c, x=y+1 • We will use the usual boolean connectives: • We will also use four temporal operators: Invariant p : Gp (aka p) Eventually p : Fp (aka p) Next p : Xp (aka p) p Until q : p. Uq , , (Globally) (Future) (ne. Xt)
LTL Properties . . . Xp p . . . Gp p p . . . Fp p. Uq p p . . . p p q
Example Properties mutual exclusion: G ( (pc 1=c pc 2=c)) starvation freedom: G(pc 1=w F(pc 1=c)) G(pc 2=w F(pc 2=c)) Given the execution path: x =((o, o, F, F, F), (o, w, F, F, T), (o, c, F, F, T)) x |= pc 1=o x |= X (pc 2=w) x |= F (pc 2=c) x |= ( turn) U (pc 2=c b) x |= G ( (pc 1=c pc 2=c)) x |= G(pc 1=w F(pc 1=c)) G(pc 2=w F(pc 2=c))
LTL Model Checking • Given a transition system T and an LTL property p T |= p iff for all execution paths x in T, x |= p For example: T |=? G ( (pc 1=c pc 2=c)) T |=? G(pc 1=w F(pc 1=c)) G(pc 2=w F(pc 2=c)) Model checking problem: Given a transition system T and an LTL property p, determine if T is a model for p (i. e. , if T |=p) Complexity: (|S|+|R|) 2 O(|f|)
Linear Time vs. Branching Time • In linear time logics we look at the execution paths individually • In branching time logics we view the computation as a tree – computation tree: unroll the transition relation Transition System s 1 s 2 s 3 Execution Paths s 4 s 3. . . Computation Tree s 3 s 4 s 3 s 1 s 2 s 3. . . s 4. . . s 1 s 2 s 3 s 1. . . s 4. . . s 1. . .
Computation Tree Logic (CTL) • In CTL we quantify over the paths in the computation tree • We use the same four temporal operators: X, G, F, U • However we attach path quantifiers to these temporal operators: – A : for all paths – E : there exists a path • We end up with eight temporal operators: – AX, EX, AG, EG, AF, EF, AU, EU
CTL Properties Transition System p s 1 s 2 s 3 |= p s 4 |= p s 1 |= p s 2 |= p s 3 Computation Tree s 3 p p s 4 s 3 |= EX p s 3 |= EX p s 3 |= AX p s 3 |= EG p s 3 |= EG p s 3 |= AF p s 3 |= EF p s 3 |= AF p s 4 p s 3 p p s 4. . . s 1 s 2 s 3 p s 1. . . p s 4. . . s 1. . .
CTL Model Checking • Given a transition system T= (S, I, R) and a CTL property p T |= p iff for all initial state s I, s |= p Model checking problem: Given a transition system T and a CTL property p, determine if T is a model for p (i. e. , if T |=p) Complexity: O(|f| (|S|+|R|)) For example: T |=? AG ( (pc 1=c pc 2=c)) T |=? AG(pc 1=w AF(pc 1=c)) AG(pc 2=w AF(pc 2=c)) • Question: Are CTL and LTL equivalent?
CTL vs. LTL • CTL and LTL are not equivalent – There are properties that can be expressed in LTL but cannot be expressed in CTL • For example: FG p – There are properties that can be expressed in CTL but cannot be expressed in LTL • For example: AG(EF p) • Hence, expressive power of CTL and LTL are not comparable
Symbolic Model Checking
![Temporal Properties Fixpoints [Emerson and Clarke 80] Here are some interesting CTL equivalences: AG](https://present5.com/presentation/fd0411d4f6aa74b82b17c4d9f0c18a76/image-42.jpg "Temporal Properties Fixpoints [Emerson and Clarke 80] Here are some interesting CTL equivalences: AG")
Temporal Properties Fixpoints [Emerson and Clarke 80] Here are some interesting CTL equivalences: AG p = p AX AG p EG p = p EX EG p AF p = p AX AF p EF p = p EX EF p p AU q = q (p AX (p AU q)) p EU q = q (p EX (p EU q)) Note that we wrote the CTL temporal operators in terms of themselves and EX and AX operators
Fixpoint Characterizations Fixpoint Characterization Equivalences AG p = y. p AX y EG p = y. p EX y AG p = p AX AG p EG p = p EX EG p AF p = y. p AX y EF p = y. p EX y AF p = p AX AF p EF p = p EX EF p p AU q = y. q (p AX (y)) p EU q = y. q (p EX (y)) p AU q=q (p AX (p AU q)) p EU q = q (p EX (p EU q))
Least Fixpoint Given a monotonic function F, the least fixpoint y. F y is the limit of the following sequence (assuming F is continuous): , F 2 , F 3 , . . . If S is finite, then we can compute the least fixpoint using the above sequence
EF Fixpoint Computation EF p = y. p EX y is the limit of the sequence: , p EX(p EX )) , . . . which is equivalent to , p, p EX p , p EX (p) ) , . . .
EF Fixpoint Computation p s 1 s 2 s 3 s 4 p Start 1 st iteration p EX = {s 1, s 4} EX( )= {s 1, s 4} ={s 1, s 4} 2 nd iteration p EX(p EX ) = {s 1, s 4} EX({s 1, s 4})= {s 1, s 4} {s 3}={s 1, s 3, s 4} 3 rd iteration p EX(p EX )) = {s 1, s 4} EX({s 1, s 3, s 4})= {s 1, s 4} {s 2, s 3, s 4}={s 1, s 2, s 3, s 4} 4 th iteration p EX(p EX ))) = {s 1, s 4} EX({s 1, s 2, s 3, s 4})= {s 1, s 4} {s 1, s 2, s 3, s 4} = {s 1, s 2, s 3, s 4}
EF Fixpoint Computation EF(p) states that can reach p p p EX(p) EX(EX(p)) . . . • • • EF(p)
Greatest Fixpoint Given a monotonic function F, the greatest fixpoint y. F y is the limit of the following sequence (assuming F is continuous): S, F 2 S, F 3 S, . . . If S is finite, then we can compute the greatest fixpoint using the above sequence
EG Fixpoint Computation Similarly, EG p = y. p EX y is the limit of the sequence: S, p EX(p EX S)) , . . . which is equivalent to S, p, p EX p , p EX (p) ) , . . .
EG Fixpoint Computation p s 1 s 2 s 3 p s 4 p Start S = {s 1, s 2, s 3, s 4} 1 st iteration p EX S = {s 1, s 3, s 4} EX({s 1, s 2, s 3, s 4})= {s 1, s 3, s 4} {s 1, s 2, s 3, s 4}={s 1, s 3, s 4} 2 nd iteration p EX(p EX S) = {s 1, s 3, s 4} EX({s 1, s 3, s 4})= {s 1, s 3, s 4} {s 2, s 3, s 4}={s 3, s 4} 3 rd iteration p EX(p EX S)) = {s 1, s 3, s 4} EX({s 3, s 4})= {s 1, s 3, s 4} {s 2, s 3, s 4}={s 3, s 4}
EG Fixpoint Computation EG(p) states that can avoid reaching p p EX(p) EX(EX(p)) . . . • • • EG(p)
![Symbolic Model Checking [Mc. Millan et al. LICS 90] • Basic idea: Represent sets](https://present5.com/presentation/fd0411d4f6aa74b82b17c4d9f0c18a76/image-52.jpg "Symbolic Model Checking [Mc. Millan et al. LICS 90] • Basic idea: Represent sets")
Symbolic Model Checking [Mc. Millan et al. LICS 90] • Basic idea: Represent sets of states and the transition relation as Boolean logic formulas • Fixpoint computation becomes formula manipulation – pre-condition (EX) computation: Existential variable elimination – conjunction (intersection), disjunction (union) and negation (set difference), and equivalence check • Use an efficient data structure for boolean logic formulas – Binary Decision Diagrams (BDDs)
Symbolic Pre-condition Computation • Remember the function EX : 2 S which is defined as: EX(p) = { s | (s, s’) R and s’ p } • We can symbolically compute pre as follows EX(p) V’ R p[V’ / V] – V : current-state boolean variables – V’ : next-state boolean variables – p[V’ / V] : rename variables in p by replacing currentstate variables with the corresponding next-state variables – V’ f : existentially quantify out all the variables in V’ from f
An Extremely Simple Example Variables: x, y: boolean Set of states: S = {(F, F), (F, T), (T, F), (T, T)} S True F, F T, F F, T T, T Initial condition: I x y Transition relation (negates one variable at a time): R x’= x y’=y x’=x y’= y (= means )
An Extremely Simple Example Given p x y, compute EX(p) F, F T, F F, T T, T EX(p) V’ R p[V’ / V] V’ R x’ y’ V’ (x’= x y’=y x’=x y’= y ) x’ y’ V’ (x’= x y’=y) x’ y’ (x’=x y’= y) x’ y’ V’ x y x’ y’ x y EX(x y) x y In other words EX({(T, T)}) {(F, T), (T, F)}
An Extremely Simple Example 3 F, F T, F Let’s compute EF(x y) 2 1 F, T The fixpoint sequence is False, x y EX(x y) , x y EX (x y EX(x y)) , . . . If we do the EX computations, we get: False, x y x y, True 0 1 2 3 EF(x y) True In other words EF({(T, T)}) {(F, F), (F, T), (T, F), (T, T)}
An Extremely Simple Example • Based on our results, for our extremely simple transition system T=(S, I, R) we have I EF(x y) hence: T |= EF(x y) (i. e. , there exists a path from each initial state where eventually x and y both become true at the same time) I EX(x y) hence: T |= EX(x y) (i. e. , there does not exist a path from each initial state where in the next state x and y both become true)
An Extremely Simple Example • Let’s try one more property AF(x y) • To check this property we first convert it to a formula which uses only the temporal operators in our basis: AF(x y) EG( (x y)) If we can find an initial state which satisfies EG( (x y)), then we know that the transition system T, does not satisfy the property AF(x y)
An Extremely Simple Example T, F 1 F, T Let’s compute EG( (x y)) F, F T, T The fixpoint sequence is 0 True, x y, ( x y) EX( x y) , … If we do the EX computations, we get: True, x y, 0 1 2 EG( (x y)) x y Since I EG( (x y)) we conclude that T |= AF(x y)
Symbolic CTL Model Checking Algorithm • Translate the formula to a formula which uses the basis – EX p, EG p, p EU q • Atomic formulas can be interpreted directly on the state representation • For EX p compute the precondition using existential variable elimination as we discussed • For EG and EU compute the fixpoints iteratively
![SMV [Mc. Millan 93] • • BDD-based symbolic model checker Finite state Temporal logic:](https://present5.com/presentation/fd0411d4f6aa74b82b17c4d9f0c18a76/image-61.jpg "SMV [Mc. Millan 93] • • BDD-based symbolic model checker Finite state Temporal logic:")
SMV [Mc. Millan 93] • • BDD-based symbolic model checker Finite state Temporal logic: CTL Focus: hardware verification – Later applied to software specifications, protocols, etc. • SMV has its own input specification language – concurrency: synchronous, asynchronous – shared variables – boolean and enumerated variables – bounded integer variables (binary encoding) • SMV is not efficient for integers, but that can be fixed – fixed size arrays
SMV Language • An SMV specification consists of a set of modules (one of them must be called main) • Modules can have access to shared variables • Modules can be composed asynchronously using the process keyword • Module behaviors can be specified using the ASSIGN statement which assigns values to next values of variables in parallel • Module behaviors can also be specified using the TRANS statements which allow specification of the transition relation as a logic formula where next state values are identified using the next keyword
Example Mutual Exclusion Protocol Two concurrently executing processes are trying to enter a critical section without violating mutual exclusion Process 1: while (true) { out: a : = true; turn : = true; wait: await (b = false or turn = false); cs: a : = false; } || Process 2: while (true) { out: b : = true; turn : = false; wait: await (a = false or turn); cs: b : = false; }
Example Mutual Exclusion Protocol in SMV MODULE process 1(a, b, turn) VAR pc: {out, wait, cs}; ASSIGN init(pc) : = out; next(pc) : = case pc=out : wait; pc=wait & (!b | !turn) : cs; pc=cs : out; 1 : pc; esac; next(turn) : = case pc=out : 1; 1 : turn; esac; next(a) : = case pc=out : 1; pc=cs : 0; 1 : a; esac; next(b) : = b; FAIRNESS running MODULE process 2(a, b, turn) VAR pc: {out, wait, cs}; ASSIGN init(pc) : = out; next(pc) : = case pc=out : wait; pc=wait & (!a | turn) : cs; pc=cs : out; 1 : pc; esac; next(turn) : = case pc=out : 0; 1 : turn; esac; next(b) : = case pc=out : 1; pc=cs : 0; 1 : b; esac; next(a) : = a; FAIRNESS running
Example Mutual Exclusion Protocol in SMV MODULE main VAR a : boolean; b : boolean; turn : boolean; p 1 : process 1(a, b, turn); p 2 : process 2(a, b, turn); SPEC AG(!(p 1. pc=cs & p 2. pc=cs)) -- AG(p 1. pc=wait -> AF(p 1. pc=cs)) & AG(p 2. pc=wait -> AF(p 2. pc=cs)) Here is the output when I run SMV on this example to check the mutual exclusion property % smv mutex. smv -- specification AG (!(p 1. pc = cs & p 2. pc = cs)) is true resources used: user time: 0. 01 s, system time: 0 s BDD nodes allocated: 692 Bytes allocated: 1245184 BDD nodes representing transition relation: 143 + 6
Example Mutual Exclusion Protocol in SMV The output for the starvation freedom property: % smv mutex. smv -- specification AG (p 1. pc = wait -> AF p 1. pc = cs) & AG. . . is true resources used: user time: 0 s, system time: 0 s BDD nodes allocated: 1251 Bytes allocated: 1245184 BDD nodes representing transition relation: 143 + 6
Example Mutual Exclusion Protocol in SMV Let’s insert an error change pc=wait & (!b | !turn) : cs; to pc=wait & (!b | turn) : cs;
% smv mutex. smv -- specification AG (!(p 1. pc = cs & p 2. pc = cs)) is false -- as demonstrated by the following execution sequence state 1. 1: a = 0 b = 0 turn = 0 p 1. pc = out p 2. pc = out [stuttering] state 1. 2: [executing process p 2] state 1. 3: b = 1 p 2. pc = wait [executing process p 2] state 1. 4: p 2. pc = cs [executing process p 1] state 1. 5: a = 1 turn = 1 p 1. pc = wait [executing process p 1] state 1. 6: p 1. pc = cs [stuttering] resources used: user time: 0. 01 s, system time: 0 s BDD nodes allocated: 1878 Bytes allocated: 1245184 BDD nodes representing transition relation: 143 + 6
Symbolic Model Checking with BDDs • BDDs are used as a data structure for encoding trust sets of Boolean logic formulas in symbolic model checking • One can use BDD-based symbolic model checking for any finite state system using a Boolean encoding of the state space and the transition relation • Why are we using symbolic model checking? – We hope that the symbolic representations will be more compact than the explicit state representation on the average – In the worst case we may not gain anything
Problems with BDDs • The BDD for the transition relation could be huge – Remember that the BDD could be exponential in the number of disjuncts and conjuncts – Since we are using a Boolean encoding there could be a large number of conjuncts and disjuncts • The EX computation could result in exponential blow-up – Exponential in the number of existentially quantified variables
Heuristics • Instead of computing a monolithic BDD for the whole transition system partition the transition relation in order to keep the BDD size small • Use good variable ordering in order to keep the BDD sizes small – Use heuristics to find good variable orderings, – Use dynamic variable ordering heuristics that change the variable ordering dynamically if the BDD size grows too much • Use other data structures (such as multi-terminal decision diagrams)
Counter-Example Generation • Remember: Given a transition system T= (S, I, R) and a CTL property p T |= p iff for all initial state s I, s |= p • Verification vs. Falsification – Verification: • Show: initial states truth set of p – Falsification: • Find: a state initial states truth set of p • Generate a counter-example starting from that state • The ability to find counter-examples is one of the biggest strengths of the model checkers
An Example • We want to check the property AG(p) • We compute the fixpoint for EF( p) • We check if the intersection of the set of initial states I and the truth set of EF( p) is empty – If it is not empty we generate a counter-example path starting from the intersection EF( p) states that can reach p p • In order to generate the p counter-example path, save the fixpoint iterations. • After the fixpoint computation converges, do a second pass to generate the counter-example path. EX( p) EX(EX( p)) . . . • • • I • • • EF( p) Generate a counter-example path starting from a state here
Automata Theoretic Model Checking
![LTL Properties Büchi automata [Vardi and Wolper LICS 86] • Büchi automata: Finite state](https://present5.com/presentation/fd0411d4f6aa74b82b17c4d9f0c18a76/image-75.jpg "LTL Properties Büchi automata [Vardi and Wolper LICS 86] • Büchi automata: Finite state")
LTL Properties Büchi automata [Vardi and Wolper LICS 86] • Büchi automata: Finite state automata that accept infinite strings – The better known variant of finite state automata accept finite strings (used in lexical analysis for example) • A Büchi automaton accepts a string when the corresponding run visits an accepting state infinitely often – Note that an infinite run never ends, so we cannot say that an accepting run ends at an accepting state • LTL properties can be translated to Büchi automata – The automaton accepts a path if and only if the path satisfies the corresponding LTL property
LTL Properties Büchi automata true Gp p p true Fp G (F p) p p The size of the property automaton can be exponential in the size of the LTL formula (recall the complexity of LTL model checking)
Büchi Automata: Language Emptiness Check • Given a Buchi automaton, one interesting question is: – Is the language accepted by the automaton empty? • i. e. , does it accept any string? • A Büchi automaton accepts a string when the corresponding run visits an accepting state infinitely often • To check emptiness: – Look for a cycle which contains an accepting state and is reachable from the initial state • Find a strongly connected component that contains an accepting state, and is reachable from the initial state – If no such cycle can be found the language accepted by the automaton is empty
LTL Model Checking • Generate the property automaton from the negated LTL property • Generate the product of the property automaton and the transition system • Show that there is no accepting cycle in the product automaton (check language emptiness) – i. e. , show that the intersection of the paths generated by the transition system and the paths accepted by the (negated) property automaton is empty • If there is a cycle, it corresponds to a counterexample behavior that demonstrates the bug
LTL Model Checking Example transition system p, q q Negation of the property G q F q 1 2 Property to be verified Gq 3 p Each state is labeled with the propositions that hold in that state Property automaton for the negated property true q q Equivalently {q}, {p, q} 1 , {p}, {q}, {p, q} 2
Transition System to Buchi Automaton Translation Example transition system p, q Corresponding Buchi automaton i 1 {p, q} 1 q 2 3 p {q} {p, q} {q} 2 Each state is labeled with the propositions that hold in that state 3 {p}
Buchi automaton for the transition system (every state is accepting) Product automaton 1, 1 {p, q} 2, 1 {p, q} 2 {q} 3, 1 {q} 3 4 {p} {q}, {p, q} 1 {q} 3, 2 4, 2 Property Automaton , {p} {p, q} , {p}, {q}, {p, q} 2 {p} Accepting cycle: (1, 1), (2, 1), (3, 1), ((4, 2), (3, 2)) Corresponds to a counter-example path for the property G q
![SPIN [Holzmann • • 91, TSE 97] Explicit state model checker Finite state Temporal](https://present5.com/presentation/fd0411d4f6aa74b82b17c4d9f0c18a76/image-82.jpg "SPIN [Holzmann • • 91, TSE 97] Explicit state model checker Finite state Temporal")
SPIN [Holzmann • • 91, TSE 97] Explicit state model checker Finite state Temporal logic: LTL Input language: PROMELA – Asynchronous processes – Shared variables – Message passing through (bounded) communication channels – Variables: boolean, char, integer (bounded), arrays (fixed size) – Structured data types
SPIN Verification in SPIN • Uses the LTL model checking approach • Constructs the product automaton on-the-fly – It is possible to find an accepting cycle (i. e. a counterexample) without constructing the whole state space • Uses a nested depth-first search algorithm to look for an accepting cycle • Uses various heuristics to improve the efficiency of the nested depth first search: – partial order reduction – state compression
Example Mutual Exclusion Protocol Two concurrently executing processes are trying to enter a critical section without violating mutual exclusion Process 1: while (true) { out: a : = true; turn : = true; wait: await (b = false or turn = false); cs: a : = false; } || Process 2: while (true) { out: b : = true; turn : = false; wait: await (a = false or turn); cs: b : = false; }
Example Mutual Exclusion Protocol in Promela #define cs 1 process 1@cs #define cs 2 process 2@cs #define wait 1 process 1@wait #define wait 2 process 2@wait #define true 1 #define false 0 bool a; bool b; bool turn; proctype process 1() { out: a = true; turn = true; wait: (b == false || turn == false); cs: a = false; goto out; } proctype process 2() { out: b = true; turn = false; wait: (a == false || turn == true); cs: b = false; goto out; } init { run process 1(); run process 2() }
![Property automaton generation % spin -f "! [] (! (cs 1 && cs 2))“](https://present5.com/presentation/fd0411d4f6aa74b82b17c4d9f0c18a76/image-86.jpg "Property automaton generation % spin -f \"! [] (! (cs 1 && cs 2))“")
Property automaton generation % spin -f "! [] (! (cs 1 && cs 2))“ never { T 0_init: /* ! [] (! (cs 1 && cs 2)) */ if : : ((cs 1) && (cs 2)) -> goto accept_all : : (1) -> goto T 0_init fi; accept_all: skip } % spin -f "!([](wait 1 -> <>(cs 1)))“ never { T 0_init: • Input formula “[]” means G “<>” means F • “spin –f” option generates a Buchi automaton for the input LTL formula /* !([](wait 1 -> <>(cs 1))) */ if : : ( !((cs 1)) && (wait 1) ) -> goto accept_S 4 : : (1) -> goto T 0_init fi; accept_S 4: if : : (! ((cs 1))) -> goto accept_S 4 fi; } Concatanate the generated never claims to the end of the specification file
SPIN • “spin –a mutex. spin” generates a C program “pan. c” from the specification file – This C program implements the on-the-fly nested-depth first search algorithm – You compile “pan. c” and run it to the model checking • Spin generates a counter-example trace if it finds out that a property is violated
%mutex -a warning: for p. o. reduction to be valid the never claim must be stutter-invariant (never claims generated from LTL formulae are stutter-invariant) (Spin Version 4. 2. 6 -- 27 October 2005) + Partial Order Reduction Full statespace search for: never claim assertion violations acceptance cycles invalid end states + + (if within scope of claim) + (fairness disabled) - (disabled by never claim) State-vector 28 byte, depth reached 33, errors: 0 22 states, stored 15 states, matched 37 transitions (= stored+matched) 0 atomic steps hash conflicts: 0 (resolved) 2. 622 memory usage (Mbyte) unreached in proctype process 1 line 18, state 6, "-end-" (1 of 6 states) unreached in proctype process 2 line 27, state 6, "-end-" (1 of 6 states) unreached in proctype : init: (0 of 3 states)
Problems/Heuristics for explicit state model checking • State space explosion: Number of states can be exponential in the number of variables and concurrent components • Heuristics used by Spin: – On the fly checking: use a depth first search that computes the product of the property automaton and the transition relation while looking for a violation – Bit-state hashing • do not store the full state information • might skip some unvisited states so it is not sound – Partial order reduction • only explore a representative subset of interleavings among the concurrent processes • this can be done in a sound manner
Software Verification Using Explicit State Model Checking with Java Path Finder
Java Path Finder • Program checker for Java • Properties to be verified – Properties can be specified as assertions • static checking of assertions – It can also verify LTL properties • Implements both depth-first and breadth-first search and looks for assertion violations statically • Uses static analysis techniques to improve the efficiency of the search • Requires a complete Java program • It can only handle pure Java, it cannot handle native code
Java Path Finder, First Version • First version – A translator from Java to PROMELA – Use SPIN for model checking • Since SPIN cannot handle unbounded data – Restrict the program to finite domains • A fixed number of objects from each class • Fixed bounds for array sizes • Does not scale well if these fixed bounds are increased • Java source code is required for translation
Java Path Finder, Current Version • Current version of the JPF has its own virtual machine: JPFJVM – Executes Java bytecode • can handle pure Java but can not handle native code – Has its own garbage collection – Stores the visited states and stores current path – Offers some methods to the user to optimize verification • Traversal algorithm – Traverses the state-graph of the program – Tells JPF-JVM to move forward, backward in the state space, and evaluate the assertion • The rest of the slides are on the current version of JPF: W. Visser, K. Havelund, G. Brat, S. Park and F. Lerda. "Model Checking Programs. " Automated Software Engineering Journal Volume 10, Number 2, April 2003.
Storing the States • JPF implements a depth-first search on the state space of the given Java program – To do depth first search we need to store the visited states • There also verification tools which use stateless search such as Verisoft • The state of the program consists of – information for each thread in the Java program • a stack of frames, one for each method called – the static variables in classes • locks and fields for the classes – the dynamic variables (fields) in objects • locks and fields for the objects
Storing States Efficiently • Since different states can have common parts each state is divided to a set of components which are stored separately – locks, frames, fields • Keep a pool for each component – A table of field values, lock values, frame values • Instead of storing the value of a component in a state store an index at which the component is stored in the table in the state – The whole state becomes an integer vector • JPF collapses states to integer vectors using this idea
State Space Explosion • State space explosion if one of the major challenges in model checking • The idea is to reduce the number of states that have to be visited during state space exploration • Here are some approaches used to attack state space explosion – Symmetry reduction • search equivalent states only once – Partial order reduction • do not search thread interleavings that generate equivalent behavior – Abstraction • Abstract parts of the state to reduce the size of the state space
Symmetry Reduction • Some states of the program may be equivalent – Equivalent states should be searched only once • Some states may differ only in their memory layout, the order objects are created, etc. – these may not have any effect on the behavior of the program • JPF makes sure that the order which the classes are loaded does not effect the state – There is a canonical ordering of the classes in the memory
Symmetry Reduction • A similar problem occurs for location of dynamically allocated objects in the heap – If we store the memory location as the state, then we can miss equivalent states which have different memory layouts – JPF tries to remedy this problem by storing some information about the new statements that create an object and the number of times they are executed
Partial Order Reduction • Statements of concurrently executing threads can generate many different interleavings – all these different interleavings are allowable behavior of the program • A model checker has to check all possible interleavings that the behavior of the program is correct in all cases – However different interleavings may generate equivalent behaviors • In such cases it is sufficient to check just one interleaving without exhausting all the possibilities – This is called partial order reduction
state space search generates 258 states with symmetry reduction: 105 states with partial order reduction: 68 states with symmetry reduction + partial order reduction : 38 states class S 1 { int x; } class First. Task extends Thread { public void run() { S 1 s 1; int x = 1; s 1 = new S!(); x = 3; }} class S 2 { int y; } class Second. Task extends Thread { public void run() { S 2 s 2; int x = 1; s 2 = new S 2(); x = 3; }} class Main { public static void main(String[] args) { First. Task task 1 = new First. Task(); Second. Task task 2 = new Second. Task(); task 1. statr(); task 2. start(); }}
Static Analysis • JPF uses following static analysis techniques for reducing the state space: – slicing – partial evaluation • Given a slicing criterion slicing reduces the size of a program by removing the parts of the program that have no effect on the slicing criterion – A slicing criterion could be a program point – Program slices are computed using dependency analysis • Partial evaluation propagates constant values and simplifies expressions
Abstraction vs. Restriction • JPF also uses abstraction techniques such as predicate abstraction to reduce the state space • Still, in order to check a program with JPF, typically, you need to restrict the domains of the variables, the sizes of the arrays, etc. • Abstraction over approximates the program behavior – causes spurious counter-examples • Restriction under approximates the program behavior – may result in missed errors • If both under and over approximation techniques are used then the resulting verification technique is neither sound nor complete – However, it is still useful as a debugging tool and it is helpful in finding bugs
JPF Java Modeling Primitives • Atomicity (used to reduce the state space) – begin. Atomic(), end. Atomic() • Nondeterminism (used to model non-determinism caused by abstraction) – int random(int); boolean random. Bool(); Object random. Object(String cname); • Properties (used to specify properties to be verified) – Assert. True(boolean cond)
Annotated Java Code for a Reader-Writer Lock import gov. nasa. arc. ase. jpf. jvm. Verify; class Reader. Writer { private int nr; private boolean busy; private Object Condr_enter; private Object Condw_enter; public Reader. Writer() { Verify. begin. Atomic(); nr = 0; busy=false ; Condr_enter =new Object(); Condw_enter =new Object(); Verify. end. Atomic(); } public boolean read_exit(){ boolean result=false; synchronized(this){ nr = (nr - 1); result=true; } Verify. assert. True(!busy || nr==0 ); return result; } private boolean Guarded_r_enter(){ boolean result=false; synchronized(this){ if(!busy){nr = (nr + 1); result=true; }} return result; } public void read_enter(){ synchronized(Condr_enter){ while (! Guarded_r_enter()){ try{Condr_enter. wait(); } catch(Interrupted. Exception e){} }} Verify. assert. True(!busy || nr==0 ); } private boolean Guarded_w_enter(){…} public void write_enter(){…} public boolean write_exit(){…} };
JPF Output >java gov. nasa. arc. ase. jpf. jvm. Main rwmain JPF 2. 1 - (C) 1999, 2002 RIACS/NASA Ames Research Center JVM 2. 1 - (C) 1999, 2002 RIACS/NASA Ames Research Center Loading class gov. nasa. arc. ase. jpf. jvm. reflection. Java. Lang. Object. Reflection Loading class gov. nasa. arc. ase. jpf. jvm. reflection. Java. Lang. Thread. Reflection =============== No Errors Found =============== -----------------States visited : 36, 999 Transitions executed : 68, 759 Instructions executed: 213, 462 Maximum stack depth : 9, 010 Intermediate steps : 2, 774 Memory used : 22. 1 MB Memory used after gc : 14. 49 MB Storage memory : 7. 33 MB Collected objects : 51 Mark and sweep runs : 55, 302 Execution time : 20. 401 s Speed : 3, 370 tr/s ------------------
Example Error Trace 1 error found: Deadlock ============ *** Path to error: *** ============ Steps to error: 2521 Step #0 Thread #0 Step #1 Thread #0 rwmain. java: 4 Reader. Writer monitor=new Reader. Writer(); Step #2 Thread #0 Reader. Writer. java: 10 public Reader. Writer( ) { … Step #2519 Thread #2 Reader. Writer. java: 71 while (! Guarded_w_enter()){ Step #2520 Thread #2 Reader. Writer. java: 73 Condw_enter. wait();
Bounded Model Checking
Bounded Model Checking • Represent sets of states and the transition relation as Boolean logic formulas • Instead of computing the fixpoints, unroll the transition relation up to certain fixed bound and search for violations of the property within that bound • Transform this search to a Boolean satisfiability problem and solve it using a SAT solver
What Can We Guarantee? • Note that in bounded model checking we are checking only for bounded paths (paths which have at most k+1 distinct states) – So if the property is violated by only paths with more than k+1 distinct states, we would not find a counterexample using bounded model checking – Hence if we do not find a counter-example using bounded model checking we are not sure that the property holds • However, if we find a counter-example, then we are sure that the property is violated since the generated counterexample is never spurious (i. e. , it is always a concrete counter-example)
Bounded Model Checking: Proving Correctness • One can also show that given an LTL property f, if E f holds for a finite state transition system, then E f also holds for that transition system using bounded semantics for some bound k • So if we keep increasing the bound, then we are guaranteed to find a path that satisfies the formula – And, if we do not find a path that satisfies the formula, then we decide that the formula is not satisfied by the transition system – Is there a problem here?
Proving Correctness • We can modify the bounded model checking algorithm as follows: – Start from an initial bound. – If no counter-examples are found using the current bound, increment the bound and try again. • The problem is: We do not know when to stop
Proving Correctness • If we can find a way to figure out when we should stop then we would be able to provide guarantee of correctness. • There is a way to define a diameter of a transition system so that a property holds for the transition system if and only if it is not violated on a path bounded by the diameter. • So if we do bounded model checking using the diameter of the system as our bound, then we can guarantee correctness if no counter-example is found.
Bounded Model Checking • What are the differences between bounded model checking and BDD-based symbolic model checking? – In bounded model checking we are using a SAT solver instead of a BDD library – In symbolic model checking we do not unroll the transition relation as in bounded model checking – In bounded model checking we do not execute the iterative fixpoint computations as in symbolic model checking – In symbolic model checking for finite state systems both verification and falsification results are guaranteed • In bounded model checking we can only guarantee the falsification results, in order to guarantee the verification results we need to know the diameter of the system
Bounded Model Checking • A bounded model checker needs an efficient SAT solver – z. Chaff SAT solver is one of the most commonly used ones • Most SAT solvers require their input to be in Conjunctive Normal Form (CNF) – So the final formula has to be converted to CNF • Similar to BDD-based symbolic model checking, bounded model checking was also first used for hardware verification • More recently, it has been applied to verification of software
Bounded Model Checking for Software CBMC is a bounded model checker for ANSI-C programs • Handles function calls using inlining • Unwinds the loops a fixed number of times • Allows user input to be modeled using non-determinism – So that a program can be checked for a set of inputs rather than a single input • Allows specification of assertions which are checked using the bounded model checking
Loops • Unwind the loop n times by duplicating the loop body n times – Each copy is guarded using an if statement that checks the loop condition • At the end of the n repetitions an unwinding assertion is added which is the negation of the loop condition – Hence if the loop iterates more than n times in some execution, the unwinding assertion will be violated and we know that we need to increase the bound in order to guarantee correctness • A similar strategy is used for recursive function calls – The recursion is unwound up to a certain bound and then an assertion is generated stating that the recursion does not go any deeper
A Simple Loop Example Original code x=0; while (x < 2) { y=y+x; x++; } Unwinding assertion: Unwinding the loop 3 times x=0; if (x < 2) { y=y+x; x++; } assert (! (x < 2))
From Code to SAT • After eliminating loops and recursion, CBMC converts the input program to the static single assignment (SSA) form – In SSA each variable appears at the left hand side of an assignment only once – This is a standard program transformation that is performed by creating new variables • In the resulting program each variable is assigned a value only once and all the branches are forward branches (there is no backward edge in the control flow graph) • CBMC generates a Boolean logic formula from the program using bit vectors to represent variables
Another Simple Example Original code x=x+y; if (x!=1) x=2; else x++; assert(x<=3); Convert to static single assignment x 1=x 0+y 0; if (x 1!=1) x 2=2; else x 3=x 1+1; x 4=(x 1!=1)? x 2: x 3; assert(x 4<=3); Generate constraints C x 1=x 0+y 0 x 2=2 x 3=x 1+1 (x 1!=1 x 4=x 2 x 1=1 x 4=x 3) P x 4 <= 3 Check if C P is satisfiable, if it is then the assertion is violated C P is converted to boolean logic using a bit vector representation for the integer variables y 0, x 1, x 2, x 3, x 4
Bounded Verification Approaches • What we have discussed above is bounded verification by bounding the number of steps of the execution. • For this approach to work the variable domains also need to be bounded, otherwise we cannot convert the problems to boolean SAT • Bounding the execution steps and bounding the data domain are two orthogonal approaches. – When people say bounded verification it may refer to either of these – When people say bounded model checking it typically refers to bounding the execution steps
Symbolic Software Model Checking with Predicate Abstraction and Counter-Example Guided Abstraction Refinement
Model Checking Programs Using Abstraction • Program model checking tools generally rely on automated abstraction techniques to reduce the state space of the system such as: – Abstract interpretation – Predicate abstraction • If the abstraction is conservative then, if there is no error in the abstracted program we can conclude that there is no error in the original program • In general the problem is to construct a finite state model from the program such that the errors or absence of errors can be demonstrated on the finite state model – Model extraction problem
Model Checking Programs via Abstraction • Bandera – A tool for extracting finite state models from programs – Uses various abstract domains to map the state space of the program to a finite set of states via abstraction • SLAM project at Microsoft Research – Symbolic model checking for C programs – Can handle unbounded recursion but does not handle concurrency – Uses predicate abstraction, counter-example guided abstraction refinement and BDDs
Abstraction (A simplified view) • Abstraction is an effective tool in verification • Given a transition system, we want to generate an abstract transition system which is easier to analyze • However, we want to make sure that – If a property holds in the abstract transition system, it also holds in the original (concrete) transition system
Abstraction (A simplified view) • How do we generate an abstract transition system? • Merge states in the concrete transition system (based on some criteria) – This reduces the number of states, so it should be easier to do verification • Do not eliminate transitions – This will make sure that the paths in the abstract transition system subsume the paths in the concrete transition system
Abstraction (A simplified view) • For every path in the concrete transition system, there is an equivalent path in the abstract transition system – If no path in the abstract transition system violate a property, then no path in the concrete system can violate the property • Using this reasoning we can verify ACTL, LTL and ACTL* properties in the abstract transition system – If the property holds on the abstract transition system, we are sure that the property holds in the concrete transition system – If the property does not hold in the abstract transition system, then we are not sure if the property holds or not in the concrete transition system
Abstraction (A simplified view) • If the property does not hold in the abstract transition system, what can we do? • We can refine the abstract transition system (split some states that we merged) • We have to make sure that the refined transition system is still an abstraction of the concrete transition system • Then, we can recheck the property again on the refined transition system – If the property does not hold again, we can refine again
Predicate Abstraction • An automated abstraction technique which can be used to reduce the state space of a program • The basic idea in predicate abstraction is to remove some variables from the program by just keeping information about a set of predicates about them • For example a predicate such as x = y maybe the only information necessary about variables x and y to determine the behavior of the program – In that case we can just store a boolean variable which corresponds to the predicate x = y and remove variables x and y from the program – Predicate abstraction is a technique for doing such abstractions automatically
Predicate Abstraction • Given a program and a set of predicates, predicate abstraction abstracts the program so that only the information about the given predicates are preserved • The abstracted program adds nondeterminism since in some cases it may not be possible to figure out what the next value of a predicate will be based on the predicates in the given set • One needs an automated theorem prover to compute the abstraction
Predicate Abstraction, A Very Simple Example • Assume that we have two integer variables x, y • We want to abstract the program using a single predicate “x=y” • We will divide the states of the program to two: 1. The states where “x=y” is true 2. The states where “x=y” is false, i. e. , “x y” • We will then merge all the states in the same set – This is an abstraction – Basically, we forget everything except the value of the predicate “x=y”
Predicate Abstraction, A Very Simple Example • We will represent the predicate “x=y” as the boolean variable B in the abstract program – “B=true” will mean “x=y” and – “B=false” will mean “x y” • Assume that we want to abstract the following program which contains only one statement: y : = y+1
Predicate Abstraction, Step 1 • Calculate preconditions based on the predicate {x = y + 1} y : = y + 1 {x = y} precondition for B being true after executing the statement y: =y+1 {x y + 1} y : = y + 1 {x y} precondition for B being false after executing the statement y: =y+1 Using our temporal logic notation we can say something like: {x=y+1} AX{x=y} Again, using our temporal logic notation: {x≠y+1} AX{x≠y}
Predicate Abstraction, Step 2 • Use decision procedures to determine if the predicates used for abstraction imply any of the preconditions x = y + 1 ? No x y x = y + 1 ? No x = y x y + 1 ? Yes x y + 1 ? No
Predicate Abstraction, Step 3 • Generate abstract code Predicate abstraction wrt the predicate “x=y” IF B THEN B : = false ELSE B : = true | false y : = y + 1 1) Compute preconditions 3) Generate abstract code {x = y + 1} y : = y + 1 {x = y} {x y + 1} y : = y + 1 {x y} 2) Check implications x = y + 1 ? No x y x = y + 1 ? No x = y x y + 1 ? Yes x y + 1 ? No
Model Checking Push-down Automata A class of infinite state systems for which model checking is decidable • Push-down automata: Finite state control + one stack • LTL model checking for push-down automata is decidable • This may sound like a theoretical result but it is the basis of the approach used in SLAM toolkit for model checking C programs – A program with finite data domains which uses recursion can be modeled as a pushdown automaton – A Boolean program generated by predicate abstraction can be represented as a pushdown automaton
Predicate Abstraction + Model Checking Push Down Automata • Predicate abstraction combined with results on model checking pushdown automata led to some promising tools – SLAM project at Microsoft Research for verification of C programs – This tool is being used to verify device drivers at Microsoft • The main idea: – Use predicate abstraction to obtain finite state abstractions of a program – A program with finite data domains and recursion can be modeled as a pushdown automaton – Use results on model checking push-down automata to verify the abstracted (recursive) program
SLAM Toolkit • SLAM toolkit was developed to find errors in windows device drivers – Examples in my slides are from the following paper: • “The SLAM Toolkit”, Thomas Ball and Sriram K. Rajamani, CAV 2001 • Windows device drivers are required to interact with the windows kernel according to certain interface rules • SLAM toolkit has an interface specification language called SLIC (Specification Language for Interface Checking) which is used for writing these interface rules • The SLAM toolkit instruments the driver code with assertions based on these interface rules
A SLIC Specification for a Lock SLIC specification: state { enum { Unlocked=0, Locked=1 } state = Unlocked; } Ke. Acquire. Spin. Lock. return { if (state == Locked) abort; else state = Locked; } Ke. Release. Spin. Lock. return { if (state == Unlocked) abort; else state = Unlocked; } • This specification states that Ke. Acquire. Spin. Lock has to be called before Ke. Release. Spin. Lock is called, • and Ke. Acquire. Spin. Lock cannot be called back to back before a Ke. Release. Spin. Lock is called, and vice versa
A SLIC Specification for a Lock SLIC specification: state { enum { Unlocked=0, Locked=1 } state = Unlocked; } Ke. Acquire. Spin. Lock. return { if (state == Locked) abort; else state = Locked; } Ke. Release. Spin. Lock. return { if (state == Unlocked) abort; else state = Unlocked; } Generated C Code: enum { Unlocked=0, Locked=1 } state = Unlocked; void slic_abort() { SLIC_ERROR: ; } void Ke. Acquire. Spin. Lock_return() { if (state == Locked) slic_abort(); else state = Locked; } void Ke. Release. Spin. Lock_return { if (state == Unlocked) slic_abort(); else state = Unlocked; }
An Example void example() { Instrumented code do { Ke. Acquire. Spin. Lock(); A: Ke. Acquire. Spin. Lock_return(); void example() { n. Packets. Old = n. Packets; do { req = dev. Ext->WLHV; Ke. Acquire. Spin. Lock(); if(req && req->status){ n. Packets. Old = n. Packets; dev. Ext->WLHV = req->Next; req = dev. Ext->WLHV; Ke. Release. Spin. Lock(); if(req && req->status){ B: Ke. Release. Spin. Lock_return(); dev. Ext->WLHV = req->Next; irp = req->irp; Ke. Release. Spin. Lock(); if(req->status > 0){ irp = req->irp; irp->Io. S. Status = SUCCESS; if(req->status > 0){ irp->Io. S. Info = req->Status; irp->Io. S. Status = SUCCESS; } else { irp->Io. S. Info = req->Status; irp->Io. S. Status = FAIL; } irp->Io. S. Info = req->Status; Smart. Dev. Free. Block(req); } Io. Complete. Request(irp); Smart. Dev. Free. Block(req); n. Packets++; Io. Complete. Request(irp); } n. Packets++; } while(n. Packets!=n. Packets. Old); } Ke. Release. Spin. Lock(); } while(n. Packets!=n. Packets. Old); C: Ke. Release. Spin. Lock_return(); Ke. Release. Spin. Lock(); } }
Boolean Programs • After instrumenting the code, the SLAM toolkit converts the instrumented C program to a Boolean program using predicate abstraction • The Boolean program consists of only Boolean variables – The Boolean variables in the Boolean program are the predicates that are used during predicate abstraction • The Boolean program can have unbounded recursion
Boolean Programs C Code: enum { Unlocked=0, Locked=1 } state = Unlocked; void slic_abort() { SLIC_ERROR: ; } void Ke. Acquire. Spin. Lock_return() { if (state == Locked) slic_abort(); else state = Locked; } void Ke. Release. Spin. Lock_return { if (state == Unlocked) slic_abort(); else state = Unlocked; } Boolean Program: decl {state==Locked}, {state==Unlocked} : = F, T; void slic_abort() begin SLIC_ERROR: skip; end void Ke. Acquire. Spin. Lock_return() begin if ({state==Locked}) slic_abort(); else {state==Locked}, {state==Unlocked} : = T, F; end void Ke. Release. Spin. Lock_return() begin if ({state == Unlocked}) slic_abort(); else {state==Locked}, {state==Unlocked} : = F, T; end
C Code: void example() { do { Ke. Acquire. Spin. Lock(); A: Ke. Acquire. Spin. Lock_return(); n. Packets. Old = n. Packets; req = dev. Ext->WLHV; if(req && req->status){ dev. Ext->WLHV = req->Next; Ke. Release. Spin. Lock(); B: Ke. Release. Spin. Lock_return(); irp = req->irp; if(req->status > 0){ irp->Io. S. Status = SUCCESS; irp->Io. S. Info = req->Status; } else { irp->Io. S. Status = FAIL; irp->Io. S. Info = req->Status; } Smart. Dev. Free. Block(req); Io. Complete. Request(irp); n. Packets++; } } while(n. Packets!=n. Packets. Old); Ke. Release. Spin. Lock(); C: Ke. Release. Spin. Lock_return(); } Boolean Program: void example() begin do skip; A: Ke. Acquire. Spin. Lock_return(); skip; if (*) then skip; B: Ke. Release. Spin. Lock_return(); skip; if (*) then skip; else skip; fi while (*); skip; C: Ke. Release. Spin. Lock_return(); end
Abstraction Preserves Correctness • The Boolean program that is generated with predicate abstraction is non-deterministic. – Non-determinism is used to handle the cases where the predicates used during predicate abstraction are not sufficient enough to determine which branch will be taken • If we find no error in the generated abstract Boolean program then we are sure that there are no errors in the original program – The abstract Boolean program allows more behaviors than the original program due to non-determinism. – Hence, if the abstract Boolean program is correct then the original program is also correct.
Counter-Example Guided Abstraction Refinement • However, if we find an error in the abstract Boolean program this does not mean that the original program is incorrect. – The erroneous behavior in the abstract Boolean program could be an infeasible execution path that is caused by the non-determinism introduced during abstraction. • Counter-example guided abstraction refinement is a technique used to iteratively refine the abstract program in order to remove the spurious counter-example traces
Counter-Example Guided Abstraction Refinement The basic idea in counter-example guided abstraction refinement is the following: • First look for an error in the abstract program (if there are no errors, we can terminate since we know that the original program is correct) • If there is an error in the abstract program, generate a counter-example path on the abstract program • Check if the generated counter-example path is feasible using a theorem prover. • If the generated path is infeasible add the predicate from the branch condition where an infeasible choice is made to the predicate set and generate a new abstract program.
Counter-Example Guided Abstraction Refinement Boolean Program: Refined Boolean Program: (using the predicate (n. Packets = npackets. Old)) void example() the boolean variable b void example() begin represents the predicate begin do (n. Packets = npackets. Old) do skip; A: Ke. Acquire. Spin. Lock_return(); b : = T; skip; if (*) then skip; B: Ke. Release. Spin. Lock_return(); skip; if (*) then skip; else skip; fi fi b : = b ? F : *; skip; fi fi while (!b); while (*); skip; C: Ke. Release. Spin. Lock_return(); end
Counter-Example Guided Abstraction Refinement • Using counter-example guided abstraction refinement we are iteratively creating more an more refined abstractions • This iterative abstraction refinement loop is not guaranteed to converge for infinite domains – This is not surprising since automated verification for infinite domains is undecidable in general • The challenge in this approach is automatically choosing the right set of predicates for abstraction refinement – This is similar to finding a loop invariant that is strong enough to prove the property of interest
Скачать презентацию A Tutorial on Automated Verification Tevfik Bultan
fd0411d4f6aa74b82b17c4d9f0c18a76.ppt