2002

Software Model Checking
Willem Visser
Research Institute for Advanced Computer Science NASA Ames Research Center
wvisser@email.arc.nasa.gov

Overview
2002
‡ ‡ Introduction to Model Checking Program Model Checking
± Major Trends ± Hardware and Software Model Checking

± A Brief History
‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡

‡ Abstraction ‡ Improved model checking technology SPIN Hand-translations State-less model checking Semi-automated translations Fully automated translations Custom-made model checkers for programs SLAM JPF Summary

± Current Trends

‡ ‡

NASA Case Studies - Remote Agent, DEOS and Mars Rover Future of Software Model Checking
© Willem Visser 2002

24 September 2002

2

2002

Model Checking The Intuition

‡ Calculate whether a system satisfies a certain behavioral property:
± Is the system deadlock free? ± Whenever a packet is sent will it eventually be received?

‡ So it is like testing? No, major difference:
± Look at all possible behaviors of a system

‡ Automatic, if the system is finite-state
± Potential for being a push-button technology ± Almost no expert knowledge required

‡ How do we describe the system? ‡ How do we express the properties? 24 September 2002 © Willem Visser 2002

3

z.y.2002 Labeled State Graph Kripke Structure x Each state represents all variable values and location counters ~p k Each transition represents an execution step in the system p y ~ p h h ~p z ~p The labels represent predicates in each state e.R.h}.{x.L) 24 September 2002 © Willem Visser 2002 4 .g.~p}. (x = 5) K = ({p.k.{x}.

Property Specifications
2002

‡ Temporal Logic
± Express properties of event orderings in time ± e.g. ³Always´ when a packet is sent it will ³Eventually´ be received

‡ Linear Time
± Every moment has a unique successor ± Infinite sequences (words) ± Linear Time Temporal Logic (LTL)

‡ Branching Time
± Every moment has several successors ± Infinite tree ± Computation Tree Logic (CTL)

24 September 2002

© Willem Visser 2002

5

Safety and Liveness
2002

‡ Safety properties
± Invariants, deadlocks, reachability, etc. ± Can be checked on finite traces ± ³something bad never happens´

‡ Liveness Properties
± Fairness, response, etc. ± Infinite traces ± ³something good will eventually happen´
24 September 2002 © Willem Visser 2002

6

Mutual Exclusion Example
2002 ‡ Two process mutual exclusion with shared semaphore ‡ Each process has three states ‡ Non-critical (N) ‡ Trying (T) ‡ Critical (C) ‡ Semaphore can be available (S0) or taken (S1) ‡ Initially both processes are in the Non-critical state and the semaphore is available --- N1 N2 S0 N1 p T1 T1 › S0 p C1 › S1 C1 p N1 › 24 September 2002 S0

||

N2 p T2 T2 › S0 p C2 › S1 C2 p N2 › S0
7

© Willem Visser 2002

Mutual Exclusion Example 2002 N1N2S0 T1N2S0 C1N2S1 T1T2S0 N1T2S0 N1C2S1 C1T2S1 T1C2S1 K AG EF (N1 › N2 › S0) No matter where you are there is always a way to get to the initial state 24 September 2002 © Willem Visser 2002 8 .

Mutual Exclusion Example 2002 N1N2S0 T1N2S0 C1N2S1 T1T2S0 N1T2S0 N1C2S1 C1T2S1 T1C2S1 K AG EF (N1 › N2 › S0) 24 September 2002 © Willem Visser 2002 9 .

Mutual Exclusion Example 2002 N1N2S0 T1N2S0 C1N2S1 T1T2S0 N1T2S0 N1C2S1 C1T2S1 T1C2S1 K AG EF (N1 › N2 › S0) 24 September 2002 © Willem Visser 2002 10 .

Mutual Exclusion Example 2002 N1N2S0 T1N2S0 C1N2S1 T1T2S0 N1T2S0 N1C2S1 C1T2S1 T1C2S1 K AG EF (N1 › N2 › S0) 24 September 2002 © Willem Visser 2002 11 .

Mutual Exclusion Example 2002 N1N2S0 T1N2S0 C1N2S1 T1T2S0 N1T2S0 N1C2S1 C1T2S1 T1C2S1 K AG EF (N1 › N2 › S0) 24 September 2002 © Willem Visser 2002 12 .

Mutual Exclusion Example 2002 N1N2S0 T1N2S0 C1N2S1 T1T2S0 N1T2S0 N1C2S1 C1T2S1 T1C2S1 K AG EF (N1 › N2 › S0) 24 September 2002 © Willem Visser 2002 13 .

The system satisfies the specification provided all the initial states are in the set.R. We often write: M f 24 September 2002 © Willem Visser 2002 14 . find the set of states in S that satisfy f: { s  S | M. some states of the concurrent system are designated as initial states.s f} ‡ Normally.Model Checking 2002 ‡ Given a Kripke structure M = (S.L) that represents a finite-state concurrent system and a temporal logic formula f expressing some desired specification.

but long execution paths can also be handled ± Can handle dynamic creation of objects/threads ± Mostly used in software ‡ Characteristics 24 September 2002 © Willem Visser 2002 . hence doesn¶t deal well with dynamic creation of objects/threads ± Mostly used in hardware 15 ± states are enumerated on-the-fly ± Forwards analysis ± Stores visited states in a hashtable ‡ Characteristics ± Memory intensive ± Good for finding concurrency errors ± Short execution paths are better.2002 ‡ Explicit State Explicit vs. Symbolic Model Checking ‡ Symbolic ± Sets of states are manipulated at a time ± Typically a backwards analysis ± Transition relation encoded by (some variant of) Binary Decision Diagrams (BDDs) or as a satisfiability problem ± Can handle very large state spaces ± Not as good for asynchronous systems ± Cannot deal well with long execution traces ± Works best with a static transition relation.

Overview 2002 ‡ Introduction to Model Checking ± Hardware Model Checking ± Software Model Checking ‡ Program Model Checking ‡ Case Studies ‡ Future of Software Model Checking 24 September 2002 © Willem Visser 2002 16 .

Motorola. hence the transition relation can be encoded efficiently ± Execution paths are typically very short ‡ The Intel Pentium bug.Hardware Model Checking 2002 ‡ BDD-based model checking was the enabling technology ± Hardware is typically synchronous and regular. was the ³disaster´ that got model checking on the map in the hardware industry ± What is it going to take in the software world? ‡ Intel. IBM. now employ hundreds 24 September 2002 © Willem Visser 2002 17 of model checking experts . etc.

and although bugs were found the field never really moved beyond some compelling casestudies ± Reality is that people write code first. not much happened until 1997 ‡ Until 1997 most work was on software designs ± Since catching bugs early is more cost-effective ± Problem is that everybody use a different design notation. rather than design ‡ The field took off when the seemingly harder problem of analyzing Visser 2002 source code was18 24 September 2002 © Willem actual first attempted .Software Model Checking 2002 ‡ Although the early papers on model checking focused on software.

but buggy programs are everywhere! ± Testing is inadequate for complex software (concurrency.Program Model Checking 2002 ‡ Why is program analysis with a model checker so much more interesting? ± Designs are hard to come by. etc. but also in verification. mostly in compiler optimization. objects. 24 September 2002 © Willem Visser 2002 19 . pointers.) ± Static program analysis was already an established field.

2002 The Trends in Program Model Checking Most model checkers cannot deal with the features of modern programming languages ‡ Bringing programs to model checking ± Abstraction (including translation) ‡ Bringing model checking to programs ± Improve model checking to directly deal with programs as input 24 September 2002 © Willem Visser 2002 20 .

Overview 2002 ‡ Introduction to Model Checking ‡ Program Model Checking ± Major Trends ‡ Abstraction ‡ Improved model checking technology ± A Brief History ± Current Trends ‡ Case Studies ‡ Future of Software Model Checking 24 September 2002 © Willem Visser 2002 21 .

return buffer[tail].2002 Program Model Checking Enabling Technology Abstraction Program void add(Object o) { buffer[head] = o. } Model Checker Input Infinite state 24 September 2002 © Willem Visser 2002 Finite state 22 . } Object take() { « tail=(tail+1)%size. head = (head+1)%size.

i. the same behaviors are present in the abstracted and original program 24 September 2002 © Willem Visser 2002 23 .e. i. i. less behaviors are present in the abstracted system than are present in the original ± Precise abstractions. and.Abstraction 2002 ‡ Model checkers don¶t take real ³programs´ as input ‡ Model checkers typically work on finite state systems ‡ Abstraction therefore solves two problems ± It allows model checkers to analyze a notation they couldn¶t deal with before.e. more behaviors are added to the abstracted system than are present in the original ± Under-approximations. ± Cuts the state space size to something manageable ‡ Abstraction comes in three flavors ± Over-approximations.e.

t.r. etc.. the property being checked. ‡ Precise abstraction.10 rather than all integer values ± Queue size 3 instead of unbounded. with no guarantee that the right behaviors are removed. may be obtained if the behaviors being removed are indeed not influencing the property 24 September 2002 © Willem Visser 2002 ± Limit input values to 0.Under-Approximation 2002 ³Meat-Axe´ Abstraction ‡ Remove parts of the program deemed ³irrelevant´ to the property being checked ‡ The abstraction of choice in the early days of program model checking ± used during the translation of code to a model checker¶s input language ‡ Typically manual. the property being checked 24 . ± Program slicing is an example of an automated under-approximation that will lead to a precise abstraction w.r. w.t.

zero} ± Replace predicates in the program by boolean variables. often called abstraction refinement © Willem Visser 2002 24 September 2002 25 . but increases the number of possible transitions.Over-Approximations 2002 Abstract Interpretation ‡ Maps sets of states in the concrete program to one state in the abstract program ± Reduces the number of states. and hence the number of behaviors ± Can in rare cases lead to a precise abstraction ± Replace int by Signs abstraction {neg. is that also an error in the original program? ± Most research focuses on this problem. and replace each instruction that modifies the predicate with a corresponding instruction that modifies the boolean. ‡ Type-based abstractions ‡ Predicate abstraction ‡ Automated (conservative) abstraction ‡ Eliminating spurious errors is the big problem ± Abstract program has more behaviors. and its counter-part the elimination of spurious errors.pos. therefore when an error is found in the abstract program.

Bringing 2002 Model Checking to Programs ‡ Allow model checkers to take modern programming languages as input. or. notations that are of similar expressive power ‡ Major hurdle is how to encode the state of the system efficiently ‡ Alternatively state-less model checking ± No state encoding or storing ‡ Almost exclusively explicit-state model checking ‡ Abstraction can still be used as well ± Source to source abstractions 24 September 2002 © Willem Visser 2002 26 .

Overview 2002 ‡ Introduction to Model Checking ‡ Program Model Checking ± Major Trends ± A Brief History ‡ SPIN ‡ Hand-translations ‡ State-less model checking ‡ Semi-automated translations ‡ Fully automated translations ± Partial-order reductions ± VeriSoft ‡ Case Studies ‡ Future of Software Model Checking 24 September 2002 © Willem Visser 2002 ± Current Trends 27 .

table-driven translations ± 1998 ‡ Automated translations still with ad-hoc abstractions ± 1997-1999 ‡ State-less model checking for C 24 September 2002 ± VeriSoft 1997 © Willem Visser 2002 28 .The Early Years 2002 ‡ Hand-translation with ad-hoc abstractions ± 1980 through mid 1990s ‡ Semi-automated.

Overview 2002 ‡ Introduction to Model Checking ‡ Program Model Checking ± Major Trends ± A Brief History ‡ SPIN ‡ Hand-translations ‡ State-less model checking ‡ Semi-automated translations ‡ Fully automated translations ± Partial-order reductions ± VeriSoft ‡ Case Studies ‡ Future of Software Model Checking 24 September 2002 © Willem Visser 2002 ± Current Trends 29 .

SPIN Model Checker 2002 ‡ Kripke structures are described as ³programs´ in the PROMELA language ‡ Automata based model checker ± Kripke structure is generated on-the-fly during model checking ± Translates LTL formula to Büchi automaton ± 10th SPIN Workshop to be held with ICSE ± May 2003 ± http://netlib.html ‡ By far the most popular model checker ‡ Relevant theoretical papers can be found here ‡ Ideal for software model checking due to expressiveness of the PROMELA language ± Close to a real programming language ‡ Gerard Holzmann won the ACM software award for SPIN 24 September 2002 © Willem Visser 2002 30 .com/netlib/spin/whatispin.bell-labs.

Hand-Translation 2002 abstraction Verification model Program translation ‡ Hand translation of program to model checker¶s input notation ‡ ³Meat-axe´ approach to abstraction ‡ Labor intensive and error-prone 24 September 2002 © Willem Visser 2002 31 .

et al.gov/havelund Translation from Lisp to Promela (most effort) Heavy abstraction 3 man months ‡ DEOS ± Penix.Penix.nasa.arc.Lowry 1997 http://ase.nasa.arc. Visser.gov/visser ± C++ to Promela (most effort in environment generation) ± Limited abstraction .2002 ± ± ± ± Hand-Translation Examples ‡ Remote Agent ± Havelund. 1998/1999 ± http://ase.programmers produced sliced system 24 September 2002 © Willem Visser 2002 32 ± 3 man months .

Overview 2002 ‡ Introduction to Model Checking ‡ Program Model Checking ± Major Trends ± A Brief History ‡ SPIN ‡ Hand-translations ‡ State-less model checking ‡ Semi-automated translations ‡ Fully automated translations ± Partial-order reductions ± VeriSoft ‡ Case Studies ‡ Future of Software Model Checking 24 September 2002 © Willem Visser 2002 ± Current Trends 33 .

lcs.sunysb.mit.edu/rivet.State-less Model Checking 2002 ‡ No state storing.bell-labs.html ‡ http://www. i.com/project/verisoft/ ‡ http://sdg. ± VeriSoft ± Rivet ‡ http://www1.edu/~stoller/ ‡ Examples include ± See also work by Scott Stoller 24 September 2002 © Willem Visser 2002 34 .cs. hence no state matching ‡ Must limit search-depth to ensure termination ‡ Enabling technology is partial-order reduction ‡ Annotate code to allow verifier to detect ³important´ statements ± Avoid revisiting some states ± Communications and nondeterministic choice ± Model checker will try all interleavings of the these statements. the last statement in any group will be an important one.e. but will consider all other statements ³atomic´.

Partial-Order Reductions 2002 ‡ Reduce the number of interleavings of independent concurrent transitions ‡ x := 1 || y := 1 where initially x = y = 0 x := 1 10 y := 1 11 00 y := 1 01 x := 1 x := 1 10 y := 1 11 00 y := 1 01 x := 1 10 y := 1 11 00 No Reductions 24 September 2002 Transitions Reduced States Reduced © Willem Visser 2002 35 .

Partial Order Reductions 2002 Basic Ideas ‡ Independence ± Independent transitions cannot disable nor enable each other ± Enabled independent transitions are commutative ‡ Partial-order reductions only apply during the on-the-fly construction of the Kripke structure ‡ Based on a selective search principle ± Compute a subset of enabled transitions in a state to execute ‡ Sleep sets (Reduce transitions) ‡ 24Persistent sets (Reduce states) September 2002 © Willem Visser 2002 36 .

Persistent Set Reductions 2002 ‡ A subset of the enabled transitions is called persistent. when any transition can be executed from outside the set and it will not interact or affect the transitions inside the set ± Use the static structure of the system to determine what goes into the persistent set ± Note. all enabled transitions are trivially persistent ‡ Only execute transitions in the persistent set ‡ Persistent set algorithm is used within SPIN ‡ See papers by Godefroid and Peled 24 September 2002 © Willem Visser 2002 37 .

VeriSoft 2002 ‡ The first model checker that could handle programs directly ± C programs running on Unix T1 || T2 T1 T2 ‡ Relies on partial-order reductions to limit the number of times a state is revisited ± Persistent sets ‡ Reduce states visited T2 T1 T1 || T2 T1 T1 || T2 T1 T2 ± Sleep sets ‡ Reduce transitions executed ‡ Paths must be replayed from the initial state to try new branches ± No check-pointing 24 September 2002 © Willem Visser 2002 T2 T2 T1 Persistent Sets Sleep Sets 38 .

State-less Example
2002
T1 T2 T3 T3 T4

T3

T1

T2 T4 T4 T2

T1

T1 ; T2 || T3 ; T4 ‡ DFS with state storage ± executes 12 transitions ‡ DFS state-less ± executes 24 transitions

24 September 2002

© Willem Visser 2002

39

State-less Example
2002
T1 T2 T3 T3 T4

T3

T1

T2 T4 T4 T2

T1

T1 ; T2 || T3 ; T4 ‡ DFS with state storage ± executes 12 transitions ‡ DFS state-less ± executes 24 transitions ‡ DFS state-less with partial-order reduction ‡ Sleep sets - executes 8 transitions (all states)
24 September 2002 © Willem Visser 2002

40

State-less Example
2002
T1 T2 T3 T3 T4

T3

T1

T2 T4 T4 T2

T1

T1 ; T2 || T3 ; T4 ‡ DFS with state storage ± executes 12 transitions ‡ DFS state-less ± executes 24 transitions ‡ DFS state-less with partial-order reduction ‡ Sleep sets - executes 8 transitions (all states) ‡ Persistent sets ± executes 4 transitions (only 5 states)
24 September 2002 © Willem Visser 2002

41

Overview 2002 ‡ Introduction to Model Checking ‡ Program Model Checking ± Major Trends ± A Brief History ‡ SPIN ‡ Hand-translations ‡ State-less model checking ‡ Semi-automated translations ‡ Fully automated translations ± Partial-order reductions ± VeriSoft ‡ Case Studies ‡ Future of Software Model Checking 24 September 2002 © Willem Visser 2002 ± Current Trends 42 .

Semi-Automatic Translation 2002 ‡ Table-driven translation and abstraction ± Feaver system by Gerard Holzmann ± User specifies code fragments in C and how to translate them to Promela (SPIN) ± Translation is then automatic ± Found 75 errors in Lucent¶s PathStar system ± http://cm.bell-labs.com/cm/cs/who/gerard/ ‡ Advantages ± Can be reused when program changes ± Works well for programs with long development and only local changes 24 September 2002 © Willem Visser 2002 43 .

gov/havelund/jpf.nasa.http://www. SMV or dSpin 24 September 2002 © Willem Visser 2002 44 .http://ase.Fully Automatic Translation 2002 ‡ Advantage ± No human intervention required ‡ Disadvantage ± Limited by capabilities of target system ‡ Examples ± Java PathFinder 1.polito.edu/santos/bandera/ ‡ Translates from Java bytecode to Promela.http://www.ksu.shtml ‡ Translates from Java to Promela (or dSpin) ± Bandera .html ‡ Translates from Java to Promela (Spin) ± JCAT .dai-arc.arc.cis.it/dai-arc/auto/tools/tool6.

Overview 2002 ‡ Introduction to Model Checking ‡ Program Model Checking ± Major Trends ± A Brief History ± Current Trends ‡ ‡ ‡ ‡ ‡ ‡ Custom-made model checkers for programs Abstraction SLAM JPF Summary Examples of other software analyses ‡ Case Studies ‡ Future of Software Model Checking 24 September 2002 © Willem Visser 2002 45 .

Program Model Checking 2002 Program void add(Object o) { buffer[head] = o. head = (head+1)%size. } Object take() { « tail=(tail+1)%size. return buffer[tail]. } Current Trends Abstract Program T1 > T2 T3 > T4 T5 > T6 « Correct Custom Model Checker Abstraction Error-trace Abstraction Abstraction refinement ‡ Custom-made model checkers for programming languages with automatic abstraction at the source code level ‡ Automatic abstraction & translation based transformation to new ³abstract´ formalism for model checker ‡ Abstraction refinement mostly automated 24 September 2002 © Willem Visser 2002 46 .

polito.dai-arc.it/dai-arc/auto/tools/tool7.shtml ± SPIN Version 4 (not released yet) ± Bandera ‡ PROMELA language augmented with C code ‡ Table-driven abstractions ‡ Translated Bandera Intermediate Language (BIR) to a number of back-end model checkers.Custom-made Model Checkers 2002 ‡ Translation based ± dSpin ‡ ‡ ‡ ‡ Spin extended with dynamic constructs Essentially a C model checker Source-2-source abstractions can be supported http://www.cis. a new BIR custom-made model checker is under development ‡ Supports source-2-source abstractions as well as propertyspecific slicing ‡ http://www.edu/santos/bandera/ 24 September 2002 © Willem Visser 2002 47 . but.ksu.

e.htm ‡ http://www.math.Custom-made Model Checkers 2002 ‡ Abstraction based ± SLAM ‡ C programs are abstracted via predicate abstraction to boolean programs for model checking ‡ http://research. during abstraction refinement don¶t abstract the whole program only certain parts ‡ http://www-cad.tau.berkeley.cs.il/~yahave/3vmc.ac.ac.com/slam/ ‡ Similar basic idea to SLAM.microsoft.il/~rumster/TVLA/ © Willem Visser 2002 24 September 2002 48 .edu/~tah/blast/ ± BLAST ± 3-Valued Model Checker (3VMC) extension of TVLA for Java programs ‡ http://www. i. but using lazy abstraction.tau.eecs.

Overview 2002 ‡ Introduction to Model Checking ‡ Program Model Checking ± Major Trends ± A Brief History ± Current Trends ‡ ‡ ‡ ‡ ‡ ‡ Custom-made model checkers for programs Abstraction SLAM JPF Summary Examples of other software analyses ‡ Case Studies ‡ Future of Software Model Checking 24 September 2002 © Willem Visser 2002 49 .

zero.r.g. set of predicates defined in concrete system © Willem Visser 2002 50 .pos) = negative | zero | pos ‡ eq(pos.positive} ± Replace all operations on the concrete variable with corresponding abstract operations ‡ add(pos.t.pos) = pos ‡ subtract(pos. Saïdi see also Uribe) 24 September 2002 ± Create abstract state-space w.Abstraction 2002 ‡ Data type based abstractions ± Abstract Interpretation (Patrick Cousot) ± E. replace integer variable with odd-even range or Signs abstraction {negative.pos) = true | false ‡ Predicate Abstraction (Graf.

ZERO)) x = Signs. Data domains int (n<0) : NEG (n==0): ZERO (n>0) : POS Signs x = ZERO.add(x. if (x == 0) x = x + 1.eq(x. if (Signs. 24 September 2002 © Willem Visser 2002 Signs NEG ZERO POS 51 .Data Type Abstraction 2002 Collapses data domains via abstract interpretation: Code int x = 0.POS).

2002 Abstract Eint v int Predicate Abstraction T F EQ = T EQ := F EQ = F bool x!y x{y EQ | (x = y) EQ | (x = y) Concrete x=0 y=0 y++ x=0 y=1 states correspond to truth values of a set of predicate ‡ Create abstract state-graph during model checking. or. whose . ‡ Create an abstract transition Visser 2002 before model checking 24 September 2002 © Willem system 52 ‡ Mapping of a concrete system to an abstract system.

2002 Example Predicate Abstraction Predicate: B | (x = y) Abstract Statement Step 1: Calculate pre-images {x = y + 1} y := y + 1 {x = y} {x { y + 1} y := y + 1 {x { y} Step 2: Rewrite in terms of predicate Step 2a: Use Decision Procedures {x = y + 1} y := y + 1 {B} x=ypx=y+1 x=ypx{y+1 {B} y := y + 1 {~B} x{ypx=y+1 x{ypx{y+1 Concrete Statement y := y + 1 Step 3: Abstract Code IF B THEN B := false ELSE B := true | false 53 24 September 2002 © Willem Visser 2002 .

ZERO)) then [2] assert(true).POS} [1] if (-2 + 3 > 0) then [2] assert(true). Signs: n < 0 -> neg 0 -> zero n > 0 -> pos [1] if(Signs.Example of Infeasible Counter-example 2002 {NEG.add(NEG. else [3] assert(false). [1]: [1]: [2]: [2]: [3]: X 54 24 September 2002 © Willem Visser 2002 .gt(Signs. else [3] assert(false).ZERO.POS).

Overview 2002 ‡ Introduction to Model Checking ‡ Program Model Checking ± Major Trends ± A Brief History ± Current Trends ‡ Custom-made model checkers for programs ‡ Abstraction ‡ SLAM ± Abstraction Refinement ‡ JPF ‡ Summary ‡ Examples of other software analyses ‡ Case Studies ‡ Future of Software Model Checking 24 September 2002 © Willem Visser 2002 55 .

head = (head+1)%size. } Simplified View Predicate Abstraction C2BP Custom Model Checker BEBOP Predicates Correct Symbolic Execution NEWTON Error-trace False Error-trace is Feasible 24 September 2002 © Willem Visser 2002 56 . return buffer[tail]. } Object take() { « tail=(tail+1)%size.SLAM 2002 C Program annotated with API usage rules void add(Object o) { buffer[head] = o.

SLAM 2002 ‡ Check API usage rules for sequential C programs ‡ C2BP ± Mostly applied to device driver code ± Inputs: C program and predicates ± Output: boolean program over the predicates ± Symbolic interprocedural data flow analysis ± Concrete CFG and BDD encoding of states ± Symbolic execution of C programs ± Using Simplify theorem prover for checking feasibility of conditionals © Willem Visser 2002 ‡ BEBOP ‡ NEWTON 24 September 2002 57 .

KeReleaseSpinLock(&devExt->writeListLock). } } while (nPackets != nPacketsOld).. if (request){ KeReleaseSpinLock(&devExt->writeListLock). request = devExt->WLHeadVa. nPacketsOld = nPackets.. nPackets++. . 24 September 2002 © Willem Visser 2002 58 .Abstraction Refinement Example 2002 Adapted from Ball & Rajamani POPL02 Property: if a lock is held it must be released before reacquiring do { //get the write lock KeAcquireSpinLock(&devExt->writeListLock).

2002 Initial Abstraction and Model Checking Boolean Program [1] do //get the write lock [2] AcquireLock().1. [6] ReleaseLock(). [3] if (*) then [4] ReleaseLock().5. Error-trace : 1.3.2.2 24 September 2002 © Willem Visser 2002 59 . fi [5] while (*).

} [5] } while (nPackets != nPacketsOld).2. The predicate nPackets == nPacketsOld is then added and used during predicate abstraction 24 September 2002 © Willem Visser 2002 60 ...5.1. Symbolic execution of 1. request = devExt->WLHeadVa. [3] if (request){ [4] KeReleaseSpinLock(&devExt->writeListLock).3. nPacketsOld = nPackets.2 shows that when 5 is execute nPackets == nPacketsOld hence the path is infeasible. nPackets++. [6] KeReleaseSpinLock(&devExt->writeListLock).Symbolic Execution 2002 [1] do { [2] KeAcquireSpinLock(&devExt->writeListLock). .

//(nPacketsOld != nPackets) [8] ReleaseLock(). // nPackets++ fi [7] while (!b). Now property holds 24 September 2002 © Willem Visser 2002 61 . [6] b = b ? False : *. b = true. // nPacketsOld = nPackets [3] [4] if (*) then [5] ReleaseLock().2002 Next Abstraction and Model Checking New Predicate b : (nPacketsOld == nPackets) [1] do [2] AcquireLock().

Overview 2002 ‡ Introduction to Model Checking ‡ Program Model Checking ± Major Trends ± A Brief History ± Current Trends ‡ Custom-made model checkers for programs ‡ SLAM ‡ JPF ± ± ± ± Abstractions Partial-order and symmetry reductions Heuristics New Stuff ‡ Case Studies ‡ Future of Software Model Checking ‡ Summary ‡ Examples of other software analyses 24 September 2002 © Willem Visser 2002 62 .

since it works with bytecodes ± Written in Java ± DFS.arc.nasa. A*. Best-first. 63 . BFS.gov/jpf 24 September 2002 © Willem Visser 2002 ± Handle all of Java.2002 Java PathFinder (2) Direct Approach ‡ Based on custom-made Java Virtual Machine ‡ Efficient encoding of states ‡ Modular design for easy extensions ‡ Supports LTL checking with properties expressed in Bandera¶s BSL notation ‡ Incorporates a number of search strategies ‡ Supports source-2-source abstractions ‡ http://ase. etc.

return buffer[tail]. } Object take() { « tail=(tail+1)%size. } Bytecode 0: 1: 2: 5: 8: 9: 10: iconst_0 istore_2 goto #39 getstatic aload_0 iload_2 aaload JAVAC JVM Model Checker Special JVM 24 September 2002 © Willem Visser 2002 64 .Java PathFinder (JPF) 2002 Java Code void add(Object o) { buffer[head] = o. head = (head+1)%size.

2002 Property Tool Bandera & JPF Architecture Abstraction Analyses Engine BIRC Translators BIR SPIN Java Parser Slicer Error Trace Display 24 September 2002 dSPIN Jimple (BC) SMV Simulator Decompile . javac JPF 65 © Willem Visser 2002 .

threads. Networks. « ‡ Must be modeled by java code instead ‡ Allows Nondeterministic Environments ± JPF traps special nondeterministic methods ‡ Checks for User-defined assertions.Key Points 2002 ‡ Models can be infinite state ± Unbounded objects. files. deadlock and LTL properties 24 September 2002 © Willem Visser 2002 66 .« ± Depth-first state generation (explicit-state) ± Verification requires abstraction ‡ Handle full Java language ± but only for closed systems ± Cannot handle native code ‡ no Input/output through GUIs.

‡ Properties ± AssertTrue(boolean cond) ‡ Miscellaneous ± Dump states. endAtomic() ‡ Nondeterminism ± int random(int). « 24 September 2002 © Willem Visser 2002 67 . boolean randomBool(). Daikon logs.Java Extensions -Verify Class2002 ‡ Used for annotations ‡ Atomicity ± beginAtomic(). print. Object randomObject(String cname).

JPF Technologies 2002 ‡ Predicate abstraction and type based abstractions ‡ Partial order and symmetry reductions ‡ Slicing ‡ Heuristic Search ‡ New Stuff ± Symbolic Execution ± Error Explanations 24 September 2002 © Willem Visser 2002 68 .

neg if (a==ZERO && b==ZERO) r=ZERO.endAtomic(). public static final int ZERO = 1.neg} if (a==ZERO && b==POS) r=POS. public class Signs { public static final int NEG = 0. © Willem Visser 2002 69 return r. pos }. -> {pos}. if (a==NEG && b==NEG) r=NEG.neg} 24 September 2002 public static int add(int a. neg if (a==NEG && b==ZERO) r=NEG. else r=Verify.beginAtomic(). }} . +abs zero pos neg zero zero pos neg pos pos pos {zero.pos. if (a==ZERO && b==NEG) r=NEG.Type-based Abstractions Abstract Interpretation 2002 abstraction Signs abstracts int TOKENS = { neg. neg if (a==POS && b==POS) r=POS. Verify. {zero. if (n > 0) return POS. zero. int b){ int r. -> {zero}. if (a==POS && b==ZERO) r=POS. if (n == 0) return ZERO.choose(2). Verify. public static final int POS = 2.pos. } abstraction mapping: n n n < 0 == 0 > 0 -> {neg}. public static int abs(int n) { if (n < 0) return NEG.

Abstract. ‡ Tool generates abstract Java program ± Using Stanford Validity Checker (SVC) ± JVM is extended with nondeterminism to handle over approximation ‡ Abstractions can be local to a class or global across multiple classes ± Abstract. x==y).remove(x).addBoolean(³EQ´.addBoolean(³EQ´. A.2002 JPF Predicate Abstraction ‡ Annotations used to indicate abstractions ± Abstract.y).remove(y). ± Dynamic predicate abstraction .x==B.works across instances 24 September 2002 © Willem Visser 2002 70 . Abstract.

Partial Order Reduction in JPF 2002 ‡ Persistent set approach ‡ Find transitions that are globally independent ± Always independent of all transitions in other threads ‡ Requires static analysis before model checking to determine global independence ± Advanced alias analysis 24 September 2002 © Willem Visser 2002 71 .

}} class SecondTask extends Thread { public void run() { int x. s1 = new S1().Partial-order Reduction 2002 class S1 {int x. S2 s2.} public class Example { public static void main (String[] args) { FirstTask t1 = new FirstTask(). x = 1. x = 3. x = 3. SecondTask t2 = new SecondTask().} class S2 {int y. S1 s1.start().start(). s2 = new S2(). }} class FirstTask extends Thread { public void run() { int x. t1. t2. }} ‡ 43 states with no reduction ‡ 18 states with partial-order reduction ‡ all statements are globally independent (safe) 24 September 2002 © Willem Visser 2002 72 . x = 2.

t1. Example.start(). S2 s2. }} class SecondTask extends Thread { public void run() { int x.x = 1. public static void main (String[] args) { FirstTask t1 = new FirstTask().start(). S1 s1. x = 3. }} ‡ 43 states with no reduction ‡ 27 states with partial-order reduction ‡ 2 statements are not globally independent 24 September 2002 © Willem Visser 2002 73 . Example.Partial-order Reduction 2002 class S1 {int x.} public class Example { public static int x = 10. x = 3. SecondTask t2 = new SecondTask(). Not Safe s1 = new S1(). s2 = new S2().x = 2.} class S2 {int y. }} class FirstTask extends Thread { public void run() { int x. t2.

s1 = new S1(). 2 }} }} 1 1 followed by 2 t1 t2 s1 s2 0 1 2 3 2 followed by 1 t1 t2 s2 s1 0 1 2 3 ‡ Map allocation to Heap position on First Path ‡ Reuse same Positions on later paths © Willem Visser 2002 24 September 2002 74 .} public class Example { public static void main (String[] args) { FirstTask t1 = new FirstTask(). s2 = new S2().Symmetry Reduction 2002 class S {int x. SecondTask t2 = new SecondTask(). t1.start(). S s2.start(). t2. }} class FirstTask extends Thread { class SecondTask extends Thread { public void run() { public void run() { S s1.

Garbage Collection 2002 class gc { public static void main (String[] args) { while(true) { System.println(³0´).out. }}} ‡ Infinite State program ‡ Garbage makes the state grow« ‡ JPF supports ‡ Mark-and-Sweep ‡ Reference counting 24 September 2002 © Willem Visser 2002 75 .

Property-directed Slicing 2002 indirectly relevant Slice mentioned in property Source program Resulting slice ‡ slicing criterion generated automatically from observables mentioned in the property ‡ backwards slicing automatically finds all components that might influence the observables. 24 September 2002 © Willem Visser 2002 76 .

notifyAll(). int bound.. removed by slicing public synchronized void add(Object o) { while ( tail == head ) try { wait(). head = (head+1) % bound. } indirectly relevant © Willem Visser 2002 77 . tail. int head.Property-directed Slicing 2002 /** * @observable EXP Full: (head == tail) */ class BoundedBuffer { Object [] buffer_. Slicing Criterion All statements that assign to head. 24 September 2002 } . } catch ( InterruptedException ex) {} Included in slicing criterion buffer_[head] = o. tail..

± Deadlock .) ‡ http://www.Slicing in JPF 2002 ‡ JPF uses Bandera¶s slicer ‡ Bandera slices w.i.r.e.t. communication statements ± Variables occurring in temporal properties ± Variables participating in race-violations ‡ Used with JPF¶s runtime analysis ‡ More examples of slicing for model checking ± Slicing for Promela (Millet and Teitelbaum) ‡ http://netlib.bell-labs.wisc.html ± Slicing for Hardware Description Languages (Shankar et al.com/netlib/spin/ws98/program.edu/~reps/ 24 September 2002 © Willem Visser 2002 78 .cs.

Heuristic Search 2002 ‡ Best-First. Beam and A* Search ‡ Heuristics based on property ± deadlock ‡ Maximize number of blocked threads ± Assertions ‡ Minimize distance to assertion ‡ Heuristics on structure of Program ± Interleaving heuristic ‡ Maximize different thread scheduling ± Branch Exploration ‡ Maximize the coverage of new branches ± Choose-free heuristic ‡ Minimize non-deterministic choice ‡ User-defined heuristics ± Full access to JVM¶s state via API ‡ Combine heuristics 24 September 2002 © Willem Visser 2002 79 .

‡ Bias the model checker ± to look only at paths that do not include instructions that introduce non-determinism ‡ JPF model checker modified ± to detect non-deterministic choice and backtrack from those points 24 September 2002 © Willem Visser 2002 80 .2002 Choose-free state space search ‡ Theorem [Saidi:SAS¶00] Every path in the abstracted program where all assignments are deterministic is a path in the concrete program.

Choose-free Heuristic 2002 ‡ Infeasible error elimination during abstraction ‡ Heuristic function returns best value for states with least number of non-deterministic choices enabled ‡ If no ³deterministic´ error exists it also searches rest of the state space X X 24 September 2002 © Willem Visser 2002 81 .

2002 Test Coverage for Model Checking ‡ Reality check: ± Model Checkers run out of memory OFTEN! ‡ If no error was found. as a heuristic ± Better heuristic values are given for least explored branches 24 September 2002 © Willem Visser 2002 82 . how confident can we be that none exists? ± Require coverage measure ‡ JPF extended with Branch coverage calculations ‡ Can the coverage measure guide the model checker ± Yes.

Concurrency Bugs 2002 ‡ Often due to unanticipated interleaving ‡ Increased thread interleaving might expose them ‡ Interleaving heuristic ± Executions in which context is switched more often are given better heuristic values 24 September 2002 © Willem Visser 2002 83 .

00 JPF DEOS Systematic Hand-translation SPIN 330.00 600.00 0.00 1000.00 Mars Rover 550.00 50.00 30.00 200.00 400.00 800.00 Remote Agent Hand-translation SPIN DEOS Java-translation JPF Autopilot JPF 1000.Scaling Program Model Checking Error-Detection 2002 1200.00 1997 24 September 2002 1998 2000 2001 2002 84 LOC analyzed per Person day © Willem Visser 2002 .

2002 JPF Most Recent Extensions ‡ Symbolic execution based model checking ‡ Error Explanations ± Sarfraz Khurshid (MIT). ± Analyze the positives and negatives ‡ Find smallest change to make a positive a negative ‡ Find invariants on positives and negatives (using Diakon) ‡ Find statements that are on all and only positives (negatives) © Willem Visser 2002 24 September 2002 85 . then try and find ³similar´ paths that lead to errors (negatives) and that don¶t lead to an error (positives). Corina Pasareanu and Doron Peled (Warwick) ± Traverses all paths and collects and solves constraints on-thefly during model checking ± Using the Omega libraries for constraint solving ± Main application is test-case generation ± Alex Groce (CMU) ± Find an error.

Overview 2002 ‡ Introduction to Model Checking ‡ Program Model Checking ± Major Trends ± A Brief History ± Current Trends ‡ ‡ ‡ ‡ ‡ Custom-made model checkers for programs SLAM JPF Summary Examples of other software analyses ‡ Case Studies ‡ Future of Software Model Checking 24 September 2002 © Willem Visser 2002 86 .

constraint solving. etc.2002 Software Model Checking Executive summary ‡ Model checking by itself cannot deal with the complexity of software ‡ Techniques from static analysis are required ± Abstract interpretation. alias&shape analysis. symbolic execution ‡ Even then. slicing. we need to borrow some more! ± Heuristic search. ‡ Abandon soundness ± Aggressive heuristics ± Runtime analysis and runtime monitoring 24 September 2002 © Willem Visser 2002 87 .

polyspace. PolySpace for C.compaq.gov/rv2002/ ‡ Analysis Toolsets ± IF (Verimag). Statecharts. ESC/Java from Compaq ‡ http://research. etc. ‡ Runtime analysis ± See Runtime Verification Workshops ‡ http://ase.com/SRC/esc/ ‡ Static analysis for runtime errors ± For example.More Software Analysis Techniques 2002 A small sample ‡ Program Verification ± For example. RSML. etc.com/ ‡ Requirements and Design Analysis ± Analysis for SCR. 24 September 2002 © Willem Visser 2002 88 . SAL (SRI).nasa.arc. Ada and Java ‡ http://www.

Overview 2002 ‡ Introduction to Model Checking ‡ Program Model Checking ‡ Case Studies ± Remote Agent ± DEOS ± Mars Rover ‡ Future of Software Model Checking 24 September 2002 © Willem Visser 2002 89 .

Case Studies 2002 DEOS Remote Agent Mars Rover 24 September 2002 © Willem Visser 2002 90 .

24 September 2002 © Willem Visser 2002 91 .Case Study: DS-1 Remote Agent 2002 Commands Spacecraft Sensors Achieve Property Subscribe Tasks Property Locks Lock Event Data base Interrupt Properties Monitor Change Event ‡ Several person-months to create verification model. ‡ One person-week to run verification studies.

´ .2002 Case Study: DS-1 Remote Agent Unexpected timing of change event Monitor Logic DB change? yes check no wait ‡ Five difficult to find concurrency errors detected ‡ ³[Model Checking] has had a substantial impact. helping the RA team improve the quality of the Executive well beyond what would otherwise have been produced.RA team ‡ During flight RA deadlocked (in code we didn¶t analyze) 24 September 2002 ± Found this deadlock with JPF © Willem Visser 2002 92 .

DEOS Operating System 2002 ‡ Integrated Modular Avionics (IMA) ± DEOS Guarantee Space and Time partitioning ‡ FAA Certification Process ± Requires Structural Testing Coverage (MC/DC) ± Inadequate for finding Time Partitioning Errors ‡ Timing Error not found by Testing occurred ‡ Behavioral Analysis of Time Partitioning ± NASA Ames and Honeywell HTC collaboration ± Model Check slice of DEOS containing timing error 24 September 2002 © Willem Visser 2002 93 .

what the error was.DEOS Analysis 2002 ‡ Translated C++ 1-to-1 to PROMELA/SPIN (1500 lines of C++ code) ± Found the time-partitioning error without any prior knowledge. ± Required very limited abstraction ± Surprised that error was found by directly checking code ± They expected NASA team to ask for smaller ³slice´ ± They now have their own model checking group building on our work ± Backwards dependency analysis from the time partitioning assertion being checked revealed candidate variables to abstract ± Applied ³range´ abstraction {0. where it was or what made it show up.many} to a specific integer variable ± Too much of an over-approximation that led to many spurious errors ± However with the choose-free heuristic the non-spurious error was found © Willem Visser 2002 ‡ DEOS Team Reaction ‡ Then translated DEOS to Java and applied JPF 24 September 2002 94 .1.

Error Trace 2002 Idle never ran User2: 16/21 idle User2: 21/60 User1: 21/60 Main: 5/20 12/20 8 15 5 timer delete 12 20 preempt 12 40 8 timer timer 60 0 24 September 2002 © Willem Visser 2002 95 .

Analysis of the K9 Mars Rover ³The Experiment´ 2002 ‡ Rover is 8000 lines of code with 6 threads ‡ Purpose ± Benchmark current state of the art in model checking. each group uses one technology on the Mars rover code to find seeded bugs ± 3 versions created and each group gets 2 days/version ± Some bugs are removed/introduced between versions ± Any new bugs discovered are not fixed. static analysis for runtime error detection and runtime analysis ± Use traditional testing as baseline ± Original code was in C++ that was translated to Java ‡ About half the code was translated to C for the static analysis that used PolySpace ± heavy use of synchronization between the threads ± Complex queue manipulation ‡ Method ± Controlled experiment: 4 groups of 2 people. only known ones 24 September 2002 © Willem Visser 2002 96 .

of the known concurrency errors and some new ones ‡ Interesting observations ± Better than any of the other teams ± Only team that could always produce not just the error but how to get to it! ± Also found all the non-concurrency errors ± Abandoned the time abstraction within the first hour for one that is closer to real-time. but one.Analysis of the K9 Mars Rover 2002 How did Model Checking do? ‡ Methodology for model checking ± Asked never to ³run´ the code. they to get Visser 2002 were flying ‡ It was too hard for them to determine if errors were spurious not knowing the code well enough . only model check it ± Code is heavily dependent on time ‡ Keep the results clean from any testing influence ‡ Given a gross over-approximation of time. had a slow 2nd version. and then found all the remaining bugs in the 1st hour of the 3rd version 97 them some time © Willemtheir framework setup. but once done. where all time-related decisions became nondeterministic ‡ Found all. but might miss errors 24 September 2002 ‡ Took ± Found a number of bugs in the first version.

Overview 2002 ‡ ‡ ‡ ‡ Introduction to Model Checking Program Model Checking Case Studies Future of Software Model Checking 24 September 2002 © Willem Visser 2002 98 .

but without this we are nowhere! ± ³Analysis is necessary.2002 ‡ ‡ ‡ ‡ ‡ ‡ The Future of Software Model Checking Abstraction based approaches Symbolic Execution ± Combine object abstractions (e.g. shape analysis) with predicate abstraction ± Automation is crucial ± Solving structural (object) and numerical constraints ± Acceleration techniques (e.g. widening) ± Test-case generation by model checking ± Runtime monitoring and model checking Model checking as a companion to testing Modular model checking for software Environment generation Result representation ± Exploiting the interface between components ± Interface automata (de Alfaro & Henzinger ) ± How to derive a ³test-harness´ for a system to be model checked ± Much overlooked. but not sufficient´ ± Jon Pincus 24 September 2002 © Willem Visser 2002 99 .

Master your semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master your semester with Scribd & The New York Times

Cancel anytime.