A Simple Method for Extracting Models From Protocol

“A Simple Method for Extracting Models From Protocol Code” David Lie, Andy Chou, Dawson Engler and David D. Dill Presented by Anna Zamansky

The Goal l Model checking for program verification is a systematic and exhaustive exploration of the system state space. Creating models manually is difficult and error prone. The goal: a method for automatically extracting models from low level software implementations.

Talk plan l l l Introduction Overview of the method and how it was applied to the FLASH cache coherence protocols. Results of the verification of FLASH. The accuracy of manually constructed models for FLASH protocols. Conclusions.

Model Checking l l Traditional method of verification: testing through trials. Difficult as the number of control paths and corner cases increases increased system cost and decreased system reliability. Solution: formal verification methods. One option: model checking - a systematic and exhaustive exploration of the system state space.

Why automatic model extraction? l l Computational complexity of model checking makes it impractical for large systems. Solutions for state explosion in model checking: – – l A great deal of manual effort human errors – – l Abstracting the system’s behavior – suppressing implementation details Scaling the system down – modeling a small instance of the system Missing bugs False alarms during verification Solution: making model checking practical by automatically extracting model descriptions from code.

The method l l The method is based on extensible compiler xg++ that performs the extraction and translation to the Mur model description. The extracted model is combined with – – – l l Model of hardware Description of correctness Initial state The whole model is checked with the Mur model checker. Case Study: cache coherence protocols of FLASH multiprocessor

FLASH protocols l l A FLASH protocol consists of a collection of event-driven software handlers that are dispatched according to requests that arrive on the various interfaces. The handlers run on the node controller and send various messages to maintain a directory of cache line states and service cache line requests.

Why FLASH as a case study? l l Manually driven model checking was applied to check FLASH protocols we can compare the new method with the traditional verification. Low level code hard to specify – – – l Highly optimized Difficult to read Representative of a variety of embedded systems A hard test: – – – Real, working Undergone years of testing under simulation and formal verification The main protocol that was checked is under constant use for 5 years

The results l Effective: – – – l 8 errors that could crash the system were found in the code 2 of the errors occur on very specific sequences of events hard to find by testing Manual verification didn’t find ANY errors! Practical: – – Extracting descriptions of 1000 lines of code from 10 K lines of implementation code No modifications to FLASH

Overview of the extraction method l l The extensible compiler xg++ allows users to write domain-specific extensions using the metal language The extensions perform – – Extraction – select sections of protocol code to be modeled Translation – translate the extracted protocol code into a Mur model description.

Advantages of user-defined extensions l l The extraction is flexible the user can use human judgment to abstract away implementation details focusing on important aspects. Using xg++ makes it feasible to construct many different models of the same system, each focusing on different functionality.

Overview of the extraction method cont. l Each extracted model is combined with – – – l l manually constructed model of the rest of the system (hardware model) correctness definition initial state The result: full Mur model description. Then the model is verified using Mur model checker.

Flow chart of model extraction and verification Protocol Code (Implementation) State Variables (Metal Slicer) Translation Patterns (Metal Printer) Correctness Properties xg++ Protocol Model Hardware Model Initial State Mur Error List

The Mur model checker l l l Mur uses a Pascal-like language for specifying models. A Mur description consists of – Declaration of constants, variables and procedures – Initial state – Collection of invariants – Rule definitions of form: Boolean expression sequence of statements Mur exhaustively searches the reachable states of the system for violations of user specified invariants: – In any given state, Mur executes all possible outcomes – Each outcome is a new state that is checked for correctness and inserted into a table of visited states

Example of a manual Mur model description Implementation Code: Void PILocal. Get(void) { /*…Boilerplate setup code…*/ head. Link. Addr = FAST_ADDRESS_TO_HEADLINKADDR(addr); FLDEBUG(‘h’, ”%u: head. Link. Addr=%llx” , proc. Num, head. Link. Addr); READ_HEADLINK(head. Link. Addr); nh. len = LEN_CACHELINE; If (!hl. Pending){ If (!hl. Dirty) { /*… 37 lines deleted…*/ ASSERT(!hl. IO); //ASSERT(hl. Local); … Hl. Local = 1; } … Partial Mur model description Rule “PI Local Get (Put)” Cache. State = Invalid & !Cache. Wait Was removed & DH. Pending from the code & DH. Dirty Begin Assert !DH. Local “PILocal. Get: L = A 0”; DH. Local : = true; Cache. Wait = true; CC_Put(Home, Memory); Endrule Hardware sets the cache state Observation: Strong correspondence between the model and the implementation code

Automatically extracted vs. manual l d Manual Mur model description Automatically extracted Mur model description Rule “PI Local Get (Put)” Cache. State = Invalid & !Cache. Wait Was removed & DH. Pending – if pending NAK from the code & DH. Dirty Begin Assert !DH. Local “PILocal. Get: L = A 0”; DH. Local : = true; Cache. Wait = true; CC_Put(Home, Memory); Endrule Rule “PI Local Get (Put)” Cache. State = Invalid & !Cache. Wait & DH. Pending & Qspace(1) Begin HG_header. nh. len : = len_data; if (hl. Pending =0) then if (hl. Dirty = 0) then mb. Result : = pi_send_func(proc. Num, nh); Local : = 1; else … nh. len : = len_nodata; … assert(nh. len = len_nodata); … Endrule Hardware sets the cache state

Automatically extracted vs. manual Benefits of the automatic extraction: 1. Takes much less time user can specify a large number of small models that check different aspects of the same implementation. 2. Is more faithful to the original implementation. Not prone to human errors.

Using xg++ to extract models xg++ breaks the extraction to tasks: 1. 2. Metal Slicer removes lines of code that do not affect the protocol state that the user is interested in modeling. Metal Printer translates the actions in the protocol implementation into abstracted actions in the model.

The Metal Slicer l l l Allows the user to match expression patterns to select the needed slice of code. Each pattern declaration selects a portion of the FLASH code to be extracted. For example, pat length indicates that the length field in the message header is to be extracted as part of the model description.

The Metal Slicer – cont. l l xg++ uses a slicing algorithm to derive all code that affects the parts selected by the Metal extensions. The algorithm is based on program dependence graph (PDG): – – The nodes of PDG represent program statements The arcs represent control and data dependencies between the program statements

The Metal Slicer – cont. l l Standard slicing techniques have difficulty producing accurate slices in the presence of such C constructs as pointers and unions. FLASH protocol – low level system code has these troublesome C features in very limited ways. The slicer is configurable we can have it automatically abstract those features by rewriting those sections. The user needn’t understand the implementation details of the compiler or the slicing algorithm.

The Metal Printer l l l Translates the selected code into a Mur model description. Allows the user to insert or rewrite actions in the model description. It is implemented by matching user-specified patterns as the slice is emitted. For example: including additional code that checks correctness properties.

A metal printer extension used to insert length field assertions Sm printer tagged_printer{ decl {scalar} data, keep, swap, wait, dec, null, type; All: /*automatically insert length assertions before each send */ {NI_SEND(type, data, keep, swap, wait, null); } { if (mgk_int_cst(data) != 0) mgk_e(“assert(nh. len = len_data); ”); else mgk_e(“assert(nh. len = len_nodata); ”_; } /*rewrite ‘len_cacheline’ and ‘len_word’ as ‘len_data’*/ | {len_cacheline} | {len_word} {ngk_e(“len_data”)}; };

Metal Printer - abstraction l Additional use of metal printer: user can abstract away implementation details using FLASH domain-specific knowledge: – – Emulation of bit operations in Mur. The printer rewrites uses of unsupported operations to call subroutines in the hardware model that emulate those actions. Limiting the number of states by abstracting data types when it is safe. For example: l l l Dyn-ptr protocol: Linked list to keep track of cache sharers Mur has no concept of pointers state explosion Solution: abstracting the linked list to an array by configuring the metal printer to rewrite calls to the linked list accessing functions.

The Hardware Model l l In manually modeled systems, actions performed by hardware and protocol can be interleaved. To model the interaction between hardware and the extracted model description in our system, hardware must be described by the user separately.

Interaction between hardware and protocol l The protocol code can make calls to hardware functional units. – l Example: SEND functions – assembler instructions that cause FLASH to transmit messages. The hardware causes certain parts of the protocol code to execute. – Example: when a FLASH node controller receives a request, a piece of code called handler is executed.

Correctness definition Examples of the invariants that were checked: l A node can not send a packet to itself l Nodes never overflow their network queues l The Real. Ptrs counter never overflows (contains the number of sharers) l If the sharing list is not empty, Real. Ptrs is greater than 0.

Initial State for FLASH State of the machine: Power on. l All valid memory is at its home node l All directory entries are blank.

Results of automatic model verification Protocol Errors found Protocol Size (lines) Extracted Model (lines) Manual Model (lines) Metal Size (lines) Dyn-Ptr 6 12 K 1100 1000 99 Bit. Vector 2 8 K 700 100 RAC 0 10 K 1500 1200 119 Coma 0 15 K 2800 1400 159

The discovered bugs l l 4 bugs were found due to the automatically inserted assertions – in the data length field. 2 counter overflow errors A message sent in the wrong network lane A false assertion

How accurate is manual modeling? l l l By using xg++, the extent to which manually described models can be inaccurate can be checked. Automatic checker that looks for semantic differences between a model and the matching protocol code. The checker was applied to the manual model of dyn-ptr protocol created by Park and Dill.

Manual models accuracy checker l l An xg++ extension tries to match the actions in each rule of Mur description to the respective actions in the code. It searches for a path in the code that will satisfy the rule’s preconditions by observing – – – l Conditionals Assignments Assertions If no such matching exists, an error message is emitted

Manual models accuracy checker Mur rule: !nh. Dirty & nh. Pending Nh. local = true; Table: nh DH … Protocol Code: If (DH. Dirty = 0 ) If (DH. Pending = 1) DH. local = 1; Error match Mur rule precondition to code path Match all actions associated with the rule along the path OK

Results of accuracy checker l 14 differences between the code and the model were found: – – l 2 semantically non-equivalent rearrangements 1 translation error that weakened the model 1 false assertion 10 incidental differences These differences illustrate the problems caused by manual modeling – – In its initial construction Modification to track implementation changes

Related work l Bandera system – – l Another system that does computer-assisted model extraction. Not flexible enough for protocols like FLASH SLAM project (Microsoft) – – – extracts a program with only boolean variables from C code. The variables represent boolean conditions in the original code Goal: check assertions in the code – too limited for FLASH

Related work l Similar approach: Path. Star system (Lucent) – – l Extraction of selected constructs using a simplified C parser Checking the result by SPIN model checker Alternative approach: Teapot system – Program Generation – – – Domain-specification language for writing cache coherence protocols Verifying the protocols with Mur After verification: translating the protocols to C

Conclusions l l A simple approach to automatically extracting models from protocol code. Advantages: – – Reduces the effort of model checking Keeps the extracted model more faithful to the protocol

Conclusions l l l The method found 8 protocol bugs that were not found by manual verification. 4 discrepancies between the manually described model and the implementation that may account for some of the missed bugs. Core of the system: an extensible compiler xg++ that provides the facilities metal slicer and metal printer

Conclusions l l l In combination with the Mur model checker, models can be quickly and easily verified. The method is not fully automatic, but some of the error prone and tedious verification tasks have been automated. The method seems applicable to other low level systems, where code is easily analyzed by tools but is difficult for humans to read.