Constraint satisfaction for random stimuli generation Yehuda Naveh

Constraint satisfaction for random stimuli generation Yehuda Naveh IBM Haifa Research Lab IBM Labs in Haifa © 2006 IBM Corporation

IBM Haifa Research Lab Constraint satisfaction problems ³ Variables: Anna, Beth, Cory, Dave, Elli, Fawn, Gill ³ Domains: Red, Green, Blue, Gray, Violet, Orange and Yellow houses ³ Constraints: ² ² ² ² ² The Red, Green, and Violet houses are in the city The Blue, Orange, Gray and Yellow houses are in the countryside The Red, Violet, and Yellow houses have two floors, the others have only one The Gray and Yellow houses are neighboring, as well as the Red and Green houses Anna and Dave have dogs, Beth owns a cat, Fawn’s got a rooster Dogs and cats cannot be neighbors Dogs must live in the countryside Roosters can live in the countryside, or in two-floor houses in the city Etc. , Etc. ³ Solution: ² Anna lives in the Blue house, Beth lives in the Red house, Cory lives in the Purple … 2 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Agenda ³ Constraint satisfaction problems (CSPs) ³ Solution algorithms ² Systematic search ² Stochastic methods ³ Simulation based verification ² NOT formal verification ³ Application of CSP to random stimuli generation ³ Cambridge walking tour 3 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Definition [ Mackworth, Freuder, Montanari, Dechter, Rossi, . . . ] ³ CSP P = {V, D, C} ³ Variables ² Anna, Beth, Cory, … ² Address, register_value ³ Domains (finite sets) for each variable ² All houses ² Address: 0 x 0000 - 0 x. FFFF ² Number of bytes in a 'load': { 1, 2, 4, 8, 16 } ³ Constraints (relations) over variables ² Dogs are not neighbors of cats ² (load n bytes) (align address to n bytes boundary) ² In a+b = c instruction, c = 0 4 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Definition ³ Solution for a CSP ² Every variable is assigned a value from its domain, such that all constraints are satisfied ± All solutions are born equal. There is no better or best solution! Example ³ Variables: a, b, c ³ Domains: A = {1, 2, 3} ; B = {2, 3, 4, 5} ; C = {1, 3, 5} ³ Constraints: ² a 2 < b ; c != b ; a

IBM Haifa Research Lab Variable assignment problems Constraints Any relation CSP flexible modeling vs. strong optimization ILP Linear Disjunction of literals SAT Boolean 6 Final & discrete Variables domains DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM Integers naveh@il. ibm. com

IBM Haifa Research Lab Beyond the traditional definition ³ What’s a solution? ² Traditionally: any assignment that satisfies the constraints ² Optimization: the “best” solution ² All solutions ² Our case: a random solution ³ Hard and soft constraints ² Some constraints are mandatory ² Others aren't: A hierarchy of constraints ² Variants: fuzzy CSP, semi-ring CSP, cost CSP, … ³ Conditional CSP ² Variable dependent problems ² (a = 2) (add variables b 1, b 2, . . . bn to the CSP) ³ Robustness, flexibility, more 7 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Applications Scheduling Circuit design Machine Vision Graph problems Machine design and manufacturing Configuration Planing genetic experiments Floor plan design 8 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM Workforce management naveh@il. ibm. com

IBM Haifa Research Lab Solution algorithms Systematic search 9 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics Stochastic search © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Systematic* search: building blocks for an algorithm * AKA as exhaustive, backtrack based, … 3. Value ordering 2. Variable ordering 10 x Red, blue, green, … y 1. Pruning z 4. Backtracking a DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Consistency: a single constraint X Y Z {1, 2, 3} R: (x, y, z) in Xx. Yx. Z, x=y+z {1, 2, 3} A constraint is consistent if every value of every variable is supported by at least one tuple of values from all other variables 11 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Solution algorithm: maintaining arc-consistency [ Mackworth, 1977 ] The process: reducing domains to single values 1. Make all constraints locally consistent ³ An iterative process ³ Achieve fixed-point 2. Choose a variable: address 3. Choose a value: address 0 x 1234 ³ 0 x 1234 in domain ( address ) 4. Go to step 1 5. On failure - backtrack ³ Failure results in an empty set / domain 12 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Sometimes, arc-consistency is not enough != != != 1, 2, 3 1, 2 1 != != != 1, 2 2 1, 2, 3 != != 1, 2 1 != != 2 But sometimes it is … 13 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Graph width a c b ³ Vertex: the number of edges from previous vertices ³ Order: max (width of vertices) ³ Graph: min (width of all orders) a b b c c b c a b a 1 14 a 1 1 2 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM Graph width: 1 naveh@il. ibm. com

IBM Haifa Research Lab Backtrack free search ³ When width equals 1: ² Make the constraint graph arc consistent ² Instantiate the variables in the graph according to the 1 -width order ² No backtracking is required ³ When width equals n: ² No backtracking required if graph is n+1 consistent [ Freuder (1982, 1985) ] 15 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Solution algorithms Systematic search 16 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics Stochastic search © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Limitations of systematic methods: an example Only solution: Local consistency at onset: Choose randomly with probability 1/N of being correct (Solution reached at 600 million years) 17 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Limitations of systematic methods: another example Propagation is computationally hard 18 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Stochastic search - the basic algorithm A cost function is defined for full assignments ³ Random initial assignment ³ Hill climbing: ² Modify the best / random variable ² Random walk* on local minima ³ After n iterations, give up and try again Essentially an optimization problem See: ³ GSAT and its variants ³ Simulated annealing 19 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Stochastic search – cont’ ³ Works well for ² Cases where local-consistency is far from global consistency ² Constraints that are hard to propagate, domains that are difficult to represent ² Randomly generated problems ³ However … ² On failure: doesn't prove solution doesn't exist ² Requires reasonable heuristics (a “good” topography) ³ Mixed paradigm approaches ² Start systematic, move to stochastic before backtracking ² The other way around: use stochastic search to find a partial assignment, continue systematically from there 20 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Solution algorithms Systematic search Stochastic search Tools and Constraints Programming (CP) 21 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Tools ³ Constraints Programming: the method of building programs (and applications) based on constraints ³ ILOG ² Provides both a C++ library and an interpreted language (OPL) ² Both CSP and ILP ² Also: adaptations to common applications (e. g. scheduling) ³ Constraints Logic Programming (CLP): prolog based environments ² SICStus, ECLi. Pse, GNU Prolog, … ³ Other: many academic languages / environments ² E. g. , Mozart / OZ 22 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Tools – cont’ IBM’s tools ³ Generation Core ³ Stocs … 23 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Stimuli generation for hardware verification As opp o. g. , m sed ³ Functional verification: Show that a design (implementation) conforms to to fo (e its specification, before cast in silicon ³ The main method today: Simulation Stimuli Generator rma l ver ifica l ch tion ecki ng) Stimuli (test-case) Specification Expected behavior 24 ode DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics ? = Implementation ? = © Copyright IBM Actual behavior naveh@il. ibm. com

IBM Haifa Research Lab The significance of functional verification ³ Roughly 70% of the design effort (time, resources, …) is invested in functional verification ³ Industry practice: verification == over 90% simulation based verification ³ A design re-spin may cost many millions of $ ² Masks ² Person-month ² Time-to-market [ Source: Synopsys 2004 user survey ] 25 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab System model: ³ What’s valid ³ What’s interesting Random stimuli generator User requirements Random stimuli generator Generate N tests N distinct tests ³ Valid, interesting ³ Satisfy user requirements A single test line * COMMENT_PPC SDr 0Mc 0Sp 0Co 0GR_0 stmd ra: 0 x 0000_671 E 0410 * len: 0 x 8 wimg: 0 x 2 ea: 0 x 0000 D 6 F 3_732 F 8410 * va: 0 x 0001_02465 BFD_532 F 8410 ps: 12 data: 90003 F 2 DC 1 F 5 B 8 B 1 * translation: on I 0000 EB 000020 FBF 90003 * EA=000002 ED 05000020 WIMG=2 stmd G 31, 0 x 0(G 25) 26 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Why CP? Constraints originate from three sources 1. Validity of the stimuli: Constraints defined by the specification 2. Verification task: Constraints defined by the user 3. Bias towards interesting tests: Soft constraints defined by domain experts Validity: Complex EA to RA translation User: EA aligned to 64 K RA in some corner memory space Effective Address: 0 x 0 B 274 FAB_0 DBC 0000 Real Address: 0 x 0002 FFC 5_90 A 4 D 000 Expert knowledge: Reuse cache row 27 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Not just IBM ³ Constraint satisfaction is the basis for modern stimuli generation across the industry ³ 42 nd DAC: ²The largest conference of the EDA industry: 6000 participants ²A tutorial about constraint satisfaction in stimuli generation “ Constraint-Driven Test Generation With Specman Elite's constraint-driven test generation, you can now automatically generate tests for functional verification. By specifying constraints, you can quickly and easily target the generator to create any test in your functional test plan …” ³ Initiated and led by IBM for more than a decade, though… 28 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Random stimuli generator (2) User requirements Generate N tests System model: ³ What’s valid ³ What’s interesting Constraint Satisfaction Problem CSP Solver Random stimuli generator N distinct tests ³ Valid, interesting ³ Satisfy user requirements 29 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab CSP characteristics and challenges • Find many random, uniformly distributed, solutions of the same CSP (many different tests from the same template) • Huge domains (e. g. , 2^64) • In conjunction with arithmetic, bit-wise, and other types of constraints • Representation and operations on sets becomes a major issue • Global, extremely complex constraints (e. g. , hardware translation tables) • Periodic, unbounded CSP (a number n of weakly-coupled, closely-similar CSP’s, where n is itself a CSP variable), conditional CSP • Path-based CSP • Large problems: Up to 10^4 variables, 10^5 constraints • Constraint hierarchy • Up to ten levels of soft constraints – according to level of interest 30 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab and performance is an issue, too … 31 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Scenarios ³ CPU instruction model ³ Very Large Instruction Word ³ Sequential execution ³ Path-based CSP ³ Vector transfer of data ³ Address translations ³ Floating point verification (computationally hard propagation( ³ more 32 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Test program constraints Quality: sum zero add R 1 R 2 + R 3 load Rx 1000 (Ry) ? ? ? , Rz mult Rz R 6 x R 7 Validity: x != y User request: same register 33 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Sequential generation ³ Instructions are generated one at a time, and then executed by an ISS ³ Cannot generate all instructions simultaneously ²Model is too complex ²Problem is too large ²Constraint propagation computationally hard Generate Configuration Initial state Generate Instruction 1 ISS State 1 Generate Instruction 2 ISS ±e. g. , MUL instruction ³ Problem: State 2 Generate Instruction 3 ²Instruction 3 may require a specific configuration ISS Final state ± move_to_special_register requires privileged mode 34 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Initial state generation: ad-hoc solutions ³ Configure initial state according to required instructions ² Intense investment of manual labor ³ Configure initial state to be the least restrictive ² Initial state is the permissive even for tests with no special requirements ² Coverage is compromised ³ Configure the initial state randomly ² Large failure rate on tests with special requirements 35 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Initial state generation: A machine-learning solution ³ Machine learning is used to calculate a favorable initial state configuration ² mimics the manual labor otherwise invested Initial state space Favorable initial state space Approximated favorable initial state 36 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Path-based CSP in systems ³ Transactions go through a number of components, via a path ³ Each component on the path adds its own constraints ² Express-bridge behaves differently than a regular bridge ² Each memory has its own address space PLB Arbiter DSP Microprocessor Interrupt Controller PLB SRAM 2 Custom Logic SRAM 1 DMA Engine Bridge Express Bridge PCI USB 37 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM EMAC naveh@il. ibm. com

IBM Haifa Research Lab Path-based constraints ³ Constraints are also imposed directly on the path ² Request for a certain component ² Request for a certain path (“two neighboring identical bridges”) ³ Biasing for collisions, and for weak links ² Use the same component in different transactions ² Use one of the known prone-to-bugs interfaces between components ³ Problem: ² Solve simultaneously for constraints on paths imposed by component properties, and imposed directly ± A large and complex CSP, with most variables being conditional on the path solution 38 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Path based CSP: solutions ³ Ad hoc: solve for the path first, fulfilling only the direct constraints, then solve the complete CSP on that path ² Large number of failures because of constraints imposed by components on the chosen path ³ A more advanced solution ² Perform a static analysis of the problem ² Use this analysis at each new generation ² Problems: ± A very long static-analysis time; needs to be re-done each time the design model changes ± Still some failures, each requiring manual intervention ³ A ‘real’ solution: Does it exist? 39 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Vector transfers of data Node #1 Cpu Clustering Adaptor Clustering Network Node #2 Cpu Clustering Adaptor Mem Sender Receiver ³ CPU #1 initializes send buffers descriptor list in memory ³ CPU #2 initializes receive buffers descriptor list in memory ³ CPU #1 kicks off the transfer via MMIO access ³ Adaptors communicate and transfer data from sender memory to receiver memory 40 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Buffer Descriptor List Head = 0 x 2000 ³ Data structures initialized in memory (point to data areas) Address = 0 x. C 800 0 x 2000 Length = 128 0 x. C 000 Next = 0 x 3000 0 x. C 1 FF Address = 0 x. C 000 Data 0 x. C 800 Length = 256 0 x. C 8 FF 0 x 3000 Next = 0 x 4000 0 x. F 000 Address = 0 x. F 000 Data Length = 1024 0 x. F 3 FF 41 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics 0 x 4000 Next = 0 x 0000 © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab User controlled variables Total Length Data Descriptors Vector Size Length Address Next 42 Instance #1 Instance #0 MM_Address Length Address MM_Address Next DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics Instance #2 Address Length MM_Address Next © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Address translation ³ Complex translation paths for addresses, as viewed by different components ² Virtual to physical addresses in processors ² Similarly exists in other types of components, e. g. , Infini. Band HCA ³ Involves huge translation tables ² Millions of entries – implies non-trivial implementation of translation constraint ³ Complex constraints, rely on all previously generated instructions ² If VA was used, use same PA; Otherwise create a new translation path ± Needs to propagate in both directions (VA t PA, PA t VA) ² Bias: reuse existing entries in translation tables ³ A complex modeling problem 43 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab A Power. PC example Effective Address 32 bit Mode 64 bit Mode Actual Effective Address Real Mode LPAR Mode Segment Translation TA Mode Limit Cross SLBs Virtual Address Exception Page Table No. Exec. Seg Page Translation Protection DAC Intermediate Real Address Final Real Address 44 No Exception DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Solution: A ‘translation table’ modeling building block ³ The modeler describes the translation table, a complex set of constraints is then automatically added (Adir et al. , MTV 2003) ² This allows for completely worked-out implementation ± The constraint can propagate in all directions ± Performance may be optimized ³ A translation table model ² Number of key attributes, number of data attributes ² Location in memory / registers ² Translation function ± Hash bits ± Offset bits ± Relation between entries ² More 45 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Floating point bugs ³ Correcting or finding workarounds for floating point bugs on silicon tends to be very difficult, if not impossible ³ Incorrect result of a floating point instruction may generate a disaster 46 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM 2+2=5 naveh@il. ibm. com

IBM Haifa Research Lab Floating point verification ³ ³ ³ Represented as mantissa and exp. ³ Limited number of bits: exp: 11 mantissa: 53 non-continuous domain, rounding mantissa: 53 exp: 11 ³ Constraints: ²'op' itself ²bit #n = '0' ²Number of '1's = m ²a in [a 1. . . a 2] ³ MAC becomes impractical ²Use stochastic search 47 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Results: Floating-point unit verification Comparison with ZChaff for floating-point multiply benchmark (133 solvable tasks) Typical task: a*b=c, a, b, c contain exactly five 1’s. ZChaff SVRH Max length 64 bit 128 bit Average time 200. 5 sec 0. 97 sec Best ratio 2861 sec 0. 3 sec Worst ratio 25 sec 5. 7 sec Quality (extreme case) 0 p 0=0 x 43 F 0000000 0 p 1=0 x. B 180000000 0 p 0=0 x 070 E 342575271 FFA 0 p 1=0 x 9560 F 399 ECF 4 E 191 Reports UNSAT Yes No 48 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab We welcome help from the Academia! ³ We are struggling daily with extremely challenging issues ² As hardware becomes more complex ² As business requirements become tighter ³ Some of the pervasive items are: ² Random uniform solutions, huge domains, hard propagators, periodic/unbounded CSP, sequential generation, … ³ The problems are REAL – they require extensive research and basic theoretical solutions ³ Any good solution will likely inflect on the quality of tomorrow’s hardware systems ² Servers, PC’s, mobile phones, set-top boxes, … 49 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com

IBM Haifa Research Lab Summary ³ Constraint satisfaction is central to stimuli generation ² And therefore to hardware verification as a whole ³ It represents specific challenges: ² Huge domains ² Uniformly distributed solutions ² Hierarchy of constraints (hard, soft) ² Path-based CSP ² Conditional CSP ² Unbounded CSP ² More ³ It provides some food for thought in walking tours ² Enjoy the tour! 50 DAC /2005 CSP Tutorial. Verification 2006 LAA Constraints and / Advanced Topics © Copyright IBM naveh@il. ibm. com