Detecting Processor Hardware Faults by Means of Automatically

Detecting Processor Hardware Faults by Means of Automatically Generated Virtual Duplex Systems Markus Jochim University of Essen Germany Markus Jochim University of Essen, CS-Department Dependability of Computing Systems 06. 18. 2002

Virtual Duplexsystem (VDS) / Diversity Compiler A Input Source A, © 2002 Smith for (i=1; i<10; i++) {. . . } VDS: 1) Execute Prog. A 2) Execute Prog. B 3) Compare results Problem: Identical wrong values ! Markus Jochim University of Essen, CS-Department Dependability of Computing Systems Compiler B Source B, © 2002 Miller Program A Program B while (i++ <10) {. . . } Output Processor 2 Fault. Injector 06. 18. 2002

DIVERSI-Tool Markus Jochim University of Essen, CS-Department Dependability of Computing Systems 3 06. 18. 2002

DIVERSI-Tool Applies modification rule to VDS (P 0, Pi) Input: • Assembler Source: VDS (P 0, P 0) • Input values for P 0 • Fault specification (FAIL) Result Database Selects a VDS and a modification rule Injects into VDS (P 0, Pi+1) Output: • Most successful VDS • Error detection rates Markus Jochim University of Essen, CS-Department Dependability of Computing Systems 4 06. 18. 2002

Modificator The Modificator: • Introduces diversity. • Pool of modification rules. • Semantical equivalence. Input: • VDS (P 0, Pi) • Rule number: r Pi Rule r Pi+1 Output: VDS (P 0, Pi+1) Modificator About 30 modification rules do exist: Examples: • Address calculations • Rearrange code blocks • Introduce data diversity • Replace conditional instructions • Introduce test functions • Alternative calculations • . . . Markus Jochim University of Essen, CS-Department Dependability of Computing Systems 5 06. 18. 2002

Modification rule Inst. 1 Inst. 2 Inst. 3 Inst. 4 Inst. 5 Inst. 6 Inst. 1 Inst. 2 VDS (P 0, P 0) Inst. 3 Inst. 4 Inst. 5 Inst. 6 Inst. 3 Inst. 4‘‘‘‘ Inst. 5 Inst. 6 Modification rule r Markus Jochim University of Essen, CS-Department Dependability of Computing Systems Inst. 1 Inst. 2‘‘ VDS (P 0, P 1) 6 06. 18. 2002

DIVERSI-Tool Applies modification rule to VDS (P 0, Pi) Input: • Assembler Source: VDS (P 0, P 0) • Input values for P 0 • Fault specification (FAIL) Result Database Selects a VDS and a modification rule Injects into VDS (P 0, Pi+1) Output: • Most successful VDS • Error detection rates Markus Jochim University of Essen, CS-Department Dependability of Computing Systems 7 06. 18. 2002

Fault Injection Many “eachs“ and “alls“ means: For each VDS produced during optimization, perform all injections for all input values. “Fault injection technique must be very fast! " • Host & Target on the same system • Insertion of FI-code into VDS • Manipulate executable VDSs • “Power-Save-Mode“ • “Pessimistic fault injection“ Fault specification: • Fault Injection Language (FAIL). • Introduction of new faults and new fault types! • 49 different fault types are specified. Markus Jochim University of Essen, CS-Department Dependability of Computing Systems 8 06. 18. 2002

Fault specification (FAIL) All my. Var 1 substitute by {%eax, %ecx, %edi, %ebp} All my. Var 2 substitute by {0, 1, 2, 6, 7, 8, 9, 15, 16, 22, 23, 28, 31} Stuck-at dice{0, 1} Bit my. Var 2 Register my. Var 1 End. All Stuck-at-faults into registers All my. Var 3 substitute by {Carry, Zero, Sign, Overflow} All my. Var 4 substitute by {cleared, inverted, set} Flag. Problem. After dice {addl, subl, cmpl} Instruction my. Var 3 is my. Var 4 Endall All my. Var 5 substitute by {text, rodata} All my. Var 6 substitute by {1, 3, 5, 9, 20, 30, 40, 50, 60, 75, 95, 100, 200, 300} Memory. Error randomly inject my. Var 5 Errors into my. Var 6 Segment End. All Markus Jochim University of Essen, CS-Department Dependability of Computing Systems 9 Processor. Statusword Memory faults 06. 18. 2002

DIVERSI-Tool Applies modification rule to VDS (P 0, Pi) Input: • Assembler Source: VDS (P 0, P 0) • Input values for P 0 • Fault specification (FAIL) Result Database Selects a VDS and a modification rule Injects into VDS (P 0, Pi+1) Output: • Most successful VDS • Error detection rates Markus Jochim University of Essen, CS-Department Dependability of Computing Systems 10 06. 18. 2002

Strategy: “Greedy Optimization“ • Very simple strategy. • Not too many costy evaluations. v. Basis = V(P 0, P 0); cycle = 1; rule. Max = Number of available rules; do { For r=1 to rule. Max V[r] = call Modificator(v. Basis, r); undetected[r] = call Injector(Vr); Next v. Basis = V[r] with min(undetected[r]); cycle++; } until( min(undetected[r]) = 0 OR no improvement in last cycle) Markus Jochim University of Essen, CS-Department Dependability of Computing Systems 11 06. 18. 2002

Experiments Experiment setup: • Diversification of 9 different programs. • Average number of VDSs: 209. • For each VDS: -5 inputs · 4800 faults = 24000 fault injections. - 49 different fault types. Total number of 4, 5 · 107 injection runs . for example • Stuck-at faults in different registers (single + multiple) • Wrong stack pointer increment / decrement • Faulty processor status word • Faulty opcode interpretation • Faulty register selection • Memory transfer faults • Faulty evaluation of jump conditions • Stuck-at in results of e. g. arithmetic operations. . Markus Jochim University of Essen, CS-Department Dependability of Computing Systems 12 06. 18. 2002

Results . Markus Jochim University of Essen, CS-Department Dependability of Computing Systems 13 06. 18. 2002

Results Program Opt. cycles Undetecte Percentag d faults e (first (best VDS) (best/first) VDS) hexint 9 1904 39 2, 05% arraywrk 8 1823 23 1, 26% adder 7 793 60 7, 57% bintree 8 2215 0 0, 00% hanoi 8 943 15 1, 59% quicksort 9 1988 160 8, 05% bubblesort 10 2175 5 0, 23% crc_table 11 3574 15 0, 42% crc 11 2710 0 0, 00% . Markus Jochim University of Essen, CS-Department Dependability of Computing Systems 14 06. 18. 2002

. Further experiments: • Design diversity & DIVERSI-Tool • Identical setup • 4 program pairs: Pconst Pvar quicksort bubblesort quicksort crc_table crc 15

Results Pconst/ Pvar Opt. cycles Undetected (first VDS) Undetected (best VDS) crc/ crc_table 8 405 0 crc_table/ crc 10 400 0 quick/ bubble 4 526 0 bubble/ quick 4 526 0 . Markus Jochim University of Essen, CS-Department Dependability of Computing Systems 16 06. 18. 2002

Conclusions • Automated VDS production is possible. • Extensive evaluation. (Fault injection techniques / language) • Substantial reduction of undetected faults. . • Lowering of costs. • Limitations: - Fail stop behaviour. - Detection of processor faults. • Combinable with other techniques (e. g. design diversity) Markus Jochim University of Essen, CS-Department Dependability of Computing Systems 17 06. 18. 2002