An Automated Approach for Software Reliability and Security

An Automated Approach for Software Reliability and Security Zhen Xiao Senior Technical Staff Member AT&T Labs – Research Joint work with Christof Fetzer

Motivation • Software reliability is becoming increasingly important. – Financial transactions, virtual office environment, life critical applications, etc. • Does the software function correctly under exceptional or stressful settings? – Field experience indicates that the error handling paths in the software typically contain most bugs. – Testing all boundary conditions before the official release of the software can be prohibitively expensive.

Example: Robustness Violation strcpy(destination, source) source length destination unmapped pages detectable with help of signal handler

Security Problems strcpy(dst, attack_string) attack_string length destination shell script attack code We want to avoid all buffer overwrites!

Why C/C++? What is a good approach for increasing software reliability?

Challenges • Transparency – Cannot assume access to the source code for applications & libraries. • Cost effectiveness & scalability – Large number of applications, shared libraries, and functions. – Applications and libraries may change often. • Flexibility – Different applications may need different levels of protection

Most Popular Libraries

Distribution of #Libraries vs. #Applications

Linux Su. SE 8. 0 Number of shared libraries Number of applications 933 15431 Number of undefined func/appl Automation is essential for protecting a large number of libraries and applications! ~49

The HEALERS Approach • Fault-containment wrappers – Intercept function calls into the dynamic link libraries. – Provide transparent protection for software without source code access. • Automated fault-injection experiments – – Find all functions defined in a library Detect arguments that cause a function to crash Derive safe argument types for each function. Prevent heap and stack buffer overflows and a large set of robustness violations.

The HEALERS Approach (Cont’d) • Micro-generator architecture – Generate a variety of wrappers through a set of microgenerators. – Applications only pay the overhead they actually need.

Extracting Libraries and Undefined Functions

Example: Unix Wrappers • Overwrite “exit” by “abort” • Wrapper: void exit(int status) { abort(); } • Start wrapper: > setenv LD_PRELOAD `pwd`/wrapper. so • No need to change existing programs! > date Wed Oct 18 13: 06: 41 EDT 2000 Abort

Approach: Wrap Functions calls Application Wrapper strcpy(d, s) calls Shared Library strcpy(d, s) check d and s

Keeping Track of the Heap w Wrap malloc, calloc, realloc, free w Use Red/Black trees to keep meta-data for each allocated block – Update Red/Black tree whenever such a function is called – Red/Black node contains address range of each block • Check if the destination buffer is sufficiently large: – Search Red/Black tree to find pointer – returns #bytes between pointer and the end of the block • Complexity: – Insert, Remove, Find: O(log(entries))

Wrapper Structure

Original Function w Wrapper needs to call the “original” function w Access to original function by “dlsym” w Example: – dlsym(RTLD_NEXT, “strcpy”); – returns address of original “strcpy” function

Wrapper Structure

Wrapper Generation Process Shared Library Fault-Injection Experiments Function Specification Phase 1: Function Spec. Generation Wrapper Generation Phase 2: Wrapper generation Wrapper

Run-time checks [SRDS 2001] • Example: function asctime(d) – wrapper checks that d is buffer of sufficient size – if not, return error code – otherwise, call original function

$Asctime Wrapper char* asctime (const struct tm* a 1) { char* ret; if (!check_R_ARRAY_NULL(a$

Asctime Wrapper char* asctime (const struct tm* a 1) { char* ret; if (!check_R_ARRAY_NULL(a 1, 44)) { errno = EINVAL ; ret = (char*) (nil); goto Post. Processing; } asm("movl %ebp, %esp"); asm("popl %ebp"); asm("jmp *libc_asctime_0"); Post. Processing: ; return ret; }

Function Specification for asctime <function> <name>asctime</name> <argument>const struct tm* <robust_type>R_ARRAY_NULL[44]</robust_type> </argument> <return_type>char *</return_type> <error_value>NULL</error_value> <errors> <errno>EINVAL</errno> </errors> <attribute>unsafe</attribute> </function> Robust Argument Type

Generation of Function Specifications Fault. Injector Header Files Manual Pages Shared Library Generator Function Declaration Fault. Injector Function Declaration

Generator Algorithm For each function in library 1. Find prototype of function: • • Parse include files given in manual pages, or Parse all header files that might contain prototype 2. Generate fault-injector using prototype • • • Define sequence of hypotheses Perform automated fault injection experiments to test each hypothesis Select non-rejected hypothesis

Prototype Extraction for glibc 2. 2 Total: 1278 global functions

Errors in Manual

Computation of Robust Argument Types • Ideal Goal: determine set of argument values that crash a function crash no crash set of all argument values wrapper accepts these values, rejects others

Idea Goal Not Realistic • Revised Goal: Accurate but not necessarily complete checks: crash no crash set of all argument values wrapper accepts these values, rejects others

Approach • Divide argument value set in disjoint subsets set of all argument values Fundamental Type

Classify Fundamental Types Using Fault-Injections • Divide argument values in disjoint subsets no value crashes f all values crash f some values crash f wrapper accepts these values, rejects others

Example: Fixed Arrays set of all argument values WONLY_FIXED RW_FIXED RONLY_FIXED UNMAPPED RONLY_FIXED[1], RONLY_FIXED[2], RONLY_FIXED[3], … NULL INVALID read only not mapped 3

asctime: injection results • Crashes for all test cases in: – – RONLY_FIXED[i] for i < 44 WONLY_FIXED[i] for any i RW_FIXED[i] for i < 44 INVALID crashes • Does not crash for test cases in: – RONLY_FIXED[i] for i >= 44 – RW_FIXED[i] for i >= 44 – NULL

Type Hierarchy • Need to be able to compute union of value sets • Define type hierarchy: – fundamental types: value sets of any two fundamental types is non-overlapping – union types: value set of this type is the union of the value set of its “subtypes”

Fixed Array Type Hierachy UNCONSTRAINED R_ARRAY_NULL[s] s≤t RONLY_FIXED[v] s≤t W_ARRAY_NULL[s] RW_ARRAY_NULL[t] R_ARRAY [t] t≤v INVALID s≤t W_ARRAY[t] t≤u NULL t≤u t≤v RW_ARRAY [u] WONLY_FIXED[v] u≤v RW_FIXED[v]

asctime: Robust Argument Type UNCONSTRAINED R_ARRAY_NULL[s] s≤t RW_ARRAY_NULL[t] R_ARRAY [44] 44≤v RONLY_FIXED[v] s≤t W_ARRAY_NULL[s] s≤t W_ARRAY[t] 44≤ 44 t≤u RW_ARRAY [44] 44≤v RW_FIXED[v]

asctime: Robust Argument Type UNCONSTRAINED R_ARRAY_NULL[44] 44≤ 44 RW_ARRAY_NULL[44] R_ARRAY [44] 44≤v RONLY_FIXED[v] 44≤ 44 W_ARRAY_NULL[s] s≤t W_ARRAY[t] NULL 44≤ 44 t≤u RW_ARRAY [44] 44≤v RW_FIXED[v]

Phase 2: Generation of Wrappers Retry Wrapper Function Specification Wrapper Generator Security Wrapper Flags Robustness Wrapper …

Micro-Generators Generator

Wrapper Types w Data Collection Wrappers: – collect failure data and usage data w Security Wrapper: [SRDS 2001] – detect buffer overflows on stack and heap w Robustness Wrappers: [DSN 2002] – prevent segmentation failures – try to keep applications running w Retry Wrapper: [ISSRE 2002] – retry failed function calls w…

Wrapper Types w Data Collection Wrappers: – collect failure data and usage data w Security Wrapper: [SRDS 2001] – detect buffer overflows on stack and heap w Robustness Wrappers: [DSN 2002] – prevent segmentation failures – try to keep applications running w Retry Wrapper: [ISSRE 2002] – retry failed function calls w…

Creating Profiling Wrappers • Create Profile Wrapper: > profile_app ls • Run wrapped ls: > wrapped_ls/ls • Output of profile wrapper in XML

Profiling Wrapper

Wrapper Types w Data Collection Wrappers: – collect failure data and usage data w Security Wrapper: [SRDS 2001] – detect buffer overflows on stack and heap w Robustness Wrappers: [DSN 2002] – prevent segmentation failures – try to keep applications running w Retry Wrapper: [ISSRE 2002] – retry failed function calls w…

Security Wrapper • Currently the wrapper can detect – heap smashing attacks (caused by C-library func. ) – stack smashing attacks • Stack Smashing Detection: – based on an approach by Lib. Safe – uses gcc frame pointers to check that return address is not overwritten

Creating Security Wrappers • Create Wrapper for individual library: > protect_library /libc. so • Create Wrapper for all libraries: > protect_all_libraries • Info about Security Wrappers in XML

Wrapper Types w Data Collection Wrappers: – collect failure data and usage data w Security Wrapper: [SRDS 2001] – detect buffer overflows on stack and heap w Robustness Wrappers: [DSN 2002] – prevent segmentation failures – try to keep applications running w Retry Wrapper: [ISSRE 2002] – retry failed function calls w…

Robustness Wrapper tested with Ballista

Performance Measurements • Pentium 3, 864 Mhz, 384 Mbytes RAM • Linux 2. 4. 4 kernel, Su. SE 7. 2 • Performance data: – each data point is 10% trimmed mean of 100 executions

Performance

Related Work • Formal analysis – Can verify deep property of the system. – Usually abstract away many implementation details. • Static analysis – Can check all the control paths in the program – Requires source code access. • Ballista – Use fault-injection experiments to evaluate the robustness of libraries and operating systems. • Xept – Language/compiler to generate wrappers

Limitations • Cannot prove correctness of the program. • Only works for applications that are dynamically linked. • Only detect faults related to library functions. • The quality of fault-injection experiments depends on the coverage of the test cases.

Conclusion/Future Work • Automation achieves scalability. • Flexible architecture pays off in the long run – Facilitates code reuse. – Easy debugging, optimization. • Future work – Collect statistics on failure causes of common applications. – Generate better test hypothesis for fault-injection experiments.