ACSAC 2005 Model Checking an Entire Linux Distribution

ACSAC 2005 Model Checking an Entire Linux Distribution for Security Violations Jacob West, Security Research Group, Fortify Software Work by Benjamin Schwarz, Hao Chen, David Wagner, Geoff Morrison, Jacob West, Jeremy Lin and Wei Tu

Outline n n n Introduction MOPS Background Analyzing Red Hat 9 ä ä ä n Vulnerability Examples ä ä n 1 Tool performance Human performance Security properties TOCTTOU Standard File Descriptors Temporary Files strncpy() Results

Introduction n n Over 50% of security vulnerabilities caused by coding errors Automated detection possible ä ä ä 2 Rapidly expanding field Academic and commercial Feasible at large scale

MOPS (MOdelchecking Programs for Security properties) n n n 3 Static analysis for security C programs Enforce temporal safety rules

Analyzing Red Hat 9: Overview n Tool performance ä ä ä n Human performance ä ä n Integration with existing build processes False positives Easy-to-review error traces Grouped error traces Security properties ä ä ä 4 Analysis of large code base feasible Compaction improves performance Reasonable resource requirements Temporal safety properties Employable by other tools Iteratively refined for low false positives

Analyzing Red Hat 9: Tool Performance n n n Red Hat 9: 839 packages, 60 million TLOC 732 packages (87%) 107 failures caused by parse errors ä ä n Compaction improves performance ä n Only consider relevant operations Reasonable resource requirements ä 5 73 packages contained C++ code 34 packages used unsupported C 99 constructs TOCTTOU takes about 10 hours on P 4 1. 5 GHZ / 1 GB

Analyzing Red Hat 9: Human Performance n Integration with existing build processes ä Integrated with rpmbuild, make Interposed on gcc ä Analyze multiple packages easily ä n False positives ä n Easy-to-review error traces ä n Navigate code quickly to verify error traces Grouped error traces ä 6 Relatively low, permits human review Understand multiple traces through representative samples

Analyzing Red Hat 9: Security Properties n Temporal safety properties ä n Pattern variables ä n n n e. g. foo(x); bar(x); where x is the same Iteratively refined to reduce false positives Employable by other tools Properties include ä ä 7 Security properties expressed as Finite State Automata (FSA) TOCTTOU: Time-of-check, to time-of-use race conditions Standard File Descriptors: Vulnerable uses of stdin, stdout and stderr Temporary Files: Insecure creation of temporary files strncpy(): Dangerous uses of strncpy()

Security Properties : TOCTTOU n n Time-of-check to time-of-use race conditions occur when a program checks the access permission of an object and, if the check succeeds, makes a privileged system call on the object. Example: if (access(pathname, R_OK) == 0) fd = open(pathname, O_RDONLY); 8

Security Properties : TOCTTOU n n 9 Checks: access(), stat(), etc. Uses: creat(), open(), unlink(), etc.

Vulnerability Example: TOCTTOU – binutils : : ar exists = lstat (to, &s) == 0; /* Use rename only if TO is not a symbolic link and has only one hard link. */ if (! exists || (!S_ISLNK (s. st_mode) && s. st_nlink == 1)){ ret = rename (from, to); if (ret == 0) { if (exists) { chmod (to, s. st_mode & 0777); if (chown (to, s. st_uid, s. st_gid) >= 0) { chmod (to, s. st_mode & 07777); }. . . 10

Security Properties: Standard File Descriptors n n Since the kernel does require that stdin, stdout and stderr point to terminal devices, an attacker may cause a victim program open one of them to a sensitive file. Example /* victim. c */ fd = open("/etc/passwd", O_RDWR); if (!process_ok(argv[0])) perror(argv[0]); /* attack. c */ int main(void) { close(2); execl("victim", "foo: <pw>: 0: 1: Super-User-2: . . . ", NULL); } 11

Security Properties: Standard File Descriptors n(… ope 12 ) States correspond to the status of the three standard file descriptors and transitions occur on a "safe" open (/dev/null and /dev/tty). ) n

Vulnerability Example: Standard File Descriptors - gnuchess void Book. Builder(short depth, . . . ){ FILE *wfp, *rfp; if (depth == -1 && score == -1) { if ((rfp = fopen(BOOKRUN, "r+b")) != NULL) { printf("Opened existing book!n"); } else { printf("Created new book!n"); wfp = fopen(BOOKRUN, "w+b"); fclose(wfp); if ((rfp = fopen(BOOKRUN, "r+b")) == NULL) { printf("Could not create %s filen", BOOKRUN); return; }. . . 13

Security Properties: Temporary Files n n Because many of the functions in the C standard library that create temporary files are insecure an adversary that is able to predict the filename can gain control of the file by precreating it. Example fd = mkstemp(action_file_name); . . . unlink(action_file_name); 14

Security Properties: Temporary Files n n 15 tmpnam(), tempnam(), mktemp() and tmpfile() are always unsafe mkstemp() is safe if the generated filename is not used

$Vulnerability Example: Temporary Files - yacc 16 static void open_files() {. . . fd$

Vulnerability Example: Temporary Files - yacc 16 static void open_files() {. . . fd = mkstemp(action_file_name); if (fd < 0 || (action_file = fdopen(fd, "w")) == NULL){. . . open_error(action_file_name); } } void open_error(char *filename) { warnx("f - cannot open "%s"", filename); done(2); } void done(int k) {. . . if (action_file_name[0]) unlink(action_file_name);

Security Properties: strncpy() n n First strncpy() encourages off-by-one errors if the programmer is not careful to compute the value of n precisely. Secondly, because the function does not automatically null-terminate a string in all cases it is a common mistake for a program to create unterminated strings during its execution. Example buf[sizeof(buf)-1] = ''; strncpy(buf, . . . , sizeof(buf)); 17

Security Properties: strncpy() 18

Vulnerability Example: strncpy() - xloadimage newopt->info. dump. type = argv[++a]; . . . dump. Image(dispimage, dump->info. dump. type, dump->info. dump. file, verbose); 19 void dump. Image(Image *image, char *type, char *filename, int verbose) { int a; char typename[32]; char *optptr; optptr = index(type, ', '); if (optptr) { strncpy(typename, type, optptr - type); typename[optptr - type] = ''; . . . }

Results Property 20 95% 41 56 61% 22 108 69% 34 Total n 790 Insecure Temporary Files n Real Bugs Standard File Descriptors n % FP TOCTTOU n Reported Warnings 954 90% 97 53/1358 79% 11/258* strncpy() 1358 strncpy() warnings; 53 audited; 11 real bugs* Projected found 2312 85% 355 200 human hours. Total 108 real bugs in 50 million lines of code Order of magnitude larger in scale than previous academic work Static analysis will be feasible and integral part of building systems

Questions? Want to talk more about software security? jwest@fortifysoftware. com