Bad Software Greg Hoglund CTO Cenzic Inc greg cenzic

Bad Software Greg Hoglund CTO, Cenzic, Inc. greg@cenzic. com

What is Bad Software? • Software that exposes confidential data to un-authenticate users • Software which crashes or grinds to a halt when exposed to faulty inputs • Software which allows an attacker to inject code and execute it • Software which executes privileged commands for an attacker

Denver Airport Baggage • Unmanned carts on a track • Bad failure recovery/detection – Piles of fallen bags would not stop the unloaders • Carts got out of sync – Full carts continue to get loaded – Empty carts get unloaded • Delayed airport opening for 11 months – $1 million dollars a day in cost due to interest bond issues

The last photo taken by the Mars Lander before it plunged to it’s death. * *This photo was found on the Internet. It has not been independently verified.

NASA Mars Lander • Failed translation – English units into metric units – major error in spacecraft's path as it approached Mars • Crashed into the planet – Shut off descent engines prematurely • Taxpayer cost: $165 Million

4 Marines Killed • MV-22 Osprey Helicopter Crash • Burst hydraulic failure • Software caused backup system to fail

Do these look alike?

Navy shoots down Civilian Airliner • IN 1988, the US Vicennes shoots down Airbus 320 • 290 human lives lost • “cryptic and misleading output displayed by the tracking software “

Microsoft’s $8. 5 billion mistake • I LOVE YOU was only possible because Microsoft Outlook was designed to execute programs that were mailed to it.

Why we have Bad Software • Networked Software is not designed to withstand a hostile environment • Development tools do not prevent simple security bugs (i. e. , buffer overflows) • QA Testing methods do not address security • Customers pay for bad software

Getting Worse • In order to compete, new services must be delivered • New technology is not being properly tested for failures • More connections, devices, and code

More Devices What happens when buffer overflows and poor access controls lead to mobile code attacks on cellular phones? Mobile code can effect distributed systems in a matter of hours

More Connections • New protocols, delivery mediums • A high degree of connectivity makes it possible for small failures to propagate and lead to massive outages – Telephone network outages – Power system grid failures

More Code • Technology is being ‘glued’ together • More feature rich, more drivers and libraries –In 1983, Microsoft word was only 27, 000 LOC Code Size 400, 000 17 million 40 million 10 million 7 million 35 million Under 5 million 1. 5 million Solaris 7 Netscape Space Station Space Shuttle Boeing 777 NT 5 Windows 95 Linux

More Exposure • Massive increase in connectivity • A vast network of relationships – Arpanet started with 12 nodes • Machines that used to work behind closed doors are now exposed – Computers are now worn on belt-loops

5 Million Backdoors • 5 – 50 bugs per 1000/lines of code [Vaos/Mc. Graw]* 1 LOC ~ 10 bytes X 30, 000 HOSTS 4. 5 Billion X 10% = 500 Million Security Bugs 500 Million X 10% = 5 Million Remote Security Bugs = 3000 EXE’s ~100 K per EXE = 10, 000 LOC / EXE 5 Bugs/1000 LOC = 50 bugs/EXE 150, 000 Bugs/ 4. 5 Billion bugs Host

Software is always in the “bleeding edge” phase • Windows 2000 shipped with 63, 000 known bugs

Software sucks because you buy it • Yes, YOU the CONSUMER play a part in demanding bad software • To demand new features in a very short time frame creates a time-to-market problem for reliable software – Will you wait two years for the features you want? – Will you pay 10 -times as much to get those features?

Deja Vu • The same software bugs just keep hanging around – We knew about buffer overflows 15 years ago • We are slow to adopt ideas – When will customers hold vendors liable for buffer overflows? – Is it reasonable to accept buffer overflows in production code?

Other Industries Get Sued • Software shops gather around to defer bugs, decide which ones to ‘patch later’, and which ones to ignore • In other industries, safety flaws that are not corrected result in major class-action suits

How come vendors don’t fix this stuff? • They can afford not to! • Hardware is expensive to replace – so huge investments are placed into testing hardware prior to release – Intel F 00 F bug cost $500 million • Software bugs can be patched and downloaded from a web-site – They pass the cost of a bug to the customer

Software is not a Steel Bridge • The methods used for testing in traditional analog systems do not apply to software • With a bridge, you extrapolate results – What happens in between a 1000 kg test and a 10, 000 kg test? – The system is continuous – State changes are gradual and predictable

Discrete systems • State changes are not predictable • Numbers can change between 00001111 and 11110000 in an instant

Let the compiler do the Diagnostics • If programmers had to book time on the mainframe two weeks in advance, they would invest countless hours checking their work • Code hackers today just bounce code off the compiler until all the errors go away – This puts the responsibility of “code review” on the compiler

Form follows Failure • Sub-synchronous resonance in power systems – The addition of series AC capacitors in high energy power systems increases electrical stability – However, due to line inductance, the capacitors create electrical oscillations that effect the mechanical generator • Mohave Generating Station, Southern Nevada, 1971 – This snapped the drive shaft on a generator twice before it was properly diagnosed – This phenomenon is now a serious consideration is any power system design

How to Fix Bad Software • Better compilers and languages – More formal, more tractable • Failure analysis and fault-injection • Hold vendors liable • Stop buying it

Security Testing Security testing requires attacking the software. The software should be tested for the unexpected and the unknown. Software will never be placed or deployed into a trusted or predictable environment.

The Missing Leg of Software Reliability Functional Reliability Performan ce Functional Security Traditional QA testing methods have never addressed security. Software systems cannot be reliable unless they are secure.

Security Testing History • • • Attack and Pen Source Code Review Network Scanning Fault Injection Full Disclosure

Fault Injection • • Source code changes require recompile Binary instrumentation requires host agent API input testing requires test harness Network input testing requires additional network node

Black Box • Can be automated • Can easily find ‘low hanging fruit’ • Automated Tools: – ISICS – Spike – Hailstorm™ – PROTOS

MSQL Overflow with Spike s_binary("12 01 00 34 00 00 00 15 00 06 01 00 1 b"); s_binary("00 01 02 00 1 c 00 0 c 03 00 28 00 04 ff 08 00 02"); //this is probably a length field s_binary("10 00 00 00"); //make this big s_string_variable("MSSQLServer"); s_binary("00 24 01 00 00");

UDP-1434 SQL Overflow Buffer Attack Injected Into Protocol Statement

0040 e 890 0040 e 895 0040 e 896 FAULT ->0040 e 898 0040 e 89 a e 87 b 8 cffff c 3 8 bc 0 8 b 10 33 c 9 call ret mov xor

White Box • IDA-Pro (reverse assemble) • More expensive and requires an expert • Very time consuming

IDA reverse of popular app-server’s “Canonicalize. URIPath”

A Fusion – Grey Box • Combines: – A runtime debugger • Soft. Ice • GDB – A white box tool • IDA – A black box tool • Hailstorm™

Using Instrumentation • Using Rational Purify™ • Using API call hooks • Using Code-coverage (gcov, etc) – Cananocalization routines – Filtering routines – Decision logic – Parsers

Hailstorm™ crashes MS-SQL 7

Input Path Tracing • Path tracing – ltrace – truss • Data tracing – Gdb breakpoints – Modified ltrace • Where is user-data getting placed? – Trusted API calls?

Boron Tagging with GDB. text: 00056140 INTutil_uri_is_evil_internal: . text: 00056140 ldsb [%o 0], %o 1. text: 00056144 mov 1, %o 3. text: 00056148 mov 2, %o 4. text: 0005614 C cmp %o 1, 0. text: 00056150 be, pn %icc, loc_561 F 4. text: 00056154 mov %o 0, %o 5. text: 00056158 mov %o 2, (gdb) x/8 s $o 0 %o 0. text: 0005615 C mov 0, %o 2 0 x 97 f 030: “/iplanet/servers/TEST_STRING”. text: 00056160 cmp 0 x 97 f 064: %o 1, 0 x 2 F "ervers/docs". text: 00056164 0 x 97 f 070: "/usr/local/iplanet/docs". text: 00056164 loc_56164: 0 x 97 f 090: "". text: 00056164 bne, a loc_561 DC 0 x 97 f 091: %icc, "227ð230" 0 x 97 f 095: "" 0 x 97 f 096: "" 0 x 97 f 097: "" TEST_STRING

Using TRUSS on Solaris # truss -u *: : -vall -xall -p 2307 2>&1 | grep –v read | grep –v poll The 2>&1 tag is required since truss does not deliver all of it’s data on the stdout pipe. The output of the command will look something like: /67: <- libns-httpd 40: __0 FT_util_strftime_conv. Pci. TCc() = 50 /67: -> libns-httpd 40: __0 FT_util_strftime_conv. Pci. TCc(0 xff 2 ed 342, 0 x 2, 0 /67: <- libns-httpd 40: __0 FT_util_strftime_conv. Pci. TCc() = 0 xff 2 ed 345 /67: <- libns-httpd 40: INTutil_strftime() = 20 /67: -> libns-httpd 40: INTsystem_strdup(0 xff 2 ed 330, 0 x 9, 0 x 41, 0 x 50) /67: -> libns-httpd 40: INTpool_strdup(0 x 9 e 03 a 0, 0 xff 2 ed 330, 0 x 0) /67: -> libc: strlen(0 xff 2 ed 330, 0 x 0) /67: <- libc: strlen() = 20 /67: <- libns-httpd 40: INTpool_strdup() = 0 x 9 f 8 b 10 /67: <- libns-httpd 40: INTsystem_strdup() = 0 x 9 f 8 b 10 /67: <- libns-httpd 40: time_cache_curr_strftime_logfmt() = 0 x 9 f 8 b 10 /67: -> libc: strcpy(0 xf 7400710, 0 x 9 f 8 b 10, 0 x 7 efefeff) /67: <- libc: strcpy() = 0 xf 7400710 /67: -> libc: strlen(0 xf 7400710, 0 x 9 f 8 b 28, 0 xf 7400710, 0 x 0) /67: <- libc: strlen() = 20 /67: -> libc: strlen(0 x 9 f 4 f 48, 0 x 34508 f, 0 x 0, 0 x 7 efefeff) /67: <- libc: strlen() = 25

Win 32 hook on strcpy

If there is code for it… • • • What if? Assume filters fail Assume API call input can be controlled Map the capability of every DLL Controlled by process permissions and access control

Every DLL that calls Set. Security. Descriptor. DACL

User Input • What can the user directly control in terms of API calls? – Authentication calls – Filesystem – Database – Command shell

Remote Capability • Do any of the native calls operate over the network? – Domain specification – Data source specification – Ip address – NTFS Path name

Authentication • Response aggregation – User/password enumeration when errors differ • No lockout – Brute force • Failed logging – Alternative requests • Can you specify a remote domain or target? – Proxied attacks

Filesystem • Can you control a filesystem path – What is the entire set of characters? • Can you create files in a target directory – Create files that will be interpreted in a server context • Can you use remote pathname – //machine_name/etc

Execution Flaws • Code Insertion • State Corruption • Fatal Exception

Architecture Flaws • Lack of randomness – Hijacking keys • No authentication – Bad configuration or design • No compartments – Use the same buffer for crypto and clear • Race conditions

Take control of the Problem • • • Test before you buy Perform independent testing on the software Perform internal testing on the software Cooperate and create a shared testing lab Create an acceptance criteria

Make the vendor responsible • Vote with your dollars • Force vendors into a comparison against competitive products • Make the vendor produce a technically credible security audit • Force vendors to accept liability associated with a security bug – Make the vendor pay the cost of a bug

It’s Ultimately Your Decision • As the customers of technology, you have the right to demand safety and reliability • Security knowledge is widespread • Reliable software is secure • Security testing is the only way to eliminate the bugs that undermine your systems

Thank You Greg Hoglund greg@cenzic. com www. cenzic. com