Can Automated Testing Tools Really Find the OWASP

Can (Automated) Testing Tools Really Find the OWASP Top 10? OWASP App. Sec Europe May 2006 Erwin Geirnaert Partner & Co-founder, ZION SECURITY erwin. geirnaert@zionsecurity. com +32478289466 Copyright © 2006 - The OWASP Foundation Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License. The OWASP Foundation http: //www. owasp. org/

Agenda

Introduction

Existing studies < Arian Evans – Tools Taxonomy – OWASP 2005 4 Personal experience with tools 4 Conclusion: still a lot of work < Dr. Holger Peine – Fraunhofer IESE 4 Test of App. Scan, Acunetix & Web. Inspect on Web. Goat and proprietary application 4 Conclusion: § A lot of false positives § A lot of false negatives § Not one tool did what it should do: find the easy vulnerabilities in Web. Goat (they can’t read the lesson hints ) < Reviews by Infoworld & e. Week OWASP App. Sec Europe 2006 4

Testing alert(document. cookie) 4 What should be the result of a test: how to detect a pop-up window in an HTML stream? 4 What should not be the result of a test: will a script tag embedded in another script tag really get executed? OWASP App. Sec Europe 2006 5

Testing

Testing

Automated Tools – Open-source

Automated Tools - Commercial

OWASP Top Ten Most Critical Web Application Security Vulnerabilities < A 1. Unvalidated Input < A 2. Broken Access Controls < A 3. Broken Authentication and Session Management < A 4. Cross Site Scripting Flaws < A 5. Buffer Overflows < A 6. Injection Flaws < A 7. Improper Error Handling < A 8. Insecure Storage < A 9. Denial of Service < A 10. Insecure Configuration Management OWASP App. Sec Europe 2006 10

Some questions

A 1. Unvalidated Input

A 1. Unvalidated Input < How to detect: examine result (and NOT error codes) and identify vulnerabilities 4 SQL Injection: parse for SQL error codes : S 4 No exception handling: parse for stacktraces? 4 Authorization bypass: is that a Admin-button? 4 Buffer overflow (Denial-of-Service? ): empty HTML-page? 4 LDAP Injection: different user attributes? 4. . . < Ultimate test: exploit vulnerability MANUALLY -> THIS REQUIRES THE TESTER TO KNOW THE ATTACK PAYLOAD < What about non-English web applications? OWASP App. Sec Europe 2006 13

A 2. Broken Access Controls

A 2. Broken Access Controls

A 3. Broken Authentication and Session Management

A 4. Cross Site Scripting Flaws < Definition: The web application can be used as a mechanism to transport an attack to an end user's browser. A successful attack can disclose the end user’s session token, attack the local machine or spoof content to fool the user. < Test: use RSnake’s cheat sheet for XSS filter evasion (http: //ha. ckers. org/xss. html) < Automated tools: some tools inject a limited XSS pattern and for some tool you don’t know what they inject and you CAN’T change it. But if you have a web site with 1000 forms they are very useful to automate the injection. But. . . If you find 1 XSS, you probably find more OWASP App. Sec Europe 2006 17

A 5. Buffer Overflows < Definition: Web application components in some languages that do not properly validate input can be crashed and, in some cases, used to take control of a process. These components can include CGI, libraries, drivers, and web application server components. < Test: replace every parameter with a lot of data: integers, strings, binary data, . . . < Automated tools: some tools inject a buffer-overflow patterns but with some tools you don’t know what they inject or you’re unable to change it. But if you have a web site with 1000 forms they are very useful to automate the injection < Results: crash of the web application, corrupt database, crash of the server so be very careful on a production environment < Automated tools: detect database corruption? OWASP App. Sec Europe 2006 18

A 6. Injection Flaws < Definition: Web applications pass parameters when they access external systems or the local operating system. If an attacker can embed malicious commands in these parameters, the external system may execute those commands on behalf of the web application. < Test: replace every parameter with command injection strings which depend on the operating system in use < Automated tools: some tools inject command injection patterns but with some tools you don’t know what they inject and it is impossible to change them. But if you have a web site with 1000 forms they are very useful to automate the injection < Results: output of the command injection must be obtained, how to automate this? E. g. Net user /add Erwin OWASP App. Sec Europe 2006 19

A 7. Improper Error Handling

A 8. Insecure Storage

A 9. Denial of Service < Definition: Attackers can consume web application resources to a point where other legitimate users can no longer access or use the application. Attackers can also lock users out of their accounts or even cause the entire application to fail. < Test: attempt to brute-force accounts, performance test, … < Automated tools: have no problem to attack accounts and they don’t execute performance tests but when attacking a site with full force it can have some unexpected side-effects OWASP App. Sec Europe 2006 22

A 10. Insecure Configuration Management

Conclusions < Automated Tools are not the silver-bullet to test for the OWASP Top 10 < They can help a security tester to assess a web application faster < Security tester must master the tools and know the limitations < Combine open-source tools with commercial tools < But automated tools will have difficulties with the latest technologies: 4 AJAX: asynchronous XML requests 4 One-time tokens like in Struts, SAP BSP, … 4 Thick clients e. g. Java Web Start 4 Web services OWASP App. Sec Europe 2006 24