Security Testing through Automated Software Tests Stephen de

Security Testing through Automated Software Tests Stephen de Vries, Principal Consultant, Corsaire stephen. de. vries@corsaire. com OWASP App. Sec Europe May 2006 Copyright © 2006 - The OWASP Foundation Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License. The OWASP Foundation http: //www. owasp. org/

Typical Iterative development life cycle 4 Sub bullet § Correcting issues is costly § The people who understand the code best, don’t perform the security tests § No buy-in from developers into the security process OWASP App. Sec Europe 2006 2

Typical Iterative development life cycle OWASP App. Sec Europe 2006 3

Typical Iterative development life cycle Integrating Security Tests in the process: ü A shorter security testing phase ü More robust applications because testing is deep ü Developer involvement in security OWASP App. Sec Europe 2006 4

Outline < < < < A taxonomy and description of testing types An introduction to JUnit and examples of its use Testing compliance to a security standard Testing security in Unit Tests Testing security in Integration Tests Testing security in Acceptance Tests Conclusion Q&A OWASP App. Sec Europe 2006 5

Use cases and Abuse cases < Use cases 4 Expected behaviour 4 Normal input 4 Functional requirements < Abuse cases 4 Unexpected behaviour 4 By Malicious agents 4 Derived from risk assessment 4 Developers should think evil which leads to… 4 Defensive programming OWASP App. Sec Europe 2006 6

Taxonomy of Automated Software Tests

Taxonomy of Automated Software Tests

Taxonomy of Automated Software Tests

Taxonomy of Automated Software Tests

Introducing JUnit “Never in the field of software development was so much owed by so many to so few lines of code. ” - Martin Fowler OWASP App. Sec Europe 2006 11

Introducing JUnit Example: A single method from a shopping cart class public void add. Item(Item item, boolean in. Stock) { Cart. Item cart. Item = (Cart. Item) item. Map. get(item. get. Item. Id()); if (cart. Item == null) { cart. Item = new Cart. Item(); cart. Item. set. Item(item); cart. Item. set. Quantity(0); cart. Item. set. In. Stock(in. Stock); item. Map. put(item. get. Item. Id(), cart. Item); item. List. get. Source(). add(cart. Item); } cart. Item. increment. Quantity(); } OWASP App. Sec Europe 2006 12

Introducing JUnit < Test that: 4 a new cart has 0 items in it 4 adding a single item results in that item being present in the cart 4 adding a single item results in the cart having a total of 1 items in it 4 adding two items results in both items being present in the cart 4 adding two items results in the cart having a total of 2 items in it 4 Test whether adding a null item results in an exception and nothing being set in the cart OWASP App. Sec Europe 2006 13

Introducing JUnit public class Cart. Test extends Test. Case { public Cart. Test(String test. Name) { super(test. Name); } protected void set. Up() throws Exception { //Code here will be executed before every test. XXX method } protected void tear. Down() throws Exception { //Code here will be executed after every test. XXX method } public void test. New. Cart. Has. Zero. Items() { // Test code goes here } public void test. Add. Single. Item() { // Test code goes here } public void test. Add. Two. Items() { // Test code goes here } public void test. Add. Null. Item() { // Test code goes here } } OWASP App. Sec Europe 2006 14

Introducing JUnit public void test. Add. Two. Items() { Cart instance = new Cart(); boolean is. In. Stock = true; Us //First add an item Item item = new Item(); item. set. Item. Id("item 01"); instance. add. Item(item, is. In. Stock); EC As. E t. Es t //Test adding a second item Item item 2 = new Item(); item 2. set. Item. Id("item 02"); instance. add. Item(item 2, is. In. Stock); //Check whether item 01 is in the cart boolean result = instance. contains. Item. Id("item 01"); assert. True("First item is in cart", result); //Check whether item 02 is in the cart result = instance. contains. Item. Id("item 02"); assert. True("Second item is in cart", result); //Check that there are 2 items in the cart assert. Equals("2 items in cart", instance. get. Number. Of. Items(), 2); } OWASP App. Sec Europe 2006 15

Introducing JUnit AB Us public void test. Add. Null. Item() { Cart instance = new Cart(); boolean is. In. Stock = true; EC As. E try { instance. add. Item(null, is. In. Stock); fail("Adding a null item did not throw an exception"); } catch (Runtime. Exception expected) { assert. True("null Item caught", true); assert. Equals("Null not in cart", instance. get. Number. Of. Items(), 0); } t. Es t } OWASP App. Sec Europe 2006 16

Introducing JUnit < Using JUnit 4 Supported in all Java IDE’s 4 Ant and Maven support 4 Part of the code-debug cycle OWASP App. Sec Europe 2006 17

Introducing JUnit < Other Unit testing frameworks 4 Java – JUnit, (www. junit. org), Test. NG (http: //beust. com/testng/), JTiger (www. jtiger. org) 4 Microsoft. NET – NUnit (www. nunit. org), . NETUnit (http: //sourceforge. net/projects/dotnetunit/), ASPUnit (http: //aspunit. sourceforge. net/), CSUnit (www. csunit. org) and MS Visual Studio Team Edition and many more. 4 PHP – PHPUnit (http: //pear. php. net/package/PHPUnit), Simple. Test (www. simpletest. org) 4 Coldfusion – CFUnit (http: //cfunit. sf. net), cfc. Unit (www. cfcunit. org) 4 Perl – Perl. Unit (http: //perlunit. sf. net), Test: : More (included with Perl) 4 Python – Py. Unit (http: //pyunit. sf. net), doctest (included in standard library) 4 Ruby – Test: : Unit (included in the standard library) 4 C – CUnit (http: //cunit. sf. net), check (http: //check. sf. net) 4 C++ – CPPUnit (http: //cppunit. sf. net), cxxtest (http: //cxxtest. sf. net) OWASP App. Sec Europe 2006 18

Web Application Security Standards < What is a Web Application Security Standard? 4 Derived from an organisation’s Security Policy 4 Similar to an Operating System Build Standard 4 Defines how the application should behave from a security point of view 4 Should include functional and non-functional security aspects OWASP App. Sec Europe 2006 19

Web Application Security Standards Example: OWASP App. Sec Europe 2006 20

Web Application Security Standard OWASP App. Sec Europe 2006 21

Testing Security in Unit Tests Test Valid Input public void test. Valid. Phone. Numbers() { //Test valid input String number = "232321"; acc. set. Phone(number); validator. validate(acc, errors); assert. False(number+" caused a validation error. ", errors. has. Field. Errors("phone")); number = "+23 232321"; acc. set. Phone(number); validator. validate(acc, errors); assert. False(number+" caused a validation error. ", errors. has. Field. Errors("phone")); number = "(44) 32321"; acc. set. Phone(number); validator. validate(acc, errors); assert. False(number+" caused a validation error. ", errors. has. Field. Errors("phone")); //etc… } OWASP App. Sec Europe 2006 22

Testing Security in Unit Tests § Test Invalid Input: public void test. Illegal. Characters. In. Phone. Number() { String number = "+(23)'; []232 - 321"; acc. set. Phone(number); validator. validate(acc, errors); assert. True(number+" did not cause a validation error. ", errors. has. Field. Errors("phone")); } public void test. Alphabetic. In. Phone. Number() { String number = "12 a 12121"; acc. set. Phone(number); validator. validate(acc, errors); assert. True(number+" did not cause a validation error. ", errors. has. Field. Errors("phone")); } OWASP App. Sec Europe 2006 23

Testing Security in Unit Tests < Advantages of testing at this layer 4 Tests are run very frequently - issues are identified quickly 4 Most granular form of test - high test coverage < Disadvantages 4 Not many security vulnerabilities can be tested at this layer OWASP App. Sec Europe 2006 24

Testing Security in Integration Tests < In-container testing 4 Realistic and complete 4 Requires specific tools 4 Overhead in starting container < Mock Objects 4 Server API is faked 4 Generic solution 4 No container overhead OWASP App. Sec Europe 2006 25

Testing Security in Integration Tests < In-container testing with Apache Cactus 4 Popular tool for testing J 2 EE applications 4 Can test EJB and Web tiers 4 Plugin’s for Jetty, Eclipse, Ant and Maven OWASP App. Sec Europe 2006 26

Testing Security in Integration Tests < Lifecycle of a single cactus test 1. 2. 3. 4. 5. begin. XXX() - setup the client side set. Up() - common server side code test. XXX() - server side test tear. Down() - common server side code end. XXX() - client side tests OWASP App. Sec Europe 2006 27

Testing Security in Integration Tests public class Test. Access. Control extends Servlet. Test. Case { public void begin. Unprivileged. User. Access. Control(Web. Request the. Request) { the. Request. set. Authentication(new Basic. Authentication("user", "password")); } public void test. Unprivileged. User. Access. Control() throws IOException, javax. servlet. Servlet. Exception { Admin. Servlet admin = new Admin. Servlet(); admin. do. Get(request, response); } public void end. Unprivileged. User. Access. Control(Web. Response the. Response) throws IOException { assert. True("Normal users must not be able to access the. Response. get. Status. Code() == 401) } /admin", } OWASP App. Sec Europe 2006 28

Testing Security in Integration Tests

Security Testing in Acceptance Tests < Tests the external API < Language agnostic < 2 types of tools: 1. Include their own HTTP client and HTML parser, e. g. : HTTPUnit, j. Web. Unit, Html. Unit, Canoo Webtest 2. Drive a browser instance, e. g. : Selenium, WATIR, Watij OWASP App. Sec Europe 2006 30

Security Testing in Acceptance Tests < Example: Testing HTML injection with j. Web. Unit public class XSSin. Search. Field. Test extends Web. Test. Case { public void set. Up() throws Exception { get. Test. Context(). set. Base. Url("http: //example. corsaire. com/ispatula/"); } public void test. Html. Injection() throws Exception { begin. At("/index. html"); assert. Link. Present. With. Text("Enter the Store"); click. Link. With. Text("Enter the Store"); assert. Form. Present("search. Form"); set. Form. Element("query", "