Testing the Tester Measuring Quality of Security Testing

Скачать презентацию Testing the Tester Measuring Quality of Security Testing

250aa1d57234cd8cd2d2eab6a8fd7883.ppt

Количество слайдов: 40

Testing the Tester Measuring Quality of Security Testing Ofer Maor CTO, Hacktics OWASP & OWASP Israel WASC App. Sec 2008 2007 Conference San Jose – Nov 2007 http: //www. webappsec. org/ Copyright © 2007 - The OWASP Foundation Permission is granted to copy, distribute and/or modify this document under the terms of the Creative Commons Attribution-Share. Alike 2. 5 License. To view this license, visit http: //creativecommons. org/licenses/by-sa/2. 5/ The OWASP Foundation http: //www. owasp. org/

Introduction <Security Testing is a Critical Element 4 Part of the system’s security lifecycle Introduction

Introduction <Creates a Huge Challenge for Organizations 4 How to choose the right security Introduction

Agenda <Quality of Security Testing 4 False Positives / False Negatives 4 Coverage / Agenda

Quality of Security Testing OWASP Israel 2008 Conference – Sep 2008 5

Quality of Security Testing <Quality of Testing is Essentially Measured by Two Elements: <False Quality of Security Testing

False Negatives: Reasons <Coverage of Tested Components 4 URL/parameter/component missed 4 Specific code section False Negatives: Reasons

False Negatives: Reasons (Cont’d)

Coverage Problems <Application Data Coverage 4 Automatic crawling problems § Practically infinite links § Coverage Problems

Coverage Problems <Test Coverage 4 Vulnerability not tested § Impossible to test (Logical by Coverage Problems

Scope of Threat Who is Trying to Attack Us? What do They Want?

False Positives: Reasons <Test Quality/Proficiency 4 Poorly defined test § Expected result not properly False Positives: Reasons

Validity <How Can We Tell if It’s Really Vulnerable? <Probe Tests – Attempt to Validity

Business Impact <How Dangerous is This Vulnerability? <Still Controversial – Do we really want Business Impact

Quality of Security Testing – Criteria Summary

Security Testing Approaches (Pros & Cons) OWASP Israel 2008 Conference – Sep 2008 16

Security Testing Approaches (Pros & Cons)

Application Vulnerability Scanners

Manual Penetration Testing <Application Data Coverage 4 Good in contextual aspects § Allows properly Manual Penetration Testing

Manual Penetration Testing <Test Quality / Proficiency 4 Potentially good – but depends greatly Manual Penetration Testing

Manual Penetration Testing <Scope of Threat 4 Depends on the actual effort and quality Manual Penetration Testing

Manual Penetration Testing <Additional Key Aspects 4 Quality varies greatly between testers 4 Quality Manual Penetration Testing

Static Code Analyzers <Application Data Coverage 4 Generally good (no crawling setbacks) 4 Problematic Static Code Analyzers

Static Code Analyzers <Test Quality / Proficiency 4 Generally good in some aspects (Depends Static Code Analyzers

Static Code Analyzers <Scope of Threat 4 Good against: § Mostly syntax attacks (Injections) Static Code Analyzers

Manual Code Review (Static) <Application Data Coverage 4 Can be problematic – Usually impossible Manual Code Review (Static)

Manual Code Review (Static) <Test Quality / Proficiency 4 Potentially excellent (if properly done) Manual Code Review (Static)

Manual Code Review (Static) <Scope of Threat 4 Good against: § High level of Manual Code Review (Static)

Choosing the Right Approach <Determining the Types of Threats <Determining the Required Frequency <Weighing Choosing the Right Approach

Testing the Tester: Evaluating Quality of Security Testing OWASP Israel 2008 Conference – Sep 2008 32

Testing the Tester <The Hardest Part – Determining the quality of security testing solution: Testing the Tester

Testing the Tester <How NOT to Determine Quality 4 Marketing material 4 Sales pitches Testing the Tester

Product Assessment <Comparative Analysis 4 Run several products on a few systems in the Product Assessment

Service Assessment <Much Trickier 4 Hiring a consultant is like hiring an employee <First Service Assessment

Service Assessment <Comparative Analysis 4 Similar to product – great way of comparing services Service Assessment

Some Techniques That (Sometimes) Help

Summary <Quality of Security Testing is Hard to Measure or Quantify <Nonetheless – It Summary

Thank You! Discussion & Questions OWASP Israel 2008 Conference – Sep 2008 40