Risk Analysis Best Practices By Gabriel Rodriguez

Agenda l l l Explanation of Risk Types of Risk Based Testing Traceability Matrix

Explanation of Risk Copyright 2006 -2007. MSQAA Federation Chapter.

Explanation of Risk l l l Concepts of Risk Test is a method for control. Controls are used to reduce risk. A risk is a condition that can result in a loss to an organization. The risk turns into a loss when some event triggers the risk. Copyright 2006 -2007. MSQAA Federation Chapter.

Explanation of Risk l The Risk is turned into a loss by threat. l A threat is the trigger that causes the risk to become a loss. l Threats are reduced or eliminated by controls. l Control can be identified as anything that tends to cause the reduction of risk. l Inadequate controls represent Vulnerability. Copyright 2006 -2007. MSQAA Federation Chapter.

Explanation of Risk l The Risk is turned into a loss by threat. l Vulnerability , therefore, can be defined as a flaw in the system of control that will enable a threat to be exploited. l The process of evaluating risk, threats, controls, an vulnerabilities is frequently called Risk Analysis. Copyright 2006 -2007. MSQAA Federation Chapter.

Types of Risk Copyright 2006 -2007. MSQAA Federation Chapter.

Types of Risk l The different types of risks are: – Risks Generated by

Types of Risk l Risks Generated by a computer The risks in a computerized environment include both the risks that would be present in manual processing and some risks that are unique in a computerized environment: – Improper use of technology. – Repetition of errors. – Cascading of errors. – Illogical processing. Copyright 2006 -2007. MSQAA Federation Chapter.

Types of Risk – – – – Inability to translate user needs into technical requirements. Inability to control technology. Incorrect entry data. Concentration of Data. Inability to react quickly. Concentration of responsibilities. Erroneous or falsified input data. Copyright 2006 -2007. MSQAA Federation Chapter.

Types of Risk – – – Misuse by authorized end users. Uncontrolled system access. Ineffective security practices for the application. Program Errors. Operating system flaws. Communication system failure. Copyright 2006 -2007. MSQAA Federation Chapter.

Types of Risk l Testing Risks Primary testing risks include: – Budget. – Number of qualified test resources. – Test environment. – Tools and procedures. – Sequence and increments of code delivery. Copyright 2006 -2007. MSQAA Federation Chapter.

Types of Risk l Other factors that add risk to the project are – Multi-vendor products that must integrate. – Multi-vendor environments. – Developed with multi-vendor tools. – End user expectations driven by products like MS Word and Excel. Copyright 2006 -2007. MSQAA Federation Chapter.

Types of Risk l l l Software Risks An effective approach to testing is to identify an evaluate the risks in a computer system. Those risks deemed important to reduce become the areas for testing. A decision can be made as to how much risk is acceptable and then a test plan design to achieve that goal. Copyright 2006 -2007. MSQAA Federation Chapter.

Types of Risk l Software Risks Types of risk associated with the development and installation of a computer system are: – Incorrect Results will be produced. – Unauthorized transactions will be accepted by the system. – Computer file integrity will be lost. Copyright 2006 -2007. MSQAA Federation Chapter.

Types of Risk – – Processing cannot be reconstructed. Continuity of processing will be lost. Service provided to the user will degrade to an unacceptable level. Security of the system will be compromised. Copyright 2006 -2007. MSQAA Federation Chapter.

Types of Risk – – Software Risks Processing will be not comply with the organizational policy or governmental regulation. Results of the system will be unreliable. Programs will be un-maintainable. System will not be portable to the other Hardware and Software. Copyright 2006 -2007. MSQAA Federation Chapter.

Types of Risk – – – Software Risks System will not be able to interconnect with other computers systems. Performance level will be unacceptable. System will be difficult to operate. Copyright 2006 -2007. MSQAA Federation Chapter.

Types of Risk l Quality Risks (Test Factors) The failure of the project team to address (control) these factors may result in loss: – Requirements comply with methodology (Methodology Test Factor). – Functional Specifications defined (Correctness Test Factor). Copyright 2006 -2007. MSQAA Federation Chapter.

Types of Risk – – – Usability Specifications Determined (Ease of Use Test Factor). Maintenance Specifications Determined (Maintainable Test Factor). Portability needs Determined (Portable Test Factor). Copyright 2006 -2007. MSQAA Federation Chapter.

Types of Risk l l Quality Risks (Test Factors) System Interface Defined (Coupling Test Factor). Performance Criteria Established (Performance Test Factor). Operational Needs Defined (Ease-of Operations Test Factor). Tolerances Established (Reliability Test Factor). Copyright 2006 -2007. MSQAA Federation Chapter.

Types of Risk l l l Authorization Rules Defined (Authorization Test Factor). File Integrity Requirements defined (File Integrity Test Factor). Reconstruction Requirements Defined (Audit Trail Test Factor). Copyright 2006 -2007. MSQAA Federation Chapter.

Types of Risk l l l Quality Risks (Test Factors) Impact of Failure Defined (continuity-of-Processing Test Factor). Desired Service Level Defined (Service Level Test Factor). Access Defined (Security Test Factor). Copyright 2006 -2007. MSQAA Federation Chapter.

Risk Based Testing Copyright 2006 -2007. MSQAA Federation Chapter.

Risk Based Testing l l The key steps to designing and planning a risk based approach are risk identification, risk strategy, risk assessment and incorporation of the risk analysis into test design. The objective of risk analysis is to identify potential problems that could affect the cost or outcome of the project. How do we define areas of risk? The objective of the strategy is to define and communicate which risks we will mitigate, and how we will mitigate those risks; what are we going to do about it? Copyright 2006 -2007. MSQAA Federation Chapter.

Risk Based Testing l Risk Identification – The activity of identifying risk answers these questions: l Is there risk to this test area or feature? l How can it be classified? – Risk identification involves collecting information about the project and classifying it to determine the amount of potential risk in the test phase and in production (in the future). – The risk could be related to system complexity, new technology or methodology involved that could cause problems, limited business knowledge or poor design and code quality. Copyright 2006 -2007. MSQAA Federation Chapter.

Risk Based Testing l Risk Strategy – After you’ve identified and assessed the risks, you’ll work toward a strategy. These are the contingency plans for possible alternative project activities or the mitigation of prioritized risks. These plans are then used to direct the management of risks during the software testing activities. It is therefore possible to define an appropriate level of testing per test area based on the risk assessment of that feature or area of functionality. This approach also allows for additional testing to be defined for features that are critical or are identified as high risk as a result of testing (due to poor design, quality, documentation, etc. ) Copyright 2006 -2007. MSQAA Federation Chapter.

Risk Based Testing l Risk Assessment – Assessing risks means determining the effects (including costs) of potential risks. A risk assessment involves asking questions such as: Is this a risk or not? How serious is the risk? What are the consequences? What is the likelihood of this risk happening? – The following sections discuss some of the tools available for performing this assessment. l Risk Assessment Checklist Copyright 2006 -2007. MSQAA Federation Chapter.

Risk Based Testing l Risk Assessment Checklist – The Risk Assessment Checklist is the highest level of the documents use for our assessment. The Checklist helps us prioritize and categorize the identified risks so that we can develop an effective and appropriate test strategy. The other listed documents help you determine, evaluate, and answer the checklist. The categories of the Risk Assessment Checklist are as follows: Complexity, New Feature, User Traffic, Bugginess, Customer Impact, Integration, and the Size of Code Change. These categories should be considered when evaluating the information gathered in your analysis. Copyright 2006 -2007. MSQAA Federation Chapter.

Traceability Matrix Copyright 2006 -2007. MSQAA Federation Chapter.

Traceability Matrix l Traceability Matrix definition from www. wikipedia. com: – – In software development, the term traceability refers to the ability to link requirements back to stakeholders' rationales and forward to corresponding design artifacts, code, and test cases. Traceability supports numerous software engineering activities such as change impact analysis, compliance verification of code, regression test selection, and requirements validation. Copyright 2006 -2007. MSQAA Federation Chapter.

Traceability Matrix l Traceability Matrix definition from www. wikipedia. com: – – – It is usually accomplished in the form of a matrix created for the verification and validation of the project. Unfortunately the practice of constructing and maintaining a requirements trace matrix [RTM] can be very arduous and over time the traces tend to erode into an inaccurate state. Alternate automated approaches for generating traces using information retrieval methods have been developed Copyright 2006 -2007. MSQAA Federation Chapter.

Traceability Matrix l Traceability Matrix definition from http: //www. jiludwig. com/Traceability_Matrix_Structure. h tmlstate: – A traceability matrix is created by associating requirements with the work products that satisfy them. Tests are associated with the requirements on which they are based and the product tested to meet the requirement. Copyright 2006 -2007. MSQAA Federation Chapter.

Traceability Matrix l l Below is a simple traceability matrix structure. There can be more things included in a traceability matrix than shown. In traceability, the relationship of driver to satisfier can be one-toone, one-to-many, many-to-one, or many-to-many. Copyright 2006 -2007. MSQAA Federation Chapter.

Traceability Matrix l l l Traceability ensures completeness, that all lower level requirements come from higher level requirements, and that all higher level requirements are allocated to lower level requirements. Traceability is also used to manage change and provides the basis for test planning. The purpose of the traceability matrix is: – To map relevant Software Development Life cycle Documentation (SDLC). (Requirements, Functional or Design specifications) with test plan (test cases). Process to generate a Traceability Matrix: – Requirements, Functional or Design documentation must exist. – Test Plan must be created. Copyright 2006 -2007. MSQAA Federation Chapter.

Traceability Matrix l Process to generate a Traceability Matrix: – Requirements documents must exist, – Functional Specification document must exist – Design documentation must exist (If applicable). – Test Plan must be created. – Remember that in order to generate an accurate traceability matrix all the documents stated above must exist. l Make sure to update the Traceability matrix as Change Control requests are generated. Copyright 2006 -2007. MSQAA Federation Chapter.

Traceability Matrix – – Recommendation to create a traceability matrix: Open a Spreadsheet in Excel and create: l 4 or 5 columns for: – Requirements – Functional specifications – Design Specifications – Test Plan – (It is not necessary to have the 3 Software Development Lice Cycle (SDLC) documentation, the traceability matrix can be created with 2 SDLC documentation. E. g. , Requirements and Functional Specifications). Copyright 2006 -2007. MSQAA Federation Chapter.

Traceability Matrix l The following slide shows an example of a Traceability Matrix. Please have the following points in mind: – Every organization designs its traceability matrix template to meet its needs. – It is not a good practice to mix the concepts of a traceability matrix with a test matrix (Test Matrix is discussed in ‘Test Design’ presentation). Copyright 2006 -2007. MSQAA Federation Chapter.

Q&A l Any questions… Copyright 2006 -2007. MSQAA Federation Chapter.

Reference l l CSTE Study Guide 2002 by QAI CSTE Study Guide 2006 by QAI www. wikipedia. com http: //www. jiludwig. com/Traceability_Matrix_Structure. htmlstate Copyright 2006 -2007. MSQAA Federation Chapter.