TQS — Teste e Qualidade de Software Software

TQS - Teste e Qualidade de Software (Software Testing and Quality) Preamble - Course Positioning João Pascoal Faria jpf@fe. up. pt www. fe. up. pt/~jpf Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 1

Index Course positioning with respect to. . . n Capability Maturity Model Integration for Software Engineering (CMMI-SW), Software Engineering Institute (SEI), Carnegie Mellon University (CMU) • To help organize quality related activities and terminology n Software Engineering Education Knowledge, Computing Curricula Software Engineering (CC-SE), IEEE Computer Society / ACM • To help organize and put in context this (educational) knowledge area n Guide to the Software Engineering Body of Knowledge (SWEBOK), IEEE Computer Society • To help organize and put in context this (4 years professional) knowledge area n Certified Software Quality Engineer (CSQE) Body of Knowledge, American Society for Quality (ASQ) • To help organize and put in context this (8 years professional) knowledge area Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 2

Capability Maturity Model Integration for Software Engineering (CMMI-SW), Version 1. 1 n n Developed by the Software Engineering Institute, Carnegie Mellon University Designed to help organizations improve their product and service development, acquisition, and maintenance processes • Judge the maturity of the software processes of an organization • Identify the key practices required to increase the maturity of these processes n Staged representation - measures process-improvement achievement across an organizational unit using maturity levels (as in traditional SW-CMM) • At maturity level n, an organization has achieved all the specific goals of the process areas assigned to maturity levels n (and generic goals assigned to levels n) • To achieve each goal a set of (best) practices must be implemented n n Continuous representation - measures process-improvement achievement within individual process areas (e. g. configuration management) using capability levels The CMMI models are the result of an effort of integration of the "traditional" CMM models (SW-CMM, etc. ) Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 3

CMMI-SW, v 1. 1, staged version Maturity Levels 1) Initial. The software process is characterized as ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort and heroics. 2) Managed. Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications. 3) Defined. The organization’s set of standard processes is established and improved over time. Projects and organizational units establish their defined processes by tailoring the organization’s set of standard processes according to tailoring guidelines. 4) Quantitatively Managed. Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled. 5) Optimizing. Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies. Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 4

CMMI-SW, v 1. 1, staged version Process Areas by Maturity Levels and Process Area Categories Level 2) Managed Level 3) Defined (*) Engineering Project Managem. Process Managem. Level 1) Initial Organizational Process Focus Organizational Process Definition Organizational Process Performance Level 5) Optimizing Organizational Innovation and Deployment Organizational Training Project Planning Integrated Project Management Project Monitoring and Control for Integrated Product and Process Development (IPPD) Supplier Agreement Management Requirements Management (REQM) Quantitative Project Management Risk Management Requirements Development (RD) Technical Solution (TS) Product Integration (PI) Verification (VER) Validation (VAL) Configuration Management (CM) Support Level 4) Quantitatively Managed Decision Analysis and Resolution (DAR) Measurement and Analysis (MA) Causal Analysis and Resolution (CAR) Process and Product Quality Assurance (PPQA) (*) also includes practices to institutionalize a defined process for each process area of level 2 Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 5

CMMI-SW, v 1. 1 Verification and Validation (V&V) n Verification - ensures that the specified requirements are satisfied • Confirms that (intermediate and final) work products properly reflect the requirements specified for them • Ensures that “you built it right” • Occurs throughout the development of the product and work products, beginning with verification of the requirements, progressing through the verification of the evolving work products, and culminating in the verification of the completed product n Validation - ensures that the product will fulfill its intended use • Confirms that the (final) product, as provided, will fulfill its intended use • Ensures that “you built the right thing” • Applied mainly to the product and product components in its intended environment, but can also be applied to work products (e. g. , requirements, designs, prototypes) selected on the basis of which are the best predictors of how well the product and product component will satisfy user needs n Both validation and verification activities often run concurrently and may use portions of the same environment Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 6

CMMI-SW, v 1. 1 Interactions Among V&V and other Engineering Process Areas REQM Requirements Product and product component requirements Alternative solutions TS RD Product components PI Product Customer Requirements Product components, verification and work products, validation reports VER VAL Customer needs Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 7

CMMI-SW, v 1. 1 Process and Product Quality Assurance (PPQA) n Quality Assurance - ensures that planned processes are implemented • Involves objectively evaluating performed processes and work products against applicable process descriptions, standards, and procedures, and communicating and ensuring resolution of noncompliance issues • Usually performed by a QA group that is independent of the project, or by peers not directly involved in the development or maintenance of the work product subject to evaluation • Should begin in the early phases of a project (with the participation of the QA group) to establish plans, processes, standards, and procedures that will add value to the project and satisfy its requirements and the organizational policies and that will be useable for performing QA evaluations, and to designate the specific processes and work products that will be evaluated n V&V and PPQA may on occasion address the same work product but from different perspectives; care should be taken to minimize duplication of effort Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 8

CMMI-SW, v 1. 1 Interactions Among PPQA and other Process Areas Organization IPPD Infrastructure organizational environment for integration OEI Ability to develop and deploy IPPD processes and supporting assets CAR Process improvement proposals IPPD knowledge and skill needs Defects and other problems Quality and noncompliance issues Measurements, analyses MA Information needs Integrated work environment and people practices Project Management process areas Process Management process areas PPQA Processes and work products, standards and procedures All process areas Selected issues Configuration items, change requests Baselines, audit reports Formal evaluations DAR CM Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 9

CMMI-SW, v 1. 1 Terminology - artifacts n Product - any tangible output or service that is a result of a process and that is intended for delivery to a customer or end user • A product is a work product that is delivered to the customer n Product components - lower level components of the product, that are integrated to “build” the product • There may be multiple levels of product components • A product component is any work product that must be engineered (requirements defined and designs developed and implemented) to achieve the intended use of the product throughout its life n Work product - any artifact produced by a process, including files, documents, parts of the product, services, processes, specifications, and invoices • Examples of processes to be considered as work products include a manufacturing process, a training process, and a disposal process for the product Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 10

CMMI-SW, v 1. 1 Terminology – actors and activities n Customer - the party (individual, project, or organization) responsible for accepting the product or for authorizing payment • The customer is external to the project, but not necessarily external to the organization • The customer may be a higher level project • Customers are a subset of stakeholders n Stakeholder - a group or individual that is affected by or in some way accountable for the outcome of an undertaking • Stakeholders may include project members, suppliers, customers, end users, and others n n Project manager - the person responsible for planning, directing, controlling, structuring, and motivating the project and satisfying the customer Project - a managed set of interrelated resources that delivers one or more products to a customer or end user • This set of resources has a definite beginning and end and typically operates according to a plan • Such a plan is frequently documented and specifies the product to be delivered or implemented, the resources and funds used, the work to be done, and a schedule for doing the work • A project can be composed of projects Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 11

Index n n Capability Maturity Model Integration for Software Engineering (CMMI-SW), Software Engineering Institute (SEI), Carnegie Mellon University (CMU) Computing Curricula - Software Engineering (CC-SE), IEEE Computer Society / ACM Guide to the Software Engineering Body of Knowledge (SWEBOK), IEEE Computer Society Certified Software Quality Engineer (CSQE) Body of Knowledge, American Society for Quality (ASQ) Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 12

Software Engineering Education Knowledge (SEEK), Computing Curriculum – Software Engineering (CC-SE), 2004 Knowledge areas and knowledge units (1/2) n Computing Essentials (172 hours) • • n n Computer Science foundations Construction technologies Construction tools Formal construction methods • • • Modeling foundations Types of models Analysis fundamentals Requirements fundamentals Eliciting requirements Requirements specification & documentation • Requirements validation Mathematical & Engineering Fundamentals (89 hours) • Mathematical foundations • Engineering foundations for software • Engineering economics for software n Professional Practice (35 hours) • Group dynamics / psychology • Communications skills (specific to SE) • Professionalism Software Modeling & Analysis (53 hours) n Software Design (45 hours) • • • Design concepts Design strategies Architectural design Human computer interface design Detailed design Design support tools and evaluation Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 13

CC-SE Knowledge areas and knowledge units (2/2) Process and Product n Software Verification & Validation (42 hours) n Assurance Software Quality (16 hours) • Software quality concepts and culture (2 h) • V&V terminology and foundations (5 h) • Software quality standards (2 h) • Reviews (6 h) • Software quality processes (4 h) • Testing (21 h) • Process assurance (4 h) • Human computer UI testing and evaluation (6 h) IPC • Product assurance (4 h) • Problem analysis and reporting (4 h) n Software Evolution (10 hours) n Software Management (19 hours) • Management concepts • Evolution processes • Evolution activities n • Project planning • Project personnel and organization Software Process (13 hours) • Process concepts • Project control • Software configuration management • Process implementation Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 14

CC-SE / SEEK Software Verification & Validation (1/2) n Software Verification & Validation (42 hours) -> 20 h "Software verification and validation uses both static and dynamic techniques of system checking to ensure that the resulting program satisfies its specification and that the program as implemented meets the expectations of the stakeholders. Static techniques are concerned with the analysis and checking of system representations throughout all stages of the software life cycle, while dynamic techniques involve only the implemented system. " n V&V terminology and foundations (5 h ) 2 h • • • Objectives and constraints of V&V Planning the V&V effort Documenting V&V strategy, including tests and other artifacts Metrics & Measurement (e. g. reliability, usability, performance, etc. ) V&V involvement at different points in the lifecycle Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 15

CC-SE / SEEK Software Verification & Validation (2/2) n Testing (21 h ) 14 h • Unit testing • Exception handling (writing test cases to trigger exception handling; designing good handling) • Coverage analysis (e. g. statement, branch, basis path, multi-condition, dataflow, etc. ) • Black-box functional testing techniques • Integration Testing • Developing test cases based on use cases and/or customer stories • Operational profile-based testing • System and acceptance testing • Testing across quality attributes (e. g. usability, security, compatibility, accessibility, etc. ) • Regression Testing • Testing tools • Deployment process (D) n n n Reviews (6 h ) 1 h • Desk checking • Walkthroughs • Inspections Human computer UI testing and evaluation (6 h) IPC • The variety of aspects of usefulness and usability • Heuristic evaluation • Cognitive walkthroughs • User testing approaches (observation sessions etc. ) • Web usability; testing techniques for web sites • Formal experiments to test hypotheses about specific HCI controls (D) Problem analysis and reporting (4 h) 3 h • Analyzing failure reports • Debugging/fault isolation techniques • Defect analysis • Problem tracking Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 16

CC-SE / SEEK Software Quality n Software Quality (16 hours) -> 14 h "Software quality is a pervasive concept that affects, and is affected by all aspects of software development, support, revision, and maintenance. It encompasses the quality of work products developed and/or modified (both intermediate and deliverable work products) and the quality of the work processes used to develop and/or modify the work products. Quality work product attributes include functionality, usability, reliability, safety, security, maintainability, portability, efficiency, performance, and availability. " n Software quality concepts and culture (2 h) 1 h • • Definitions of quality Society's concern for quality The costs and impacts of bad quality A cost of quality model Quality attributes for software (e. g. dependability, usability, etc. ) The dimensions of quality engineering Roles of people, processes, methods, tools, and technology Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 17

CC-SE / SEEK Software Quality n Software quality standards (2 h) 2 n • The nature of process assurance • Quality planning • Organizing and reporting for process assurance • Techniques of process assurance • Quality Management Systems • ISO/IEEE Standard 12207 Software Life Cycle Processes • Organizational implementation of standards • IEEE software quality-related standards (D) n Software quality processes (4 h) 4 • Software quality models and metrics • Quality-related aspects of software process models • Introduction/overview of ISO 15504 and the SEI CMMs • Quality-related process areas of ISO 15504 • Quality-related process areas of the SW-CMM and the CMMIs • The Baldrige Award criteria as applied to software engineering (O) • Quality aspects of other process models (O) Process assurance (4 h) 4 n Product assurance (4 h) 3 • The nature of product assurance • Distinctions between assurance and V&V • Quality product models • Root cause analysis and defect prevention • Quality product metrics and measurement • Assessment of product quality attributes (e. g. usability, reliability, availability, etc. ) Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 18

CC-SE / SEEK A proposed course on Software Quality Assurance (1/2) n Software Quality Assurance (37 hours) 34 h • Broad coverage of software quality and testing. • Course Description: - Quality: how to assure it and verify it, and the need for a culture of quality. Avoidance of errors and other quality problems. Inspections and reviews. Testing, verification and validation techniques. Process assurance vs. Product assurance. Quality process standards. Product and process assurance. Problem analysis and reporting. Statistical approaches to quality control. • Learning objectives Upon completion of this course, students will have the ability to: - Conduct effective and efficient inspections - Design and implement comprehensive test plans - Apply a wide variety of testing techniques in an effective and efficient manner - Compute test coverage and yield, according to a variety of criteria - Use statistical techniques to evaluate the defect density and the likelihood of faults - Assess a software process to evaluate how effective it is at promoting quality Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 19

CC-SE / SEEK A proposed course on Software Quality Assurance (2/2) • Suggested sequence of teaching modules: - Introduction to software quality assurance Inspections and reviews Principles of software validation Software verification Software testing Specification based test construction techniques White-box and grey-box testing Control flow oriented test construction techniques Data flow oriented test construction techniques Cleanroom approach to quality assurance Software process certification • Sample labs and assignments - Use of automated testing tools Testing of a wide variety of software Application of a wide variety of testing techniques Inspecting of software in teams; comparison and analysis of results Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 20

Index n n Capability Maturity Model Integration for Software Engineering (CMMI-SW), Software Engineering Institute (SEI), Carnegie Mellon University (CMU) Computing Curricula - Software Engineering (CC-SE), IEEE Computer Society / ACM Guide to the Software Engineering Body of Knowledge (SWEBOK), IEEE Computer Society Certified Software Quality Engineer (CSQE) Body of Knowledge, American Society for Quality (ASQ) Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 21

Guide to the Software Engineering Body of Knowledge (SWEBOK), Trial Version 1. 0, 2001 n Developed by the IEEE Computer Society n Established with the following objectives: 1. Promote a consistent view of software engineering worldwide. 2. Clarify the place—and set the boundary—of software engineering with respect to other disciplines such as computer science, project management, computer engineering, and mathematics. 3. Characterize the contents of the software engineering discipline. 4. Provide a topical access to the Software Engineering Body of Knowledge. 5. Provide a foundation for curriculum development and individual certification and licensing material. • Describes knowledge expected after 4 years of experience as software engineer Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 22

SEWBOK - Knowledge areas (1/2) Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 23

SEWBOK - Knowledge areas (2/2) Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 24

SEWBOK Software Testing Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 25

SEWBOK Software Quality Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 26

SWEBOK Terminology n Verification & Validation versus Software Quality Assurance • SQA governs the procedures meant to build the desired quality into the products by assuring that the process is well-planned and then applied as prescribed • V&V is aimed more directly at product quality, in that it is based on testing that can locate deviations and fix them. But it also validates the intermediate products and therefore the intermediate steps of the software engineering process • SQA ensures that appropriate types of tests are planned, developed, and implemented, and V&V develops test plans, strategies, cases and procedures n n Testing is a technique for evaluating product quality and also for indirectly improving it, by identifying defects and problems Software testing consists of the dynamic verification of the behavior of a program on a finite set of test cases, suitably selected from the usually infinite executions domain, against the specified expected behavior Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 27

Index n n Capability Maturity Model Integration for Software Engineering (CMMI-SW), Software Engineering Institute (SEI), Carnegie Mellon University (CMU) Computing Curricula - Software Engineering (CC-SE), IEEE Computer Society / ACM Guide to the Software Engineering Body of Knowledge (SWEBOK), IEEE Computer Society Certified Software Quality Engineer (CSQE) Body of Knowledge, American Society for Quality (ASQ) Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 28

Certified Software Quality Engineer (CSQE) Body of Knowledge, American Society for Quality (ASQ) n n The CSQE is a professional who has comprehensive understanding of software quality development and implementation, a thorough understanding of software inspection, testing, verification, and validation, and can implement software development and maintenance processes and methods. Requires 8 years of experience (3 if master's or doctorate) Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 29

CSQE Body of Knowledge I. GENERAL KNOWLEDGE, CONDUCT, and ETHICS A. Quality philosophy and principles 1. Benefits of software quality (C) Describe how software quality engineering can benefit an organization. 2. Prevention vs. detection (C) Describe how quality engineering methodologies can reduce the length of time for testing and can influence other defect detection methods. 3. Organizational and process benchmarking (An) Identify, analyze, and model best practices at the macro (organizational) and micro (process and project) levels. Identify and develop business objectives, use metrics to monitor their achievement, and provide feedback to close the process improvement loop. B. Standards, specifications, and models (Ap) Identify and use software process and assessment models, including ISO 9001, ISO 15504, IEEE software standards, IEEE/EIA 12207, SEI Capability Maturity Model Integrated (CMMI), etc. , in a variety of situations. C. Leadership tools and skills (Ap) 1. Organizational leadership 2. Team management 3. Team tools 4. Facilitation skills 5. Communication skills D. Ethical conduct and professional development 1. ASQ Code of Ethics (E) 2. Software liability and safety issues (Ap) 3. Professional training and development (Ap) Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 30

CSQE Body of Knowledge II. SOFTWARE QUALITY MANAGEMENT - A A. Goals and objectives 1. Quality goals and objectives (E) Describe, analyze, and evaluate quality goals and objectives for programs, projects, and products. 2. Outsourced services (E) Define, analyze, and evaluate the impact of acquisitions, subcontractor services, and other external resources on the organization's goals and objectives. 3. Planning (E) Identify, apply, and evaluate scheduling and resource requirements necessary to achieve quality goals and objectives. 4. Software quality management (SQM) systems documentation (C) Identify and describe various elements related to SQM system documentation. 5. Customer requirements (E) Analyze and evaluate customer requirements and their effect on programs, projects, and products. Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 31

CSQE Body of Knowledge II. SOFTWARE QUALITY MANAGEMENT - B B. Methodologies 1. Review, inspection, and testing (E) Define, describe, evaluate, and differentiate between these defect detection methods. 2. Change management methods (E) Identify and apply various methods appropriate for responding to changes in technology, organizations, environment, human performance, etc. 3. Cost of quality (COQ) (An) Define, differentiate, and analyze COQ categories (prevention, appraisal, internal failure, external failure) and their impact on products and processes. 4. Quality data tracking (E) Define, describe, select, and implement information systems and models used to track quality data in various situations. 5. Problem reporting and corrective action procedures (E) Define, describe, analyze, and distinguish between these procedures for software defects, process nonconformances, and other quality system deficiencies. 6. Quality improvement processes (E) Define, describe, analyze and distinguish between various defect prevention, detection, and removal processes, and evaluate process improvement opportunities in relation to these tools. Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 32

CSQE Body of Knowledge II. SOFTWARE QUALITY MANAGEMENT - C C. Audits 1. Program development and administration (C) Identify roles and responsibilities for various audit participants, including team leader, team members, auditee, auditor, etc. 2. Audit preparation and execution (C) Define and distinguish between various audit types, including process, compliance, supplier, system, etc. Define and describe various steps in the audit process, from scheduling the audit through the closing meeting and subsequent follow-up activities. Define and identify various tools and procedures used in conducting audits. 3. Audit reporting and follow up (Ap) Identify, describe, and apply the steps of audit reporting and follow up, including the need for and verification of corrective action. Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 33

CSQE Body of Knowledge III. SOFTWARE ENGINEERING PROCESSES A. Environmental conditions 1. Life cycles 2. Systems architecture B. Requirements management 1. Requirements prioritization and evaluation 2. Requirements change management 3. Bi-directional requirements traceability C. Requirements engineering 1. Requirement types 2. Requirements elicitation 3. Requirements analysis and modeling 4. System and software requirements specifications D. Analysis, design, and development methods and tools 1. Software design methods 2. Types of software reuse 3. Clean room and other formal methods 4. Software development tools E. Maintenance management 1. Maintenance types 2. Operational maintenance Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 34

CSQE Body of Knowledge IV. PROGRAM AND PROJECT MANAGEMENT A. Planning 1. Project planning elements 2. Goal-setting and deployment 3. Project planning tools 4. Cost and value data B. Tracking and controlling 1. Phase transition control techniques 2. Interpreting and reporting cost of quality (COQ) data 3. Tracking elements and methods 4. Project reviews C. Risk management 1. Risk management planning methods 2. Risk probability 3. Product release decisions 4. Software security, safety, and hazard analysis issues Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 35

CSQE Body of Knowledge V. SOFTWARE METRICS, MEASUREMENT, AND ANALYTICAL METHODS –A, B A. Metrics and measurement theory 1. Definitions 2. Basic measurement theory and techniques 3. Psychology of metrics B. Process and product measurement 1. Process, product, and resource metrics (Ap) 2. Commonly used metrics (Ap) Define and use metrics to measure various aspects of software, including software complexity, lines of code (LOC), non-commented lines of code (NCLOC), design defects, requirements volatility, system performance, etc. 3. Software quality attributes (C) Identify and describe various criteria for measuring attributes such as maintainability, verifiability, reliability, usability, reusability, testability, expandability, etc. 4. Defect detection effectiveness measures (Ap) Define, describe, and use defect detection measures such as cost, yield, customer impact, etc. , and track their effectiveness. Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 36

CSQE Body of Knowledge V. SOFTWARE METRICS, MEASUREMENT, AND ANALYTICAL METHODS - C C. Analytical techniques 1. Data integrity (S) Define, use, and interpret various techniques to ensure the quality of metrics data, its accuracy, completeness, timeliness, etc. 2. Quality tools (An) Define, select, and use quality analysis and problem-solving tools such as flow charts, Pareto charts, cause and effect diagrams, check sheets, scatter diagrams, control (run) charts, histograms, root cause analysis, affinity diagrams, tree diagrams, process decision program charts (PDPCs), matrix diagrams, interrelationship digraphs, prioritization matrices, activity network diagrams. 3. Sampling theory and techniques (An) Describe, differentiate, and analyze various sampling techniques for use in auditing, testing, product acceptance, etc. Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 37

CSQE Body of Knowledge VI. SOFTWARE VERIFICATION AND VALIDATION (V&V) - A A. Theory 1. V&V planning procedures and tasks (S) Identify and select various methods for verification and validation, including static analysis, structural analysis, mathematical proof, simulation, etc. Identify and analyze which tasks should be iterated as a result of proposed or completed modifications. 2. V&V program (An) Describe and analyze methods for managing and reviewing a V&V program, including technical accomplishments, resource utilization, program status, etc. 3. Evaluating software products and processes (S) Analyze and select various ways of evaluating documentation, source code, test and audit results, etc. , to determine whether user needs and project objectives have been satisfied. 4. Interfaces (C) Identify various interfaces used with hardware, user, operator, and software applications. 5. Program performance and process effectiveness (An) Identify and use various methods of examining performance and effectiveness. Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 38

CSQE Body of Knowledge VI. SOFTWARE VERIFICATION AND VALIDATION (V&V) - B B. Reviews and inspections 1. Types (Ap) Define, describe, and use various types of reviews and inspections, including desk-checking, walk-throughs, Fagan and Gilb inspections, technical accomplishments, resource utilization, future planning, etc. 2. Items (Ap) Identify, describe, and use various review and inspection items, including proposals, project charters, specifications, code, tests, etc. 3. Processes (Ap) Define, describe, and use various review and inspection processes to examine objectives, criteria, techniques, methods, etc. 4. Data collection, reports, and summaries (Ap) Define, describe, and use terms related to data collection, including preparation rates, defect density yield, phase containment, etc. Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 39

CSQE Body of Knowledge VI. SOFTWARE VERIFICATION AND VALIDATION (V&V) - C C. Test planning and design 1. Types of tests (S) Select, apply, and develop various types of test, including functional, performance, regression, certification, environmental load, stress, worst case, perfective, exploratory, etc. 2. Test tools (C) Define and describe the application and capabilities of commonly used test tools such as acceptance test suites, utilities (for memory, screen capture, string-finding, file viewer, file comparison, etc. ), and diagnostics (for hardware, software, configuration, etc. ). 3. Test strategies (S) Identify, analyze, and apply various test strategies, including top-down, bottom -up, black-box, white-box, simulation, automation, etc. 4. Test design (Ap) Identify, describe, and apply various types of test design including fault insertion, fault-error handling, equivalence class partitioning, boundary value, etc. Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 40

CSQE Body of Knowledge VI. SOFTWARE VERIFICATION AND VALIDATION (V&V) - C C. Test planning and design (cont. ) 5. Test coverage of specifications (S) Identify, apply, and develop various test coverage specifications, including functions, states, data and time domains, etc. 6. Test environments (S) Identify various environments and use tools such as test libraries, drivers, stubs, harnesses, etc. , in those environments, and describe how simulations can be used in test environments. 7. Supplier components and products (Ap) Identify the common risks and benefits of incorporating purchased software into other software products. Use various methods to test supplier components and products in the larger system. 8. Test plans (Ap) Identify, describe, and apply methods for creating and evaluating test plans including system, acceptance, validation, etc. , to determine whether project objectives are being met. Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 41

CSQE Body of Knowledge VI. SOFTWARE VERIFICATION AND VALIDATION (V&V) - D D. Test execution and evaluation 1. Test implementation (Ap) Define, describe, and use various implementation elements, including scheduling, freezing, dependencies, V-model, error repair models, acceptance testing, etc. 2. Test documentation (Ap) Define, describe, and use various documentation procedures, including defect recording and tracking, test report completion metrics, trouble reports, input/output specifications, etc. 3. Test reviews (S) Describe, develop, and analyze various methods of reviewing test efforts, including technical accomplishments, future planning, risk management, etc. 4. Code coverage metrics (Ap) Define and apply various metrics including branch-to-branch, condition, domain, Mc. Cabe's cyclomatic complexity, boundary, etc. 5. Customer deliverables (S) Identify and select various methods for testing the accuracy of customer deliverables, including packaged or downloaded products, license keys, user documentation, marketing and training materials, etc. 6. Severity of anomalies (E) Identify and select various methods for evaluating severity of anomalies in software operations. Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 42

CSQE Body of Knowledge VII. SOFTWARE CONFIGURATION MANAGEMENT A. Configuration infrastructure 1. Configuration management 2. Library/repository processes 3. Defect tracking and library tools B. Configuration identification 1. Configuration items 2. Baselines 3. Configuration identification methods 4. Software builds C. Configuration control 1. Item and baseline control 2. Proposed modifications 3. Review and configuration control boards (CCBs) 4. Concurrent development 5. Traceability 6. Version control 7. Configuration item interfaces D. Configuration status accounting 1. Status reporting 2. Changes to configuration items and baselines 3. Documentation control E. Configuration audits 1. Functional configuration audit 2. Physical configuration audit F. Release and distribution issues 1. Product release process issues 2. Packaging, production, and distribution Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 43

CSQE Body of Knowledge Levels of Cognition (Bloom's Taxonomy) n n n Knowledge Level (K) - Being able to remember or recognize terminology, definitions, facts, ideas, materials, patterns, sequences, methodologies, principles, etc. Comprehension Level (C) - Being able to read and understand descriptions, communications, reports, tables, diagrams, directions, regulations, etc. Application Level (Ap) - Being able to apply ideas, procedures, methods, formulas, principles, theories, etc. , in job-related situations. Analysis (An) - Being able to break down information into its constituent parts and recognize the parts’ relationship to one another and how they are organized; identify sublevel factors or salient data from a complex scenario. Synthesis (S) - Being able to put parts or elements together in such a way as to show a pattern or structure not clearly there before; identify which data or information from a complex set is appropriate to examine further or from which supported conclusions can be drawn. Evaluation (E) - Being able to make judgments regarding the value of proposed ideas, solutions, methodologies, etc. , by using appropriate criteria or standards to estimate accuracy, effectiveness, economic benefits, etc. Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 44

References and further reading n http: //www. sei. cmu. edu/cmmi/ • Capability Maturity Model Integration for Software Engineering (CMMI-SW), Software Engineering Institute (SEI), Carnegie Mellon University (CMU) n http: //sites. computer. org/ccse/ • Computing Curricula - Software Engineering (CC-SE), IEEE Computer Society / ACM n http: //www. swebok. org/ • Guide to the Software Engineering Body of Knowledge (SWEBOK), IEEE Computer Society n http: //www. asq. org/cert/types/csqe/index. html • Certified Software Quality Engineer (CSQE) Body of Knowledge, American Society for Quality (ASQ) Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004 45