Independent Verification and Validation IV V Techniques for Object

Independent Verification and Validation (IV&V) Techniques for Object Oriented Software Systems SAS meeting July 2003

Outline IV&V of OO artifacts Identifying Risks Proposed Techniques GQM (Goal Question Metric) Application BBN (Bayesian Belief Network) Application COMPANY PROPRIETARY 2 APC 0342

Problem Statement NASA has considerable experience applying IV&V to traditional function-based software systems However the emerging use of OO brings unique challenges and risks There is a need to develop IV&V techniques for addressing these risks Verify these techniques COMPANY PROPRIETARY 3 APC 0342

Objective Understand risks unique to large OO systems Establish a framework for identifying, evaluating, and implementing IV&V techniques for mitigating these risks COMPANY PROPRIETARY 4 APC 0342

Multi -Year Goals Identify Risks Develop base set of IV&V Techniques Apply & evaluate techniques Focus additional research and match emerging techniques to prioritized needs COMPANY PROPRIETARY 5 APC 0342

Approach Phased Approach – Identify Risks, Develop Base set of Techniques, Evaluate Base set – Focused research to adapt techniques for a broader spectrum of OO projects Reasoning – Level of maturity of IV&V’s understanding of the problem, developers processes, and V&V techniques specific to OO system development – Limited availability of defect data specific to OO development COMPANY PROPRIETARY 6 APC 0342

2) Develop Base IV&V Techniques – Requirements – Analysis – Design 3) Evaluate Base IV&V Techniques Current Phase Continuous research & evaluation of emerging techniques 1) Identify OO Development Risks Focused research to map, adapt, or develop techniques to prioritized IV&V activities based on experienced and perceived needs for improvement in efficiency, accuracy, and capacity. Next Phase COMPANY PROPRIETARY 7 APC 0342

Activities to-date Evaluation of current & emerging OO IV&V techniques Identification of Risks with OO development Investigating framework for OO IV&V Developing base set of IV&V techniques COMPANY PROPRIETARY 8 APC 0342

State of IV&V Practice Rudimentary manual techniques for Use cases or OO analysis Techniques limited to static analysis No automated tools or metrics being used for requirements, analysis & design Manual techniques ensure • Specifications use UML compliant notation • Design is adequately documented • Design specifications will meet requirements Techniques do not address the quality of OO design objectives • Encapsulation, coupling, cohesion, polymorphism etc. COMPANY PROPRIETARY 9 APC 0342

Emerging Techniques Some examples of emerging techniques – Dynamic Analysis Techniques e. g. Yacoub et. al. "Dynamic Metrics for Object Oriented Designs" – Semantic Metrics e. g. Etzkorn et. al. “Towards a Semantic Metrics Suite for OOD” – Application of BBN e. g. Fenton et. el. “Bayesian Belief Network Model for the Safety Assessment of Nuclear Computer-Based Systems” State of practice – These techniques offer exciting promise for improving IV&V products – Need further enhancement for practical IV&V application COMPANY PROPRIETARY 10 APC 0342

Risks to OO Development Risk to OO development fall into three major categories – Programmatic Risks • Project termination, software component integration, utilization of tools, and project failure – Technology Risks • Issues with technology used to build the product, such as: Operating system, Compilers, Libraries, and middleware – Methodology Risks • Type of OO methods being used, incorrect application of a method, and issues related to lifecycle phases COMPANY PROPRIETARY 11 APC 0342

Programmatic Risks Project termination – Sub par schedule and performance of product Methods abandonment – Lack of progress or OO related benefits High learning curves – Insufficient budget and schedule for training Not meeting OO objectives – Autonomy between organizations, absence of short iterative development cycle Dependencies on Tools – Cost and tool deployment requirements COMPANY PROPRIETARY 12 APC 0342

Technology Risks Object Model – Technology does not fully support the object model e. g. asynchronous communication between objects in relational databases Persistence – Poor support for object persistence e. g. OODB, relational DB, ad-hoc flat files COMPANY PROPRIETARY 13 APC 0342

Methodology Risks Incorrect application of methodology, insufficient methodology, or no methodology – Design mixed with requirements/use cases , insufficient information in design models Software performance – Language/hardware combination fails to supports performance High defect level – System usability, reliability, and safety COMPANY PROPRIETARY 14 APC 0342

GQM Approach Identify software product metrics that can support analysis goals For Use Case diagrams – GOALS- completeness, traceability, correctness – Questions-How to achieve the goals – Metrics-use case metrics such as by Marchesi Later plans for application to Class diagrams COMPANY PROPRIETARY 15 APC 0342

BBN Approach For Use Case diagrams – Use of BBN to determine readiness for project transition into the next lifecycle phase – Actors, actions, communication association, as part of network topology – Work is on going for determining characteristics of nodes, e. g. weight, child and parent levels – CPT tables Assumption: Use Cases completely expanded COMPANY PROPRIETARY 16 APC 0342

Near term Goals Develop base set of IV&V techniques for – Requirements phase products assuming Use Case Artifacts. – Analysis phase products assuming class and sequence diagrams. – Design phase products assuming class, sequence, and deployment diagrams and state charts. Solidify feasibility of applying – GQM for Requirements and Analysis Phases – BBN to gauge readiness for lifecycle phase transition COMPANY PROPRIETARY 17 APC 0342