Automation for System Safety Analysis Jane T Malin

Automation for System Safety Analysis Jane T. Malin, Principal Investigator Project: Automated Tool and Method for System Safety Analysis Software Assurance Symposium September, 2008 Complex systems typically fail because of the unintended consequences of their design, the things they do that were not intended to be done. - M. Griffin, System Engineering and the “Two Cultures” of Engineering, March 28, 2007 1 SAS_08_Automation_for_System_Safety_Analysis_Malin

Overview • • • Problem/NASA Relevance Technical Approach and Overview 2008 Target Capability Crew Exploration Vehicle (CEV) Launch Abort System Case Data – Constellation (Cx) failure modes and effects analysis/critical items lists (FMEA/CILs) • Technical Challenges – Information Extraction – Semi-Automated Model Construction – Analysis and Test Case Generation • 2009 Planned Capability • Potential Applications 2 SAS_08_Automation_for_System_Safety_Analysis_Malin

Problem and NASA Relevance • NASA needs early evaluation of software (SW) requirements and design, to reduce software-system integration risks – Assess system failures and anomalous conditions that may challenge software in system integration testing – Identify robustness issues early (and often) – Identify requirements gaps early (and often) • Project test case: NASA Constellation (Cx) Launch Abort System (LAS) for Pad Abort PLANT and Environment FAULTS/Reliability ‘Activate’ Faults and Influence Failures Operations and Stresses SOFTWARE FAULTS/Reliability 3 SAS_08_Automation_for_System_Safety_Analysis_Malin

Technical Approach Systematic semi-automated extraction and analysis for early evaluation and rapid update – Capture model of the controlled system architecture • Abstract physical architecture models with subsystems, functions, interfaces, connections • Extract directly from requirements and design text and data – Capture risks and hazards in model • Constraints, hazards, risks from requirements and design • Risk and failure libraries – Analyze and simulate to identify risks and constraints • Analyze and simulate hazard/risk propagation in the system • Use operational and off-nominal scenarios and configurations – Identify possible test scenarios for virtual system integration testing 4 SAS_08_Automation_for_System_Safety_Analysis_Malin

Technology Overview Hazard Identification Tool (HIT) Aerospace Ontology CONFIG Hybrid Simulation Extract and Model Information Extractor Identify Test Cases Requirements Text Analyze, Simulate and Test Early - Identify interaction-propagation paths - Investigate influence of timing - Perform Virtual Tests Virtual System Integration Laboratory (VSIL) 5 SAS_08_Automation_for_System_Safety_Analysis_Malin

Modeler: Architecture Model and Visualization of a Set of Requirements [C. 1] Telecommunication Subsystem (Tele. Sub) • • • [C. 1. 1] The CDHC sends the Tele. Sub a compressed picture. [FG. 1] [Tele. Sub C. 1. 4] [C. 1. 2] The CDHC sends the Tele. Sub telemetry. [FG. 2] [FR. 1] [FR. 5] [Tele. Sub C. 1. 5] [C. 1. 3] The CDHC sends In View of Ground alerts to the Tele. Sub. [DP. 5. 6] [Tele. Sub C. 1. 6] [C. 1. 4] The CDHC receives plan files from the Tele. Sub. [FR. 3] [Tele. Sub C. 1. 3] [C. 1. 5] The CDHC receives ground commands from the Tele. Sub. [FR. 3] [Tele. Sub C. 1. 2] [C. 1. 6] The CDHC receives the Tele. Sub operating state from the Tele. Sub. [DP. 5. 5] [Tele. Sub C. 1. 1] … [C. 2] Camera Subsystem • • • [C. 2. 1] The CDHC sends the Camera a "take picture" command. [FG. 2] [FR. 1] [FR. 3] [C. 2. 2] The CDHC sends the Camera x, y and z gimballing coordinates. [FG. 2] [FR. 1] [FR. 3] [C. 2. 3] The CDHC sends a turn on command to the Camera. [DP. 5. 3] [H Constraint 1. 1. 4] [C. 2. 4] The CDHC sends a turn off command to the Camera. [DP. 5. 3] [C. 2. 5] The CDHC receives a compressed picture file from the Camera. [FG. 1] [FG. 2] [FR. 1] … [C. 4] Attitude Determination Subsystem (ADS) • [C. 4. 1] The CDHC receives an In View of Ground alert from the ADS. [DP. 5. 6] [ADS] • [C. 4. 2] The CDHC receives the ADS operating state from the ADS. [DP. 5. 5] [ADS] Note: CDHC is Command Data Handling Computer Physical/Functional Architecture Visualization 6 SAS_08_Automation_for_System_Safety_Analysis_Malin

CONFIG Simulation: Assess Timed Scenarios CONFIG simulation tool used for software virtual validation testing for NASA 1997 90 -day manned Lunar Life Support Test • Software: Intelligent control for gas storage and transfer • Models: Gas volumes and processing systems controlled by software; mixed fidelity, discrete and continuous • Testing: Simulated failures and imbalances that would not be tested in hardware-software integration • Too slow to develop, too expensive, too destructive • Results: Identified software requirements deficiency due to unintended consequences of integrating gas processing systems 7 SAS_08_Automation_for_System_Safety_Analysis_Malin

Virtual System Integration Lab (VSIL) Models and Test Definitions • Triakis has used VSIL in >25 avionics verification projects • Project Output to VSIL: Models and test definitions DE: detailed executable, the simulation of the embedded controller hardware RAM/ROM: memories ES: executable specifications I/O: input/output V&V: verification and validation CPU: processor 8 SAS_08_Automation_for_System_Safety_Analysis_Malin

2008 Target Capability • Integration: Information extraction, architecture modeling and test generation – Model parts extracted from requirements and FMEA/CIL texts • XML output, including reference traces • Components, physical hierarchy, connections, interface components, flows/resources, time or phase context • Functions, vulnerabilities, limits, failures, causes – Ontology for model extraction and semi-automated modeling • Identify types of components, functions, problems, resources • Paths: A provides power to B; C receives command data from B • Functions and failures: B processes command data; B failure mode is No Output command to D; cause of no output is B does not receive power. – Semi-automated model development from extracted model parts • Component model library: Resource producer; Data processor… • Generic functions, failures and influences: Resource problem, Stressor, Data rate problem, Data Integrity problem… – Model visualization for overview and completeness checking – Simulation and path analysis to identify hazardous configurations, scenarios and test cases • Where failure or degradation of required functions results from unintended system interactions • Project Participants – CEV Flight Software Engineering, Abort Decision Logic, Abort Sequence – Orion Software Safety and Mission Assurance 9 SAS_08_Automation_for_System_Safety_Analysis_Malin

CEV Launch Abort System (LAS) Case Crew Exploration Vehicle (CEV) Pad Abort Sequence - notional 10 SAS_08_Automation_for_System_Safety_Analysis_Malin

CEV Launch Abort System Case • CEV Crew Module (CM) software controls the Pad Abort Sequence – LAS events, trajectory – No direct command feedback • Components for the case – Paths and interactions for commands firing separation pyros during pad abort sequence • CM computer → Remote Interface Unit → LAS Pyros • Possible Addition Case: Inertial measurement unit (IMU) → GN&C → Abort motor – Summer Systems Engineering Intern manually built TEAMS models of pyros and Remote Interface Unit (RIU) from FMEA/CILs • Need to perform simulation rather than pure path analysis on the LAS case – Timing is important in aborts – Hazard Identification Tool (HIT) path analysis models do not capture timing – CONFIG simulations can use timing 11 SAS_08_Automation_for_System_Safety_Analysis_Malin

Data – Con. Ops, Requirements, Safety Analyses • Developing tools and methods using documents and data from CEV sources, both NASA and Orion contractor • Met with NASA expert on Orion software that controls Launch Aborts – Identified key CEV documents and confirmed analysis approach • Orion Contractor’s Concept of Operations – Best guess at the Abort Sequence • • Interface Requirements Documents Interface Control Documents, when they become available Project Orion Flight System Safety Hazard Analysis FMEA/CILs (preliminary now) – Determined that much key information (sensors, feedback) is TBD, and being defined by the Cx Integrated Abort Team 12 SAS_08_Automation_for_System_Safety_Analysis_Malin

Technical Challenges • Limitations of early life-cycle requirements, design, hazard analysis and FMEA/CIL as sources for – Automatic extraction of model information from requirements and design text – Semi-automatic construction of models from extracted information, for simulation and visualization • Combining graph analysis and simulation to identify possible hazard paths and off-nominal test scenarios for complex system interaction models • Maturation Challenge: Develop mature software prototypes that can be used to develop products for broader use 13 SAS_08_Automation_for_System_Safety_Analysis_Malin

Information Extraction • • Objective: Extract information from CEV sources for semi-automated model construction Information Extraction Evaluation – Success Criterion: % of available model information extracted, compared to % of model information available • Types of Extractions (from text to XML) – Interface Requirements → Components, connections, flows/resources, reference trace • Some interface components, vulnerabilities, functions, limits, context (time or phase) – FMEA/CILs → Components, system to component hierarchy, interfaces, subcomponents, functions, failures, causes, effects – Architecture Descriptions → Components, interfaces, hierarchy, functions; some design parameters, acronyms • Challenges – Multiple document formats require definition of data structures for each document, some difficult sentence parsing – Indirect access to Cradle requirements via PDF documents • Progress: – Parser improved by incorporating NESC-funded parser from Univ. Central Florida – Experience coding multiple document data structures, leading to format specification approach – Successful extraction from Cradle-based PDF docs. 14 SAS_08_Automation_for_System_Safety_Analysis_Malin

Example Information Extraction Benchmarking LAS Description: The LAS consists of LAS Components a nose cone, a canard section which Component: Nose cone enables the LAS to reorient the CM for Component: Canard Section parachute deployment following an Function: ? abort, three propulsive motors (attitude Component Enables Function: LAS: reorient the CM Function Enables Function: LAS? : deploy parachute control, jettison, and abort), a bi-conic (following an abort) adapter which provides the structural Component Group: Three propulsive motors interface to the CM, and a boost protective cover (BPC) sized for ascent Component: attitude control motor Enables Function: LAS? : control attitude heating to protect CM thermal protection … system (TPS) coatings. Component: A bi-conic adapter Ideal Model Extraction Benchmark: Top Level: LAS Function: reorient the CM Agent: LAS Action: reorient Operand: CM Function: control attitude Agent: LAS? Action: control Operand: ? Variable: attitude … Function: deploy parachute Agent: LAS? Action: deploy Operand: parachute Connection Connector: bi-conic adapter From: LAS To: CM Type: Structural Component: Boost protective cover Acronym: Boost protective cover = BPC Design parameter: size Determined by: ascent heating Function: protect CM thermal protection system coatings Agent: Boost protective cover Action: protect Operand: CM thermal protection system coatings Other: CM Component: Thermal protection system Acronym: thermal protection system = TPS 15 SAS_08_Automation_for_System_Safety_Analysis_Malin

Model Construction • Objective: Use extracted model information for semi-automatic model construction • Challenges – LAS case with timing issues requires CONFIG simulation, not just HIT architecture model – Missing information in pre-PDR documents • Operating modes, vulnerabilities, side effects etc. • Progress – CONFIG provides visualization and supports libraries of generic components – Concept for use of Aerospace Ontology hierarchies with CONFIG library for generic components, operating modes, functions, side effects, problems 16 SAS_08_Automation_for_System_Safety_Analysis_Malin

Analysis and Test Case Generation • Objectives: – Identify and evaluate failure and hazard propagation in the system model • Unintended system interactions and unanalyzed propagation of failure effects – Generate corresponding off-nominal test cases (configurations, scenarios) • Evaluations – Hazard Analysis Success Criterion: Are new hazards and failures identified, compared to standard method? – Test Case Success Criterion: Does model-based hazards and failures analysis make test generation easier than current methods? • Challenges – Path analysis algorithm for HIT models needs to be adapted, because LAS case will use CONFIG simulation rather than HIT – Extracting information for operational and failure scenarios has not yet been addressed • FMEA/CIL effects information can provide parts of failure scenarios • Progress – Concept for combining HIT and CONFIG models, using CONFIG Inner Models for subsystem details 17 SAS_08_Automation_for_System_Safety_Analysis_Malin

2009 Planned Capability • Capabilities should be valuable from pre-PDR through operations • Continue tool enhancements focusing on – Off-nominal test scenario discovery and evaluation – Component model library and generic defaults • Use on a new CEV case – more complex interactions and more complete system information • Deliver – Tool prototype files – Information extraction tool, Aerospace ontology, HIT graph modeling and analysis, CONFIG simulation modeling, model libraries – Documentation – methods, tools, user manuals 18 SAS_08_Automation_for_System_Safety_Analysis_Malin

Future Applications • Improve efficiency and repeatability of system and software risk analysis – Reduce time spent reanalyzing when specifications and designs change • Visualize integrated requirements – Combined success and failure spaces – Combined system and operation/event spaces • Validate requirements and perform integration tests early with low-fidelity and multi-fidelity simulation • Validate FMEAs and fault trees • Evaluate completeness and consistency of requirements and risk – Support requirements traceability evaluations – Enhance analysis with reliability and event probability information 19 SAS_08_Automation_for_System_Safety_Analysis_Malin