Sys ML Assessment Roadmap Approach SE DSIG

Sys. ML Assessment & Roadmap Approach SE DSIG Meeting Reston March 25, 2014 Yves Bernard Sanford Friedenthal

Purpose &Background § Purpose – Establish an approach to continue to assess Sys. ML and develop a Sys. ML roadmap in support of evolving MBSE needs • Scope includes more than Sys. ML specification updates, but can include tool integration, …. § Background – Previous Roadmap discussion at Reston, March 2012 (refer to minutes-syseng-2012 -03 -02, and roadmap presentation-syseng-2012 -03 -07) – Action item from Yves Bernard Roadmap presentation at Santa Clara SE DSIG in Dec 2013 to distill inputs and refine approach (refer to SE DSIG minutes-syseng/2013 -12 -01) – Working Group Established To Formulate Approach 2

Topics § § § MBSE Directions Proposed Assessment Approach Starting Point for Assessment Building the Roadmap Next Steps 3

MBSE Directions 4

MBSE Definition “Model-based systems engineering (MBSE) is the formalized application of modeling to support system requirements, design, analysis, verification and validation activities beginning in the conceptual design phase and continuing throughout development and later life cycle phases. ” INCOSE SE Vision 2020 (INCOSE-TP-2004 -02), Sept 2007 MBSE is SE 5

MBE To-Be State Source: NDIA MBE Final Report dated February 2011 Operational Models System Models Analysis Models ò )U (G s Hardware Models Component Models S SET Q R CLR Q Operational Models Analysis Models U (s) G (s) ò System Models Component Models Program Management Test Operational Models System Models Manufacturing Systems Hardware Logistics Customer Software Configuration Management § Needs § Current Capabilities § Budget/Schedule MBE Enhances Affordability, Shortens Delivery and Reduces Risk Across the Acquisition Life Cycle

SE Vision 2020 INCOSE-TP-2004 -02 September, 2007 § Domain-specific modeling languages and visualization that enable the systems engineer to focus on modeling of the user domain § Modeling standards based on a firm mathematical foundation that support high fidelity simulation and realworld representations § Extensive reuse of model libraries, taxonomies and design patterns § Standards that support integration and management across a distributed model repository § Highly reliable and secure data exchange via published interfaces 3/19/2018 7

INCOSE MBSE Roadmap MBSE Capability Reduced cycle times System of systems interoperability Design optimization across broad trade space Cross domain effects based analysis Extending Maturity and Capability Institutionalized MBSE across Academia/Industry Distributed & secure model repositories crossing multiple domains Well Defined MBSE Maturity Defined MBSE theory, ontology, and formalisms Architecture model integrated with Simulation, Analysis, and Visualization Matured MBSE methods and metrics, Integrated System/HW/SW models Ad Hoc MBSE Document Centric Emerging MBSE standards 2010 Refer to activities in the following areas: • Planning & Support • Research • Standards Development • Processes, Practices, & Methods • Tools & Technology Enhancements • Outreach, Training & Education 2020 2025

Proposed Assessment Approach 9

Continuing Sys. ML Assessment & Roadmap Approach § § § § § UML for SE RFP Sys. ML RFI (2009) Sys. ML RTF Priorities SE DSIG Inputs MBSE Adoption Issues MBSE Use Cases MBSE Usability MB Engr Environment Other 10

UML for SE RFP: evaluation criteria (p 44 -46) § § Ease of use Unambiguous – The language should be based on well-defined semantics. – specified well-formedness rules. § Precise – The language should specify the semantics, which can be translated into a formal mathematical based § Complete – The language support system specification, design, analysis, and verification. § Scalable – The language should provide support for modeling abstractions, elaborations, and refinements of complex systems. § Adaptable to different domains – The language should provide the capability to extend the semantics and notation of model elements to support specific domains (e. g. , aerospace, telecom, automotive). 11

UML for SE RFP: evaluation criteria (p 44 -46) § Evolvable – The language should be designed for change, and support backward compatibility with previous versions. § Capable of model interchange – support mapping to both an XMI schema and to the AP-233 neutral data exchange format (technical scope only) to exchange semantic information between tools. This is also intended to include the exchange of model version control information. § § Capable of diagram interchange Process and method independent – The language should be capable of supporting industry standard systems engineering technical processes, including EIA 632 and ISO 15288 and not overly constrain the choice of a specific process or method. § Compliant with the UML metamodel – The language should be consistent with the approved UML specification and should base the customization of UML for SE on extension mechanisms that UML defines. § Verifiable – Against RFP requirements 12

Starting Point For Assessment 13

Sys. ML v 1. 4 14

Update evaluation criteria and prioritize 1. Update/refine/clarify UML for SE RFP Criteria 2. Prioritize and weight the criteria 3. Use as reference for building the roadmap 15

UML for SE RFP Traceability Matrix Summary § Traceability matrix developed for original UML for SE RFP § Updated to reflect Sys. ML v 1. 4 § Use to identify potential gaps 16

UML for SE RFP Traceability Matrix Example 17

Sys. ML RFI Summary (2009) § § Web based survey – Used Survey. Monkey. com web survey service RFI Survey was open to any practitioner – RFI Survey advertised using • Introduced at INCOSE International Symposium MBSE sessions in Singapore • Posted on two Sys. ML forums (Yahoo, Google) • Posted on two UML forums • Several of the Linked. In groups • Large number of INCOSE members, including the MBSE list • SE Domain Special Interest Group (SE DSIG) • Sys. ML Revision Task Force (RTF) • OMG Sys. ML Discussion Group Survey consisted of two parts – Part I – questions on Sys. ML language • Language effectiveness, issues, recommendations – Part II – questions on how Sys. ML is used to support a model-based systems engineering approach • Methods, tools, training, metrics Contact information obfuscated to protect individuals and companies

Sys. ML RFI Example Result Key Finding #11: BDD and IBD Diagrams § Block Definition Diagrams (BDD’s) and Internal Block Diagrams (IBD’s): – Used the most – Valued the most – Hardest for stakeholders to understand • Confusion on the use of ports and interfaces. To what extent were the following diagram types used relative to the total modeling effort? 4. 22 Block definition diagram Internal block diagram 4. 03 Use case diagram 3. 81 3. 69 Activity diagram Sequence diagram 3. 58 State machine diagram 3. 04 Requirement diagram 2. 92 Package diagram 2. 58 Parametric diagram 0. 00 2. 20 0. 50 1. 00 1. 50 2. 00 2. 50 3. 00 3. 50 4. 00 4. 50

Sys. ML v 1. 4 RTF Priorities 20

Previous Sys. ML 1. 4 RTF Priorities Topic Comments X Variant modeling Out of scope The RTF declines to add specialized notation for variants ~ Sys. ML Formalization Sys. ML annex in the PSCS proposal ~ Allocation concept (13840) Some improvements made ~ Instance modeling Instance specification added in 1. 3 Semantic issue still pending. Might implies an UML enhancement Some remaining issue about specification of individuals 0 Property-Based requirement No noticeable progress Controversing 0 Sys. ML/MARTE convergence No noticeable progress WG no more active 0 Timing Diagram No work reported 0 Higher Status Conceptual Model of Sys. ML No work reported Prametrics improved in 1. 4 Port/Parameter relation Resolved in 1. 4 X Reference nested property (14055) Resolved in 1. 4 X Element Group (13928) Resolved in 1. 4 0 Parsing Text in Requirements (13939) No work reported ~ View/Viewpoint alignment with ISO 42010 Veiw/viewpoint improvement in 1. 4 0 Additional UML construct in UML 4 Sys. ML No work reported 0 Item flows on sequence diagrams No work reported X Align Sys. ML with QUDV Resolved in 1. 4 Model lib for ISO 80000 parts Resolved in 1. 4 X Ambiguous block hierarchy (14447) Resolved in 1. 4 ~ Easily disposed issues Improvments in 1. 4 0 Concepts from AUTOSAR (tech arch) No work reported 0 Alignment with UML Testing profile No work reported 0 HSUV example in Sys. ML tool No work reported 0 Requirements interchange issue (13177) No work reported 0 Automatic test case generation No work reported 0 Datasets (13219) No work reported X Standard alignment (URL, etc. . ) Resolved in 1. 4 (TBC) Missing references to issues, if any ~ Production issues SVN available Document production remains problematic X Constraining decomposition hierarchy Resolved in 1. 4 0 Lower Simplifying Parametrics Diagrams X Priority ~ X Profile constraint No noticeable progress 0: not started/stopped - Legend ~: in progress X: done No clear requirement regarding « simplification » To be re-evaluated against the capabilities added No precise set defined X: aborted 21

Source: Derived from SE DSIG Minutes Dec 2011 Presented at SE DSIG Dec 2012 as part of Roadmap Discussion Systems Modeling Directions & Needs § Rich diagrammatic syntax with standard symbol libraries for domain specific applications (e. g. Visio libraries) § Extensive viewing capability to query the model and present the results. (e. g. , similar to building architecture layers) § Extensive modeling checking and analysis capability to reason about the system model and confirm its integrity § Extensive reuse libraries

Source: Derived from SE DSIG Minutes Dec 2011 Presented at SE DSIG Dec 2012 as part of Roadmap Discussion Systems Modeling Directions & Needs § Cross domain model integration through transformation technology and interchange standards § Scalable model management part of PLM to include configuration control , change management & workflow § Built in modeling metrics to effectively estimate productivity, quality, and risk

MBSE Use Cases § Incrementally develop MBSE use cases based on accepted SE Processes – (refer to SEBo. K and ISO 15288 as top level framework) § Assess how well Sys. ML supports each use case 24

MBSE Use Cases Product Support ò Test Plan G(s) Analysi s Spec U(s) Verification Models System Architectural Model Analytical Models System Engineering Development Environment Performance, RMA, SWa. P, Cost, etc. Mechanical & Electrical Models Manufacturing S SET Software Models Q R CLR Q To measure Sys. ML effectiveness we need to understand the context of how it is used Source: John Watson © 2014 Lockheed Martin Corporation. All Rights Reserved.

Perform System Security Engineering UC Create and share validated reference libraries View external reference documents 1. 2. 3. Define and Organize UCs Create Activities to expose vulnerabilities Integration with threat analysis tools Update Structure and interface definitions Verify requirements are satisfied with test cases 1. 2. Define a document structure and content Publish document Source: John Watson Create a domain specific profile that includes concepts and iconic representations compatible with Sys. ML 1. 2. Define and Organize Requirements Create Activities 1. 2. Define and Organize Requirements Add domain attributes to Interfaces 1. 2. 3. Assessing change impacts Include and overlay impacted domain views Remove un-impacted domain views Execute defined behavior

MBSE Adoption Issues § § § § § More focus on mechanical engineering Provide more examples/guidance Availability of libraries of reusable models Availability of patterns Language stability Increased analysis capabilities A clear value assessment from using Sys. ML Model consistency Domain specific icons Support for continuum of models that support early concepts and more detailed formal models 28

MBSE Adoption Issues (cont. ) § § § Agility of modeling Dynamic (i. e. simulation) and static analysis capabilities Capture of trade studies Reduce the number of ways things can be modeled. This is a source of confusion to modelers Ability to represent model in textual form Better handling of large number of requirements FMEA capabilities Consider industries which are not highly regulated Consider how to model humans Make the model invisible (transparent) to support other discipline engineers MDA for Sys. ML 29

Building the Roadmap 30

Synthesizing the Assessment Results § Prioritize assessment results based on evaluation criteria § Develop roadmap strategies based on results from above § Roadmap scope can extend beyond Sys. ML specification updates and include – – – – Model libraries Tool integration Modeling tool capabilities Document generation capabilities Other related modeling standards (OSLC) Domain specific profiles … 31

How to organize the contributions? contributions Means Higher Topics RTF regular work RTF specific WG X Formalization of the constraints X Time support ? Precise Semantics (x. Sys. ML) X X Helpers for metrics X X System/HW/SW integration X Test domain integration X Cross domains integration X Domain specific concrete syntax X Standard Model Libraries Lower Specific RFCs/RFPs X Simplification Sys. ML/UTP Convergence WG X Sys. ML conceptual model Priority Sys. ML document production Sys. ML/MARTE Convergence WG Viewing capability (language aspects only) X X 32

Next Steps § Initiate Working Group § Identify lead for each data source – – – § § UML for SE Requirements Traceability Matrix Gaps Sys. ML RTF Issues Sys. ML RFI Findings MBSE Adoption Issues MBSE Use Cases Perform assessment Review findings Synthesize and prioritize findings Develop preliminary roadmap 33