M S Based System Development and Testing in a

M&S Based System Development and Testing in a Joint Net-Centric Environment Bernard P. Zeigler, Ph. D. , Co-Director, Arizona Center for Integrative Modeling and Simulation www. acims. arizona. edu and Joint Interoperability Test Command Fort Huachuca, AZ 85613 -7051

There is an acute need for a new Net-centric testing paradigm at the enterprise level where joint and coalition operations are conducted (Editorial, ITEA Jnl, Sept, 2005) Net-Centric T&E places an increased emphasis on standards conformance self-testing over the Global Information Grid Service Oriented Architecture (GIG-SOA ) and emphasizes services rather than systems

Simulation-based acquisition requires the use of M&S in all phases of the system development life-cycle Concept Refinement Modeling and Simulation Technology Development Systems of Systems System Development & Demonstration Production & Deployment especially starting early in system development when defects can be caught and corrected with low cost Operations & Support

Simulation-based development implies the need for simulation-based testing Specified as abstract model, e. g. UML Connecting middle ware, e. g. HLA System Under Test (SUT) send/receive messages Test Device send/receive messages Network -- an infrastructure to test a system that is first formulated as an abstract model is offered by distributed simulation -- both the System under Test (SUT) and the test device are coupled by exchanging data packets on a network Raises the question: How is the Test Device developed in an authoritative manner?

Problem: Testing Polar Opposite Requirements To deal with the increasing complexity and advanced decision capabilities of C 4 ISR systems =>testing methodology has to become more rigorous, in-depth and thorough To keep up with the rapid change and short development life cycles expected from the system builders => tests have to be ready to conduct in time scales compatible with the agile development strategies of new systems. Solution: employ formal M&S • to increase capabilities for simulation-based testing and • as a basis to increase the automation of testing processes.

Testing of interface standards is a focus area for automated simulation-based testing. Link-16 is required in all Joint and multi-national operations. Theater Warning DSP/SBIRS ABL AWACS JLENS F-15 THAAD TEL PATRIOT MEADS SIS(MSCS) AVENGER ATACMS AEGIS (CEP) The Joint Interoperability Test Command (JITC) has developed an automated test generation methodology as its core technology for testing conformance of systems to Link-16 This methodology is fundamentally enabled by the DEVS formalized modeling and simulation approach Link-16 specification

Recent Successful Application The JITC employed M&S-based testing for the initial major milestone evaluation of the Integrated Architecture Behavior Model (IABM) developed by the Joint Single Integrated Air Picture (SIAP) System Engineering Organization (JSSEO) in 2005. The test exercise produced significant results that uncovered flaws in the model design and added acknowledged value to the model development. The ACTGen Development Team, NGIT & ACIMS was selected as the winner in the Cross-Function category for the 2004/2005 M&S Awards presented by the National Training Systems Association (NTSA).

Types of Distributed Simulation Testing Description Example standards conformance test whether system conforms to standard supports interoperability Tactical Data Link Standards, Link-16, VMF, USMTF, interoperability test whether systems can interoperate (at the syntactic, higher, levels) Joint Translator Forwarder (JXF) – air to/from land exchange of tactical data mission/capabilities test whether system Joint Close Air of systems have Support (JCAS) capabilities required for mission

Levels of Distributed Simulation Testing Level/Variation Description Example One-on-One Stimulus/ Response - managed time - real time Test Driver sends message stimuli to which SUT responds IABM SIAP One-on-Many Federation ------------- managed time - real time Test Driver sends message stimuli and controls supporting federates including sensors IABM Peer to Peer Architecture Multiplatform (Many-to-Many interaction) ------------- managed time - real time Active mode (test driver controls scenario interaction) JIT 06 -03 live C 2 systems from USAF, USMC, Navy in remote locations Passive Mode (test manager opportunistically observes interaction Joint Combined Hardware-in -the-Loop (HWIL) Event (JCHE) Phase 3 CNI pilot event “Runs for Record” (~Aug 06) (CNI = Navy Common Network Interface) JXF tests

Multiplatform Distributed Simulation – controlled testing Platform (System, Component) Test Driver controls the scenario

Multiplatform Distributed Simulation - uncontrolled testing Platform (System, Component) Observer Test Coordinator Distributed Observers look for opportunities to test

Joint Test Enterprise Requirements • Must respond to requirements for increasing complexity, evolving from standards conformance, to interoperability, to joint operations capabilities testing • Top down test development methodology – aim for maximum reuse of test assets by supporting configuration of assets to meet objectives of each new exercise • Requires support of appropriate concepts and levels of abstraction – e. g. , Objectives, capabilities, functionalities, rule sets, test model federates • Requires development of software at levels of abstraction with appropriate flexibility to interface with configurable middleware and hardware to meet conditions of tests, e. G. , High speed/accuracy implies real-time operating system replaces default windows OS • Develop new assets when no configuration of existing assets can satisfy objectives; then develop with reusability and fit within methodology in mind

Joint Test Enterprise Methodology • Determine objectives up front and drive test configuration process • Determine capabilities to be tested • Develop scenarios/mission threads to involve SUT use of identified capabilities • Identify lower level functions supporting capabilities • Decompose functions into relevant sub sets of rules • Develop testable sequences of rules from relevant rule sets • Apply automated development of test sequences as test model federates

Integrated Development and Testing Process Reference Master Model Standard Provides Behavior Requirements at one or more levels of System Specification Formalization by mapping into DEVS representations Corresponding levels of System Specification Simulation based testing in net-centric environment Semi-automated test suite design based on Experimental Frames Real-time implementation and execution

Summary • Interoperability is mandated throughout the system development process to achieve mission effectiveness • The emerging challenges of testing in a joint environment require a new paradigm • A test methodology at the enterprise level that scales to net-centric services on the GIG is needed • Automation of the generation of test environments is needed – illustrated for tactical data link standards conformance testing