Multilevel Simulation of Discrete Network Models John A

Multilevel Simulation of Discrete Network Models John A. “Drew” Hamilton, Jr. , Ph. D. Lieutenant Colonel, United States Army Director, Joint Forces Program Office https: //cipo. spawar. navy. mil

Research Objectives JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • Specify discrete event network simulation components at different levels of abstraction with transparent consistency. • Examine extracted run-time components of a simulation. • Alter simulation model components at various levels of abstraction. • Validate the developed network simulation results against those obtained by using classical simulation procedures. Drew Hamilton http//www. drew-hamilton. com 2

Why Study Networks? JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • Computer networks provide the underlying support for the information explosion that is revolutionizing academia, industry and government. • This increase in capability brings a concomitant increase in complexity. • The current lack of easy to use , large scale network monitoring and modeling tools makes systematic study of networks difficult. Drew Hamilton http//www. drew-hamilton. com 3

Research Prerequisites JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • The network components must be decomposable into reusable generic objects. • Varying levels of detail are possible. • Accurate traffic can be represented. Drew Hamilton http//www. drew-hamilton. com 4

Abstraction in Simulation JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • Abstraction is the selective examination of certain aspects of a problem. The goal of abstraction is to isolate those aspects that are important for some purpose and suppress those aspects that are unimportant. Rumbaugh, et al, Object-Oriented Modeling and Design, Prentice-Hall. • Model abstraction is the identification of relationships between models described at different levels of detail and with deriving more abstract relationships from more detailed ones. Sevinc, “Theories of Discrete Event Model Abstraction, ” Proceedings of the 1991 Winter Simulation Conference. Drew Hamilton http//www. drew-hamilton. com 5

Relationships JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization Tradeoffs: detail vs. abstraction decomposition vs. aggregation Wall, “Multilevel Abstraction of Discrete Models in an Interactive Object-Oriented Simulation Environment, ” Ph. D. Dissertation, Texas A&M University, 1993. Drew Hamilton http//www. drew-hamilton. com 6

Network Complexity Factors JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • The number and variety of managed resources. • The distribution of devices and physical structure of the network, as well as subnetting. • The number and variety of communications services and distributed applications. • The degree to which services are integrated and the associated quality of service (Qo. S). • The number of organizational and administrative units. • Mission of the organization. Hegering, Abeck & Wies, “A Corporation Operation Framework for Network Service Management, ” IEEE Communications, Jan. 1996. Drew Hamilton http//www. drew-hamilton. com 7

Network Monitoring Tools JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • There a variety of commercial network analysis tools on the market. Law and Mc. Comas, “Simulation Software for Communications Networks: The State of the Art, ” IEEE Communications, Vol. 32, No. 3, Mar. 1994, pp. 44 - 50. • Many of these products are very expensive, putting them out of bounds for many researchers. • Fortunately, there are some free alternatives. • Network monitoring provides the measurements that ultimately validate the accuracy of a network simulation. • The monitoring process is neither easy nor inexpensive and is limited to the configuration already installed and operational. Drew Hamilton http//www. drew-hamilton. com 8

Network Monitoring JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • An important step in simulating a complex system is to observe the system (if possible). • Systematic observation also yields data for validating the model. • Additionally, monitoring provides the means to employ historical validation. Sargent, “Simulation model verification and validation, ” Proceedings of the 1991 Winter Simulation Conference, p. 40. Drew Hamilton http//www. drew-hamilton. com 9

Network Monitoring JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • Drawbacks: – This is not predictive, just lessons learned by the hard taskmaster of experience. – Experimenting on an operational network is: • risky • costly • interfering • Meaningful measurement efforts must be made over time and will produce very large amounts of data to be interpreted. • Theoretical systems cannot be directly observed. Drew Hamilton http//www. drew-hamilton. com 10

Network Simulators JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • NETSIM – M. I. T’s Network Simulator • Ma. RS – Maryland Routing Simulator (University of Maryland) • LANSF – Local Area Network Simulation Facility (University of Alberta) • SMURPH – System for Modeling Unslotted Real-Time Phenomena (University of Alberta) Drew Hamilton http//www. drew-hamilton. com 11

Commercial Simulators JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • General-purpose simulation languages such as: MODSIM, BONe. S DESIGNER, GPSS/H, SIMSCRIPT II. 5, SLAMSYSTEM, SES/workbench. • Communications-oriented simulators such as BONe. S Plan. Net, COMNET III, L*NET II. 5 and NETWORK II. 5. • Communications-oriented simulation language such as: OPNET Modeler. Drew Hamilton http//www. drew-hamilton. com 12

Simulation Granularity JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization Drew Hamilton http//www. drew-hamilton. com 13

Simulation Granularity JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • The major component of resolution is the level of detail that the simulation can receive as input and return as output. • If the resolution of a network model is at the packet level, then there is no information provided at the bit level. • Bits submitted as input to the model cannot be processed unless the bits are collected and formed as packets prior to processing. • The resolution of the output will be at the packet level unless an external synthesizing function of some sort is used to manipulate the output. • The output will be produced based on the granularity of the simulation, but the output may be manipulated by an external function which may change the granularity. Drew Hamilton http//www. drew-hamilton. com 14

Variable Resolution JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • Davis outlines methods of varying simulation resolution: – Using high resolution to provide a picture when the lower-resolution depiction seems too abstract. – Invoking high resolution for special processes within the course of an otherwise low resolution simulation. – Using high resolution to establish bounds for parametric analysis using lower-resolution models. (e. g. , the number of retries to deliver a packet. ) – Using high resolution to calibrate lower-resolution recognizing that knowledge of the world comes at all levels of detail. – Using low resolution for decision support, including rapid analysis of alternative courses of action. Davis, “An Introduction to Variable-Resolution Modeling and Cross-Resolution Model Connection, ” RAND Report R-4252 -DARPA, The RAND Corporation, Santa Monica, Calif. , 1992. Drew Hamilton http//www. drew-hamilton. com 15

Network Simulation JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • The complexity of network simulation requires multiple forms of abstraction control. • Three methods of interest are: – Multilevel Simulation – Hierarchical Abstraction – Aggregation Drew Hamilton http//www. drew-hamilton. com 16

Multilevel Simulation JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization Drew Hamilton http//www. drew-hamilton. com 17

Hierarchical Abstraction JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization Drew Hamilton http//www. drew-hamilton. com 18

Aggregation JOINT FORCES PROGRAM OFFICE • Representational aggregation • Absolute consistency Drew Hamilton a Joint Command & Control Integration & Interoperability Group organization http//www. drew-hamilton. com 19

Resolution Summary JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • Model resolution is a critical component in determining the utility of a simulation. Model fidelity is a closely related concept but one does not imply the other. • Simulation models are under development which allow the analyst to dynamically alter the resolution of the various components. • There is a fundamental need for variable resolution models in which there is true consistency across resolution levels and for concepts and methods making it easier to do cross-resolution work, including models not originally designed to be compatible. Davis & Blumenthal, “The Base of Sand Problem, A White Paper on the State of Military Combat Modeling, ” RAND Report N-3148 -OSD/DARPA, 1991. • An understanding of abstraction, resolution & multimodeling provides the basis for an open simulation architecture OSA. Hamilton & Pooch, “An Open Simulation Architecture for Force XXI, ” Proceedings of the 1995 Winter Simulation Conference, Washington, D. C. , Dec. 3 - 6, 1995, pp. 1296 - 1303. Drew Hamilton http//www. drew-hamilton. com 20

OPNET JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • Communications-oriented scripting language. • Provides access to source code. • Uses a multilevel modeling structure. * *State Transition Diagram Drew Hamilton http//www. drew-hamilton. com 21

Network Model (Upper Level) JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization Drew Hamilton http//www. drew-hamilton. com 22

Network Model (Lower Level) JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization Drew Hamilton http//www. drew-hamilton. com 23

Node Model JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization Drew Hamilton http//www. drew-hamilton. com 24

Process Model JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization Drew Hamilton http//www. drew-hamilton. com 25

Validation Structure JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization Does the model implementation correctly reflect the network? Do the predicted approximate the actual results? A modification to Knepell and Arangno’s validation framework Does the software perform correctly? Drew Hamilton http//www. drew-hamilton. com 26

Conceptual Model Validity JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • Three subnets: 133, 134, 135 were monitored and 250, 000 packets were collected from each subnet. • Data used as simulator input and three performance measures computed: – packet throughput in packets per second – mean packet length – utilization in terms of throughput * mean packet length divided by bandwidth Drew Hamilton http//www. drew-hamilton. com 27

Observed versus Expected JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization Expected Observed Drew Hamilton http//www. drew-hamilton. com 28

Conceptual Validation | Results JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • Used the Smith-Satterthwaite procedure – for comparing means with unequal variances. – computer test statistic for a t test. • Packet lengths had identical means and variances, no further analysis needed. • Observed throughput and utilization data had high variances, expected data had low variances. • Throughput and utilization data for all subnets passed t-tests using the Smith-Satterthwaite (i. e. paired) procedure. Drew Hamilton http//www. drew-hamilton. com 29

Steady State Computation JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • Three stations with varying loads were selected. • The packet throughput for each station was collected during three sixty second runs each using a different random number seed. • After discarding the first ten seconds of transient observations, the means of the remaining observations were computed. • From Udo W. Pooch. Ph. D. , “if the number of observations in which the output is greater than the average is about the same as the number in which it was less, then steady state conditions are likely to exist. ” Drew Hamilton http//www. drew-hamilton. com 30

Steady State on Guru JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization Drew Hamilton http//www. drew-hamilton. com 31

Experimentation Plan JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • High resolution: script or table driven • Low resolution: distribution driven • Exponentially distributed synthetic workloads were generated for high resolution nodes. • Nodes set to either low or high resolution. Drew Hamilton http//www. drew-hamilton. com 32

Operational Validity JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • Objective Approach: – Utilization data statistically tested and found to be valid. • Subjective Approach: – Throughput data found to have similar means but significantly different variances. Drew Hamilton http//www. drew-hamilton. com 33

Summary JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • Multilevel, mixed-resolution simulation can effectively expand the problem domains studied through simulation in the following ways: – improved cost efficiency – ability to model notional network components – ability to improve simulation run times by eliminating unnecessary detail • Representational aggregation of nodes into subnets and subnets into collections of subnets provides abstraction mechanisms that allow the analyst to focus on areas of specific interest. • Lack of data may dictate simulating entities at low resolution. Drew Hamilton http//www. drew-hamilton. com 34

Original Recommendations JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • Operational use of the MMRNS requires an automated topology builder. • The OPNET environment provides a powerful GUI, but individually modeling hundreds of individual nodes is tedious. • Gateways between the MMRNS and other open architecture tools should be built. • Mixed-resolution simulation should be used to model communications networks still under development. Drew Hamilton http//www. drew-hamilton. com 35

Results over Time JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • Army DISC 4 adopted OPNET as a standard due to its open architecture – tactical version under development. • Joint Staff initiates NETWARs program. • Army SIGCEN has implemented a network modeling and simulation block of instruction in their FA 24 course. • Army ISEC has been using mixed resolution simulation to support base ops network design in their Technology Integration Center. • ISEC in partnership with University of Arizona form Arizona Center for Integrative Modeling and Simulation. Drew Hamilton http//www. drew-hamilton. com 36

Conclusion JOINT FORCES PROGRAM OFFICE a Joint Command & Control Integration & Interoperability Group organization • The Defense Department has demonstrated multiple interests in the extension of this research. • This research demonstrates the efficacy of mixed resolution simulation. The flexibility provided by mixed resolution simulation may be safely utilized by following the methodology described herein. • It was specifically demonstrated that utilization could be accurately simulated with as many as 75% of the stations operating in low resolution mode. Drew Hamilton http//www. drew-hamilton. com 37