1 st Annual Workshop NAS-Wide Simulation in Support

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Probabilistic NAS Platform George Hunter, Fred Wieland Ben Boisvert, Krishnakumar Ramamoorthy Sensis Corporation December 10, 2008

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Outline • • • What is PNP? Team and development history Example uses of the model Software processes and testing Validation 2

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Outline • • • What is PNP? Team and development history Example uses of the model Software processes and testing Validation 3

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 What is PNP? • An fast-time and flexible NAS-wide simulation tool – Real-time or fast-time modes • Half-hour runtime on a laptop, to simulate a day in the NAS – Physics-based: trajectories computed through integrating aerodynamic energy balance equations by varying the time-step size – System uncertainties (weather, security, operations …) – Plug-and-play architecture • Dynamic clients (TFM, DAC, AOC, …) – An ATC community resource – Formal software development processes in place – Adaptable to current system or Next. Gen future concepts • Uses – Environment in which to design, build and test decision support tools • TFM, DAC, AOC, … • Fast-time, real-time, shadow-mode – Potential NAS tool • Service provider, operator, collaborative uses – Benefits assessment tool • Fast-time tool to evaluate improved infrastructure, technology, procedures … • Evaluates historic and future traffic scenarios in weather 4

Graphical User Interface Plan View Display Reports NAS Database Flight Data MATLAB® Scripting Interface Probabilistic NAS Platform (PNP) A fast-time physics-based (trajectory-based) NAS-wide modeling tool Weather Data NAS Simulation 1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 PNP Architecture Performance Data 5

Graphical User Interface Plan View Display Reports NAS Database Flight Data Probabilistic NAS Platform (PNP) MATLAB® Scripting Interface Weather Data NAS Simulation Performance Data Sim. Objects MATLAB® Client Java Client As Middleware Prob-TFM A fast-time physics-based (trajectory-based) NAS-wide modeling tool External Client (Any Language) Decision making 1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 PNP Architecture 6

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 PNP Client Development • TFM client development – Prob. TFM (Sensis internal development) • TFM client integration – C 2 (algorithms from and used with permission of Bob Hoffman, Metron) – Constrained LP (algorithms from and used with permission of NASA, Joey Rios) in progress • DAC client integration – Mx. DAC (algorithms from and used with permission of Min Xue, NASA/UARC) • AOC client development – Gaming behaviors (collaboration with GMU/Lance Sherry) in progress 7

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Capabilities Summary • • • • • Real-time Fast-time Airport weather impact models Airspace weather impact models Weather-integrated decision making Probabilistic modeling / decision making Traffic flow management Dynamic airspace configuration Surface traffic modeling Terminal area modeling Super density operations Fuel burn modeling Emissions modeling Trajectory-based operations Separation assurance Plug-n-play Fast run-time Existing Can Support √ √ √ √ √ 8

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Outline • • • What is PNP? Team and development history Example uses of the model Software processes and testing Validation A fast-time physics-based (trajectory-based) NAS-wide modeling tool 9

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Team and Development History 2003 George Hunter 2004 2005 Krishnakumar Ramamoorthy 2006 Ben Boisvert 2007 Diego Escala 2008 Tae Lee Michelle Lu Huina Gao People Project Env’nment System lead Projects Users Software System Software Java/real-time Matlab Data Funct’ality Software lead System Web 2. 0 WSI collaboration for real-time weather feed NAS-wide, probabilistic JPDO Internal Wx modeling and routing Client architecture NASA NRAs Dynamic clients FAA NASPAC NWA GMU 10

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Outline • • • What is PNP? Team and development history Example uses of the model Software processes and testing Validation A fast-time physics-based (trajectory-based) NAS-wide modeling tool 11

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Example Project Uses • JPDO Modeling and Analysis – Next. Gen performance evaluation with weather • FAA NASPAC Weather Modeling – Convection impact modeling for NASPAC • NASA Gaming NRA – Evaluation of Next. Gen gaming with AOC clients • NASA Meta. Simulation NRA – Investigation of TFM + DAC interactions • NASA SLDAST RFA – Evaluation of Next. Gen TFM concepts and models • NASA Market-Based TFM NRA – Evaluation of Next. Gen market-based TFM concepts 12

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Next. Gen Sensitivity Studies Next. Gen Performance Sensitivity Analysis Benefit of Using Clear Weather Forecasts Benefit of Improved Wx Forecasts NAS Performance Sensitivity Case 2: No distinction between clear and heavy weather forecast accuracy Persistence forecast 11/16/06 Case 1: Take advantage of improved forecast accuracy in clear weather George Hunter, Fred Wieland " Sensitivity of the National Airspace System Performance to Weather Forecast Accuracy, " Integrated Communications, Navigation and Surveillance Conference (ICNS), Herndon, VA, May, 2008 Kris Ramamoorthy, George Hunter, "Evaluation of National Airspace System Performance Improvement With Four Dimensional Trajectories, " AIAA Digital Avionics Systems Conference (DASC), Dallas, TX, October, 2007 13

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Market-Based TFM Studies Delay SCC UAL 233 Delay Cost NAS Access Valuation Models George Hunter, et. al. , "Toward an Economic Model to Incentivize Voluntary Optimization of NAS Traffic Flow, " AIAA ATIO Conference, Anchorage, AK, September, 2008. 14

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Dynamic Airspace Configuration Nov 12, 2006, LAT=2, #Gen=40 ZFW FAA sectors George Hunter, "Preliminary Assessment of Interactions Between Traffic Flow Management and Dynamic Airspace 15 Configuration Capabilities, " AIAA Digital Avionics Systems Conference (DASC), St. Paul, MN, October, 2008.

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 AOC Dispatch Use Case Reroute with low probability of delay 16

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Outline • • • What is PNP? Team and development history Example uses of the model Software processes and testing Validation A fast-time physics-based (trajectory-based) NAS-wide modeling tool 17

Processes and Testing Cycle 1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Project Monitoring & Control Development Tracking Quantitative Project Management Branch Configuration Management Regression Testing Unit and System Testing Trunk Configuration Management 18

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Project Monitoring and Control • JIRA is used to track issues – Project Manager and Lead Software Engineer assign task priorities, due dates, and personnel. • Weekly telecoms keep distributed team apprised of PNP and communications open • Project Manager maintains a master schedule in MSProject 19

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Development Tracking • Software engineers use JIRA to track and status development efforts. 20

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Branch Configuration Management • Software Engineers are responsible for creating branches from the trunk to develop fixes/enhancements. • The Configuration Management of the software is accomplished with Subversion – Subversion is an open source version control system (http: //subversion. tigris. org/) 21

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Unit and System Testing • Software Engineers are responsible for creating unit tests to verify the correctness of their code. The JIRA issue number is to be used throughout the code and unit tests for tracking purposes. • Software Engineers are responsible for running their own system/function tests to verify their software. • Once testing is validated, code is merged back on to the trunk. 22

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Trunk Configuration Management • Once all validated JIRA issues are merged unto the trunk, regression testing is performed. 23

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Regression Testing • Regression testing • Aggregate results – – – Total delay Total congestion Traffic volume #TFM initiatives Runtime • Different scenarios – Truncated demand set – Full demand set – Weather • Automated – Weekly or as required • Archived • Graphical quick-look 24

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Quantitative Project Management • Regression testing validation is performed and the release letter is updated. • Release is tagged in Subversion. • JIRA issues are closed. • Documentation is updated to reflect changes in software. 25

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Outline • • • What is PNP? Team and development history Example uses of the model Software processes and testing Validation A fast-time physics-based (trajectory-based) NAS-wide modeling tool 26

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 System-Level Engineering Validation • ASPM / ETMS verification tests √ – Compare ASPM/ETMS data with simulation data • Calibrate concept to match aggregate field observations – Models • Trajectory data • Airport capacities (VMC / IMC) • Sector capacities in weather – Aggregate performance • Mean flight delay • Sector and airport overloadings – Detailed performance • Flight delay by airport and time of day • Overloading and delay patterns (Spatial and temporal) § Delays by airport and time of day § Sector and airport loading by time of day § Spatial loading patterns – Light and heavy weather days 27

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 System-Level Software Verification • Cross check sums – – – SFlights = SOperations at all airports SFlight time = SMinutes from sector loads SSector load by sector = SSector load by time SAirport ops = SFlights using the airport in demand set SDelays by flight = SDelays by time; and reroutes • Weather data checks – – Compare PNP/Metar airport capacity with ASPM AAR/ADR Compare PNP/Metar airport capacity with ASPM IFR periods Ensure SEn route convection versus time of day is smooth Ensure Wx. MAP ≤ MAP for all sector time bins • Graphical – Ensure reroutes overlaid on weather make sense • TFM Performance – Number of delays per flight, min and max flight delay – Maximum airport and sector overloading (ensure are reasonable) 28

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 System-Level Engineering Validation • ASPM / ETMS verification tests – Compare ASPM/ETMS data with simulation data • Calibrate concept to match aggregate field observations – Models √ • Trajectory data • Airport capacities (VMC / IMC) • Sector capacities in weather – Aggregate performance • Mean flight delay • Sector and airport overloadings – Detailed performance • Flight delay by airport and time of day • Overloading and delay patterns (Spatial and temporal) § Delays by airport and time of day § Sector and airport loading by time of day § Spatial loading patterns – Light and heavy weather days 29

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Trajectory Model Validation • Compared to ETMS flight data (May 2008) N: 316 Mean: 0. 321 min Std dev: 11. 95 min Detrended for Range George Hunter, Ben Boisvert, Kris Ramamoorthy, "Advanced Traffic Flow Management Experiments for National Airspace Performance Improvement, " 2007 Winter Simulation Conference, Washington, DC, December, 2007 Mean: 0. 80 min Std dev: 6. 51 min R 2: 0. 012 30

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Prob. TFM Performance • ASPM / ETMS verification tests – Compare ASPM/ETMS data with simulation data • Calibrate concept to match aggregate field observations – Models • Trajectory data • Airport capacities (VMC / IMC), actual and forecasted • Sector capacities in weather, actual and forecasted √ – Aggregate performance • Mean flight delay • Sector and airport loadings – Detailed performance • Flight delay by airport and time of day • Overloading and delay patterns (Spatial and temporal) § Delays by airport and time of day § Sector and airport loading by time of day § Spatial loading patterns – Light and heavy weather days 31

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Compare With Field Observations • Compare to ETMS/ASPM – Forecast accuracies, Decision making horizon, Delay distribution January 7, 2007 (Similar results with other days) LAT = 60 minutes (14. 5, 1657) LAT = 30 mins LAT = 0 32

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Verification of Results • ASPM / ETMS verification tests – Compare ASPM/ETMS data with simulation data • Calibrate concept to match aggregate field observations – Models • Trajectory data • Airport capacities (VMC / IMC), actual and forecasted • Sector capacities in weather, actual and forecasted – Aggregate performance • Mean flight delay • Sector and airport loadings √ – Detailed performance • Flight delay by airport and time of day • Overloading and delay patterns (Spatial and temporal) § Delays by airport and time of day § Sector and airport loading by time of day § Spatial loading patterns – Light and heavy weather days 33

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 System Loading Patterns Prob. TFM predicted, 14: 45 GMT ETMS Actual, 14: 45 GMT ETMS Underloading Overloading ETMS Prob. TFM loading 34

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Verification of Results • ASPM / ETMS verification tests – Compare ASPM/ETMS data with simulation data • Calibrate concept to match aggregate field observations – Models • Trajectory data • Airport capacities (VMC / IMC), actual and forecasted • Sector capacities in weather, actual and forecasted – Aggregate performance • Mean flight delay • Sector and airport loadings – Detailed performance • Flight delay by airport and time of day • Overloading and delay patterns (Spatial and temporal) § Delays by airport and time of day § Sector and airport loading by time of day § Spatial loading patterns √ – Light and heavy weather days, control days 35

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Conclusion • The development of PNP has benefited from lessons learned over past two decades in NAS system wide modeling – Plug and play simulation architecture – Supports both analytical and HITL studies – Adaptable to simulate current system as well as Next. Gen future concepts – Fast-time, physics-based – Formal software development processes in place – Probabilistic decision making and extensive weather modeling explicitly incorporated in tool 36

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. Publications George Hunter, "Preliminary Assessment of Interactions Between Traffic Flow Management and Dynamic Airspace Configuration Capabilities, " AIAA Digital Avionics Systems Conference (DASC), St. Paul, MN, October, 2008. George Hunter, et. al. , "Toward an Economic Model to Incentivize Voluntary Optimization of NAS Traffic Flow, " AIAA ATIO Conference, Anchorage, AK, September, 2008. George Hunter, Fred Wieland " Sensitivity of the National Airspace System Performance to Weather Forecast Accuracy, " Integrated Communications, Navigation and Surveillance Conference (ICNS), Herndon, VA, May, 2008. George Hunter, Kris Ramamoorthy, "Integration of terminal area probabilistic meteorological forecasts in NAS-wide traffic flow management decision making, " 13 th Conference on Aviation, Range and Aerospace Meteorology, New Orleans, LA, January, 2008. Kris Ramamoorthy, George Hunter, "The Integration of Meteorological Data in Air Traffic Management: Requirements and Sensitivities, " 46 th AIAA Aerospace Sciences Meeting and Exhibit, Reno, NV, January, 2008. George Hunter, Ben Boisvert, Kris Ramamoorthy, "Advanced Traffic Flow Management Experiments for National Airspace Performance Improvement, " 2007 Winter Simulation Conference, Washington, DC, December, 2007. Kris Ramamoorthy, George Hunter, "Evaluation of National Airspace System Performance Improvement With Four Dimensional Trajectories, " AIAA Digital Avionics Systems Conference (DASC), Dallas, TX, October, 2007. Kris Ramamoorthy, Ben Boisvert, George Hunter, "Sensitivity of Advanced Traffic Flow Management to Different Weather Scenarios, " Integrated Communications, Navigation and Surveillance Conference (ICNS), Herndon, VA, May, 2007. George Hunter, Ben Boisvert, Kris Ramamoorthy, "Use of automated aviation weather forecasts in future NAS, " The 87 th American Meteorological Society Annual Meeting, San Antonio, TX, January, 2007. Kris Ramamoorthy, George Hunter, "Probabilistic Traffic Flow Management in the Presence of Inclement Weather and Other System Uncertainties, " INFORMS Annual Meeting, Pittsburgh, PA, November, 2006. Kris Ramamoorthy, Ben Boisvert, George Hunter, "A Real-Time Probabilistic TFM Evaluation Tool, " AIAA Digital Avionics Systems Conference (DASC), Portland, OR, October, 2006. George Hunter, Kris Ramamoorthy, Alexander Klein "Modeling and Performance of NAS in Inclement Weather, " AIAA Aviation Technology, Integration and Operations (ATIO) Forum, Wichita, KS, September 2006. Kris Ramamoorthy, George Hunter, "A Trajectory-Based Probabilistic TFM Evaluation Tool and Experiment, " Integrated Communications, Navigation and Surveillance Conference (ICNS), Baltimore, MD, May, 2006. Kris Ramamoorthy, George Hunter, "Avionics and National Airspace Architecture Strategies for Future Demand Scenarios in Inclement Weather, " AIAA Digital Avionics Systems Conference (DASC), Crystal City, VA, October, 2005. George Hunter, Kris Ramamoorthy, Joe Post, "Evaluation of the Future National Airspace System in Heavy Weather, " AIAA Aviation Technology, Integration and Operations (ATIO) Forum, Arlington, VA, September 2005. James D. Phillips, “An Accurate and Flexible Trajectory Analysis, ” World Aviation Congress (SAE Paper 975599), Anaheim, CA, October 13 -16, 1997. 37

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Questions? 38

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Backup 39

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 PNP Systems Requirements • System requirements – PNP is a Java application – Hardware • Memory: minimum 1 GB, preferred 2 GB • CPU: Pentium (4) 3. 2 GHz or better • Video card: 128 MB memory, preferred 256 MB – Software • Java JDK 6 http: //java. sun. com/javase/downloads/index. jsp • My. SQL Server 5. 0 http: //dev. mysql. com – Third party licenses • Eurocontrol BADA usage license 40

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Weather Days • Ten weather days, two control days 41

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Weather Days • Weather days – Spectrum of weather days • Variation in weather type and intensity • Variation in season – Support real-world comparison • Support same sector data • Variation in traffic demand volume and structure § Different days of week, holidays • Control days 42

43 1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Next. Gen Performance Sensitivity Analysis 44

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 En Route and Terminal Area Combined Sensitivities - 2025 45

46 1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Benefit of Improved Convection Forecasts 47

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Investment Analysis 48

49 1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Benefit of Using Clear Weather Forecasts 50

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Benefit Evaluation Case 2: No distinction between clear and heavy weather forecast accuracy Case 1: Take advantage of improved forecast accuracy in clear weather Persistence forecast, 11/16/06 51

52 1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Market-Based TFM: Valuation of NAS Access 53

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Congestion-Delay Relationship • Unconstrained sector congestion cost (SCC) for zero lookahead time (blue) and PNP-Prob. TFM simulated delay (black) time histories for all en route NAS sectors and flights, respectively. Delay SCC 54

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Aggregate Delay Model • Hypothesize a first-order lag transfer function Simulated delay Modeled delay 55

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Aggregate Delay Model • Hypothesize a second-order transfer function Simulated delay Modeled delay 56

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Transfer Functions Summary 57

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Explicit Cost Model • Evaluate cost of NAS access by removing the flight • Remove one flight – 11/16/06, UAL 233, A 320 – Morning departure from Bradley International (KBDL) to Chicago O’Hare airport (KORD) – Relatively high cost flight • 90. 02 SCC 58

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Remove UAL 233 • Delay reduction by time bin in simulation run – Delay reduction of 8141 minutes 59

60 1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 NAS Performance Sensitivity Studies • Performance sensitivity to: • Delay distribution policy (most important factor) • TFM system agility • System forecasts (least important factor) Nov 12, 2006 ETMS/ASPM Non Agile Delay Distribution Non Minim Agile um D Mini elay mum Dela y Delay Distribution 61

62 1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Dynamic Airspace Configuration 63

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 NAS Sectorization • Nov 12, 2006 64

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Mx. DAC Afternoon Sectorization • Nov 12, 2006, LAT=6, #Gen=20 65

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Coeff_peak_ac_var=0. 0 Coeff_avg_ac_var=0. 0 Coeff_crossings=0. 0 Coeff_transition_time=0. 0 Coeff_residual_capacity=1. 0 Mx. DAC Midday Sectorization • Nov 12, 2006, LAT=2, #Gen=40 66

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Delay-Congestion Performance Mx. DAC on, LAT = 4 hrs Mx. DAC on, LAT = 2 hrs Mx. DAC off 67

68 1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Equity Analysis: Cost of Delay Distribution 69

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Cost of Distributing Delay • RMS delay can be reduced by spreading delay to more flights – But at the cost of increased total delay Nov 12, 2006 $65/minute Increased delay distribution 70

71 1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 AOC Dispatch Use Case 72

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Dispatcher Successfully Finds a Reroute with low probability of delay 73

74 1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Project Monitoring and Control • JIRA is used to track issues – Project Manager and Lead Software Engineer assign task priorities, due dates, and personnel. • Weekly telecoms keep distributed team apprised of PNP and communications open • Project Manager maintains a master schedule in MSProject 75

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Development Tracking • Software engineers use JIRA to track and status development efforts. 76

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Branch Configuration Management • Software Engineers are responsible for creating branches from the trunk to develop fixes/enhancements. • The Configuration Management of the software is accomplished with Subversion – Subversion is an open source version control system (http: //subversion. tigris. org/) 77

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Unit and System Testing • Software Engineers are responsible for creating unit tests to verify the correctness of their code. The JIRA issue number is to be used throughout the code and unit tests for tracking purposes. • Software Engineers are responsible for running their own system/function tests to verify their software. • Once testing is validated, code is merged back on to the trunk. 78

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Trunk Configuration Management • Once all validated JIRA issues are merged unto the trunk, regression testing is performed. 79

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Regression Testing • Regression testing • Aggregate results – – – Total delay Total congestion Traffic volume #TFM initiatives Runtime • Different scenarios – Truncated demand set – Full demand set – Weather • Automated – Weekly or as required • Archived • Graphical quick-look 80

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Quantitative Project Management • Regression testing validation is performed and the release letter is updated. • Release is tagged in Subversion. • JIRA issues are closed. • Documentation is updated to reflect changes in software. 81

1 st Annual Workshop NAS-Wide Simulation in Support of Next. Gen, 12/10/08 Risk Management • Lessons learned analysis – A wrap up meeting is held to discuss all issues on a project in which proactive steps can be documented to avoid the same mistakes 82