Volpe Center CDM Data Quality Update A D Meeting

Volpe Center CDM Data Quality Update A&D Meeting 18 April 2001 Mike Golibersuch (Volpe/EG&G) AUTOMATION APPLICATIONS DIVISION 4/18/01 DTS - 56 1

Volpe Center Overview • Analysis of OAG-Only Time Out Cancellations (CDM Meeting Action Item) • • Status of Volpe Data Quality Web Site Proposed CDM Data Quality Metrics AUTOMATION APPLICATIONS DIVISION 4/18/01 DTS - 56 2

Volpe Center Analysis of OAG-Only Timeout Cancellations • Request for this analysis arose during discussion of Daily Download at CDM Meeting. • The Problem: some flights are created in ETMS from OAG data, but no other messages are received and the flight does not operate. Eventually the flight Time Out Cancels. • “Check and Correct” could reduce problem by detecting incorrect OAG-scheduled flights in ETMS database. • Airlines asked that Volpe quantify the problem, to provide an indicator of one type of benefit obtainable with “Check and Correct”. • OAG-Only TOCs were analyzed for the 60 -day period from 2/5/01 – 4/5/01 AUTOMATION APPLICATIONS DIVISION 4/18/01 DTS - 56 3

Volpe Center OAG-Only TOCs Per Week By Airline (2/5/01 – 4/5/01) ACA 3 xxx CDN (removed rule -- 4/11) Strike – 3/27 USA 49 xx PDC (added rule -- 4/11) AUTOMATION APPLICATIONS DIVISION 4/18/01 DTS - 56 4

Volpe Center Data Quality Web Site – OAG-TOC Report AUTOMATION APPLICATIONS DIVISION 4/18/01 DTS - 56 5

Volpe Center Frequency Distribution of OAG-TOC Flights AUTOMATION APPLICATIONS DIVISION 4/18/01 DTS - 56 6

Volpe Center Distribution of OAG-Only TOCs AUTOMATION APPLICATIONS DIVISION 4/18/01 DTS - 56 7

Volpe Center Conclusions and Implications • Problems with OAG data are episodic. Continuous monitoring is needed. • “Check and Correct” capability could improve things, but benefit may be moderate. • Other potential benefits of “Check and Correct” are less easily quantified. AUTOMATION APPLICATIONS DIVISION 4/18/01 DTS - 56 8

Volpe Center Volpe Data Quality Web Site • • Provides access to detailed message history by flight. • Next Steps n Performance Issues and TOC Issues Are Being Addressed n Data Quality Metrics Will Be Added • • Two products available now to help find data problems: n OAG Time-outs: highlights OAG schedule errors. n GDP Time-outs: daily report of any flight in a GDP that time-out cancelled. Access to the Site n Web Site Address is 152. 122. 152. 33: 8000 n Those who want access must provide client IP address to Volpe n Volpe will provide User Name and Password Each Airline May View n Summary Data for All Other Airlines n Detailed Flight Data Only for Its Own Flights AUTOMATION APPLICATIONS DIVISION 4/18/01 DTS - 56 9

Volpe Center Data Quality Web Site -- GDP Time-outs AUTOMATION APPLICATIONS DIVISION 4/18/01 DTS - 56 10

Volpe Center Data Quality Web Site -- Flight History AUTOMATION APPLICATIONS DIVISION 4/18/01 DTS - 56 11

Volpe Center ETMS/CDM Data Quality Metrics • Several candidate CDM data quality metrics were discussed at the last A&D Meeting • Based on general consensus, 3 Metrics are proposed for initial use as measurements of CDM data quality • Each metric will be tracked and reported on the Volpe Data Quality Web Site • After the measures have been tracked for a period of time, standard benchmarks will be defined • Several other indicators will be tracked for analysis purposes, but at least initially will not be treated as formal data quality measures • Each airline will have access to summary data (averages, trends) for all airlines, but will have access to detailed data only for its own flights AUTOMATION APPLICATIONS DIVISION 4/18/01 DTS - 56 12

Volpe Center Data Quality Metrics 1) Timeout-Cancelled Flights As percent of: w All flights planned to operate (FS, FC, or FM) w All cancelled flights 2) Airline-Cancelled Flights (FX, SI) Flights that Operate As percent of: w All flights planned to operate w Airline-cancelled flights Reinstated flights (FX followed by FC) are OK, and are not counted 3) Unscheduled Flights (no OAG data) that Operate without a CDM Message As Percent of: w All flights that operate w Unscheduled flights that operate AUTOMATION APPLICATIONS DIVISION 4/18/01 DTS - 56 13

Volpe Center Metric #3. Unscheduled Flights that Operate without CDM Message AUTOMATION APPLICATIONS DIVISION 4/18/01 DTS - 56 14

Volpe Center How Will the Data Be Displayed • Bar Charts comparing average values by airline over a userspecified time interval • Trend Line showing weekly value over 4 -month timespan for a selected airline • Airline may request a list of its flights that match the criteria during a specified time interval • Airline may select a flight from the list to display flight details and message history AUTOMATION APPLICATIONS DIVISION 4/18/01 DTS - 56 15

Volpe Center Other Indicators 4) Scheduled flights that operate without a CDM message • • As Percent of all scheduled flight that operate Useful for detecting problems with daily download 5) Flights that depart at least 15, 30, 45, or 60 minutes after EDT* • As Percent of: – All flights that operate – All delayed flights (I. e. , actual departure after OAG schedule) 6) Controlled flights that depart at least 15, 30, 45, or 60 minutes after EDT* • As Percent of controlled flights that operate ¬ For measurement purposes, EDT is determined as follows: • • Use ETMS EDT 10 minutes prior to actual departure. I. e. , for measuring prediction accuracy, disregard last minute updates From this EDT, remove any Time Out Delay that was automatically added by ETMS AUTOMATION APPLICATIONS DIVISION 4/18/01 DTS - 56 16