Development of an Industry Standardized Web-based Surveillance

Development of an Industry Standardized Web-based Surveillance & Auditing Tool (Web. SAT) to minimize maintenance Errors Prior to Dispatch by Airlines Department of Industrial Engineering Clemson University FAA Program Manager: Dr. William “Kip” Krebs Industry Partner: Fed. Ex Clemson Web. SAT Team: Dr. Anand K. Gramopadhye Dr. Joel S. Greenstein Pallavi Dharwada Kunal Kapoor Nikhil Iyengar

Outline http: //www. hf. faa. gov/docs/508/docs/maint. Tool 051904. ppt • Research Requirement and Execution Plan • Premise • Project Background • First Year Objectives • Methods • Results and Deliverables • Problems Encountered and Best Accomplishment • Expected Conclusion • Dissemination to-date

Aviation Maintenance TCRG Requirement • • • Develop existing PC /paper based auditing/surveillance tool to web-based application in performing auditing/surveillance/monitoring and validation of oversight of maintenance to ensure a consistent level of oversight is maintained. This system can proactively identify contributing factors of improper maintenance before aircraft is dispatched once work is complete. In addition, portions of this web-based surveillance auditing tool can be used by aircraft manufacturers before delivery of aircraft to their customers.

Aviation Maintenance TCRG Execution Plan Execution plan states: (1) identify an exhaustive list of impact variables that affect aviation safety and transcend across various aircraft maintenance organizations; (2) develop data collection/reduction and analysis protocol to analyze errors for the identified set of impact variables; and (3) using the results of the aforementioned activity develop and implement a surveillance/monitoring tool to ensure all appropriate data is used to identify safety improvements in the maintenance environment. FY 03 $64. 4 K FY 04 $142. 3 K FY 05 $163. 7 K FY 06 $129. 4 K

Premise (1 of 3) • Maintenance Error is a crucial factor in aircraft accidents • In a recent study: – 23% of accidents studied involved incorrect removal or installation of components – 28% involved a manufacturer or vendor maintenance/inspection error – 49% involved error due to an airline’s maintenance policy – 49% involved poor design that led to maintenance errors Source: Rankin, W. , Hibit, R. , Allen, J. , Sargent, R. , Development and evaluation of the maintenance error decision aid (MEDA) process, International Journal of Industrial Ergonomics, 26, 2000

Premise (2 of 3) • Maintenance accidents are analyzed after they have occurred – HFACS • • There is opportunity for improvement of maintenance procedures Lack of standardization of data collection American Airlines Flight 1420 (June 1, 1999)

Premise (3 of 3) • Analysis of aircraft maintenance – Closely monitoring and evaluating aircraft maintenance and inspection activities • Surveillance and auditing activities – Level of compliance – Objectives – Findings – Importance

Web. SAT – Project Background • Addresses the need for a system that ensures quality performance of surveillance and auditing activities • Identifies proactively the various factors that contribute to improper maintenance • Develops and promotes a standardized format within and across all airlines for – Data collection – Data reduction – Data analysis

First Year Objectives (09/03 to 09/04) • Data collection • Identify customer needs • Identification of relevant process measures (previously referred to as impact variables) • Submit quarterly reports periodically • Submit annual report

Methods (1 of 2) • Task analytic & user centered software lifecycle methodology guides through the following Phases I: Identifying process measures II: Design and develop Web. SAT prototype III: Develop data analysis module

Methods (2 of 2) • Phase I: Identifying process measures – Discover • Understand surveillance and auditing functions • Conduct interviews, focus groups and observations – Identify • Determine process measures – Validate • Ensure measures are representative of those used across maintenance facilities • Survey other stakeholders to obtain consensus on process measures – Construct • Identify Web. SAT modules • Develop module goals

Results and Deliverables (1 of 5) • Discover: – Three Interview visits with the Quality Assurance departments at Fed. Ex in Memphis, TN. – Two Observation visits at Mobile Aerospace Engineering in Mobile, AL. – Three trip reports documenting the information gathered. • Identify: – Translated voice of the user to user need statement – Developing metrics corresponding to user needs – Defining process measures with Fed. Ex Quality Assurance departments. • Validate: – Developing web-based survey to validate process measures with other stakeholders

Results and Deliverables (2 of 5) • Results: – Developed preliminary version of Web. SAT framework: Tier-1 Tier-2 Tier-3

Results and Deliverables (3 of 5) • Results: Identified probable process measures Surveillance: (i) In-process Surveillance (ii) Verification Surveillance (iii) Safety/Fire protection (iv) Shelf Life Auditing: (i) Administration (ii) Safety (iii) Manuals (iv) Records Airworthiness Directives Control Group: (i) Compliance (ii) Publication Changes (iii) Verification (iv) Applicability (v) Authorization (vi) EO/WIC Readability Currently in the process of developing a consensus on these aforementioned variables

Results and Deliverables (4 of 5) • • Deliverables: – Two Quarterly reports – December’ 03 and March’ 04 – Three Trip Reports – October’ 03, January’ 04 and March’ 04 – Four conference papers submitted to IERC, SAHI and HFES presenting the ongoing research initiative – Two research posters displaying the Web. SAT research presented at Clemson University’s Research Forum poster competitions Work-in-progress: – Web-based survey to validate process measures – Identify Web. SAT modules – Visit Fed. Ex headquarters at Memphis, TN – Visit Fed. Ex maintenance facility, Greensboro, NC

Results and Deliverables (5 of 5) Screen Shot of Web-based survey tool which is in process of development

Problems Encountered & Best Accomplishments • • Problems – Developing and sustaining interests of other airlines – Agreeing upon terminology Accomplishments – Identified probable process measures – Translated customers’ voice to need statements

Expected Conclusion • Web. SAT is expected to achieve the following: – Support of maintenance related surveillance, auditing and airworthiness directive activities – Identification of maintenance related process measures and related safety indices – Identification of potential problems at geographically dispersed maintenance sites through data standardization – Accessible display of trends in data

Backup Slides

Dissemination to-date • Dharwada, P. , Iyengar, N. , Kapoor, K. , Gramopadhye, A. K. , and Greenstein, J. S. , Web-Based Surveillance and Auditing Tool (Web. SAT): A Proactive System to Capture Maintenance Errors, Proceedings of Safety Across High-Consequence Industries, St. Louis, Missouri, 2004. • Iyengar, N. , Kapoor, K. , Greenstein, J. S. , and Gramopadhye, A. K. , Selection of Data Gathering Methodologies for the Aviation Maintenance Industry, Proceedings of the Industrial Engineering Research Conference, Houston, 2004. • Kapoor, K. , Dharwada, P. , Iyengar, N. , Greenstein, J. S. , and Gramopadhye, A. K. , Standardized Auditing and Surveillance of the Aircraft Maintenance Operations, Proceedings of the Industrial Engineering Research Conference, Houston, 2004. • Kapoor, K. , Dharwada, P. , Iyengar, N. , Greenstein, J. S. , and Gramopadhye, A. K. , A Strategy for the Development of a Web-based Tool to Reduce Aviation Maintenance Errors, Proceedings of the Human Factors and Ergonomics Society, New Orleans, 2004.