Registration Management Committee RMC AQMS Accreditation Programs Lessons

Registration Management Committee (RMC) AQMS Accreditation Programs Lessons Learned Company Confidential Auditor Workshop Atlanta, GA July 22 -23, 2010 1

Registration Management Committee (RMC) Objectives • Provide an overview of the NCR’s identified from the witness audits. • Present in process approach. • Discuss what to expect when witnessing Atlanta, GA July 22 -23, 2010 2

Registration Management Committee (RMC) Pre-Audit Assignment • PREVIOUS NCRS: – Assigning Auditors who either did not have the proper NACE code or AS 91 XX qualifications or potential conflicts (such as ITAR). – Client not made aware of the OP assessor. • AWARENESS: – Verify auditors are qualified for the full dynamics of the audit. – Communicate clearly with the client. – Applicable to both witness audits and office assessments Atlanta, GA July 22 -23, 2010 3

Registration Management Committee (RMC) Pre-Audit Planning • PREVIOUS NCRS: 19011, 6. 4. 1. c – (4) The audit plan did not ID the organizations functional units and processes to be audited. • AWARENESS: – The audit plan ensure the client will have the proper resources available to the audit. – Clearly demonstrates what processes are intended to be audited. – Applicable to both witness audits and office assessments Atlanta, GA July 22 -23, 2010 4

Registration Management Committee (RMC) Pre-Audit Planning • PREVIOUS NCRS: 17021. 9. 3. 2. 1 – (2) Audit Planning not effective in assuring the AQMS is assessed to the min requirements for surveillance audits. • AWARENESS: – Ensure the auditors understand where the full auditing requirements are for surv audits if not clearly ID in the audit plan. – Applicable to both witness audits and office assessments Atlanta, GA July 22 -23, 2010 5

Registration Management Committee (RMC) Pre-Audit Planning • PREVIOUS NCRS: MD 5 and AS 9104 – (5) Insufficient Auditor Days in the planning without clear justification. • AWARENESS: – If deviation…. . JUSTIFY. – If the audit is being witnessed, provide to ANAB in the pre-audit information. – Applicable to both witness audits and office assessments Atlanta, GA July 22 -23, 2010 6

Registration Management Committee (RMC) On-site Audit Depth of auditing • PREVIOUS NCRS: – Sampling part of an element and making a conclusion on the whole requirement. • AWARENESS: – Audit plan and report accuracy is critical. Atlanta, GA July 22 -23, 2010 7

Registration Management Committee (RMC) On-site Audit Depth of auditing • PREVIOUS NCRS: – Auditing to the scope of certification is not adequate. • AWARENESS: – Ensuring the scope of certification, scope of audit and audit plan is consistent. – Validate the exclusions. Is scope consistent with the QM? – Ensure the address and scope on the Certificate is correct. Atlanta, GA July 22 -23, 2010 8

Registration Management Committee (RMC) On-site Audit Depth of auditing • PREVIOUS NCRS: – Not following audit trails when objective evidence suggest otherwise. • AWARENESS: – A plan is a plan. The CB should follow the trail to its natural conclusion if potential findings are evident. – The CB should keep head on a swivel, don’t ignore clear findings. Atlanta, GA July 22 -23, 2010 9

Registration Management Committee (RMC) On-site Audit Depth of auditing • PREVIOUS NCRS: – Not fully verifying the effectiveness of the actions taken on nonconformities identified during the previous audit. • AWARENESS: – Ensure the evidence is more than a record review. Atlanta, GA July 22 -23, 2010 10

Registration Management Committee (RMC) On-site Audit Understanding • PREVIOUS NCRS: – Not fully auditing Outsourcing when applicable. • AWARENESS – Standard clearly requires “control of such outsourced processes shall be identified” and therefore subject to audit. Not just a document verification. Atlanta, GA July 22 -23, 2010 11

Registration Management Committee (RMC) On-site Audit Understanding • PREVIOUS NCRS: – Changes to the capability listing of the client. • AWARENESS – Similar to the scoping discussion. The CB should verify any changes to the capability listing as this could affect the scope of certification or introduce new processes and technology. Atlanta, GA July 22 -23, 2010 12

Registration Management Committee (RMC) On-site Audit Decisions • PREVIOUS NCRS: – Calling the Clients CA/PA system effective with ineffective/inadequate correction, root cause or corrective action. • AWARENESS: – The CB should deep dive the CA/PA system. Validate the information. If it doesn’t make sense…. the CB should document the NCR. Atlanta, GA July 22 -23, 2010 13

Registration Management Committee (RMC) On-site Audit Decisions • PREVIOUS NCRS: – Soft Grading – Clearly stating findings during the audit and not raising the NCR or improper categorization. – Accepting informal correction during the audit. • LEARNINGS – The CB should learn the definitions and follow them. Keep track of the “verbal” findings identified during the audit. Accepting correction to verbal findings without reporting is consulting. – As an auditor it is not their job to justify an NCR down. Atlanta, GA July 22 -23, 2010 14

Registration Management Committee (RMC) On-site Audit Decisions • PREVIOUS NCRS: – Continuing the audit when the audit objectives are clearly unattainable. • AWARENESS: – Only the client can make the decision to continue an audit. – There should be a clear conclusion that the objectives are unattainable and reported to the client with options. » At the time this is realized not at the end!! Atlanta, GA July 22 -23, 2010 15

Registration Management Committee (RMC) On-Site Audit Closing • PREVIOUS NCRS: – Closing meeting does not address all of the requirements. • AWARENESS – The CB should clearly address the CA follow up and impacts on the existing cert. – The CB should ensure the CA are presented to the correct requirements. Atlanta, GA July 22 -23, 2010 16

Registration Management Committee (RMC) On Site/Off-Site Audit Reporting • PREVIOUS NCRS: – Report not in conformance with AS 9014/AS 91 XX • AWARENESS – The report should include: » Justification / explanation all NA. » Clear evidence to support the conclusion. » Clear evidence to support the SCOPE. » Detailed notes – Applicable to both witness audits and office assessments Atlanta, GA July 22 -23, 2010 17

Registration Management Committee (RMC) Post Audit NCR Closure • PREVIOUS NCRS: Improper closing of NCR’s and Improper Certification Decision. • AWARENESS: – Clear evidence of Correction, RC and CA. – Ensure the NCR’s are closed with appropriate evidence PRIOR to the cert decision. Atlanta, GA July 22 -23, 2010 18

Registration Management Committee (RMC) Office Assessments OASIS • PREVIOUS NCRS: OASIS data base admin /accuracy • AWARENESS: – Must have the CB log in to see OASIS information – Verify » Accuracy (reports, scoring, certificates) » Timing Atlanta, GA July 22 -23, 2010 19

Registration Management Committee (RMC) Office Assessments Transfers • PREVIOUS NCRS: – Transfer requirements not met. • AWARENESS: – Specific requirements for pre-transfer review » Validating the certificate » Collecting audit records for last full system (initial or re-cert) and subsequent surveillances. » NCR’s should be closed (or closed by new CB before certification) – If pre-transfer review is unsuccessful full audit must be conducted. Atlanta, GA July 22 -23, 2010 20

Registration Management Committee (RMC) Office Assessments Decision Process Certification • PREVIOUS NCRS: – Improper qualification /competency of AS cert decision maker. • AWARENESS: – Cert decision maker must include someone with aerospace knowledge – Must be independent of the audit being reviewed (including independent of the corrective action reviews) Atlanta, GA July 22 -23, 2010 21

Registration Management Committee (RMC) Office Assessments Decision Process Certification • PREVIOUS NCRS: – Decision made with incomplete or inadequate certification information. • AWARENESS: – Records not available (report, CA’s, etc) – Records not complete or inaccurate Atlanta, GA July 22 -23, 2010 22

Registration Management Committee (RMC) Office Assessments Certification Documents (e. g. Certificate) • PREVIOUS NCRS: – Certificates are incorrect or inaccurate. • AWARENESS: – Scope statements not correct or misleading (not specific enough) or incorrect for multiple sites. – Addresses incorrect or does not include all addresses for multiple sites. – Text in regards to being accredited under the registration management program. – Unaccredited certificates not allowed Atlanta, GA July 22 -23, 2010 23

Registration Management Committee (RMC) Questions Atlanta, GA July 22 -23, 2010 24