Microgravity Program Support by E TS and OSAT Surveillance

Microgravity Program Support by E&TS and OSAT Surveillance Process for MRDOC Presentation to Project Managers’ Working Group August 7, 2000 Dennis Rohn (E&TS) and Gary Kelm (OSAT) Page 1

Microgravity Program Support by E&TS and OSAT Agenda • Introduction • MSD Contract Management Approach • Surveillance Organization and Roles • Proposed Surveillance Process • Surveillance Classification Guidelines • OSAT/E&TS Organization and Estimated Staffing • Feedback Page 2

Microgravity Program Support by E&TS and OSAT Introduction • MRDOC is a Performance Based Contract, but the government is still responsible for ensuring that contract and delivery order requirements are met. • The process of doing this is called “surveillance”, which involves monitoring and communicating with the contractor to assure requirements are properly interpreted and verifying they are satisfied. • A draft Surveillance Plan is currently part of MRDOC, but a more specific “Implementation Plan” was needed. • A team comprising Tom St. Onge, Joyce Wanhainen (MRDOC SEB Chair), Dennis Rohn and Gary Kelm developed a proposed surveillance process. • The team presented this process to Steve Simons (MRDOC COTR) and is now presenting it to the MSD Project Managers for feedback. Page 3

Microgravity Program Support by E&TS and OSAT MSD Contract Management Approach Microgravity Science Program (Directorate Level) • Workforce Planning • Make-Buy Decisions • Inter-directorate Issue Resolution MSD Management Team (MMT) • • • Information Exchange Personnel Actions Programmatic Strategy Make-Buy Recommendations ISO Management Issue Resolution MSD Change Board (MCB) • • • Approve Contract Changes Define DO Allocations Approve Delivery Orders Approve Fee Requests Issue Technical Direction Approve Deliverables MSD Surveillance Board (MSB) Project Surveillance Teams • • • Generate/Closeout SRFAs Evaluate Deliverables Interpret Requirements Identify Risk Areas Support ERB/FRB Attend Maj Milestone Rev Page 4 • • SRFA Review & Closeouts Identify Trends/Estab Metrics Initiate Special Studies Monitor/Assess System Reqts & Program Risk • Recommend Actions to MMT • Support Project Surveil Teams

Microgravity Program Support by E&TS and OSAT Surveillance Board • Participants: – – – • • • COTR or Designee Exhibit Representatives E&TS System Engineer Representative OSAT Representative Business Management Representative Secretary (1 or 2) (5) (1) (1) Caucuses monthly (or as required). Evaluates problems, issues and concerns (e. g. , Surveillance RFAs); requests additional surveillance activities, as needed. Reviews contractor metrics, trends and performance indicators and takes corrective or preventative action as required to reduce risk or bring contract into compliance. Page 5

Microgravity Program Support by E&TS and OSAT Surveillance Teams Primary Goal: Ensure what the contractor proposes and builds meets contract requirements and customer needs. Teams are led by Project Managers. Team members support the PM and need to: • • • Become familiar with MRDOC, including SOWs, Data Item Deliverables and technical requirements (i. e. , science, safety, integration). Become thoroughly familiar with the MRDOC Exhibits and/or Delivery Orders (DOs) to be monitored. Work closely with the Project Manager. – – – • Work closely with the contractor team. – – – • Create mutual understanding and agreement on what is needed and how requirements should be interpreted. Conduct independent analyses, as needed or directed, to assure validity of contract deliverables. Identify areas of risk or concern and report to the Project Manager. Conduct audits or other assessments as directed by the Project Manager or Surveillance Board. Evaluate contractor performance against established metrics. Suggest new metrics that might promote contractor performance. Help contractor interpret and understand requirements. Offer suggestions, where appropriate, to help contractor understand requirements. Serve as a consultant: propose potential options or alternatives for contractor consideration, without explicitly telling the contractor how to do the job. Participate in contract reviews, meetings, telecons, etc. Monitor or witness contractor fabrication, assembly, test or other activities to assess progress and problems. Utilize personal expertise or recruit other NASA personnel with relevant skills and experience to evaluate/approve DIDs, investigate problems, etc. Page 6

Microgravity Program Support by E&TS and OSAT Surveillance Team Roles • What are our roles? – Extremely important! – Independent Verification and Validation – Requirements Interpretation (We are the Customer) – Providing Added-Value to the Contractor (they would like to know if they may not meet a requirement as early as possible to avoid loss of P-R-O-F-I-T) • Do’s – Attend Contractor meetings (you have the right!) that pertain to your area – Provide “advice”, add-value to their process (“consultant” model – they don’t have to follow our advice) – Report to Project Manager on any identified problems (generate Surveillance RFAs, when necessary) – Perform analysis and review contractor analysis for independent validation • Don’ts – Argue with the contractor – Provide technical direction (we have the right to provide Technical Direction, but this needs to be done through MRDOC Change Board and could involve $$$) – Perform contractor’s work – they are responsible – could be a problem later! Page 7

Microgravity Program Support by E&TS and OSAT Surveillance is as Easy as ABC 2) The builder formally nominates options, also known as design decision options, that they are willing to stand behind The associate (or surveillance team member) works here to make sure that the options offered up will meet the needs of the customer. Customer Role Works for 3) The customer confirms the precoordinated design decision Associate Role Works with Builder Role Informally operated option generation and assessment {formally organized by the builder role] In which any and all may participate 1) The customer invites the builder to nominate options All ideas only enter here - never from above The three meeting purpose dimensions: (1) To generate and assess design options [formally organized but informally operated--no boss, tables-of-three, efforts]; (2); To present the results [by information briefings, lectures, conferences, and etc. ] of the formally organized but informally operated option generation and assessment efforts [that include formally organized but informally operated "independent" assessments and reviews]; and (3) To confirm pre-published results and planned decisions Page 8

Microgravity Program Support by E&TS and OSAT MSD Surveillance Process Project Team Project Issue Lead Project Contractor Microgravity Change Board (MCB) Microgravity Surveillance Board (MSB) GRC MSD Management Team (MMT) START Issue identified by team member Make Issue Known to contractor PM reviews for clarity return to originator for update and distribution Clarification provided, issue goes away Draft issue on RFA format, forward to PM for review Update RFA and submit original to PM. Copies to PAM, CSE, Functional Mgr, contractor PM, MSB Review Monthly Issue reports for trends Cognizant of issues through monthly program discipline reviews Initiates investigation of issue and assesses risk (comparison to risk database). Reports status at monthly reviews. Review contractor position and PM or designee make recommendation to issue. Continue discussions with contractor to try to resolve. A Define contractor position and make recommendation to issue Page 9

Microgravity Program Support by E&TS and OSAT MSD Surveillance Process Project Team Project Issue Lead Project Contractor Microgravity Change Board (MCB) A YES Agreeable solution to all parties, resolution reached NO PM requests inhouse alternative study, prepare results Supports in-house study. PM requests ERB/FRB to assess options and recommend approach ERB/FRB reports recommendation from review. B Page 10 Microgravity Surveillance Board (MSB) GRC MSD Management Team (MMT)

Microgravity Program Support by E&TS and OSAT MSD Surveillance Process Surveillance Project Contractor Microgravity Change Microgravity Project Team Project Issue Lead Board (MCB) Board (MSB) GRC MSD Management Team (MMT) B YES Is the Issue dismissed with no further action NO Is there a minority view, decision outside scope of project, increased risk, or resource constraints? Resolved by MMT YES NO Is there a cost, schedule, or performance impact requiring a contract mod. , wavier, or task close-out? YES Submit RFA form to MCB for resolution Resolved by MCB. NO PM documents agreement on RFA and distributes copies to CAM, CSE, originator, functional Mangers, contractor PM, MSB Updates risk database as needed. END Review closeout reports. Page 11

Microgravity Program Support by E&TS and OSAT SRFA Form Page 12

Microgravity Program Support by E&TS and OSAT SRFA Form Page 13

Microgravity Program Support by E&TS and OSAT Surveillance Activities Classification Guidelines • Class 1 (typical for FCF): >5 FTE E&TS/OSAT Support – – – • Class 2 (typical for Shuttle, Single-PI Experiments, Multi-PI Experiments, Exhibit 2, SAMS, Sounding Rockets): 2 -5 FTE E&TS/OSAT Support – – – • Engineering review of contractor deliverables Independent engineering analysis and assessments System requirements assessments Support PM/contractor TIMs Major review support Change board support Engineering review of contractor deliverables System requirements assessments Support PM/contractor TIMs Major review support Change board support Class 3 (typical for reflights, Gloveboxes, and PI Experiments for Multi-PI Experimental Apparatus): <1 FTE E&TS/OSAT Support – System requirements assessments – Major review support – Change board support Page 14

Microgravity Program Support by E&TS and OSAT MSD EDAD/SED Page 15

Microgravity Program Support by E&TS and OSAT Staffing Page 16

Microgravity Program Support by E&TS and OSAT EDAD/SED Organization Supporting Surveillance MSD FCF Projects OSAT Page 17 One person from the team will be assigned to provide primary surveillance, and coordinate team surveillance activities for each project

Microgravity Program Support by E&TS and OSAT EDAD/SED Surveillance Staffing • Level of effort dictated by Make/Buy Committee at about 21 -22 Civil Servants for FY 01 Page 18

Microgravity Program Support by E&TS and OSAT EDAD/SED Work Authorization • • • BMS requirement is to have an Engineering Request EDAD/SED responds with two additional forms that have to be filled out, one of which captures the monthly budget estimate. Proposal is to have a single Engineering Request (see form) to cover all surveillance needs – one JO, one set of BMS forms A Surveillance Engineering Support Request (see form) form has been developed to define what support is requested and when, for each project, including any special tasks Information for the form is developed between the Project Manager/Requester and the Engineering Surveillance Lead (or Chief Systems Engineer – Microgravity) Page 19