The National Ignition Facility NIF Software Engineering Efforts

The National Ignition Facility (NIF) Software Engineering Efforts Presentation to SIR Computers and Technology Group March 17, 2016 Eric Stout NIF Controls Software Architect

The NIF mission is to study high-energy-density physics and fusion ignition • NIF is the world's largest and most energetic laser • NIF’s goal is to demonstrate nuclear fusion with energy gain for the first time in the laboratory NIF explores extreme conditions in a laboratory setting 100 million degrees K 100 billion times atmospheric pressure LLNL-PRES-657542 2

The mission of NIF is to produce high-energy-density conditions and, ultimately, fusion ignition High Energy Density Physics Fusion Ignition Basic Science • Helping ensure the nation's security without nuclear weapons testing • Research for a safe, virtually unlimited, carbon-free energy future • Achieving breakthroughs in a wide variety of scientific disciplines, including astrophysics, materials science, radioactive and hazardous waste treatment, particle physics, and x-ray and neutron science LLNL-PRES-657542 3

Inertial confinement fusion will be used to produce ignition and energy gain NIF’s 192 energetic laser beams will be used to compress very small deuterium-tritium fusion targets the size of a BB LLNL-PRES-657542 4

Target chamber hanging

LLNL-PRES-657542

The NIF is all about the data • The NIF has just one product – data — The NIF is a very dynamic environment and the management of data is essential in order to meet it’s scientific goals • There are several different categories of data used on the NIF — Setup data — Configuration data — Calibration data — Analysis data • A large number of tools have been implemented to handle this ever growing amount of data The purpose of the support tools is to facilitate taking high quality shots on the NIF LLNL-PRES-657542

Three dozen diagnostic systems were deployed by 2010 for the National Ignition Campaign SPIDER POLAR DIM Static x-ray Imager DANTE CRYO-Target Positioner Target Alignment Sensor Positioner MRS DIM Chamber Center Reference System Final Optics Damage Inspection Neutron Time of Flight 20 m Target Positioner Neutron Time of Flight 20 m Full Aperture Backscatter Station LLNL-PRES-657542 8

Data is at the heart of the NIF Input Output (NIF Archive) (Oracle DB) Campaign Management & Shot Setup Reports Shot Planning & Loop Analysis Configuration of Installed Parts Conduct Experiment Integrated Computer Control System (ICCS) Automated Analysis Automated Shot Data Analysis IDL Cluster Calibration Values Shot Data & Analysis Results Manual Analysis Downloads Visualization Laser Performance Operations Model The NIF archive is the foundation of the support tools LLNL-PRES-657542

The NIF project has five teams developing the support tools Shot Planning & Loop Analysis Tools Campaign Management Enterprise Configuration Management Data Analysis & Visualization Laser Performance Operations Model Lane Mgr Campaign Management Tool Work Permits Analysis Archive (CMS) Physics model interface Damage Calculator Target diagnostic Template Mgr Logs Analysis engine Optics Loop Viewers Approval Mgr Calibration Archive (CDMS) Visualization Data source management HIF Estimated Exposure Tool Configuration Checker Site Registry (Glovia) Archive Exporter Reports Facility Calendar Pulse Shape Editor Maintenance Tracking (SMART) Experimenter Website Verification Data Loader (ETL) Beam Pointing Assistant Radiological Tracking System Data Loader (ETL) Target Selection Manager Configurator Parts List Manager Document Control (ECMS) These toolsets constitute over 40 integrated applications LLNL-PRES-657542

A significant effort has been needed in order to implement the support tools Code count by support tool Languages used Laser Performance Operations Model; 287, 598 Shot Planning & Loop Tools; 181, 433 Data Analysis & Visualization; 656, 908 Campaign Management; 948, 211 IDL 5% C/C++ 6% Other 7% HTML 8% Java 50% SQL 11% Configuration Management; 2, 550, 690 XML 13% 4. 3 M lines of code in order to implement the current capabilities LLNL-PRES-657542

The CMT shot setup allows users to configure the Dante Diagnostic CMT allows the specification of over 18, 000 setup parameters of which over 500 are applicable to Dante LLNL-PRES-657542

The Lane Manager is used to schedule experiments based on Dante availability Lane Manager is the primary means of communicating the shot schedule LLNL-PRES-657542

The Shot Report page indicates the success of each diagnostic analysis Automated analysis supports over a dozen diagnostics LLNL-PRES-657542

Multiple views of the Dante data are visualized not just the final result The tools can visualize data across many shots and compare them to simulated data LLNL-PRES-657542

The Laser Performance Operations Model allows scientists to verify that the laser met it goals Key data items: Laser Use Laser Energy Laser Timing Laser Pulse Shape Laser Power Balance LPOM data is used in Dante analysis to adjust the timing of the results based upon the actual laser timing LLNL-PRES-657542

ICCS software provides software controls automation across all NIF laser subsystems Beam Control Master Oscillator Cryogenic Optics Inspection Amplifier & Power Conditioning Target Area Positioners Optical Switch (PEPC) LLNL-PRES-657542 Injection Laser Diagnostics 17

NIF is a 192 beam laser organized into “clusters”, “bundles”, and “quads” LLNL-PRES-657542

Requirements of the NIF Integrated Computer Control System (ICCS) • Core duties — Configure facility for an experiment — Execute experiment — Store experimental results • Implementation challenges — Control of 50, 000 device points — Beam alignment to <50 microns on target — Beam timing within <30 psec — Flexible automation with oversight by 14 operators — Shot-to-shot goal of <4 hours • Assure — Situational awareness — Machine protection A highly data-driven, modular, automated, and distributed controls architecture was needed for a system the size of NIF LLNL-PRES-657542 19

The Integrated Computer Control System (ICCS) robustly operates NIF • Highly data-driven — Device configuration — Experiment definitions — Experiment model — Experiment results NIF Control Room • Highly distributed — 35 Framework & Supervisory servers — 3 compute clusters (110 nodes) — 950 Front-End Processors — 950 embedded controllers — 2, 400 processes • Industry-standard software design — Object-oriented design patterns — CORBA as a communication middleware Automatic control of 192 -beam shots is overseen by 14 operator stations Early key design decisions have proven successful and are now the foundation for controlling the NIF LLNL-PRES-657542 20

ICCS consists of 3. 5 million lines of software running on 1, 900 distributed computers throughout NIF ICCS computer and control hardware LLNL-PRES-657542 21

A segmented and layered architecture implements ICCS control system Framework Services Oracle Database Experiment Automation System Software Distribution Bus (CORBA over network) Safety (Allen-Bradley, Rockwell) Utilities (Allen-Bradley, Rockwell) FEP (Vx. Works, Ada) FEP (Solaris, Ada) FEP (Windows, Ada/Java) PLC/IO Vacuum PLC Devices PLC/IO Cooling PLC Embedded Controller Argon PLC 16 k Points LLNL-PRES-657542 Graphical User Interfaces NIF, Target, and Bundle Supervisory Systems Local PC 50 k Points 22

Common frameworks in all processes simplify maintenance and operation of ICCS C T O C B Application Objects R P Connection Objects Object System Status Message Factory Manager Monitor Log Alerts Events Reservations Shot Machine Archive History Config/ Name Service Framework Agents A Startup/Shutdown Heartbeat • All ICCS processes have the same core behaviors • Consistency provides flexibility in software development staff assignments LLNL-PRES-657542 23

Object-oriented, data-driven design abstracted 50, 000 control points to 300 discrete control classes Motor Example Control Points Quantity Stepping motors 10, 500 Mirror surface actuators 7, 500 Binary actuators 2, 200 Precision timing triggers 1, 600 Cameras 1, 000 Photodiodes 900 Resulting reduction in code base has significantly improved the maintainability LLNL-PRES-657542 24

ICCS primary requirement is to automatically align, fire and diagnose laser shots with a goal of <4 hours Fire pre-amplifier main amplifiers S Amplifier cooling h o t Archive S h o t Verify Shot Preparation 30 -m 20 -m S h o t Archive 2 -s <4 hours Automated Shot Cycle • Acquire campaign shot goals from laser physics model • Perform automatic alignment and wavefront correction • Configure diagnostics and laser performance settings • Conduct countdown (software: 4 -min and timing: 2 -sec) • Assess shot outcome and archive shot data LLNL-PRES-657542 25

Automation software execute NIF experiments with data driven workflow and re-useable ‘macro steps’ Shot Director Layer Collaboration Layer Shot Cycle States Implement_Plan Laser Diagnostics Shot Model With Macro Step Sequences Laser Alignment LLNL-PRES-657542 Align Main Laser (Perform Phase) Align PABTS (step) Substeps Status & Control Layer Macro Step Phases Steps Shot Supervisor Layer Set Filter 30% Open Shutter Setpoints Goto 30% Transmission Goto Open Filter Shutter 26

Setpoints simplify verification and management of device control settings • A device setpoint consists of — A unique, device-specific name — One or more configuration settings • Setpoints can be — Pre-defined in the database — Created on demand • Experiment Automation System uses setpoints to configure, verify, and lock devices in “shot” position • Outside shots, operators can view, change, and command devices to known setpoints • Over 100, 000 device setpoint reconfigurations are performed on each shot cycle The setpoint abstraction provides a system-wide language for device state management LLNL-PRES-657542 27

A user interface framework ensures consistent presentation layer and simplifies development • Navigator — Starting point for operators — Provides database-driven “table of contents” for traversing GUI hierarchy • Broadviews — Scalable schematic diagrams for a subsystem — Live control and status updates for collections of devices • Control Panels — Specialized interface for a single device type — Detailed status and control • All user interfaces: — Update automatically when device state changes — Alert users of connectivity problems LLNL-PRES-657542 28

Drilldown Load Inventory Drilldown Database LLNL-PRES-657542 29

ICCS Cornerstone: Rigorous software engineering practices • Documented governance plan for Software Quality Assurance • IEEE standards used as best practice guidelines • Peer reviews for designs and code changes • Integrated product testing • Independent Verification and Validation team • Dedicated Configuration Management team The ICCS software engineering process is a key component of NIF equipment protection LLNL-PRES-657542 30

Rigorous quality controls at every development phase assure delivery of reliable software Development Team Quality Control Team Release Plan Software Development Operations Team Test Procedures Software Integration Acceptance Test NIF Install Off-line Test & Training On-line Test Commissioning Software Change Requests Software Change Control Board Stakeholder Steering Committee Work Authorization Point System Ready LLNL-PRES-657542 31

Incremental deployment strategy delivers functionality, while iteratively managing risk • Target high-level project milestones — Bi-monthly software releases (previously 3 -6 months) — Flexibility for interim software patches and data changes as required • Release content driven by — Enhanced capabilities and operational improvements — Defect corrections — Framework and code migration • Overlapped 2 -month development, test and deployment lifecycle — Stakeholders prioritize release content — SW Leads collect detailed requirements — Developers design, code, unit test and integrate — Independent test team performs offline and online qualification tests LLNL-PRES-657542 32

Software schedules are planned and tracked to ensure successful releases Release Scope Closure Rates Using historical rate projections to manage release content has greatly increased success in meeting deployment schedule LLNL-PRES-657542 33

Future work activities continue to focus on increasing controls availability to NIF operations • Availability — NIF operational support (20 -30%) — Laser preparation shots (concurrent shot cycles) — Maintenance operations automation (AMT & Loop 1 commissioning shots) — Reduce backlog of RAM software change requests — Predictive monitoring for off-normal system states • Maintainability — Continued multi-year code base migration to Java • Controls Enhancements — Beamline pulse delays up to 1 micro second (Long Delays) — Short Pulse backlighter capability commissioning (ARC) LLNL-PRES-657542 34

Technology refresh is an active effort • Software and hardware options are constantly improving — Industry trend is toward open source software • Migrating codebase from Ada to Java — Hardware platform independence — Large inventory of vendor and open-source libraries — Significantly better debugging, profiling, monitoring tools — Easier to find industry expertise when staffing • Migrating processes to Linux — x 86 commodity hardware — Virtual machines • Developing strategy for FEP technology refresh ICCS software architecture was designed for evolutionary change with minimal impact LLNL-PRES-657542 35

Port all ICCS software to a single language platform Now Future Accu. Rev, Apex Code Repository Accurev Code Repository Rational Ada, GNAT Ada Orb. Express, Java, Jacorb Operating Systems Technology Refresh Operating Systems Vx. Works, Linux, Solaris, Windows Platforms Sun, X 86, PPC, VME, field buses LLNL-PRES-657542 Linux, Windows, Virtual Machines x 86, VME, field buses 36

ICCS is successfully operating at full scale • A distributed architecture — Supports continual evolutionary change with minimal impacts • A data-driven architecture — Allows modification of run-time behavior without new software releases • Model-driven Experiment Automation System — Provides the flexibility and parallelism required to meet all NIF program missions • Actively leverage new technologies — Fewer and more mainstream tools — Runtime diagnostics for evaluating system performance A versatile, data-driven software architecture using client/server distribution will support NIF operation for decades LLNL-PRES-657542 37

38