Machine Checkout and Setup Periods Jan Uythoven Thanks

Machine Checkout and Setup Periods Jan Uythoven Thanks to: O. Aberle, R. Bailey, F. Bordry, O. Brunner, L. Bottura, E. Carlier, P. Charrue, E. Ciapala, S. Claudet, R. Denz, D. Forkel. Wirth, B. Goddard, M. Jimenez, R. Jones, M. Lamont, V. Mertens, P. Ninin, B. Puccio, S. Ramberger, F. Rodriguez Mateos, R. Schmidt, L. Serio, B. Todd, M. Zerlauth 1

Machine Checkout: WHEN ? u Year u u 1: Following Hardware Commissioning Assume all equipment tests are done in the past 18 months Most of the system tests are done in the past 18 months u Subsequent u Assume most equipment tests are done u Machine u u years: Following Shutdown Work Checkout First time machine complete cold and closed All systems running together from the control room u Before First Beam Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 2

Tasks - Summary u Redo a subset of tests performed during the Hardware Commissioning period u u u New Tests u u u Because some tests more than a year ago Because the final configuration of the system might be different, especially the interface to other equipment If hardware commissioning period was squeezed in time, this will have been at the cost of the system tests As a result: will need more time for the Machine Checkout Additional system tests All systems together – machine wide, interdepedencies Machine completely closed, last access / final checks Under the responsibility of the Operations Group Green light from DSO, info for INB Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 3

Time Phases of Machine Checkout Some phases Hardware Commissioning Sectors 1 -2 and 2 -3 Cooldown of all sectors Close machine ELQA tests completely; Machine Checkout phase I Equipment tests Contr. Access per 2 sectors: Interlocks, Power Converters RF, kickers, BDI, . . Machine Checkout phase III Jan Uythoven, AB/BT in parallel Chamonix@CERN 2005, Machine Checkout Complete Machine Cycle from CCC DSO tests First beam 4

Phase I u Infrastructure for the complete machine u u Water, Ventilation, Power, Controls Network Radiation Protection (RAMSES) u Check response ionisation chambers u Interface with other systems UPS, AUG When full inventory u Cooldown of the complete machine of He available? u Sectors T up to 200 K: need 10 days for cooldown u If T > 80 K: need to redo Electrical Quality Assurance tests (ELQA), first time 3 weeks u If T < 100 K: cooldown needs about 5 days u Up to about 5 weeks for sectors warmed up > 80 K. Need to start cooldown in parallel with Hardware Commissioning. Manpower. Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 5

Phase I, continued Access System u Close the complete machine u u u New tasks – before only done per sector (during HC) u u u Test of LHC Access Control System (LACS) Test of LHC Access Safety System (LASS) New tasks: interface adjacent sectors, machine wide u Important Safety Element in point X must interface over the global machine and beam dump in point 6 Interface with Radiation Protection equipment: access veto Interface with HR data base, biometrical data Some equipment was not in final state during Hardware Commissioning tests Exercise procedures for ‘patrouille’, giving access, back from access, … Start first with sectors needed for Machine Checkout equipment tests phase II u If necessary, time to sort out problems parallel to phase II programme Ideally only controlled access from phase I onwards Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 6

Phase II u Per u u two sectors Check interlocks: PIC, WIC, QPS u See Test all power converters, two sectors at a time u See u Simultaneously Determines the pace of tests in Phase II in other sectors u Other equipment checks, see Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 7

Phase II Interlock Systems u u Before power converters can be tested, test interlock system Quench Protection System (QPS) u u Powering Interlock Controller (PIC) u u Visual check of equipment, half day per sector Automatic test procedure, 8 hours (same as monthly procedure): fires the quench heaters without current in magnets Test interface to BIS -> check in phase III Takes information from the QPS Automated test procedures and visual control Needs about 8 hours for testing Warm Interlock Controller (WIC) u u No automated procedures foreseen Some manual testing and visual control: 8 hours Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 8

Phase II Power Converters u Followed by testing of Power Converters two sectors at a time u u Run all converters to full power, perform tests at full power Calibration already done during Hardware Commissioning 12 h continuous running at full power Need about half a week per two sectors u Proceed Jan Uythoven, AB/BT around the ring Chamonix@CERN 2005, Machine Checkout 9

Phase II Control System u Control u u System Communication Mechanisms Field buses operational machine wide. All elements to be checked: need about 8 days Already start during phase I u Front. Ends, u Should have been made available during the Hardware Commissioning period sector by sector and left running u Timing u generation and distribution 6 - 8 days for the first year u Databases u Back. Ends and general services access 1 day, at least Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 10

Phase II Control System, ctd. u Application u u u software Tested continuously during the last months / years Specific projects need testing u Interlocks, realtime feedback, logging, postmortem, safe beam flag etc. u Estimate 10 days for test …. . ? ? ? Final tests when complete machine available: See phase III I): d BD an , OP NS CO nce any ( DRY RU m perie m ex ge fro ry runs & uoting TI 8 a Q Mess UNS, d R DRY Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 11

Other Equipment tests: RF u u Basic work of testing cavities, cryo, LLRF and control during hardware commissioning period Number of tests which would clearly accelerate RF operations and improve performance: Ed’s Wish List - Reliability of RF system: power & cryogenics - Software and diagnostics developments / improvements - New conditioning hardware for subsequent years - RF noise and ripple measurements RF w - Customisation of klystrons (positioning of resonance) ill us time - Linearisation of klystron response mad e a stability e - Effect of detuning and coupler position on loop vaila ble - Optimisation of loops for higher intensity - Fine adjustment of trip level for fast transients - Setting-up of the longitudinal damper - Preparation of feed-forward and 1 -T Checkout Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine feedback the 12

Other Equipment tests: Warm Magnets u Visual u u inspection No spanners lying around… All protection covers in place u Cooling u u system o. k. All valves open No visible leaks u Expect Jan Uythoven, AB/BT 1 day work per sector, 8 days total Chamonix@CERN 2005, Machine Checkout 13

Other Equipment Tests: Vacuum u Assume u Check u u interlocks Sector valves Connection to BIC u Not u machine already cold & closed 3 days per octant 2 teams in parallel 12 days to forget Vacuum interlocks on transfer lines: 2 days Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 14

Other Equipment tests: Collimators and Beam Absorbers u Redo some of the hardware commissioning tests in local & remote u u Motor control, position switches Temperature sensors, cooling system u Interfaces with more intelligent software from the CCC to manage settings u Not defined yet, should include BIC u Couple u of days In parallel with other tests Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 15

Other Equipment tests: Pulsed Magnets u u Beam dump kickers (MKD and MKB point 6) Injection kickers (MKI points 2 and 8) Tune and Aperture kickers (MKQA point 4) For all of them u u u Verification of local interlocks Pulse at full power – MKI some re-conditioning time Interface with many systems across the machine u RF point 4 (rev. freq. ) u Energy from dipoles either side point 6 u Timing, controls, machine protection, post mortem u OASIS, logging u Cross connections between MKI and MKD Maximum in HC, start early, profit from ILITY RUN Jan Uythoven, AB/BT RELIAB Chamonix@CERN 2005, Machine Checkout Couple of days per system Couple of WEEKS 16

Other Equipment tests: BDI u Large distributed systems: BPMs & BLMs u u u Interlocked Systems – check interface BIC (phase III) u u Hardware fully checked during HC, but re-check Main tasks: software and logging, links to other users CONTROL SYSTEM DRY RUNS, distributed over about 2 months, so start BEFORE the machine checkout BLM, BCT point 4, BPMs point 6, LDM (Longitudinal Density Monitor) E own h that as Q and Q’ measurements, Schottky systems, L Sychrotron radiation A W AYS BDI monitors, OTR screens, Wire scanners to ha ve a wants n acce other ss xper Diagnostics systems – no special requirements ienc sh e u Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 17

Phase III u Status after Phase I & II u u All equipment and systems (re-)tested, sector by sector Individual interlocks tested Controlled access Phase III u u u Machine wide tests from the CCC Go through complete machine cycles without beam Accent will be on control system and application software Interleaved with final tests of access system and green light by DSO In principle no access should be necessary any more Control System Tests: Build on experience ! u To be tested throughout the year. S with the maximum number of ‘dry runs’ u TI 8 tests with beam u Hardware Commissioning of all sectors including system tests u Sector test with beam Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 18

Phase III Complete Machine Cycle Without Beam u Initial machine cycle (45 minutes) to get to initial field conditions u u u Generate functions for the power converters Download all functions to power converters – synchronous ramping of all power converters including energy ramping of beam dumping system u u u All fixed displays, video, trajectory and orbit correction Feedback applications: trajectory, tune, chromaticity, transverse, longitudinal Logging of all parameters during the cycle Vacuum displays, alarm screens, radiation monitoring Interface with the Beam Interlock System u u u Timing (slow and fast), OASIS to set timing of kickers Interface with RF, injection kickers, beam dump kickers, machine protection system Applications – communication with all equipment u u Settings management, trimming of functions Machine wide timing of the systems, pre-pulses etc. u u No further tests possible – real magnetic field to be found out with beam Safe beam flag depending on energy Post mortem after beam dump Some kind of sequencer to automate tasks (LHCexec) Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 19

Phase III, continued u Now obligatory to have machine completely closed u u u Followed by Beam Interlock System (BIS) tests (or BIC) u u Need “Green Light” from all systems – towards the end of test period 16 different users, 150 ‘user boxes’ u Every box needs to receive an interlock, provoked by the equipment connected to it. Check 4 x dump request per equipment. Estimate 3 x 8 hours to test – 2 days Rubber stamping of systems by DSO – data for INB u u If search broken during phase II: Complete patrouille with several teams: 2 days Re-test complete access system before DSO test: 2 x 12 hours u Machine protection system has probably been touched upon during phase II Need about 1 day, if everything goes fine Rubber stamping by ‘Mr. Interlock’ u Need formal test procedures and acceptance criteria Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 20

Time Required u Phase u Cooldown and closing machine u Phase u u u II: 3 weeks minimum Interlocks, equipment & system tests u Phase u I: 1 week minimum Partially parallel with hardware commissioning III: 2 weeks minimum Complete cycles without beam Final test access system and BIS DSO tests Jan Uythoven, AB/BT Minimum of 6 weeks on planning! Equipment setup with beam, see Chamonix@CERN 2005, Machine Checkout 21

Equipment Tests With Beam u BDI u u u RF u u u Check interlocks of the system Safe beam flag uou ks / sly d u mo nth ring t he f s of irst ope rati on wee Beam Dumping System u u u Logging beam data, post mortem Steering – feedback Machine protection tests u u Low level control (typically 3 weeks) Phasing Con Feed back tin Control system u u Calibration BPMS, BCTs Set-up of tune and chromaticity measurements Beam dump to be set up at the different energies Verification of post mortem, logging and analysis Cryogenics with significant beam load Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 22

Not to underestimate: Transfer Lines u TI 8 will need recommissioning u u u Tested before u With beam up to TI 8 TED end 2004, u Not tested yet: last 100 m + collimators u Sector test end 2006 But will need to be recommissioned before first beam TI 2: never fully commissioned before u u u Will probably only be ready for commissioning at about the same time as the last LHC sector (transport of all LHC magnets through the TI 2 tunnel) Could be done in parallel with hardware commissioning of the last sector: preferred solution but impact on HC schedule Other option is to do this in parallel with ring 2 beam commissioning: higher risk in case of problems Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 23

Machine Checkout For Subsequent Years u u No Hardware Commissioning period but shutdown work to be finished Outline stays the same with similar tests Some tests should be faster, because systems are better known Foresee about 5 weeks minimum: longer than SPS / LEP because systems are more complex, machine protection to be taken more seriously Compare LEP and SPS: About 2 weeks of pre-closure and 2 weeks of real machine checkout when machine is for OP 2 weeks of LEP / SPS ‘cold checkout’ similar to 2 weeks of phase III of what is presented for the LHC Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 24

Conclusions u After the hardware commissioning period the machine is not ready for beam u u u At least 6 weeks of tests need to figure on the LHC planning for Machine Checkout u u u Start earlier for cooling down the sectors warmed up > 80 K for ELQA tests, in parallel with hardware commissioning Start earlier for software and machine protection: DRY RUNS to be planned and scheduled Many tasks in parallel: need careful planning and good coordination Some running in of equipment with beam is also required u u Confirmed by experience from LEP & SPS, need a period to get ready for beam under the responsibility of OP Even with all equipment groups 100 % ready on time (!): complete systems tests remain and their interdependencies need to be verified BDI being the obvious candidate, but also RF, beam dumping system Don’t forget the transfer lines Same applies every year after a shutdown Jan Uythoven, AB/BT Chamonix@CERN 2005, Machine Checkout 25