x TCA for Physics Standards Progress Applications

x. TCA for Physics Standards Progress & Applications Summary for Project X Collaboration September 11 -12, 2009 Ray Larsen SLAC National Accelerator Laboratory ARD-LC Department SLAC ATCA/u. TCA Evaluation R&D Page

SLAC LC Global Controls R&D • L-Band RF Station Controls & Interlocks • Standard HA Platform Evaluations • PICMG x. TCA for Physics Standards Initiative • New Applications for Experiments & Controls SLAC ATCA/u. TCA Evaluation R&D Page 2

L-Band RF Station Interlocks & Controls 1 – Based on new FPGA VME module processing both fast & slow interlocks – Phase 1 –VME Implementation (Fast Fault Finder F 3) – Phase 2 - Port to ATCA shelf via ATCA-VME Adapter Phase 1 VME Phase 2 ATCA SLAC ATCA/u. TCA Evaluation R&D Page 3

RF Interlocks & Controls 2 F 3 Board • 4 Fast (20 MHz), 10 Slow (10 KHz) 16 bit ADC channels • Fast inputs via FP coax, Slow inputs via differential shielded cable on RTM (Rear Transition Module) • All trip levels programmable via EPICS panels • Chained Fault Outputs to klystron, modulator SLAC ATCA/u. TCA Evaluation R&D Page 4

RF Interlocks & Controls 3 • VME System Software – VME system fail-safe but non-redundant hardware, software – EPICS Vx. Works driver for F 3 FPGA based Interlock module, hi-lo threshold programmed on fast & slow interlocks – VME system in final test • ATCA system plan includes redundant auto-failover hwe, shelf mgr SLAC ATCA/u. TCA Evaluation R&D Page 5

RF Interlocks & Controls 4 – Phase II • VME Adapters in 5 -Slot ATCA Shelf Phase II F 3 ATCA Implementation SLAC ATCA/u. TCA Evaluation R&D Page 6

Standard Platform Evaluations • Topics of Interest for Physics – ATCA Form Factor Card, Chassis, Rear Transition Module – ATCA Carrier Cards – AMC Mezzanine Cards – Micro. TCA Chassis for AMC Cards – Optional AMC card formats for physics (e. g. 2 -wide with additional I/O connector – Micro. TCA Rear Transition Modules – IPMI – Intelligent Platform Management Interface VME F 3 w/ Rear Transition Module SLAC ATCA/u. TCA Evaluation R&D Page 7

x. TCA Components in hand for Evaluation SLAC ATCA/u. TCA Evaluation R&D Page 8

Example IPMI Information from Micro. TCA MTC 5070 6 -Slot Shelf via Web Link Push buttons for module & system diagnostics & control: Hot Swap, fan speeds, temp, voltage thresholds, power supply temp, etc Temp Readouts Threshold Readouts SLAC ATCA/u. TCA Evaluation R&D Page 9

PICMG x. TCA for Physics Initiative PURPOSE • Develop extensions to Telecom specs for Physics • Need analog signal I/O. timing and sync lines, Rear Transition Modules • Coordinating Committee formed March 2009 • Hardware & Software Working Groups being formed • Specifications developed for approval by full PICMG membership • Goal: New AMCMicro. TCA physics specs by end 2009 SLAC ATCA/u. TCA Evaluation R&D Contact: larsen@slac. stanford. edu Page 10

x. TCA for Physics CC Membership Rev. 021209 SLAC ATCA/u. TCA Evaluation R&D Page 11

Possible x. TCA Applications SLAC ATCA/u. TCA Evaluation R&D Page

Generic ATCA for Detectors • Initiative by M. Huffer w/ G. Haller Group • Full ATCA data processing unit w/ RTM for extremely high throughput DAQ systems – TB/sec per ATCA Crate, plus companion Hub switcher card – Generic data processing unit targets several very high throughput experiments such as LHC detector upgrades (CERN), LSST ground-based telescope, several others – IPMI not implemented on carrier but has “hooks” to do so; no hooks to RTM – Dozen or so units built and being distributed to other labs for evaluation SLAC ATCA/u. TCA Evaluation R&D Page 13

ATCA-VME Adapter • First proposed as fast way to use existing VME IO module functionality in ATCA platform – Design contracted to SAIC – Design completed, board built via SLAC, first unit being tested at SAIC – IPMI involves some Vadatech components – Progress greatly delayed by funding cancellation in 2008; resumed in 2009 but progressing very slowly – Lack local engineering & software to work on it SLAC ATCA/u. TCA Evaluation R&D Page 14

Micro. TCA Evaluation • Started evaluation of Micro. TCA platform late 2008 – Performance Tech shelf ~6 AMC slots w/IOC controller purchased to test concepts – TEWS IO modules: 8 -ch 14 bit 105 MSps ADC, IP adapter modules for slow 16 bit ADC-DAC for interlocks & controls – Rear Transition adapters needed to get existing cable plants to front panel connectors (pending invention of rear IO version) – Committee working on specifications for extensions to MTCA standards including Micro. RTM SLAC ATCA/u. TCA Evaluation R&D Page 15

SLAC Linac Controls Upgrade • Possible opportunity for MTCA solution – 240 RF stations over 30 Sectors, 3 km length – Phase I: LCLS S 30 -30; Phase II: S 0 -19 plus Damping Rings – Current system: • • Distributed Intel micros on Multibus Custom CAMAC Serial Branch Driver on Multibus CAMAC instrumentation crates ~ 3 -4 per Sector (x 30) Custom low level RF system, instrumentation, interlocks chassis and CAMAC modules to drive 50 MW peak power klystrons • Interlock protection, temp, water flow etc. via custom chassis & CAMAC monitoring – 10 sectors for LCLS to be replaced over next 2 -3 years – Recent LCLS upgrades on VME including Magnet power systems, beam monitors, will not be replaced – Remaining 20 sectors to be upgraded in follow-on program SLAC ATCA/u. TCA Evaluation R&D Page 16

DESY x. TCA Evaluations • Spin-off from DESY ILC program: XFEL – Fraction of German ILC proposal approved to be built for X-Ray light source, 1 km linac, 30 10 MW L Band RF stations – Both ATCA and Micro. TCA platforms in prototype development for RF system, LLRF, fast & slow interlocks, monitoring and controls – In-house developed IPMI implemented on prototypes; systems demonstrated at shelf level – Partnering with industry for module development as well initiating designs in-house – Developed stacked AMC for IO from carrier, plus pushing for 2 -wide AMC for MTCA with µRTM SLAC ATCA/u. TCA Evaluation R&D Page 17

PICMG x. TCA for Physics Progress • Overall Goal: HWG, SWG Technical Subcommittees established late May early June – HWG Chair: Robert Downing – SWG Interim Chair: Stefan Simrock • HWG Goal: Specify key extensions by end October 2009; develop prototypes with industry in parallel to get samples by end of 2009 • HWG Progress • ATCA Managed RTM – Specified “preferred” ZD connector interface including alignment pin, managed power, JTAG, IPMI & high speed data pins on ZD connectors – Interconnect details close to finalized – Ready to proceed with Draft Specification Document SLAC ATCA/u. TCA Evaluation R&D Page 18

HWG Progress 2 • Clock & Trigger Timing Distribution – Agreed on principle of selecting “spare” dual star lanes in serial backplane to distribute physics clocks, triggers – Variants are handled in x. TCA by physical keying and E-keying for shelf management – Need to select specific lines based on survey of user requirements (number of lines needed), then proceed with Draft spec SLAC ATCA/u. TCA Evaluation R&D Page 19

HWG Progress 3 • ATCA AMC Carriers with IO & Timing – DESY design of stacked carrier with separate IO board seems sound; separates analog grounds – Stacked connectors are standard, multi-source – Gets IO signals via RTM – Also requires selecting clock, trigger lines in AMC extended options region SLAC ATCA/u. TCA Evaluation R&D Page 20

HWG Progress 3 • Micro. TCA • MTCA Form factors – Stacked AMC will work in MTCA crate with matching backplane, single-wide RTM – Double wide AMC also proposed by DESY (single board deemed too small analog space) – RTM double wide, facilities to manage under IPMI – Would use similar connectors to ATCA RTM (3 rows vs. 4 rows) – Attractive design, simpler than stacked version that requires additional analog/digital connector between stacked boards SLAC ATCA/u. TCA Evaluation R&D Page 21

HWG Progress 4 • MTCA Clocks & Trigger Distribution – Assign existing backplane lines to avoid additional backplane • Mechanics – RTM’s mate directly to associated module which provides power, JTAG & IPMI, keying & E-Keying – Main connector issues recently resolved – At least 1 vendor has mechanical design of shelf in hand • Status – Need to decide whether 1 or 2 versions of MTCA; 2 seems likely – Fast systems need ATCA backplane for fast direct communications between carriers with AMC’s; this feature for MTCA not available without something like SRIO between MTCA’s – Goal: resolve main topology choices in next 1 -2 months; begin draftin specs as soon as decisions made SLAC ATCA/u. TCA Evaluation R&D Page 22

SWG Progress • Late starting; mainly exploring following: – Protocol latencies for fast feedback (related to hardware choices) – Understanding IPMI, especially: • Large system topologies with a central control room • Extended uses of IPMI for calibration of applications modules • Support of FPGA’s and other firmware devices during operation of large system, including remote error checking in background, alarms, uploading of new releases – Exploring architectures for high availability software • Standard interface products like SAF HPMI (Hardware Platform Management Interface) • Commercial software products, operating systems, IPMI systems, development tools etc. SLAC ATCA/u. TCA Evaluation R&D Page 23

Summary Goal: COTS Support • Community hope to leverage telecom standards & design investments to take advantage of COTS availability – Already benefit from COTS shelf, IPMI, power architecture, backplane design, IOC’s – Some analog appearing and more I&C companies seem interested to expand into x. TCA • With vendor support via standards committees could establish infrastructure for lab developments, enhance growth toward more common COTS elements both hardware and software SLAC ATCA/u. TCA Evaluation R&D Page 24