HCAL Readout and DAQ using the ECAL Readout

HCAL Readout and DAQ using the ECAL Readout Boards Paul Dauncey Imperial College London, UK 28 February 2003 Paul Dauncey - HCAL Readout 1

ECAL readout electronics overview CALICE ECAL has 30 layers, 18 18 channels/layer, 9720 total • Each gives analogue signal, 14 -bit dynamic range • Very-front-end (VFE) ASIC (Orsay) multiplexes 18 channels to one output line • VFE-PCB handles up to 12 VFEs (216 channels) • Cables from VFE-PCBs go directly to UK VME readout boards 28 February 2003 Paul Dauncey - HCAL Readout 2

VFE-PCB control and timing VFE-PCB needs several control and timing signals • • Analogue: 12 differential input, 2 differential output from readout board Digital: 4 LVDS input, 9 LVDS output from readout board, plus 7 uncommitted LVDS spares (could be in or out; assume out here) 28 February 2003 Paul Dauncey - HCAL Readout 3

Readout board overview Readout board structure • Front End (FE) FPGA controls all signals on VFE-PCB cable • Back End (BE) FPGA gathers and buffers all event data from FE and provides interface to VME • Trigger logic in BE for timing and backplane distribution; only active in one board • First readout board prototypes this summer, final boards end 2003; cost per board ~ € 10 k 28 February 2003 Paul Dauncey - HCAL Readout 4

Readout board features • Board clock ~40 MHz • • Sample-and-hold timing needs to be better than 10 ns • • DAC fed back for internal as well as VFE calibration No data reduction planned in hardware • • Effectively the VFE-PCB trigger; VFE shaping time of 180 ns sets maximum trigger latency also Fine-tune rising edge of signal on 8 clock, ~3 ns steps Each cable can be timed independently Dual 16 -bit ADCs and 16 -bit DAC • • Almost all signals edges timed to 25 ns periods Read out all ECAL channels for every event 2 Bytes 1728 channels/board = 3. 5 k. Bytes/board 2 Bytes 9720 channels = 19 k. Bytes total On-board buffer memory; 8 Mbytes • Allows up to ~2 k events for ECAL during beam spill 28 February 2003 Paul Dauncey - HCAL Readout 5

Trigger/DAQ logic Idea is to implement trigger/DAQ interface for all subsystems 28 February 2003 Paul Dauncey - HCAL Readout 6

Trigger handling • Multiple (4) trigger and “activity” (background) inputs • • Need clean events with no pile-up • • Distribution to other boards in crate using custom cable Point-to-point LVDS (probably) Trigger also configurably delayed and output to connector • • • Reset through VME; single reset for system so no synchronisation issues Trigger is fanned out and sent to backplane connector • • Activity prevents subsequent triggers within configurable time period Trigger records following activity; read out with event Trigger latches off logic, preventing further triggers • • Can be enabled and disabled through VME For distribution to other systems (beam monitoring, HCAL? ) Assuming 16 outputs, LVDS; we have a LVDS NIM converter available Beam on signal allows buffering during spill, readout after spill 28 February 2003 Paul Dauncey - HCAL Readout 7

Motivations for reuse of readout boards We have tried to keep design flexible enough to allow readout boards to be used for HCALs also • • Aim for single crate (or at least a single DAQ PC) if possible • • No need for inter-PC communication Single VME interface reduces amount of code needed • • Assuming the on-detector electronics is compatible Makes offline analysis more uniform for different HCALs also Trigger use and distribution is more uniform • • Particularly if single VME crate Common buffering method if not reading every event before next trigger One downside may be less flexibility during testing before beam • • (Semi-)custom 9 U crate (CMS tracker standard); would need to buy specific crate for tests ~ € 6 k Is rate high enough with single VME crate? See later… 28 February 2003 Paul Dauncey - HCAL Readout 8

Use for analogue HCAL Tile AHCAL has ~1500 analogue channels (AFAIK) • Each ECAL readout board has 96 ADCs; 16 boards, need 2 crates • Multiplex AHCAL to fit into smaller number of boards? • • Are systems compatible? • • • Can something like twelve (multiplexed) signals be fed into each cable? Is multiplex control compatible with number of LVDS signals available? What is maximum allowed trigger latency for AHCAL? Are ADC input, DAC output ranges compatible? Trivial if using (slightly modified) VFE chip • • E. g. 8 -fold multiplexing could fit into two readout boards But who is designing equivalent of VFE-PCB? Otherwise would probably need (non-UK) firmware development • • FE FPGA only; rewrite cable signal handling, keep BE interface BE and VME would be identical; retains common VME interface 28 February 2003 Paul Dauncey - HCAL Readout 9

Use for digital HCAL Digital HCAL has ~350 k channels (AFAIK) • Assume zero suppression, as data volume otherwise is high (45 k. Bytes/event), and data concentrator on detector • Use LVDS lines for clock, trigger, configuration, control and data readout of concentrator • • What is maximum allowed trigger latency for DHCAL? Clearly requires FE firmware development, as for AHCAL But BE and VME are still uniform across system Are the number of lines compatible with concentrator interface? • • Minimum would be clock, trigger, serial in, serial out per concentrator Each cable could handle 4 concentrators, i. e. 12 outputs, 4 inputs per readout board, so each readout board handles 32 concentrators E. g. with 1 k channels per concentrator, see talk by Gary Drake Feb 6, this requires 11 readout boards (What sets number of channels per concentrator? ) Leaves three free VME slots in the crate for beam monitoring readout 28 February 2003 Paul Dauncey - HCAL Readout 10

Transfer rates to readout board • ECAL • • Analogue HCAL • • 18 channels multiplexed per line; all lines read in parallel ADC maximum rate is 500 k. Hz; actually needs around 3 ms per channel Total time around 50 ms per event With multiplexing at or below 18 channels per line, will be less than or equal to the above (assuming able to multiplex output at 500 k. Hz) Digital HCAL • • • Time set by maximum data volume of all concentrators for each event For 4 Bytes/channel, assume average maximum data volume per concentrator is 1 k. Byte = 250 channels (pessimistic? ) With single serial line at 40 MHz, rate is 40 Mbits/s = 5 MBytes/s, so time required is 200 ms 28 February 2003 Paul Dauncey - HCAL Readout 11

Modes of operation Readout board can run in any of three modes • Single trigger readout • • Semi-buffered readout • • Read all data between each trigger; slow but sure Read minimal information from each board per event; e. g. trigger number and buffer occupancy Must read any unbuffered electronics for each event Buffer rest of data and read later (after beam spill) Fully-buffered readout • • • Nothing read from readout boards per trigger; only unbuffered electronics All readout board data buffered until end of spill A missed trigger corrupts whole spill of data Semi-buffered is to avoid the need for a more sophisticated fast control and timing system 28 February 2003 Paul Dauncey - HCAL Readout 12

DAQ readout rates Assumptions: • Average event data: ECAL 19 k. Bytes, DHCAL 5 k. Bytes (? ) (AHCAL 3 k. Bytes), other (beam monitoring, etc) 2 k. Bytes (? ? ) • Average time for clear/trigger/interrupt from end of one event to start of read of next: 0. 1 ms. Implies beam rate >10 k. Hz • VME data speeds: non-block transfer 4 MBytes/s, block transfer 16 MBytes/s • Semi-buffered readout of ECAL and HCAL is 400 Bytes per event total, read out with non-block transfer so takes 0. 1 ms • All beam monitoring, etc, data is always unbuffered and is read out with non-block transfer so takes 0. 5 ms per event • • • Significantly bigger than data transfer time to readout board of 0. 2 ms Transfer time is not a limiting factor Disk write speed: ~ 20 MBytes/s; always outperforms VME 28 February 2003 Paul Dauncey - HCAL Readout 13

DAQ readout rates (cont) Rates for the different modes: • Single trigger: 24 k. Bytes block transfer 1. 5 ms, total 2. 3 ms per event; rate limited to 430 Hz • Semi-buffered: 400 Bytes takes 0. 1 ms so 0. 9 ms per event during spill; rate limited to 1. 1 k. Hz. 1. 5 ms per event outside of spill; rate limited to 670 Hz • Fully-buffered: Only time saved is readout of 100 Bytes, so 0. 8 ms per event during spill; rate limited to 1. 2 k. Hz. 1. 6 ms per event outside of spill; rate limited to 630 Hz No significant gain from using fully-buffered mode over semibuffered mode • Simpler trigger, better error detection and event verification in semi-buffered mode 28 February 2003 Paul Dauncey - HCAL Readout 14

Total data sample sizes • Want to save unsuppressed data for noise and pedestal studies but also want processed data for analysis • • Total disk space needed per event ~ 30 k. Bytes Want to run in many configurations • • • Particle type, energy, HCAL, preshower, angle, etc; ~ 100 configurations Need ~ 106 events per configuration Total event sample ~ 108 events ~ 3 TBytes • • Raw event size ~ 26 k. Bytes (assuming no significant “gzip” compression) Processed event size ~ 4 k. Bytes (assuming DHCAL can be reduced? ) Just brought 3. 2 TBytes for Ba. Bar for ~ € 6 k We have budgeted ~ € 12 k for DAQ disk Processed data is ~ 400 GBytes DESY have said they will host data long term 28 February 2003 Paul Dauncey - HCAL Readout 15

What if no zero suppression in DHCAL? How would these numbers change if DHCAL raw bits are read out? • Event data size is now fixed at 45 k. Bytes per event • Transfer time to readout board reduced to 0. 03 ms as now always transfering 1 kbit = 128 bytes • Buffer memories will contain 45 k. Bytes spread over 11 boards, i. e. ~ 4 k. Bytes per board, so can still buffer ~ 2 k events • VME readout modes: single trigger becomes 210 Hz, semibuffered and fully buffered become 250 Hz out of spill but are still above 1 k. Hz during the spill • The total event size is 66 k. Bytes per event, giving a total data sample size of 7 Tbytes; processed data size is unaffected • Could always do zero suppression in DAQ software so disk space as before Worth considering if this simplifies (and reduces cost) of ondetector electronics significantly 28 February 2003 Paul Dauncey - HCAL Readout 16

Summary • With some firmware effort, the ECAL readout boards may be able to be used for both the AHCAL and DHCAL options • • Rates of around 1 k. Hz during a spill, 600 Hz outside of a spill would be possible • • Buffering up to 2 k events during the spill on-board Data volumes would be large but affordable • • Savings in software effort, code uniformity, trigger distribution, etc. Even if DHCAL is read out unsuppressed BUT… important to cross check the assumptions made here are realistic • • • Compatibility of different HCAL readouts Amounts of HCAL data Beam spill timings 28 February 2003 Paul Dauncey - HCAL Readout 17