First group planning meeting Introduction — Ted Liu

First group planning meeting: Introduction - Ted Liu, Jan. 30 th, 2003 • Introduce new people • A brief history of time • Goals of this meeting • Project overview • Roadmap • Guideline for the project planning …real talks by others… • Some starting points for discussion session(see web). Initial thoughts (or targets for you to shoot at): near term hardware needs/production plan manpower/organization/near term goals … future meetings & L 2 upgrade e-log

This project is attracting the best people/groups @ CDF … New people who joined the project recently: • Burkard Reisert • Cheng-Ju Lin • Paul Keener • Joe Kroll & his $$ • Kristian Hahn • Shawn Kwang • Bob Blair • John Dawson • Jimmy Proudfoot • Franco Spinella (FNAL new posdoc) (FNAL posdoc) (Upenn, DAQ expert) (Upenn, Nigel’s student) (UC, Mel’s student) (ANL) (INFN) “Old” people who is eager to come back: • Sakari Pitkanen (still waiting for visa @ Finland) => youngest collaborator 2

A brief history of time … Oct. 2001: FNAL decided to build Pulsar board Uof. C decided to provide engineer support soon Upenn purchased obsolete interface components Mar. 2002: INFN built SVT-TSI-PCI daughter card Aug. 2002: ANL possibility with Atlas Ro. IB Oct. 2002: Pulsar works with ANL Gigabit Ethernet mezzanine card (essentially plug&play) ANL/FNAL/INFN/Uof. C/Upenn We have a strong group, now we need a good plan 3

Goals of this meeting • Focus on planning, not technique details • Roadmap • Near term goals and long term plan • Manpower availability (current/future) • Who is interested in working on what&when • Get a clear picture of the issues/tasks involved will have a few technique talks 4

Project Overview may roughly divide into four “parallel” efforts: • E 1: Pulsar hardware/firmware/VME software • E 2: Design spec. for data format/algorithm/system interface • E 3: Ro. IB option effort (see Bob’s talk) • E 4: PCI/CPU architecture/OS/Infrastructure SF/L 2 D SF … E 1 E 2 E 3 E 4 CDF Run. IIb L 2 Atlas L 2 Phase A Phase B Phase C 5

Roadmap may divide into three phases/stages: • Phase A: Teststand, core firmware, HW prod/testing Algorithm firmware specification PCI/CPU/OS/infrastructure software will show some details • Phase B: Rx algorithm firmware implementation/testing, system integration in standalone mode • Phase C: System integration in parasitic mode, test runs 6 with cosmic and beam

Pulsar design 9 U VME (VME and CDF ctrl signals are visible to all three FPGAs) P 1 T S T R K P 2 SLINK signal lines P 3 spare lines Control/ Merger L 1 T R K 3 Altera APEX 20 K 400_652 (BGA) FPGAs 502 user IO pins each Data IO Mezz card connectors SRAM 128 K x 36 bits Data IO SRAM 3 APEX 20 K 400 FPGAs on board = 3 Million system gates/80 KB RAM per board 7 2 128 K x 36 pipelined SRAMs with No Bus Latency: 1 MB SRAM (~5 ns access time)

Pulsar is designed to be fully self-testable: at board level as well as system level For every input/output, there can be an output/input, All interfaces are bi-directional (except VME) TSI SVT/XTRP SRAMs RF clock one for all and all for one Gigabit Ethernet It is this feature which allows us to develop&tune an upgrade system in standalone mode 8 Overall planning should be based on this feature

An old slide from an old talk on Dec. 7 th, 2001: original motivation to build Pulsar: reduce impact on running exp. The way we have been debugging Why diagnostic tools? the Level 2 decision crate: Reces HP scope very often need to use CDF+Tevatron L HUGE amount of work has been done 1 C this way by a few hardworking experts X S L I T TS V I S R T S O P T Chicago L 2 decision crate L 2 inputs α Logical HP scope Analyzer The idea is to build test stand tools to “replace” CDF and Tevatron, to make life MUCH easier. Booster CDF DØ Tevatron p source Main Injector (new) 9

A slightly modified version for funding agency Pulsar can be used as Weapons against mass distraction to reduce impact on running experiment CDF can be used as Weapons of top mass construction Chicago real data Booster CDF DØ Tevatron p source Main Injector (new) emulation L 2 T 10 Pulsar can be used to emulate both CDF/Tevatron and ATLAS/LHC for L 2 T

Hardware setup during Phase A: (1) Pulsar Tx => Rx =>PCI=>PC w/ 4 hotlink mezzanine cards (muon/L 1/SVT/XTRP/TSI paths) Fully develop teststand capability (should test with Run. IIA boards) Tx Develop core firmware for all cases Robustness/Production test setup (see Burkard’s talk) (2) Pulsar Tx => Rx => PCI=>PC 4 Taxi mezzanine cards (Reces path) (3) Pulsar Tx => Rx => PCI=>PC 2 Hotlink (Cluster) & Rx 2 Taxi (Isolation) PC 11

Core firmware example: Data. IO FPGA (Rx case) => Control FPGA is similar VME responses SRAM interface CDF Ctrl interface downstream Mezzanine Card interface SLINK Play/record formatter RAM/ Spy buffers DAQ buffers Filter algorithm • Firmware was designed this way from day one Play/record upstream RAM/ Spy buffers Blocks in green: common to all FPGA types, blocks in white are data path specific (core firmware only deal with the 12 simplest case: pack ALL data into SLINK format)

Well organized firmware effort is one of the keys to have a system: clean/easy-to-understand/robust & with minimal maintenance effort Common core firmware SVT XTRP LVDS + ext. FIFO L 1 LVDS TSI VME responses SLINK formatter DAQ buffers CDFCtrl responses SRAM ctrl Mezzanine interfaces Diagnostic RAMs Spy buffers … muon cluster hotlink Taxi Isolation Reces Knowing one data path Pulsar knows all Pulsars Data specific algorithm difference is just minor details 13 Need dedicated people responsible for developing the core firmware

Summary of what I have said or Guideline for the project planning: Project planning should follow Pulsar design philosophy (1) In God we trust, everything else we test (2) One for all and all for one Anyone who has a problem with this is in the wrong project 14

Goals of this meeting • Focus on planning, not technique details • Roadmap • Near term goals and long term plan • Manpower availability (current/future) • Who is interested in working on what&when • Get a clear picture of the issues/tasks involved will have a few technique talks… By now, hopefully you are all warmed up… 15

Agenda • Introduction (TL 20 mins) • Pulsar hardware status (Mircea, 5 mins) • Automating Pulsar test procedures (Burkard, 10 mins) • L 2 data format/real data patterns (Cheng-Ju, 10 mins) • Pulsar algorithm firmware spec. for Iso&Reces (Bob, 20 mins) • Status/plan of S 32 PCI 64/CPU/OS study (Paul, 20 mins) • ATLAS Ro. IB option (Bob, 20 mins) • Discussions (ALL, 60 mins) • Summary (5 mins) • dinner at 6 pm 16

Material (some initial thoughts) for discussion session • Some details on hardware needs during phase A/B/C • Manpower/organization • Near term goals for each effort • Future meetings & L 2 upgrade e-log • More material can be found on the meeting web page http: //hep. uchicago. edu/~thliu/projects/Pulsar/ L 2_upgrade_meeting. html 17

Some details Hardware setup during Phase A 1: (1) Current setup (since Nov 02): Pulsar Tx => Rx =>PCI=>PC with 4 hotlink mezzanine cards (for muon/L 1/SVT/TSI path) Tx • uses CERN AUX card • uses old SLINK to PCI card & • uses ANL Gigabit Ethernet card • uses an old PC Rx Will need to: • test with our AUX card (next month) • need 4 more hotlink mezz cards as reference cards to test new Pulsars PC 18

Hardware setup during Phase A 2: (2) Next setup (March 03 -- ): Pulsar Tx => Rx =>PCI=>PC with 4 Taxi mezzanine cards (for Reces path) Tx • 2 more Pulsars from prototype run (ready for testing next week) • 4 Taxi mezz. cards (next month) Will need to: • test the two new Pulsars with the 4 hotlink mezz. reference cards first Rx PC 19

Hardware setup during Phase A 3: (3) Next setup (~April 02): Pulsar Tx => Rx =>PCI=>PC with 2 hotlink/Taxi cards (for Cluster/Isolation paths) Tx • 2 more Pulsars from pre-production • 2 hotlink/Taxi mezz. cards Once this setup is fully developed&tested: time for production Rx Will need to: • test the two new Pulsars with the 4 hotlink mezz. reference cards first • new PC with new PCI card PC 20

Pulsar hardware needs during Phase A (summary): (1) Current setup: Pulsar Tx => Rx =>PCI=>PC with 4 hotlink mezzanine cards (for muon/L 1/SVT/TSI paths) * 4 more hotlink mezzanine cards as reference cards for new motherboards testing * one AUX card (5 V) (2) Next setup: Pulsar Tx => Rx => PCI=>PC with 4 Taxi mezzanine cards (Reces path) * 2 more Pulsar boards from prototype batch (next week) * 4 Taxi mezzanine cards from prototype run * one more AUX card (3. 3 V) (3) Next setup: Pulsar Tx => Rx => PCI=>PC with 2 Hotlink (Cluster) & 2 Tax(Isolation) * 2 more Pulsar boards from pre-production run * 2 Hotlink/Taxi mezzanine cards Production starts after (1) –(3) fully tested 21

Hardware setup for Phase B: Does Ro. IB need a separate crate ? T M RR U OA C C Rx O N E R C L U / I S O R E C E S R E C … E S T M RR U OA C C Tx O N E R C L U / I S O R E C E S Teststand Area • Initial system level integration and performance studies • Finalize data path specific firmware, reduce the data size as necessary • online monitoring software • … TSI 22

Hardware setup for Phase C: T M RR U OA C C Rx O N E R C L U / I S O R E C E S R E C … E S L 2 Central Crate in Trigger Room In addition to the teststand area setup in standalone mode, setup the full system (Rx) in L 2 central crate next to the current L 2 decision crate • Initial system level integration with real system • fine tune FW/SF/performance • parasitic running if possible • special test runs with cosmic and beam • … 23

Project Overview may roughly divide into four “parallel” efforts: • E 1: Pulsar hardware/firmware/VME software • E 2: Design spec. for data format/algorithm/system interface • E 3: Ro. IB option effort (see Bob’s talk) • E 4: PCI/CPU architecture/OS/Infrastructure SF/L 2 D SF … E 1 E 2 E 3 E 4 Phase A Phase B Phase C 24

(E 1) Pulsar hardware/firmware/VME software Pulsar/mezz/Aux production: UC shop Production testing: Burkard/Sakari+Kristian + … Technical support: UC(Harold/Mircea)/FNAL(Bob Demaat) Infrastructure support: Crates/TSI/Testclk/Tracer: CJLin/PWittich Core firmware spec/implementation: Burkard/Sakari + others Core VME software(testing, online): Burkard/Kristian/Sakari subsystem specific algorithm firmware/VME software: muon path: Shawn with support from Sakari/Burkard Reces path: ? … Cluster path: Kristian … Isolation: Kristian … L 1/XTRP/SVT: part of the core firmware Overall coordinator: Burkard Reisert Consultant: Mel Shochet/Bob Blair/John Dawson/Peter Wilson/Jonathan Lewis. . . + volunteers 25

E 1: possible near term goals (1) Feb: have the automated testing procedure fully working for 16 hotlink channels (muon path): Tx->Rx->PC; (2) March: finish testing 2 more Pulsar boards with hotlink/Taxi mezzanine cards and AUX cards; (3) April: initial proof of principle test with S 32 PCI 64 with test setup (4) May: core firmware (Tx/Rx) fully developed and tested with hotlink/taxi mezzanine cards; (5) June: production/testing for Pulsar/mezz cards/AUX cards (6) July: have Tx->Rx-PC fully developed for Reces(taxi), cluster/Isolation (hotlink/taxi); (7) Aug: document everything above: cdfnote 26

(E 2) Firmware spec. /SLINK data format/data patterns System Level(TS handshake, DAQ etc): Peter/Mel/Bob Blair/John + others Subsystem specific: muon: Mel/Cheng-Ju/Shawn Kwang Reces: Bob Blair/Cheng-Ju/? Cluster: Cheng-Ju/Mel/Kristian Isolation: Bob/Cheng-Ju/Kristian L 1/XTRP/SVT: Cheng-Ju/Burkard/Mel Overall coordinator: Cheng-Ju Lin Consultant: Jonathan Lewis/Peter Wilson/Henry Frisch/Jimmy Proudfoot/. . . + volunteers 27

E 2: possible near term goals (1) (2) (3) (4) Feb: March: April: May: (5) June: (6) July: document ALL L 2 data format; software ready to derive all real data patterns; initial algorithm specification for each path; software modeling based on algorithm specifications online monitoring software finalize algorithm/system firmware specification document everything above --> cdf note ready 28

(E 4) PCI/CPU/OS/infrastructure software/L 2 D software need more thinking here for the organization…need a posdoc in this area ? R&D setup at Upenn: Paul Keener PCI/CPU/OS/infrastructure software development Upenn has funding from the university for this R&D effort. Testing/integration setup at ANL: Bob Blair/John Dawson +? Testing/integration setup at FNAL: Kristian + ? L 2 algorithm software: Peter Wittich/Bob/Cheng-Ju/Kristian/Burkard. . . Overall coordinator: Peter Wittich Consultant: Jim Patrick/Rick van Berg/Joe Kroll/Franco Spinella/Bill Ashmanskas. . . + 29

E 4: possible near term goals (1) Feb: at the very least repeat CERN's results on S 32 PCI 64 performance; (2) March: pick one PC platform/OS with reasonable performance (doesn't have to be the best) (3) April: basic infrastructure software ready early April and transfer to FNAL teststand, along with a PC/OS. This will be used for proof of principle test at FNAL in April and teststand needs (4) May-June: more studies to see if there is any better choice for PC platform/OS. study on how to send the L 2 decision back to Pulsar (5) July: final decision on PC platform/OS. . . and infrastructure software. . . (6) Aug: final decision on how to send L 2 decision back to Pulsar. (7) Sept: document everything above --> cdfnote ready 30

Future meetings • Group meetings: Once per month is reasonable I would suggest that each group ANL(Bob)/FNAL/INFN(Luciano)/UC(Mel)/Upenn(Joe) take turn to organize/chair the meetings • Subgroup meetings during phase A: bi-weekly regular meetings or let each coordinator decide when to call the meetings? • create a L 2 upgrade e-log? too many meetings too much BS We have good people who want to spend more time to do real work Based on years of observation: much less BS on e-log 31 an effective way to improve communication

Run. IIA and Run. IIB 2 B or not 2 B? 32