Implementing White Rabbit in your design Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 1
Outline: My background: • KM 3 Ne. T / FPGA / Serial Communication White Rabbit introduction • Technology / Implementations WR PTP Core in detail How to implement WR in your design Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 2
KM 3 Ne. T Artist impression & detection method 640 strings 18 DOM/string 11. 520 DOMs 357. 120 PMTs v ko en er Ch ~ 860 m Peter Jansweijer Nikhef Amsterdam Electronics. Technology m 100 m Volume: ~5 km 3 n 11. 520 DOMs in the deep sea at 3 -5 km depth 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 3
The very beginning… “Measuring propagation delay over a coded serial communication channel using FPGA. ” 0111010110000010101101110101 3 1 0 Rx. Rec. Clk Bit. Slide(4: 0) Peter Jansweijer Nikhef Amsterdam Electronics. Technology 0 0011 0000 0001 = 1 3 0 Start/Stop delay Presented at the VLVn. T 09, October 15, 2009 in Athens Elsevier Nuclear Instruments and Methods: http: //dx. doi. org/10. 1016/j. nima. 2010. 04. 126 FPGA transceivers propagation delay study: http: //www. nikhef. nl/pub/services/biblio/technicalreports/ETR 2010 -01. pdf 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 4
White Rabbit Introduction Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 5
Layer 1 Syntonization = “The adjustment of two electronic circuits or devices in terms of frequency” All network devices use the same physical layer clock. Clock is encoded in the Ethernet carrier and recovered by the receiver chip. Phase detection allows sub-ns delay measurement Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 6
Precision Time Protocol (IEEE 1588) Frame-based synchronization protocol. Synchronizes local clock with master clock. Link delay evaluated by measuring and exchanging frames with tx/rx timestamps Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 7
Digital Dual Mixer Time Difference (DDMTD) Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 8
Using the DDMTD as phase detector and phase shifter Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 9
White Rabbit: an extension of Ethernet Two separate services (enhancements to Ethernet) Peter Jansweijer Nikhef Amsterdam Electronics. Technology provided by WR: Synchronization: accuracy better than 1 ns precision (tens of ps sdev skew max) Deterministic, reliable and low-latency Control Data delivery 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 10
Standardization Ongoing standardization effort to include WR into PTPv 3 Extension to PTP (IEEE 1588) – defined as PTP profile Compatible with “standard” PTP According to ISPCS Plug Fest results… “… White Rabbit is the most accurate PTP implementation in the world!” Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 11
White Rabbit Switch http: //www. ohwr. org/projects/white-rabbit/wiki/Switch Central element of WR network 18 1000 BASE-BX 10 ports Open design (H/W and S/W) Commercially available Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 12
White Rabbit Node example 1: SPEC http: //www. ohwr. org/projects/spec/wiki Peter Jansweijer Nikhef Amsterdam Electronics. Technology FMC-based Hardware Kit All carrier cards are equipped with a White Rabbit port. Mezzanines can use the accurate clock signal and “TAI” (synchronous sampling clock, trigger, time tag, …). Starting kit: http: //www. ohwr. org/projects/wr-starting-kit/wiki 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 13
Other White Rabbit Node examples: CUTE-WR (LHAASO) Central Logic Board (KM 3 Ne. T) Peter Jansweijer Nikhef Amsterdam Electronics. Technology CRIO-WR (CERN) SVEC (CERN) 3 rd International VLBI Technology Workshop SPEXI (CERN) 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 14
White Rabbit PTP Core http: //www. ohwr. org/projects/wr-cores/wiki/Wrpc_core T-Sense / Unique ID Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 15
White Rabbit PTP Core in detail Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 16
White Rabbit PTP Core in detail T-Sense / Unique ID PHY: Interface to the physical network, with deterministic phase relationship between gigabit clock and system clock Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 17
White Rabbit PTP Core in detail T-Sense / Unique ID Endpoint: Ethernet MAC, with precision time stamping capabilities of in- and outbound Ethernet frames Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 18
White Rabbit PTP Core in detail Redirector: Redirects all PTP Ethernet packets to the Mini-NIC T-Sense / “Normal” packets Unique ID Receives the payload of the PTP packets and stores payload data in memory Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 19
White Rabbit PTP Core in detail T-Sense / Unique ID Embedded Processor system (Lattice. Mico 32 + Bus interface + Memory) running a PTP-daemon Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 20
White Rabbit PTP Core in detail Now the CPU has “knowledge” of time. It can adjust the local time and update the proper TAI time and control the Pulse Per T-Sense / Second (PPS) generation Unique ID Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 21
White Rabbit PTP Core in detail T-Sense / Unique ID The local oscillator that is used for TX is locked onto the RX receive clock. A PLL is made using Digital Dual Mixer Time Difference (DDMTD) technique and software on the Lattice. Mico 32 (hence Soft-PLL) to tune the oscillator. DDMTD allows for < 1 ns phase adjustment under control of the Lattice. Mico 32 => a result of calculations of the round trip delay (Timing Synchro) Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 22
White Rabbit PTP Core in detail T-Sense / Unique ID 1 -Wire is used to fetch a unique MAC address UART is used for debug only Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 23
How to implement WR in your design Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 24
How to implement White Rabbit in your design FPGA SFP SPI REF clock generator 125 MHz CLK 125 M_PLLREF CLK 20 M_VCXO Peter Jansweijer Nikhef Amsterdam Electronics. Technology DAC VCXO DMTD clock generator Your own stuff 20+ MHz CLK_DMTD VCXO (1 Gbps) GTREFCLK Unique-ID (MAC-addr) (optional) DAC DATA PHY PLL Timing (TAI[ns], REFCLK) http: //www. ohwr. org/projects/white-rabbit/wiki/WRReference. Design 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 25
White Rabbit & CLBv 2(KM 3 Ne. T) Management & Config. Flags Tx_data 2 buf Tx Stream Select Tx_pkt 2 mac Pause Frame Tx. Packet Buffer 32 KB Rx. Port_m Tx. Port 1 Tx. Port 2 Tx. Port_m TDC 30 S Management & Control Fifo S S 31 PMTs TDC 0 Fifo Rx. Port 1 Rx. Port 2 Fifo ADC Management & Control Nano Beacon M M M S WB Crossbar M M (1 x 7) M M M Peter Xilinx MEM S Jansweijer Kintex-7 Nikhef Amsterdam Data Electronics. Control Technology Wishbone bus M M M WB Crossbar (3 x 2) M S S M 2 nd CPU LM 32 S SPI UTC time & Clock (PPS, 125 MHz) Point to Point interconnection SPI 3 rd International VLBI Technology Workshop Flash S S UART I 2 C Debug RS 232 Temp S I 2 C Debug LEDs S GPIO Compass Hydrophone Flags 31 TDCs State Machine Rx_mac 2 buf Rx. Packet Buffer 64 KB Time Slice Start IP/UDP Packet Buffer Stream Selector (IPMUX) Start Time Slice UTC & Offset counter since Rx_buf 2 data Rx Stream Select FPGA Tilt 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 26
White Rabbit Group: • Peter Jansweijer Nikhef Amsterdam Electronics. Technology WR Users http: //www. ohwr. org/projects/white-rabbit/wiki/WRUsers Picture taken @ 4 th WR workshop (2011), GSI Darmstadt http: //www. ohwr. org/projects/white-rabbit/wiki Many slides in this presentation orginate from the WR group: Thanks! 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 27
Thank you Peter Jansweijer Nikhef Amsterdam Electronics. Technology 3 rd International VLBI Technology Workshop 10 - 13 November 2014, Groningen/Dwingeloo, the Netherlands 28
Скачать презентацию Implementing White Rabbit in your design Peter Jansweijer
87cfba86f6bc2decf39617107c719049.ppt