Deep wide field VLBI imaging The need for

Deep wide field VLBI imaging “The need for speed” Harro Verkouter Joint Institute for VLBI in Europe Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht]

Information content VLBI datasets contain much more information than is extracted Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 2

Information content Usually a field of 50 x 50 synthesized beams is imaged Still only 0. 1% of information! Typically corresponds with a field-ofview of 1” x 1” Primary beam of Effelsberg is 6’ (@18 cm) Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 3

Field of view limitations 1. Pre-correlation a) Sensitivity Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 4

Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 5

Field of view limitations 1. Pre-correlation a) Sensitivity 2. Post-correlation b) Time smearing c) Spectral resolution Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 6

Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 7

Field of view limitations(2) VLBI datasets >> connected element array datasets Almost obsessive need for averaging to make datasets manageable Time smearing/spectral smearing scale with baseline length! Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 8

What can we do about it? 1. Pre-correlation: a) EVN: actively support development for more sensitivy, e. g. PCEVN, Mark. V. b) EVN: 12 hrs @512 Mbit 10 Jy/beam 2. Post-correlation: a lot more! a) Decrease correlator integration time b) Increase spectral resolution Need enhanced output data rate! Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 9

And there’s more… Once the data is correlated, is that it? ‘re-correlate’ by mathematics; produce multiple phasecentres Experiment with the data: apply algorithms at will Helps with near real time space VLBI Apply filters before exporting to user Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 10

And there’s more (2) Applicability of the algorithms is limited by the aforementioned time- and bandwidth smearing Need enhanced output datarate! Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 11

$Requirements Want to map out considerable fraction of Ef primary beam (6’ @18 cm):$

Requirements Want to map out considerable fraction of Ef primary beam (6’ @18 cm): Field-of-view of 1’x 1’ needs Integration time 0. 25 s 62 k. Hz channelwidth 2 k channels for 64 MHz bandwidth Integration time ~ 1/ Channelwidth of 62 k. Hz remains Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 12

Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 13

Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 14

EVN Mk. IV Correlator limits Integration time Cycle time of 0. 015 s (actually, 1/64 th of a second) Spectral resolution 131072 complex lags per readout = 65536 spectral points per readout Divided over 32 products leads to 2048 spectral channels per product Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 15

Datarate Each lag occupies 5 bytes of data (4 data+1 overhead) Hence, ¼ million lags at 64 readouts/second equals 16 Mlags/second Which works out to be 160 MByte/s Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 16

The EVN Mk. IV Correlator Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 17

The PCInt project Short for Post Correlator Integrator We want to capture the full output of the EVN Mk. IV correlator to disk Need to replace output datapath Steve Parsley Marcel Loose Harro Verkouter Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 18

The PCInt project (2) High speed readout of the correlator was already prepared Via DSP powered serial port Need hardware and software to enable this Receiving end of serial port Gbit ethernet for transfer from correlator rack to datacollection host Fast disk subsystem in order to support 160 MByte/s (parallel RAID arrays in a Storage Area Network) Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 19

Correlator Board (x 8) VME High Speed Serial C 40 COMM PCI Ethernet Card SBC (x 2) Correlator rack (x 4) 100 TX 1000 FX (x 8) Switch CCC DDD (xn) Fibre Channel EEE (xk) Aug. 23 rd 2002 FC Switch Raid Array (xm) need for speed [URSI XXVII GA - Maastricht] Current situation RT System 20

RT System Correlator Board (x 8) VME High Speed Serial PCI Ethernet Card SBC (x 2) Correlator rack (x 4) 100 TX 1000 FX (x 8) Switch CCC DDD (xn) Phase 0 C 40 COMM Fibre Channel EEE (xk) Aug. 23 rd 2002 FC Switch Raid Array (xm) need for speed [URSI XXVII GA - Maastricht] 21

RT System Correlator Board (x 8) VME High Speed Serial PCI Ethernet Card SBC (x 2) Correlator rack (x 4) 100 TX 1000 FX (x 8) Switch CCC DDD (xn) Phase 1 C 40 COMM Fibre Channel EEE (xk) Aug. 23 rd 2002 FC Switch Raid Array (xm) need for speed [URSI XXVII GA - Maastricht] 22

RT System Correlator Board (x 8) VME High Speed Serial PCI Ethernet Card SBC (x 2) Correlator rack (x 4) 100 TX 1000 FX (x 8) Switch CCC DDD (xn) Phase 2 C 40 COMM Fibre Channel EEE (xk) Aug. 23 rd 2002 FC Switch Raid Array (xm) need for speed [URSI XXVII GA - Maastricht] 23

Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 24

Post-processing issues Data volume = huge 1 hour @160 MByte/s equals 560 GBytes of data Use a cluster of nodes Need automated processing Use a processing pipeline Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 25

Processing pipeline Correlated data is saved to disk Apply automatically Normalization correction Flagging Produce dataset with requested phasecentre(s) for user Can also apply filtering Export to FITS Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 26

Processing pipeline (2) Use AIPS++ JIVE data quality assessment is already done using AIPS++ supports parallel processing Export of data to user is also already done using AIPS++ Aug. 23 rd 2002 need for speed [URSI XXVII GA - Maastricht] 27