Global Scale Tele-Immersion Network Performance Activities Jason Leigh

Global Scale Tele-Immersion Network Performance Activities Jason Leigh, Oliver Yu, Linda Winkler, Alan Verlo, Tom De. Fanti Yong-joo Cho, Ray Fang, Javier Girado, Liujia Hu, Tomoko Imai, Naveen Krishnaprasad, Michael Lewis, Ya Ju Lin, Dave Pape, Kyoung Park, Chris Scharver, Brenda Silva, Liang Wang Josh Eliason, Jinghua Ge, Eric He, Atul Nayak, Shalini Venkatamaran Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Common Characteristics of Teleimmersive Applications Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Characterization of Tele-Immersive Streams Estimated bandwidth (bits/s) UDP avatar Diff. Serv Types 6 K x n (15 fps) Burstiness Latency Jitter Error sensitive Constant Y Y N Brief Y Y N Constant Y Y YN depends Noninteractive Real-time Constant Y N YN 7 K x n Reliable Brief YN YN Y depends Best Effort or Deadline Delivery Sustained burst N N Y UDP audio stream 64 K x n UDP video stream 10 M (2 -way only) UDP stream With Playback TCP control data TCP bulk data Interactive Real-time Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Network Research Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Maximizing Bandwidth Utilization over Long Fat Networks • Even if Qo. S via Diff. Serv or Int. Serv is available, it still does not solve the Long Fat Network problem • Problem is small TCP window sizes (well known problem but still no widely accepted solution) • On SGI’s change in window size requires kernel rebuild • Size of window should be set to current available BW of the network • CAVERNsoft’s Parallel Socket Striping works well but is considered “irresponsible” use of networks Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

64 K Window Size Amsterdam to Chicago Bursty as max bw reached but performance is still good Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

64 K Window Size CERN to EVL Bursty as max bw reached but performance still good Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Window Size: EVL = 1. 85 M, SARA = 64 K EVL to SARA When window size is large enough no real benefit to using parallel sockets Sending client determines the window size SARA to EVL Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Window size: EVL = 1. 85 M, CERN = 640 K EVL to CERN Similar story at CERN to EVL Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Anomalies • Theoretical BW from EVL to SARA is 100 Mbps • Netperf UDP shows reasonable performance: – EVL to SARA 85 Mbps – SARA to EVL 65 Mbps (5 more hops via Abilene) • Netperf and Parallel sockets TCP shows only: – 30 Mbps • Perhaps due to asymmetric tcp window size settings? • Argument for UDP-based schemes? E. g. Forward Error Correction Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Forward Error Correction scheme for lowlatency delivery of error sensitive data • Transmit error correction data over high bandwidth networks that can be used for correcting UDP streams to achieve lower latency than TCP but higher reliability. • Transmit error correction data to improve quality of streamed video by correcting for lost packets. • Not intended for bulk data transfer but in light of TCP results this might hold some promise. Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

FEC Experiments • EVL to SARA- Amsterdam (45 Mb/s 100 ms RT latency) • Broader Ques: – Can FEC provide a benefit? How much? – Tradeoff between redundancy and benefit? • Specific Ques: – TCP vs UDP vs FEC/UDP – How much jitter does FEC introduce? – High thru put UDP vs FEC/UDP to observe loss & recovery Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

` goal FEC greatest benefit is in small packets. Larger packets impose greater overhead. As redundancy decreases FEC approaches UDP. Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Goal Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Packet Loss over UDP vs FEC/UDP between Chicago & Amsterdam UDP FEC Data Rate (bits/s) 1 M Packet Size (Bytes) 128 Packet Loss Rate in UDP (%) Packet Loss Rate in FEC over UDP (%) 0. 4 0 1 M 256 0. 2 0 1 M 1024 0. 2 0 10 M 128 30 4 10 M 256 25 3 10 M 1024 21 1. 5 Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Human Factors in Tele-Immersion Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Collaborative Coordination Experiments between Chicago and Singapore CAVE to CAVE (STAR TAP) • Audio via Phone call • Scramnet (adjustable latency, 0 jitter) • LAN Ethernet (~ 10 ms) • Local ISDN (~ 200 ms) • STAR TAP (~ 250 ms) • Predict STAR TAP similar to performance over ISDN Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Collaborative Coordination Experiments between Chicago and Singapore • • 200 ms RTT is the threshold where performance begins to suffer Roughly RTT to Asia. Results to Singapore similar to local ISDN 200 ms RTT with 0 jitter is same as 10 ms RTT with 7 ms jitter Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Diff. Serv Experiment 1 + background + Diff. Serv Bandwidth recovery good EVL 100 Mbps x fore back 25 Mbps 80 Mbps Latency recovery good x 42 Mbps ANL x 100 Mbps 42 Mbps x 100 Mbps Small packet loss Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Diff. Serv Experiment 2 + background + Diff. Serv Bandwidth recovery good EVL 100 Mbps x 25 Mbps 80 Mbps Latency recovery not good x fore 42 Mbps ANL x 100 Mbps 42 Mbps x back 100 Mbps Packet loss double Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Application of Research Results CAVERNsoft G 2 applications at i. Grid 2000 in Yokohama Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Tele-Immersion Middleware The CAVERNsoft G 2 Toolkit • G 2 is C++ toolkit for building Tele-Immersive applications with special emphasis on networking • Networking: – – – UDP, TCP, Multicast, HTTP. UDP reflector and multicast bridge. TCP reflector. Remote procedure calls. 32 and 64 bit Remote file I/O. Parallel 32 & 64 bit TCP socket striping for high throughput data delivery. – FEC library. – Client/Server distributed shared memory persistent database. – Threading, Mutual Exclusion. – Built-in Instrumentation of networking services. – Qo. S via GARA and MCSP underway. Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Tele-Immersion Middleware The CAVERNsoft G 2 Toolkit • • • Audio streaming. Articulated Avatars. VR navigation. VR menus. Speech recognition with IBM Via. Voice. Collaborative application shell to jumpstart development. Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

TIDE • Teleimmersive Data Explorer (TIDE) • In collaboration with National Center for Data Mining • General framework for collaborative visualization of massive data-sets • Current data-set is ozone data from NOAA Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

CIBRView • Collaborative Image Based Rendering Viewer (CIBRview) • In collaboration with Wes Bethel and Steve Lau at Lawrence Berkeley Lab • Accesses volume data 512 x 256 x 256 frames ~ 40 Gig data-sets • Generates image slices that are distributed to collaborating clients. • Sent about 500 slices/files from Chicago to Japan Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Virtual Harlem University of Missouri Virtual Harlem Reconstruction of Harlem during the Harlem Renaissance 1920 -40 Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Earthquake Hypocenters Space Physics & Aeronomy Research Collaboratory (U of Michigan) A demonstration at Telecom 2000 and SC 2000 between Israel, Dallas, Chicago, Michigan Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Network Visualization Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Qo. S Internet Monitoring Tool Qo. SIMoto Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

STAR TAP Network Visualization Electronic Visualization Laboratory (EVL) University of Illinois at Chicago

Future Work • Diff. Serv and RSVP from EVL to CERN in collaboration with NWU • Reliable UDP for high throughput bulk data transmission • Integrated Collaboratory for Analysing Networks (i. CAN): i. CAN-Monitor, i. CAN-Visualize, i. CAN-Manage, i. CAN-Active Test, i. CAN-Collaborate Electronic Visualization Laboratory (EVL) University of Illinois at Chicago