How is the Internet Performing Les Cottrell

How is the Internet Performing? Les Cottrell – SLAC Lecture # 2 presented at the 26 th International Nathiagali Summer College on Physics and Contemporary Needs, 25 th June – 14 th July, Nathiagali, Pakistan Partially funded by DOE/MICS Field Work Proposal on Internet End-to-end Performance Monitoring (IEPM), also supported by IUPAP 1

Overview • Internet characteristics – packet sizes, protocols, hops, hosts … – complexity, flows, applications • Application requirements • How the Internet worldwide is performing as seen by various measurements and metrics • How well are requirements met? • Many sources of measurements CAIDA/Skitter Matrix Ping. ER/IEPM Surveyor 2

Packet size • primarily 3 sizes: Measured Feb 2000 at Ames Internet e. Xchange Cu, mulative probability % – close to minimum=telnet and ACKs, 1500 (max Ethernet payload, e. g. FTP, HTTP); ~ 560 Bytes for TCP implementations not using max transmission unit discovery Mean ~ 420 Bytes, median ~ 80 Bytes Packets ~ 84 M packets, < 0. 05% fragmented Bytes Packet size (bytes) 3

Internet protocol use • There are 3 main protocols in use on the Internet: SLAC protocol flows ICMP TCP Flows/10 min In Out – UDP (connectionless datagrams, best effort delivery), – TCP (Connection oriented, “guaranteed” delivery) – ICMP (Control Message protocol) TCP dominates today UDP Time Feb-May 2001 4

Web use characteristics • Size of web objects varies from site to site, server to server and by time of day. – Typical medians vary from 1500 to 4000 bytes • Also varies by object type, e. g. medians for – movies few 100 KB to MBs, postscript & audio few 100 KB – text, html, applets and images few thousand KB Big peaks for error messages Bytes 5

Hops • Hop counts seen from 4 Skitter sites (Japan, S. Cal, N. Cal, E. Canada, i. e. 10 -15 hops on average RTT Weak RTT dependence on hop count 95% 50% 5% Hops Hop Count 6

Autonomous Systems (AS) Disperson • Color indicates the AS responsible for the router at the hop, height is number of probes for that route • Seen by Skitter at Palo Alto US (F root name server) Hop number 7

Country dispersion • Seen from Japan • After 3 to 4 hops most goes to US. Probes – In some cases goes US & back to jp – Some goes to UK & onto other European countries Hops 8

Route maps • Simple routes from TRIUMF, Canada to several sites already gets quite complex TRIUMF DESY UW SLAC FNAL CERN KEK 9

Getting more complex • Ping. ER Beacon sites in US seen from TRIUMF, Vancouver (from Andrew Daviel, TRIUMF) 10

Connections by country NL IT US Unknown RU UK JP DE 11

Richness of connectivity • Angle = longitude of AS HQ in whois records • Radius=1 -log(outdegree(AS)+1)/(maxoutdegree + 1) – Outdegree = number of next Hops As’ accepting traffic • Deeper blue & red more connections • All except 1 of top 15 AS’ are in US, exception in Canada • Few links between ISPs in Europe and Asia 12

Hosts by regions • Jan 2001, 109 Million hosts – Source: Internet Software Consortium (www. isc. org) • see web site also for hosts/population Notes: • Many. com are in N. America • S. Asia = in (36 K), pk (6 K), lk, bd • E. Asia=jp, cn, my, sg, tw, hk, th, id, bn, mm • Mid East=il, kw, lb, ae, tr, sa • TLDs with hosts~238 • Total TLDs~258 13

Backbone utilization Shows utilization of I 2/Abilene backbone links, NB Backbone < 30% loaded Most losses at exchange points & edges 14

Flow sizes SNMP Real A/V AFS file server Heavy tailed, in ~ out, UDP flows shorter than TCP, packet~bytes 75% TCP-in < 5 k. Bytes, 75% TCP-out < 1. 5 k. Bytes (<10 pkts) UDP 80% < 600 Bytes (75% < 3 pkts), ~10 * more TCP than UDP Top UDP = AFS (>55%), Real(~25%), SNMP(~1. 4%) 15

Flow lengths TCP outbound flows Measured by Netflows tied off at 30 mins Active time in secs • 60% of TCP flows less than 1 second • Would expect TCP streams longer lived – But 60% of UDP flows over 10 seconds, maybe due to heavy use of AFS at SLAC – Another (CAIDA) study indicates UDP flows are shorter 16 than TCP flows

Typical Internet traffic by Application • CERFnet link • Dominated by WWW (http) Mail WWW FTP Real. Audio 17

SLAC Traffic profile Mbps in SLAC offsite links: OC 3 to ESnet, 1 Gbps to Stanford U & thence OC 12 to I 2 OC 48 to NTON HTTP Profile bulk-data xfer dominates Last 6 months Mbps out iperf 2 Days FTP SSH bbftp 18

SLAC Internet Application usage Ames IXP: approximately 60 -65% was HTTP, about 13% was NNTP Uwisc: 34% HTTP, 24% FTP, 13% Napster 19

What does performance depend on? • End-to end internet performance seen by applications depends on: – round trip times – packet loss – jitter – reachability – bottleneck bandwidth – implementation/configurations – application requirements • Data transmitted in packets 20

Application requirements • Based on ITU Y 1541 • The Vo. IP loss of 10^-3 used to be 0. 25 but that assumed random flat loss – actual loss is often bursty • Tail drop in routers • Sync loss in circuits, bridge spanning tree reconfiguration, route changes 21

RTT from ESnet to Groups of Sites RTT ~ distance/(0. 6*c) + hops * router delay Router delay = queuing + clocking in & out + processing ITU G. 114 300 ms RTT limit for voice 20%/year 22

RTT Region to Region OK White 0 -64 ms Green 64 -128 ms Yellow 128 -256 ms NOT OK Pink 256 -512 ms Red > 512 ms OK within regions, N. America OK with Europe, Japan 23

Brazil 300 ms E. Coast Europe & S. America RTT (ms) E. Coast US W. Coast US Frequency RTT from California to world Europe 0. 3*0. 6 c 300 ms RTT (ms. ) Longitude (degrees) Source = Palo Alto CA, W. Coast 24 Data from CAIDA Skitter project

RTT(ms) RTT from Japan to world Longitude Seen from Japan 25

• Gives quality measure • Seen from San Diego, US Skitter • Steeper = less jitter, i. e. better • Small values better Cumulative % Cumulative RTT distributions RTT ms 26

Routes are not symmetric Advanced to U. Chicago RTT ms • Min, 50% & 90% RTT measured by Surveyor • Notice big differences in RTTs • May be due to different paths in the 2 directions or to different loading U. Chicago to Advanced 27

Loss seen from US to groups of Sites 50% imp rove men t / ye ar ETSI DTR/TIPHON-05001 V 1. 2. 5 threshold for good speech 28

Detailed example of improvements Increase of bandwidth by factor of 460 in 6 years, more than kept pace - factor of 50 times improvement in loss Note valleys when students on vacation 29

Loss to world from US Using year 2000, fraction of world’s population/country from www. nua. ie/surveys/how_many_online/ 30

How are the U. S. Nets doing? In general performance is good (i. e. <= 1%) ESnet holding steady, still better than others Edu (v. BNS/Abilene) &. com improving 31

Losses for 28 days in May 2001 % Loss DNS Internet WWW ISP • Measured by MIDS to 583 DNS services, 383 Web services, 1367 Internet (ping) hosts, & 1225 ISPs (routers) 32

Losses between Regions 33

Bulk throughput • Important for long TCP flows where we want to copy large amounts of data from one site to another in a relatively short time, e. g. file transfer • Depends on RTT, loss, timeouts, window sizes 34

Throughput quality TCPBW < 1/(RTT*sqrt(loss)) Note E. Europe catching up Macroscopic Behavior of the TCP Congestion Avoidance Algorithm, Matthis, Semke, Mahdavi, Ott, Computer Communication Review 27(3), July 1997 35

Throughput also depends on window • Optimal window size depends on: – Bandwidth end to end, i. e. min(BWlinks) AKA bottleneck bandwidth – Round Trip Time (RTT) – For TCP keep pipe full • Window (sometime called pipe) ~ RTT*BW – Can increase bandwidth by orders of magnitude Src Rcv • If no loss Throughput ~ Window/RTT CK A t = bits in packet/link speed RTT 36

“Jitter” from N. America to W. Europe “Jitter” = IQR(ipdv), where ipdv(i) =RTT(i) – RTT(i-1) 214 pairs ETSI: DTR/TIPHON-05001 V 1. 2. 5 (1998 -09) good speech < 75 ms jitter 37

“Jitter” between regions ETSI: DTR/TIPHON-05001 V 1. 2. 5 (1998 -09) 75 ms=Good 125 ms=Med Jitter varies with loading 225 ms=Poor 38

SLAC-CERN Jitter ETSI/TIPHON delay jitter threshold (75 ms) 39

Reachability Within N. America, & W. Europe loss, RTT and jitter is acceptable for Vo. IP But what about reachability 40

Reachability – Outage Probability Surveyor probes randomly 2/second Measure time (Outage length) consecutive probes don’t get through Heavy tailed outage lengths (packet loss not Poisson) http: //www-iepm. slac. stanford. edu/monitoring/surveyor/outage. html 41

Europe seen from U. S. Monitor site Beacon site (~10% sites) HENP country Not HENP & not monitored 200 ms 1% loss 7% loss 650 ms 10% loss 42

Asia seen from U. S. 10% loss 3. 6% loss 0. 1% loss 250 ms 640 ms 450 ms 43

Latin America, Africa & Australasia 4% Loss 170 ms 220 ms 700 ms 2% Loss 350 ms 44

Animated monthly 2000 20% loss 200 ms RTT Big is Bad 20% unreachable 45

RTT worldwide from the Matrix 46

More Information • IEEE Communications, May 2000, Vol 38, No 5, pp 120 -159 • IEPM/Ping. ER home site – www-iepm. slac. stanford. edu/ • CAIDA/Skitter home site – www. caida. org/home/ • Matrix Net home site – www. matrix. net/index. html • Surveyor home site: – www. advanced. org/csg-ippm/ 47