Internet Topology COS 461 Computer Networks Spring 2006

Internet Topology COS 461: Computer Networks Spring 2006 (MW 1: 30 -2: 50 in Friend 109) Jennifer Rexford Teaching Assistant: Mike Wawrzoniak http: //www. cs. princeton. edu/courses/archive/spring 06/cos 461/ 1

Returning the Midterm Exam • Exam scoring break down – Range: 70 -100 – Average: 89 – Median: 92 • See the course Web site – Exam – Answer key 2

Goals of Today’s Lecture • Internet’s two-tiered topology – Autonomous Systems, and connections between them – Routers, and the links between them • AS-level topology – Autonomous System (AS) numbers – Business relationships between ASes • Router-level topology – Points of Presence (Po. Ps) – Backbone and enterprise network topologies • Inferring network topologies – By measuring paths from many vantage points 3

Internet Routing Architecture • Divided into Autonomous Systems – Distinct regions of administrative control – Routers/links managed by a single “institution” – Service provider, company, university, … • Hierarchy of Autonomous Systems – Large, tier-1 provider with a nationwide backbone – Medium-sized regional provider with smaller backbone – Small network run by a single company or university • Interaction between Autonomous Systems – Internal topology is not shared between ASes – … but, neighboring ASes interact to coordinate routing 4

Autonomous System Numbers AS Numbers are 16 bit values. Currently just over 20, 000 in use. • • • Level 3: 1 MIT: 3 Harvard: 11 Yale: 29 Princeton: 88 AT&T: 7018, 6341, 5074, … UUNET: 701, 702, 284, 12199, … Sprint: 1239, 1240, 6211, 6242, … … 5

AS Topology • Node: Autonomous System • Edge: Two ASes that connect to each other 4 3 5 2 7 6 1 6

What is an Edge, Really? • Edge in the AS graph – At least one connection between two ASes – Some destinations reached from one AS via the other d d AS 1 Exchange Point AS 2 AS 3 7

Interdomain Paths Path: 6, 5, 4, 3, 2, 1 4 3 5 2 1 7 6 Web server Client 8

Business Relationships • Neighboring ASes have business contracts – How much traffic to carry – Which destinations to reach – How much money to pay • Common business relationships – Customer-provider E. g. , Princeton is a customer of AT&T E. g. , MIT is a customer of Level 3 – Peer-peer E. g. , Princeton is a peer of Patriot Media E. g. , AT&T is a peer of Sprint 9

Customer-Provider Relationship • Customer needs to be reachable from everyone – Provider tells all neighbors how to reach the customer • Customer does not want to provide transit service – Customer does not let its providers route through it Traffic to the customer Traffic from the customer d provider advertisements provider traffic customer d customer 10

Peer-Peer Relationship • Peers exchange traffic between customers – AS exports only customer routes to a peer – AS exports a peer’s routes only to its customers – Often the relationship is settlement-free (i. e. , no $$$) Traffic to/from the peer and its customers advertisements peer d traffic peer 11

Princeton Example • Internet: customer of AT&T and USLEC • Research universities/labs: customer of Internet 2 • Local residences: peer with Patriot Media • Local non-profits: provider for several non-profits AT&T USLEC Internet 2 peer Patriot 12

AS Structure: Tier-1 Providers • Tier-1 provider – Has no upstream provider of its own – Typically has a national or international backbone – UUNET, Sprint, AT&T, Level 3, … • Top of the Internet hierarchy of 12 -20 ASes – Full peer-peer connections between tier-1 providers 13

Efficient Early-Exit Routing • Diverse peering locations Customer B – Both costs, and middle • Comparable capacity at all peering points Provider B – Can handle even load • Consistent routes multiple peering points Early-exit routing – Same destinations advertised at all points – Same AS path length for a destination at all points Provider A Customer A 14

AS Structure: Other ASes • Tier-2 providers – Provide transit service to downstream customers – … but, need at least one provider of their own – Typically have national or regional scope – E. g. , Minnesota Regional Network – Includes a few thousand of the ASes • Stub ASes – Do not provide transit service to others – Connect to one or more upstream providers – Includes vast majority (e. g. , 85 -90%) of the ASes 15

Characteristics of the AS Graph • AS graph structure – High variability in node degree (“power law”) – A few very highly-connected ASes – Many ASes have only a few connections CCDF 1 All ASes have 1 or more neighbors 0. 1 0. 01 Very few have degree >= 100 0. 001 1 10 1000 AS degree 16

Characteristics of AS Paths • AS path may be longer than shortest AS path • Router path may be longer than shortest path 2 AS hops, 8 router hops d s 3 AS hops, 7 router hops 17

Intra-AS Topology • Node: router • Edge: link 18

Hub-and-Spoke Topology • Single hub node – Common in enterprise networks – Main location and satellite sites – Simple design and trivial routing • Problems – Single point of failure – Bandwidth limitations – High delay between sites – Costs to backhaul to hub 19

Princeton Example • Hub-and-spoke – Four hub routers and many spokes • Hub routers – Outside world (e. g. , AT&T, USLEC, …) – Dorms – Academic and administrative buildings – Servers 20

Simple Alternatives to Hub-and-Spoke • Dual hub-and-spoke – Higher reliability – Higher cost – Good building block • Levels of hierarchy – Reduce backhaul cost – Aggregate the bandwidth – Shorter site-to-site delay … 21

Backbone Networks • Backbone networks – Multiple Points-of-Presence (Po. Ps) – Lots of communication between Po. Ps – Accommodate traffic demands and limit delay 22

Abilene Internet 2 Backbone 23

Points-of-Presence (Po. Ps) • Inter-Po. P links – Long distances – High bandwidth Inter-Po. P Intra-Po. P • Intra-Po. P links – Short cables between racks or floors – Aggregated bandwidth • Links to other networks – Wide range of media and bandwidth Other networks 24

Where to Locate Nodes and Links • Placing Points-of-Presence (Po. Ps) – Large population of potential customers – Other providers or exchange points – Cost and availability of real-estate – Mostly in major metropolitan areas • Placing links between Po. Ps – Already fiber in the ground – Needed to limit propagation delay – Needed to handle the traffic load 25

Customer Connecting to a Provider 1 access link Provider 2 access routers Provider 2 access links Provider 2 access Po. Ps 26

Multi-Homing: Two or More Providers • Motivations for multi-homing – Extra reliability, survive single ISP failure – Financial leverage through competition – Better performance by selecting better path – Gaming the 95 th-percentile billing model Provider 1 Provider 2 27

Shared Risks • Co-location facilities (“co-lo hotels”) – Places ISPs meet to connect to each other – … and co-locate their routers, and share space & power – E. g. , 32 Avenue of the Americas in NYC • Shared links – Fiber is sometimes leased by one institution to another – Multiple fibers run through the same conduits – … and run through the same tunnels, bridges, etc. • Difficult to identify and accounts for these risks – Not visible in network-layer measurements – E. g. , traceroute does not tell you links in the same ditch 28

Learning the Internet Topology • Internet does not have any central management – No public record of the AS-level topology – No public record of the intra-AS topologies • Some public topologies are available – Maps on public Web sites – E. g. , Abilene Internet 2 backbone • Otherwise, you have to infer the topology – Measure many paths from many vantage points – Extract the nodes and edges from the paths – Infer the relationships between neighboring ASes 29

Inferring an Intra-AS Topology • Run traceroute from many vantage points – Learn the paths running through an AS – Extract the hops within the AS of interest 1 169. 229. 62. 1 inr-daedalus-0. CS. Berkeley. EDU 2 169. 229. 59. 225 soda-cr-1 -1 -soda-br-6 -2 3 128. 32. 255. 169 vlan 242. inr-202 -doecev. Berkeley. EDU 4 128. 32. 0. 249 gig. E 6 -0 -0. inr-666 -doecev. Berkeley. EDU 5 128. 32. 0. 66 qsv-juniper--ucb-gw. calren 2. net 6 209. 247. 159. 109 POS 1 -0. hsipaccess 1. San. Jose 1. Level 3. net 7 209. 247. 9. 170 8 66. 185. 138. 33 AOL pos 8 -0. hsa 2. Atlanta 2. Level 3. net pop 2 -atm-P 0 -2. atdn. net 9 66. 185. 142. 97 Pop 1 -atl-P 3 -0. atdn. net 10 66. 185. 136. 17 pop 1 -atl-P 4 -0. atdn. net 11 64. 236. 16. 52 www 4. cnn. com 30

Challenges of Intra-AS Mapping • Firewalls at the network edge – Cannot typically map inside another stub AS – … because the probe packets will be blocked by firewall – So, typically used only to study service providers • Identifying the hops within a particular AS – Relies on addressing and DNS naming conventions – Difficult to identify the boundaries between ASes • Seeing enough of the edges – Need to measure from a large number of vantage points – And, hope that the topology and routing doesn’t change 31

Inferring the AS-Level Topology • Collect AS paths from many vantage points – Learn a large number of AS paths – Extract the nodes and the edges from the path • Example: AS path “ 1 7018 88” implies – Nodes: 1, 7018, and 88 – Edges: (1, 7018) and (7018, 88) • Ways to collect AS paths from many places – Mapping traceroute data to the AS level – Measurements of the interdomain routing protocol 32

Map Traceroute Hops to ASes Traceroute output: (hop number, IP) 1 169. 229. 62. 1 AS 25 2 169. 229. 59. 225 AS 25 Berkeley 3 128. 32. 255. 169 AS 25 4 128. 32. 0. 249 AS 25 5 128. 32. 0. 66 AS 11423 Calren 6 209. 247. 159. 109 AS 3356 7 * AS 3356 8 64. 159. 1. 46 AS 3356 9 209. 247. 9. 170 AS 3356 10 66. 185. 138. 33 AS 1668 11 * AS 1668 12 66. 185. 136. 17 AS 1668 13 64. 236. 16. 52 AS 5662 CNN Level 3 AOL 33

Challenges of Inter-AS Mapping • Mapping traceroute hops to ASes is hard – Need an accurate registry of IP address ownership – Whois data are notoriously out of date • Collecting diverse interdomain data is hard – Public repositories like Route. Views and RIPE-RIS – Covers hundreds to thousands of vantage points – Especially hard to see peer-peer edges Sprint AT&T d 1 Harvard ? ? ? Harvard B-school d 2 34

Inferring AS Relationships • Key idea – The business relationships determine the routing policies – The routing policies determine the paths that are chosen – So, look at the chosen paths and infer the policies • Example: AS path “ 1 7018 88” implies – AS 7018 allows AS 1 to reach AS 88 – AT&T allows Level 3 to reach Princeton – Each “triple” tells something about transit service • Collect and analyze AS path data – Identify which ASes can transit through the other – … and which other ASes they are able to reach this way 35

Paths You Should Never See (“Invalid”) Customer-provider Peer-peer two peer edges transit through a customer 36

Challenges of Relationship Inference • Incomplete measurement data – Hard to get a complete view of the AS graph – Especially hard to see peer-peer edges low in hierarchy • Real relationships are sometime more complex – Peer is one part of the world, customer in another – Other kinds of relationships (e. g. , backup and sibling) – Special relationships for certain destination prefixes • Still, inference work has proven very useful – Qualitative view of Internet topology and relationships 37

Conclusions • Two-tiered Internet topology – AS-level topology – Intra-AS topology • Inferring network topologies – By measuring paths from many vantage points • Next class – Vivek Pai guest lecture See reading assignment on the course Web site – Mike Wawrzoniak talking about assignment #2 Start the assignment so you can ask questions • Next week – Intradomain and interdomain routing 38