QBone Service Specification and TE Early Diff. Serv Deployment Experiences and Reduced Expectations Ben Teitelbaum <ben@internet 2. edu> January 25 th, 2001 TEQUILA Workshop on Internet Design for SLS Delivery
“The Holy Grail of computer networking is to design a network that has the flexibility and low cost of the Internet, yet offers the end-toend quality of service guarantees of the telephone network. ” - S. Keshav Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 2
The Paradox of Internet Qo. S Internet’s success largely due to its lack of Qo. S! Best-Efforts: scalability on steroids • Routers: complexity of forwarding very low • L 2: minimal requirements • Providers: “magic”-free provisioning and low user expectations • Developers: E 2 E transport and simplicity of Berkeley socket interface Qo. S complicates everything (costs), but opens many opportunities (benefits) • Developers: new apps exploit the ubiquity of the Internet • Providers: may make more efficient use of the network Big Question: How can we build upon Diff. Serv without compromising scalability? Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 3
How We Got Here (short version) • Began chanting: “enable advanced applications, …” • Assessed requirements “What do you want? ” “What can you give us? ” • Recommended Diff. Serv • Selected “Premium” service to meet demands of loss/jitter sensitive apps • Charted QBone initiative • Specified QBone architecture • Now proceeding to implement it and tweak the architecture deployment Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 4
QBone Architecture A Service: QBone Premium Service • Built on Expedited Forwarding (EF) (RFC 2598) • Assurance: near-zero loss & low, bounded jitter for marked traffic conforming to a specified peak rate – a. k. a. “virtual leased line”, “virtual wire” Reservation Setup Protocol • Now: long-lived, manual setup • Proposed: SIBBS protocol between QBone domains; RSVP end-to-end between hosts QBone Measurement Architecture • Uniform collection of Qo. S metrics • Uniform dissemination interface Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 5
QBone Measurement Architecture 1/2 Collection • metrics, EF and BE. . . • Active metrics (paths) – One-way delay-variation – One-way loss – Traceroutes – e. g. IPPM Surveyors • Passive metrics (interfaces) – Load – EF reservation load – Discards (suggested) – Link bandwidths (suggested) – e. g. OCx. Mon, RTFM, MIBs Active Measurements AM node MIB-based statistics Boundary Router Intra-Domain Premium Path Inter-Domain Premium Path PM node Passive Measurements QBone Domain 2 QBone Domain 1 QBone Domain 3 Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 6
QBone Measurement Architecture 2/2 Dissemination • Standard URL query syntax: label : : = <alphanum> { <alphanum> } router : : = label"-ROUTER” probe : : = label"-PROBE” sniffer : : = label "-SNIFFER” host : : = router | probe | sniffer path : : = host "/" host PHB : : = "BE" | "EF” metric : : = "LOSS" | "ONEWAY" | "PING" | "IPDV" | "LOAD" | "TRACEROUTE" | "COMMITMENT" | "RESERVATION" year : : = digit month : : = "01" | "02" |. . . | "12” day : : = "01" | "02" |. . . | "31” YYYYMMDD : : = year month day aggregation : : = <unsigned_integer> prefix : : = “http” | “ftp” | <other> query : : = prefix ”: //" path "/" PHB "/" metric "/” YYYYMMDD "/" metric "/" aggregation • whois server to learn canonical names for QBone domains, routers, sniffers, etc Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 7
Simple Interdomain Bandwidth Broker Signaling (SIBBS) Basic Idea • Simple domain-to-domain reservation request/response protocol to signal changes in the aggregate reservations of globally well-known services Design Goals • Simple & extensible • Bootstrap wide-area experimentation with Qo. S-needy apps, while providing hooks for clouds to experiment with various Qo. S TE approaches • Integrate with end-to-end signaling capabilities of hosts Status • Protocol draft nearing completion (see: http: //qbone. internet 2. edu/bb/) • New protocol for now; looking at mapping to COPS Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 8
SIBBS: Basic Reservation Setup Web interface RSVP DIAMETER COPS RYO CLI SNMP COPS Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 9
Virtual Peerings & Core Tunnels BB SIBBS BB “Virtual” SIBBS peering BB BB SIBBS BB Src “Core Tunnel” RTR Dest RTR Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 10
SIBBS Approach to SLSs Basic approach: • Define services and reservations, but punt on SLSs Services • Globally specified, standardized, and named • Include many parameters from draft-tequila-sls (scope, traffic envelope, performance guarantees, etc) Reservations • Identify a service ID • Specify service parameters SLSs • Each SLS along a reservation path should be regarded as a black box that answers admissions questions and extends reservations across the peerings Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 11
Simple Interdomain Bandwidth Broker Signalling (SIBBS) BB BB BB S LS 1 -2 DS Domain 1 S LS 2 -3 R H R R DS Domain 2 H Control Data R DS Domain 3 Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 12
QBone E 2 E Picture Campus A Giga. Po. P A Campus B Backbone Campus C Key X Kbps of QPS from hither to………. . yon Giga. Po. P B Campus D Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 13
Abilene Topology January, 2001 • 47 connectors • 183 participants • 34 connections to 20 peer networks Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 14
Developing International Peerings STARTAP Abilene v. BNS ESnet DREN NISN SEATTLE Abilene ESnet CA*net 3 AARnet CERNET IUCC GEMNET APAN Renater CERN SINET Singaren Tanet NYC (Telehouse 25 Broadway) NYC (60 Hudson) LA Abilene Singaren CA*net 3 SURFnet MIRnet NORDUnet REUNA CUDI HARNET SINET Courtesy: Linda Winkler, STAR TAP Abilene ESnet v. BNS Miami DFN INFN DANTE CA*net 3 CERN SINET JAnet SURFnet NORDUnet Argentina Brazil Chile Columbia Brazil Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 15
Abilene Load Snapshot Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 16
APS Participation Goal: • Make APS a reference implementation of the QBone architecture Current Participants • MAGPI (U. Penn) • i. CAIR • PSC (Penn State) • OARNet (Ohio State) • NASA EOS • ANL • UIUC • DOE Science GRID (peering transit network) Others in the wings • TF-NGN (through DANTE) • MIRnet • Various other international Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 17
Initial Engineering Plan (obsolete) Sweetwater Midland Odessa Pecos Measurement (Surveyor + SNMP + HTTP) Edge Policing (“Firehose” CAR) Manual Setup (Whiteboard + CLI) EF Core Forwarding (MDRR) EF Edge Forwarding (MDRR) Automated Setup (BB) Shaping (GTS) Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 18
APS Measurements Status: • Collecting EF/BE loads and CAR conform/exceed stats • Not currently monitoring IPDV, but Abilene Surveyor nodes now OC-3 connected and operational • Ohio-ITEC hosting APS measurements and QBonewide whois server Near Future: • IPDV along edge-to-edge QBone paths • Abilene Surveyor timing improvements – Better NTP – New CDMA timing sources (can't get GPS in Qwest POPs) • Collection of AS-level traffic matrices Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 19
Committed Access Rate (CAR) Classifies traffic based on certain matching criteria (interface, DSCP, or ACL) and meters it to a leaky bucket traffic profile Depending on metering result, different actions applied (drop, transmit, set DSCP, …) Syntax: rate-limit {input | output} [access-group [rate-limit] acl-index] bps burst-normal burst-max conform-action exceed-action Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 20
CAR Experience For the most part, CAR is exactly what the Diff. Serv doctor ordered However, there are some limitations… • PPS performance cost • Quirky constraints on token bucket depths • Not easy to do "virtual trunk" style classification Problem: we want. . . "firehose" But what we get is… "virtual trunk" Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 21
Abilene Architecture Limitation: “Porous” Edge Problem Do. Qo. S problem with current architecture MDRR (EF forwarding) on interior interfaces easily subverted by unpoliced connectors Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 22
How to “Crisp” the Edge? Problem: EF requires that all connectors be policed Solution: • Short term: Stochastically detect illegal EF traffic with Net. Flow and/or OCx. Mon passive monitoring; giga. Po. P would be asked to police • Longer term: Wait for E 3 edge cards, deploy them aggressively, and police everywhere • Road not taken: Re-write all non-participant traffic with DSCP 000000 (using PIRC hack); need to pass some DSCP values (reason why coming up…) Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 23
Looking Forward: Resource Accounting and Admissions Problem: how to account for link EF capacity and commitment in routed network? Potential solution: Diff. Serv-Aware MPLS-TE • Basic idea: – Edge-to-edge MPLS tunnels – LSP setup constrained by available EF bandwidths – OSPF augmented to carry Qo. S link state attributes – See draft-lefaucheur-diff-te-reqts • “Solves” Diff. Serv admissions problem • Does not solve general Diff. Serv provisioning problem • Status: completed extensive lab test of Cisco’s implementation of this idea; considering field trial Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 24
Current Engineering Plan Sweetwater Midland Odessa Pecos Measurement (Surveyor + SNMP + HTTP + whois + traffic matrices) Edge Policing (CAR + QPPB) Manual Setup (Whiteboard + CLI) EF Core Forwarding (MDRR) EF Edge Forwarding (MDRR) Automated Setup (SIBBS? + DSTE? ) Shaping (GTS) Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 25
QPS Status (Good News) The Good News: • With enough elbow grease, E 2 E services can be built • Diff. Serv functionality in most modern routers • Many hosts now support Qo. S signaling (RSVP) • Numerous testbed trials • Partial implementations in ESnet and Abilene will form nucleus of QBone Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 26
SC 2000 Interdomain Qo. S Demo November 6 -9, 2000 Premium service over two wide-area paths • LBNL-ESnet-Abilene-SCinet-Internet 2 booth • Stanford-Cal. REN 2 -Abilene-SCinet-Internet 2 booth Congestion induced at multiple points CD-quality interactive audio application shown with/without Qo. S SC 2000 Network Challenge Winner: Award for "Most Captivating and Best Tuned" Demo Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 27
Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 28
QPS Status (Bad News) The Bad News: • Router vendor hype/reality mismatch • Deploying QPS “requires upgrading the world” • Low demand for QPS (app/net chicken/egg dynamic) • Elbow grease in short supply in most campus IT shops Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 29
Bottom Line Progress on QPS will continue to be slow Internet 2 will continue with a QPS work program in 2001… • Deploy incrementally where it counts • Revise/complete architecture (joint work with TF-NGN) • Work to build Qo. S-sensitive application user communities who understand their needs . However, QPS will no longer be the only Internet 2 Qo. S activity • Need to reap lower hanging fruit from DS (I 2 SS) • Much stronger push on measurements/monitoring • Big Internet 2 E 2 E performance initiative ramping up Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 30
E 2 E Performance and Measurements Internet 2 E 2 E Performance Initiative • Typical BE TCP throughputs often much less than one would expect • Common problems – Broken TCP stacks – Ethernet auto-negotiation failures • Evolving attributes of initiative – Performance Emergency Response Teams (PERTs) – Sharper tools for measurement, monitoring, and analysis Measurement arsenal • WEB 100 (Mathis et al. ) • Open Internet 2 measurement architecture (à la QBone) • Open source and protocols for one-way delay measurement • Application-level performance fault-isolation (reflector infrastructure) Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 31
Internet 2 Scavenger Service Basic idea • A lower priority class of best-effort • Voluntary marking hints to network that degraded service is OK (think of a "nice" for the network) Intended uses • Non-time-critical traffic (e. g. server-to-server NNTP, anonymous FTP, network backups) • Bulk data transfers using TCP • Non-mission applications (e. g. Napster, games, etc) • New kinds of distributed applications that attempt to use idle network capacity Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 32
I 2 SS Service Specification Rigorously defining the E 2 E I 2 SS is difficult! • Hoping to define relative to best effort I 2 SS traffic indicated by DSCP 001000 • Modification of class selector PHB • Note that the I 2 SS codepoint has global significance I 2 SS domain requirements • Traffic leaving must be marked I 2 SS, if it entered so Router requirements (all SHOULDs) • Forward I 2 SS independently giving it a lower probability of timely forwarding OR forward in same manner as BE • Offer I 2 SS a very small minimum departure rate (unlike BH PDB) • Offer I 2 SS all un-used bandwidth Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 33
For more information. . . Internet 2 Home: • http: //www. internet 2. edu/ Internet 2 Qo. S Working Group Home: • http: //www. internet 2. edu/wg/qos/ QBone Home: • http: //qbone. internet 2. edu/ Abilene Premium Service Home: • http: //www. internet 2. edu/abilene/qos/ Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 34
Service Specification and TE for the QBone—Amsterdam (January 25 th, 2001) 35
Скачать презентацию QBone Service Specification and TE Early Diff Serv
742ce01dbd3dd82a05e7d53e7894b855.ppt