Cross-layer Visibility as a Service Ramana Rao Kompella

Cross-layer Visibility as a Service Ramana Rao Kompella Albert Greenberg, Jennifer Rexford Alex C. Snoeren, Jennifer Yates 1

Layering in the current Internet OVERLAYS MPLS IP Ethernet Optics Fiber-spans Fiber 2

Layering is a mixed blessing Ø Layering allows us to contain complexity Ø Each layer evolves independently without affecting any other layer Ø Allows us to focus on one layer at a time Ø There associated challenges too… Ø Routine operational tasks need associations across layers Ø Example: mapping an IP link to optical circuit, overlay link to an IP path Ø Lack of accurate cross-layer associations can affect the reliability of the network 3

Intended planned maintenance Seattle Planned maintenance on optics Boston Denver San Francisco Chicago New York St Louis Los Angeles Dallas Orlando 4

Intended planned maintenance Ø Optical component is on circuit id A LA to San Traffic from LA to Ø Lookup database to Francisco link is Dallas is rerouted congested map circuit id A to IP via Denver link Planned maintenance can Denver induce faults Ø Due to mis-association, if San Francisco accurate associations are incorrectly maps it to LA not maintained to Dallas Los Angeles Ø Increase OSPF weight Dallas High on LA to Dallas link OSPF Ø Disconnect component weight Ø Causes failure X 5

Customer Fault Tolerance New York Internet Shared optical element Customer diversity Customer in NJ information requires INTRA-CARRIER Philadelphia accurate cross-layer DIVERSITY Sprint associations, sometimes Level across 3 domains New York Internet INTER-CARRIER DIVERSITY Going through same conduit or Holland tunnel ? Customer in NJ 6

Fault diagnosis Seattle Because of a bug, IP forwarding path changed, but MPLS did not ! Boston Denver San Francisco Los Angeles Chicago Diagnosing faults requires accurate cross-layer associations. St Louis X Dallas New York Orlando MPLS circuit between LA and New York What happened ? !! 7

Why is it hard ? Ø Can’t the operators maintain associations in a centralized fashion ? Ø Maintain database as links are provisioned Ø Update as and when interfaces are re-homed Ø Hard due to flux in topology Ø Churn because of dynamic topology changes Ø Human errors during re-homing interfaces Ø Operational realities – separation of concerns 8

How it is done today ? Ø A combination of non-standard databases Ø Human-generated inventory data Ø Measurement data obtained from probes Ø Configuration state from network elements Ø Policies implemented in network elements Ø Higher complexity and overhead Ø No compatibility across ASes Ø Difficult to evolve a network Ø Difficult to integrate two networks after acquisition Ø Difficult to incorporate third-party tools 9

Why not concentrate on restoration? Ø Advantages of lower-layer restoration Ø Hides lower-layer failures from impacting upper layers Ø Obviates to some extent need for cross-layer visibility Ø Cross-layer visibility still important Ø Lower-layer restoration more expensive than IP restoration Ø Subtle performance changes (e. g. , RTT) need diagnosis 10

Why not fatten the interfaces ? Ø Fattening interfaces to make layers aware of the entire topologies above and below Ø Layers discover and propagate mappings automatically Ø Management system can query the network to obtain mappings Ø Fattening results in high complexity Ø Interoperability is a big challenge – long design and test cycles Ø Wider interfaces impact security 11

Architecture for cross-layer visibility BOW-TIE OVERLAYS Backbone planning MPLS Cross-layer Policy Server IP Ethernet Optics DB Fiberspans Ping Trace-route Customer diversity Backbone maintenance Fault diagnosis Fiber IP HOUR-GLASS MANAGEMENT APPLICATIONS 12

Standardize what goes in ! OVERLAYS Standardize what goes in (e. g. IP topologies) MPLS IP Optics FIBER, FIBERSPAN Facilitates interaction between ISP policy servers AS 1 OVERLAYS MPLS IP Optics FIBER, FIBERSPAN AS 2 13

Advantages of the bow-tie Ø Topology, routing information and other associations can be queried for maintenance, diversity, and fault diagnosis Ø Cooperation across ASes to present better visibility across domains Ø Policies easily enforced through the server Ø Lower overhead on network elements Ø Caching of common queries possible Ø Historical questions can be answered 14

Evolution path to improve accuracy Ø A lot of room for improvement Ø Architecture accommodates evolution so that accuracy can be improved over time Ø Evolution path for individual layers Ø Fiber & Fiber-spans Ø Optical components Ø IP links Ø MPLS and overlay paths 15

Fiber & Fiberspans Ø Automated mechanisms [sebos 02] FIBER GPS OPTICAL TAPS / RFID DB Ø Inject labels through fibers or use RFID Ø GPS to determine the location of fibers Ø Transmit this information to the DB Ø More coverage results in better accuracy but expensive FIBER 16

Optical components Ø Manual mechanisms Ø Basic consistency checks Ø Automatic correlation mechanisms such as [kompella 05 nsdi] to output errors Ø Automatic mechanisms Ø Neighbor discovery for active optical devices Ø Configuration state from “intelligent” optical networks (that support dynamic restoration) 17

Optical components Configuration state during restoration Neighbor discovery through periodic broadcasts at optical layer Intelligent Optical Network ROUTER B ROUTER A DB 18

Other layers Ø IP layer Ø Periodically obtain configuration information to construct topology Ø Automatically collect up/down messages to provide up-to-date view Ø MPLS and overlay paths Ø Static paths obtained from configuration Ø Dynamic paths obtained by monitoring signaling messages 19

Summary Ø Accurate associations critical to many operational tasks Ø A bow-tie architecture for cross-layer visibility Ø Provides the cross-layer associations as a service to various applications Ø Allows better cooperation among ASes through standardizing what goes into the database Ø Policy controlled export of these associations Ø Lower overhead on network elements Ø Allows for innovation while containing complexity 20

Future research directions Ø Design automated mechanisms at each layer to improve cross-layer visibility Ø What frequency should information be obtained? Ø How do we resolve conflicts (minimal edits) in the database? Ø Identify higher-level models that we need to standardize Ø Devise incentives for cooperation among ASes Ø Define a language to specify policies 21

Questions ? 22