The SAHARA Project Composition and Cooperation in the

The SAHARA Project: Composition and Cooperation in the New Internet Randy H. Katz, Anthony Joseph, Ion Stoica Computer Science Division Electrical Engineering and Computer Science Department University of California, Berkeley, CA 94720 -1776

Presentation Outline • • • Service Architecture Opportunity SAHARA Project Motivation SAHARA Reference Architecture Mechanisms for Service Composition Summary and Conclusions

Presentation Outline • • • Service Architecture Opportunity SAHARA Project Motivation SAHARA Reference Architecture Mechanisms for Service Composition Summary and Conclusions

Traditional View of Networking • All about protocols and the OSI layers – Protocol details: link-state vs. distance vector, TCP – Protocol layering – Multiaccess technology – Switching and routing – Naming – Error control – Flow control & scheduling – Special topics like multicast and mobility

The New Opportunity • New things you can do inside the network • Connecting end-points to “services” with processing embedded in the network fabric • Not protocols but “agents, ” executing in places in the network • Location-aware, data format aware • Controlled violation of layering necessary! • Distributed architecture aware of network topology • No single technical architecture likely to dominate: think overlays, system of systems

Distributed Service Architectures for Converged Networks • Converged Networks – – Public Switched Telephone Network (PSTN) Internet/Public Switched Data Network (PSDN) Mobile Internet Converged Structure? • Distributed Service Architecture – Services • • “-Ility” connectivity Rich call: new call “features” Infrastructure services: proxies, search, commerce Enablers for distributed apps: event & content distribution

Services in Converged Networks

Services in Converged Networks

New Kind of Communications. Oriented Service Architecture • Emerging, still developing, in a highly heterogeneous environment – Rapid development/deployment of new services & apps – Delivered to radically different end devices (phone, computer, info appliance) over diverse access networks (PSTN, LAN, Wireless, Cellular, DSL, Cable, Satellite) – Exploiting Internet-based technology core: clients/server, applications level routers, TCP/IP protocols, Web/XML formats – Beyond traditional “call processing” model: client-proxy-server plus application-level partitioning – Built upon a new business model being driven by the evolution of the Internet: traditional “managed” networks and services versus emerging “overlay” networks and services structured on top of and outside of the above – Composition via cooperation or brokering to achieve enhanced performance and reliability

Presentation Outline • • • Service Architecture Opportunity SAHARA Project Motivation SAHARA Reference Architecture Mechanisms for Service Composition Summary and Conclusions

Scenario: Service Composition Restaurant Guide Service JAL UI Babblefish Translator NTTDo. Co. Mo Zagat Guide User Tokyo Sprint Salt Lake City User

The “Sahara” Project • • • Service Architecture for Heterogeneous Access, Resources, and Applications

Sahara Research Focus • New mechanisms, techniques for end-to-end services w/ desirable, predictable, enforceable properties spanning potentially distrusting service providers – Tech architecture for service composition & inter-operation across separate admin domains, supporting peering & brokering, and diverse business, value-exchange, accesscontrol models – Functional elements • • • Service discovery Service-level agreements Service composition under constraints Redirection to a service instance Performance measurement infrastructure Constraints based on performance, access control, accounting/billing/settlements • Service modeling and verification

Problems and Solutions “The Network Effect” • Creating and deploying new services – Development and deployment expense • Cost of 3 G licenses and networks • “Even if I had $1 billion and set up 1000 s of locations, I could never in my network have a completely ubiquitous footprint. ” —Sky Dayton, founder of Boingo • Composition, cooperation, overlays • Achieving desirable end-to-end properties – Control of the end-to-end path – Cooperation, peering, overlays (brokering) • Evolving network services – Difficult to change global operational infrastructure – Overlays, cooperation

Internet Connectivity and Processing Cable Modem Premisesbased Access Networks LAN Transit Net Premisesbased Core Networks WLAN Private Peering Transit Net WLAN Operatorbased Cell Regional Internet Datacenter NAP Public Peering Data Voice Analog Transit Net H. 323 RAS H. 323 PSTN DSLAM Data Voice Wireline Regional

Interconnected World: Agile or Fragile? • Baltimore Tunnel Fire, 18 July 2001 – “… The fire also damaged fiber optic cables, slowing Internet service across the country, …” – “… Keynote Systems … says the July 19 Internet slowdown was not caused by the spreading of Code Red. Rather, a train wreck in a Baltimore tunnel that knocked out a major UUNet cable caused it. ” – “PSINet, Verizon, World. Com and Above. Net were some of the bigger communications companies reporting service problems related to ‘peering, ’ methods used by Internet service providers to hand traffic off to others in the Web's infrastructure. Traffic slowdowns were also seen in Seattle, Los Angeles and Atlanta, possibly resulting from re-routing around the affected backbones. ” – “The fire severed two OC-192 links between Vienna, VA and New York, NY as well as an OC-48 link from, D. C. to Chicago. … Metromedia routed traffic around the fiber break, relying heavily on switching centers in Chicago, Dallas, and D. C. ”

Internet Service Composition Applications (Portals, E-Commerce, E-Tainment, Media) Appl Infrastructure Services (Distribution, Caching, Searching, Hosting) AIP ISV Application-specific Servers (Streaming Media, Transformation) ASP Internet Data Centers ISP CLEC Application-specific Overlay Networks (Multicast Tunnels, Mgmt Svrcs) Global Packet Network Internetworking (Connectivity)

Competition vs. Cooperation • Internet Service Providers: Competition – Peering for packet transport: BGP protocol – Charging based on traffic volumes ISP A Peering Point Hot Potato Routing ISP B Peering Point

Composition and Cooperation: Mobile Virtual Network Operator MVNO has everything but its own physical network

Mobile Virtual Network Operator: Composition and Cooperation Inter. Call one 2 one Competition one 2 one 1 -to-1 Relationship M-to-N Relationships

GPRS Transit: Peering, Cooperation, Composition Operator A DNS BG Operator C BG GGSN GPRS Peering Network R R GRX R R R DNS R Operator B DNS SGSN DNSDNS R. gprs BG R R GRX DNS R DNS SGSN R Operator C BG DNS SGSN Per Johannson, Ericsson Research

Peering Policy-Based Routing • Multi-homing – Reliability of network connectivity – Traffic discrimination Primary Transit Network Berkeley Dorm Campus End Network Traffic Alternative New Primary Transit Research Fail-over Network Traffic Peer Cal. REN Peer Networks Network

Overlays Creating New Interdomain Services • Deploy new services above the routing layer – E. g. , interdomain multicast management and peering – E. g. , alternative connectivity for performance, resilience Isolated Intra-cloud service Administrative domain Admin domain Traditional unicast peering Steve Mc. Canne

Overlays Brokered Resources for Applications • Examples: – Multicast management and peering at application level – Implement performance qualities at overlay level Steve Mc. Canne

Composition: Wireless ISPs (w. ISPs) • T-Mobile Wireless Broadband (Mobile. Star), Way. Port – Traditional network ISP, subscription-based services in public places – Hotels (Wayport), airports (Wayport @ SJ airport), airport clubs (TMobile @ AA Admirals Club), and cafes (T-Mobile @ Starbucks) – Diverse billing models: e. g. , 24 -hour subscription at a hotel • Boingo, Joltage, here. Uare, Net. Near. U – “Aggregator” of access, e. g. , Boingo aggregates Wayport, here. Uare – Client s/w including network sniffer/location finder, back-end authentication/secure VPN/settlement services – Revenue sharing with micro ISPs/single local network (SLN) – Diverse billing models: subscriptions as well as pay per use • Sputnik – Cooperative wireless neighbor-to-neighbor networks • Ipass, GRIC – Secure remote access for mobile employees – Simplify connection establishment and login, wireless VPN support

Composition of Wireless Infrastructure Services VPN Operator, Client-Software WISP Aggregator Private Brand Net Operator (MVNO) Single Sign-on Unified Billing, ECommerce Authentication Inter-site Mobility SLN Aggregator Single Location Network Operator Single Location (SLN) Network Operator Cooperative (SLN) Networking Revenue Sharing Full Service Network Full Service Operator Network Full Service Premises-based Operator Network Operator Access

Presentation Outline • • • Service Architecture Opportunity SAHARA Project Motivation SAHARA Reference Architecture Mechanisms for Service Composition Summary and Conclusions

Technical Challenges • Trust management and behavior verification – Meet promised functionality, performance, availability • Adapting to network dynamics – Actively respond to shifting server-side workloads and network congestion, based on pervasive monitoring & measurement – Awareness of network topology to drive service selection • Adapting to user dynamics – Resource allocation responsive to client-side workload variations • Resource provisioning and management – Service allocation and service placement • Interoperability across multiple service providers – Interworking across similar services deployed by different providers

Service Composition Models • Cooperative – Individual component service providers interact in distributed fashion, with distributed responsibility, to provide an end-to-end composed service • Brokered – Single provider, the Broker, uses functionalities provided by underlying service providers, encapsulates these to compose an end-to-end service • Examples – Cooperative: roaming among separate mobile networks – Brokered: JAL restaurant guide

Service Composition Models Cooperative Negotiation & control path Service Data flow Brokered Negotiation & control path Broker Service Data flow Service

Layered Reference Model for Service Composition Middleware Services End-to-End Network With Desirable Properties Enhanced Paths Enhanced Links IP Network Connectivity Plane Service Composition Applications Services Application Plane End-User Applications

Layered Reference Model for Service Composition Composed Service at Layer i Measurement-based Adaptation Interoperabilty Dynamic Resource Allocation Policy Management Trust Management/ Verification Underlying Composition Techniques Services at Layer i-1 Component Services at Layer i-1 Other Services at Layer i

Layered Reference Model for Service Composition • Connectivity Plane – End-to-end network with desirable properties composed on top of commodity IP network – Enhanced Links & Paths: Qo. S and protocol verification within and between connectivity service providers • Applications Plane – Services strategically placed and actively managed within the network topology – Applications and Middleware Services: end-client oriented vs. infrastructure oriented

Presentation Outline • • • Service Architecture Opportunity SAHARA Project Motivation SAHARA Reference Architecture Mechanisms for Service Composition Summary and Conclusions

Mechanisms for Service Composition • Measurement-based Adaptation – Examples • General-purpose third party end-to-end Internet host distance monitoring and estimation service • Universal In-box: Application-specific middleware measurement layer to exchange network and server load using link-state algorithm • Content Distribution Networks: measurement-based DNS-based server selection to redirect client to closest service instance

Mechanisms for Service Composition • Utility-based Resource Allocation Mechanisms – Examples • Auctions to dynamically allocate resources; applied for spectrum/bandwidth resource assignments to MVNO from underlying competiting MNOs • Congestion pricing: influence user behavior to better utilize scarce resources; applied in: – Voice port allocation to user-initiated calls in H. 323 gateway/Voice over IP service management – Wireless LAN bandwidth allocation and management – H. 323 gateway selection, redirection, and load balancing for Voice over IP services

Mechanisms for Service Composition • Trust Mgmt/Verification of Service & Usage – Authentication, Authorization, Accounting Services • Authorization control scheme w/ credential transformations to enable cross-domain service invocation • Federated admin domains with credential transformation rules based on established peering agreements • AAA server makes authorization decisions, liberating providers from preparing rules for each affiliated domain – Service Level Agreement Verification • Verification and usage monitoring to ensure properties specified in SLA are being honored • Border routers monitoring control traffic from different providers to detect malicious route advertisements

Mechanisms for Service Composition • Policy Management – Visibility into local policies to better coordinate global policies among (cooperating) service providers – Developing inter-AS architecture for load balancing, performance and failure mode policies to be applied throughout the network • Internet topology discovery through AS relationship map of the Internet plus measurement infrastructure • Policy agent framework for inter-AS negotiation to manage incoming traffic

Mechanisms for Service Composition • Interoperability through Transformation – Interoperability of data, protocols, policies among composed service providers – Example • Broadcast federation: global multicast service composed from multicast implementations in different provider domains • Protocol transformation gateways between admin domains employing non-interoperable multicast protocol implementations

Presentation Outline • • • Service Architecture Opportunity SAHARA Project Motivation SAHARA Reference Architecture Mechanisms for Service Composition Summary and Conclusions

Summary and Conclusions • Goal: Evolve (mobile) Internet architecture to better support multi-network/multi-service provider model – Dynamic environment, location-based implies larger numbers of service providers & service instances • Status: architectural specification driven by selected applications and underlying wide-area services • Focus: – Composition across confederated vs. independent service providers: peer-to-peer vs. brokering – Explore new techniques/technologies: • Market-based mechanisms • Trust management, SLA verification, perf. monitoring

Recent Publications • • C. Chuah, L. Subramanian, A. D. Joseph, R. H. Katz, “Qo. S Provisioning Using A Clearing House Architecture, ” 8 th International Workshop on Quality of Service (IWQOS 2000), Pittsburgh, PA, (June 2000). S. Zhuang, B. Zhao, A. Joseph, R. H. Katz, J. Kubiatowicz, “Bayeux: An Architecture for Wide-Area, Fault-Tolerant Data Dissemination Protocol, ” ACM NOSSDAV 2001, New York, (June 2001). Z. Mao, W. So, R. H. Katz, “Network Support for Mobile Multimedia Using a Self-Adaptive Distributed Proxy, ” ACM NOSSDAV 2001, New York, (June 2001). Y. Chen, A. Bargteil, R. H. Katz, “Quantifying Network Denial of Service: A Location Service Case Study, ” Third International Conference on Information and Communication Security (ICICS’ 2001), Xi’an, China, (November 2001).

Recent Publications • • • J. Shih, R. H. Katz, “Pricing Experiments for a Computer-Telephony. Service Usage Allocation, ” IEEE Globecom 2001, San Antonio, TX, (November 2001). Y. Chen, R. H. Katz, J. Kubiatowicz, “Replica Placement for Scalable Content Delivery, ” Proceedings First International Conference on Peer -to-Peer Systems (IPTPS’ 02), Cambridge, MA, (March 2002). T. Suzuki, R. H. Katz, “An Authorization Control Framework to Enable Service Composition Across Domains, ” Proceedings Eleventh World Wide Web Conference (WWW 2002), Honolulu, HI, (May 2002). M. Caesar, D. Ghosal, R. H. Katz, “Resource Management for IP Telephony Networks, ” Proceedings 10 th International Workshop on Quality of Service (IWQo. S), Miami Beach, FL, (May 2002). S. Machiraju, M. Seshadri, I. Stoica, “A Scalable and Robust Solution for Bandwidth Allocation, ” Proceedings 10 th International Workshop on Quality of Service (IWQo. S), Miami Beach, FL, (May 2002).

Recent Publications • • Y. Chawathe, M. Seshadri, “Broadcast Federation: An Applicationlayer Broadcast Internet, ” Proceedings Network and Operating System Support for Digital Audio and Video (NOSSDAV’ 02), Miami Beach, FL, (May 2002). L. Subramanian, V. Padmanabhan, R. H. Katz, “Geographic Properties of Internet Routing, ” USENIX Conference, Monterey, California, (June 2002). Z, Mao, C. Cranor, F. Douglis, M. Rabinovich, O. Spatscheck, J. Wang, “A Precise and Efficient Evaluation of the Proximity between Web Clients and their Local DNS Servers, ” USENIX Conference, Monterey, California, (June 2002). L. Subramanian, S. Agarwal, J. Rexford, R. H. Katz, “Characterizing the Internet Hierarchy from Multiple Vantage Points, ” IEEE Infocomm Conference, New York, NY, (June 2002).

Recent Publications • • J. Shih, R. H. Katz, “Evaluating Tradeoffs of Congestion Pricing for Voice Calls, ” Extended Abstract, ACM Sigmetrics Conference, San Diego, California, (July 2002). J. Shih, R. H. Katz, “Evaluating the Tradeoffs of Congestion Pricing for Voice Calls, ” 2002 International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS 2002), San Diego, California, (July 2002). B. Raman, R. H. Katz, “Emulation-based Evaluation of an Architecture for Wide-Area Service Composition, ” 2002 International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS 2002), San Diego, California, (July 2002). Z. Mao, R. Govindan, S. Shenker, R. H. Katz, “Route Flap Damping Exacerbates Internet Routing Convergence. ” ACM SIGCOMM Conference, Pittsburgh, PA, (August 2002).

Recent Publications • • B. Raman, S. Agrawal, Y. Chan, M. Caesar, W. Cui, P. Johannson, K. Lai, T. Lavian, S, Machiraju, Z. Mao, G. Porter, T. Roscoe, M. Seshadri, J. Shih, K. Sklower, L. Subramanian, T. Suzuki, S. Zhuang, A. D. Joseph, R. H. Katz, I. Stoica, “The SAHARA Model for Service Composition across Multiple Providers, ” Pervasive Computing 2002, Zurich, Switzerland, (August 2002). Z. Mao, R. H. Katz, “A Framework for Universal Service Access using Device Ensembles, ” CRA Grace Murray Hopper Celebration of Women in Computer Science Conference, Vancouver, BC, (October 2002).

SAHARA: A Revolutionary Service Architecture for Future Telecommunications Systems Randy H. Katz, Anthony Joseph, Ion Stoica Computer Science Division Electrical Engineering and Computer Science Department University of California, Berkeley, CA 94720 -1776

Work in Progress • • Enhanced Links Enhanced Paths Middleware Services Applications Services

Work in Progress • Enhanced Links – Congestion Pricing for Access Links – Auction-based Resource (Bandwidth) Allocation – Traffic Policing/Verification of Bandwidth Allocation

Congestion Pricing at Access Links Computer Local Area Access Internet Router Network Qo. S Computer $ • Setup – 10 users – 3 Qo. S (Slow-going, Moderate, & Responsive) differ on degree of traffic smoothing – 24 tokens/day, 15 minutes of usage per charge • Acceptable – Users make purchasing decision at most once every 15 minutes • Feasible – Changing prices cause users to select different Qo. S • Effective – If entice half of users to choose lower Qo. S during congestion, then reduce burstiness at access links by 25%

Auction-based Resource Allocation Application Bidder Auctioneer Resource • Problem – Efficiently and effectively allocate resources according to application’s dynamic requirements • Approach – Leveraging auction schemes and work -load predictions • Features – Bidders can place bids based on application requirements and contention level. – Bidders can place bids for near future resource requirements based on recent history. – Bidders can express both utility and priority to auctioneer. – Auctioneer can dynamically change application’s priority by changing the token allocation rate. • Status – On-going work – First application: bandwidth allocation in ad hoc wireless networks

Bandwidth Allocation • Problem: scalable (stateless) and robust bandwidth allocation • Control Plane: – Soft state – Per-router per-period certificates for robustness without per-flow state – Random sampling to prevent duplicate refreshes • Data Plane: – Monitor aggregate flows – Recursively split misbehaving aggregates R 1 attaches new certificate to the refresh message misbehaving aggregate – split it

Work in Progress • Enhanced Paths – – – BGP Route Flap Dampening BGP Policy Agents Backup Path Allocation in Overlay Networks Host Mobility Multicast Interoperation

BGP: Stability vs. Convergence • Problem: – Stability achieved through flap damping[RFC 2439] – Unexpected: flap damping delays convergence! Topology: clique of routers n n Solution: selective flap damping [sigcomm 02] n Duplicate suppression: Ignore flaps caused by transient convergence instability n Still contains stability n n Eliminates undesired interaction!

Policy Management for BGP • 3 -15 minute failover time • Slow response to congestion • Unacceptable for Internet service composition • • • Lack of distributed route control Need distributed policy management Explicit route policy negotiation 20 AS’s 129 AS’s • • • Identified current routing behavior Inferred AS relationships, topology Next : gather traffic data, finish code, emulate 897 AS’s 971 AS’s 8898 AS’s

Backup Path Allocation in Overlay Networks The Overlay Network The Underlying Network • Challenge • Problem • Approach • Status – Disjoint primary and backup path in the overlay network may share underlying links because the overlay network cannot control underlying links used by a path – Find a primary and backup path pair with minimal failure probability based on correlated overlay link failures – Decouple backup path routing from primary path routing – Route backup paths based on failure probability cost which measures the incremental path failure probability caused by using a link in the path – Finished work, submitted to ICNP’ 02

Host Mobility Using an Internet Indirection Infrastructure • The Problem (ID, data) (ID, R) – Internet hosts increasingly mobile; Sender (S) need to remain reachable (ID, data) – Flows should not be interrupted – IP address represents unique host ID & net location • ROAM (Robust Overlay Architecture for Mobility) – – (ID, R) Receiver (R) Leverages i 3: overlay network triggers & forward packets Efficiency, robustness, location privacy, simultaneous mobility No changes to end-host kernel or applications Cost: i 3 infrastructure, and proxies on end-hosts • Simulation & Experimental Results – Stretch lower than MIP-bi able to choose nearby triggers – 50 -66% of MIP-tri when 5 -28% domains deploy i 3 servers – Even 4 handoffs in 10 seconds have little impact on TCP performance

Multicast Broadcast Federation • Goal : compose different non- interoperable multicast domains to provide an end-to-end multicast Broadcast Domains service. SSM Approach : overlay of Broadcast Gateways (BGs) – BGs establish peering between domains. – Inside a domain, local multicast capability is used. – Clustered gateways for scalability. – Independent data flows and control flow. CDN IP Mul – Should work for both IP and App-layer protocols. • Source Clients BG Data • Implementation : Peering – Linux/C++ event-driven program – Easily customizable interface to local multicast capability (~700 lines) – Upto 1 Gbps BG thruput with 6 nodes. – Upto 2500 sessions with 6 nodes.

Work in Progress • Middleware Services – Measurement and Monitoring Infrastructure – Robust Service Composition – Authorization Interworking

Internet Distance Monitoring Infrastructure • Problem: N end hosts in different administrative domains, how to select a subset to be probes, and build an overlay distance monitoring service without knowing the underlying topology? • Solution: Internet Iso-bar – Clustering of hosts perceiving similar performance Cluster C • Good scalability • Good accuracy & stability – Tested with NLANR AMP & Keynote data Cluster B Cluster A • Small overhead • Incrementally deployable • [SIGMETRICS PAPA 02] & [CMG journal 02] Monitor Distance from monitor to its hosts End Host Distance measurements among monitors

Text to audio Text Source Availability in Wide-Area Service Composition • Issue: Multi-provider WA composition • Poor availability of Internet path Poor service availability for client Text Source • Fix: detect and recover from failures using service replicas • Highlight of results: – Quick detection (~2 sec) possible – Scalable messaging for recovery (can handle simultaneous failure recovery of 1000 s of clients) – See SPECTS’ 02 paper • More recent results on load balancing across service replicas… • >15 sec outage • Note: BGP recovery could take several minutes [Labovitz’ 00] • End-to-end recovery in about 3. 6 sec: 2 sec detection, ~600 ms signaling, ~1 sec state restoration WA setup: UCB, Berk. (Cable), SF (DSL), Stan. , CMU, UCSD, UNSW (Aus), TU-Berlin (Germany)

Authorization Control Across Administrative Domains Domain 1 Trusted third party Should grant access? Service Decision Request Policy compliance check - certificates - credentials Authorization Authority Verification Credential transformation User Domain 2 Trust peering agreement - credential transformation rule • Authorization authority • Trust peering agreement – Provides authorization decision service. – Manages different verification methods and credentials. – Credential transformation rule – Acceptable verification method Certificates Credentials

Work in Progress • Applications Services – Voice Over IP – Adaptive Content Distribution – (Universal In-Box)

IP Telephony Gateway Selection ITG LS LS ITG Gateway (ITG) IP Terminal LS Location Server (LS) LS Load Advertisement Call Session Goal: High quality, economically efficient telephony over the Internet n Questions: How to n Perform call admission control? n Route calls thru converged net? n n Results: Congestion sensitive pricing decreases unnecessary call blocking, increases revenue, and improves economic efficiency n Hybrid redirection achieves good Qo. S and low blocking probability n

SCAN: Scalable Content Access Network • Problem: Provide content distribution to clients with small latency, small # of replicas and efficient update dissemination • Solution: SCAN – Leverage P 2 P location services to improve scalability and locality – Simultaneous dynamic replica placement & app-level multicast tree construction • Close to optimal # of replicas wrt latency guarantee • Small latency & bandwidth for sending updates • [IPTPS 02] & [Pervasive 02] data source data plane replica cache always update adaptive coherence Web server SCAN server client Tapestry mesh network plane