Design Principles and Architectures for End-to-End Quality of

Design Principles and Architectures for End-to-End Quality of Service Chris Gill cdgill@cs. wustl. edu Center for Distributed Object Computing Department of Computer Science Washington University, St. Louis, MO http: //www. cs. wustl. edu/~cdgill/End_to_End. ppt

Theme: Layering in Distributed Systems • End-to-end layers – Application components, middleware services, and transport protocols distributed over multiple end-systems • Hop-to-hop layers – Network: internetworking e. g. , IP routing between LANs – Data Link: e. g. , frame management in LANs, bridges – Physical: EE aspects for a medium: voltages, timing • Different time scales – End-to-end vs. hop-to-hop, in-band vs. out-of-band mechanisms • Design Trade-offs – How to decide what functionality belongs where

The End-to-End Argument • J. H. Saltzer, D. P. Reed and D. D. Clark, “End-To-End Arguments in System Design”, ACM Transactions on Computer Systems, 4(4): 277 -288, November 1984. • Design Principle: “A lower level subsystem that supports a distributed application may be wasting its effort providing a function that must by nature be implemented at the application level anyway” - depends on other design forces • Examples – FTP correctness, message delivery, data encryption, duplicate message suppression, FIFO message ordering, transactions

The End-to-End Argument: FTP Correctness • Question: where to perform bit error detection and/or correction for file transfer in a distributed system? • End-to-end path: I/O device to application to OS to protocol stack to LAN to router to protocol stack to router to LAN to protocol stack to OS to application to I/O device • Hop-to-hop error detection/retransmit, or error correction is overly expensive unless link is highly unreliable, and will not solve the entire problem, as sender crashes, I/O bit errors, and other threats still require end-to-end solution • Application-specific solution is an end-to-end checksum every so many bytes and at EOF

The End-to-End Argument: Message Delivery • A message is much shorter than a byte stream (e. g. , FTP) and is at less risk for corruption • However, there are still many threats along the path – E. g. , packet dropping due to router congestion • Hop-to-hop ACK/retransmit may be useful for congestion control depending on link reliability and router capabilities, but still cannot ensure “perfect” reliability – E. g. , receiver crashes right after receipt, unbounded priority inversion on receiver, etc. • Need an application-specific end-to-end solution such as a transactional commit protocol

The End-to-End Argument: Secure Transmission • If hop-to-hop encryption is performed, network nodes will need to securely manage encryption keys • Unless the application also performs decryption, the message will be vulnerable on the destination end-system • The application will also need to perform authentication of the messages it receives • Therefore, intermediate encryption is redundant here • However, intermediate encryption still might be useful to ensure no unauthorized exposure of information occurs, say outside a departmental firewall: additional requirement

The End-to-End Argument: Duplicate Messages • For efficiency, and possibly correctness, it is useful to discard duplicate messages • Even if the network discards all duplicate messages, it has no way to detect duplication at the application level – E. g. , a repeated user login request due to slow system response • Since the application needs a way to suppress duplicates anyway, that function can also be used to suppress duplicates originating in lower layers • Therefore eliminating duplicates in lower layers is wasteful, unless motivated by another design force

The End-to-End Argument: FIFO Ordering • Independent paths are useful for robustness in the face of link failures (e. g. , spanning tree re-convergence for routing), for priority-preserving virtual circuits, etc. • Latency differences (e. g. , due to congestion, retransmission) between consecutive messages routed along independent paths may result in reordering • However, distributed applications over multiple endsystems are asynchronous, so for many application if the order of messages is important, then a higher level message synchronization protocol is needed anyway

The End-to-End Argument: Applied • Distributed reads and writes in the SWALLOW distributed data storage system - anticipates DOC applications (1984) • Object id plus message version is sufficient to detect duplicate writes at application layer of server, plus duplicate replies for reads are easily discarded at client • Writes on server are ACKed back to client after data is stored safely, and the read value response is the ACK • Performing duplicate elimination and acknowledgement at the application level simplifies the lower layers, and improves performance by reducing both in-band overhead and message traffic from lower level ACKs

The End-to-End Argument: Endpoints • The message delivery and secure transmission examples highlight a more fundamental issue: where a path ends • Different applications may apply requirements to different path segments, and may have different overall paths – Client-server application: two-way reply appears on the screen – Manufacturing application: remote robot arm rotates 15 clockwise • Different inherent semantics for various approaches – CORBA 2 -way calls vs. transactional MOM • Different applications may produce different decisions WRT the end-to-end argument

Qo. S Architectures • Aurrecoechea, C. , Campbell, A. T. and L. Hauw, "A Survey of Qo. S Architectures, " ACM/Springer Verlag Multimedia Systems Journal , Special Issue on Qo. S Architecture, Vol. 6 No. 3, pg. 138 -151, May 1998 • Presents a set of Qo. S design principles • Presents an architecture (they use the term framework) for Qo. S specification and enforcement • Surveys other Qo. S management architectures, circa 1998 • Terrific bibliography (modulo a few erroneous dates) will highlight a few papers that proposed design principles

Qo. S Design Principles • Integration principle - Qo. S must be configurable, predictable, and maintainable over all layers – Kurose, J. F. “Open Issues and Challenges in Providing Quality of Service Guarantees in High Speed Networks”, ACM Computer Communications Review, vol. 23, no. 1, pp. 6 -15, Jan 1993 • End-to-end Qo. S depends on all modules traversed in the end-to-end path, each of which must offer: – Qo. S configurability based on some specification – Resource guarantees and enforcement mechanisms – Maintenance of ongoing flows

Qo. S Design Principles, Continued • Separation principle - media transfer, control, and management are three functionally distinct activities – Lazar, A. A. , “A Real-time Control, Management, and Information Transport Architecture for Broadband Networks”, Proc. International Zurich Seminar on Digital Communications, pp. 281295, 1992 • Three activities at three scales of complexity – Media transfer - simple transmission: hop-to-hop forwarding – Control - more complex but related to media transfer: routing, queue scheduling, fine-grain monitoring etc. – Management - distributed, possibly end-to-end: reservations, reconfiguration, large-scale monitoring, etc.

Qo. S Design Principles, Continued • Transparency principle - applications should be shielded from the complexity of the underlying Qo. S policies and management mechanisms • Reduces accidental complexity by decoupling Qo. S aspects from application functionality • Delegates inherent complexity to a layer below the application – Combined with the end-to-end argument, this makes a good case for Qo. S management in middleware

Qo. S Design Principles, Continued • Multiple time scales principle - guides division of Qo. S control and management functionality between modules – (also from) Lazar, A. A. , “A Real-time Control, Management, and Information Transport Architecture for Broadband Networks”, Proc. International Zurich Seminar on Digital Communications, pp. 281 -295, 1992 • Three activities at three scales of time – Media transfer - rapid transmission – Control - additional overhead but still close to media time scale – Management - distributed, possibly end-to-end time scales • Balancing design force to the end-to-end argument

Qo. S Design Principles, Continued • Performance principle - a collection of widely agreed design idioms pertaining to overall system performance – The end-to-end argument – Avoiding multiplexing • Tennenhouse, D. L. , “Layered Multiplexing Considered Harmful”, Protocols for High-Speed Networks, Elsevier Science Publishers (North-Holland), 1990 – Structuring communications protocols • Clark, D. and Tennenhouse, D. L. , “Architectural Consideration for a New Generation of Protocols”, ACM SIGCOMM 1990 – Pushing protocol processing down into hardware

Qo. S Architecture: Specification • Each category captures a class of application level Qo. S requirements as declarative (what rather than how) policies that are enforced by layer-specific mechanisms – Flow synchronization - degree of frame tightness between flows • interestingly, applies to frame-based event-driven applications – Flow performance - throughput, delay, jitter, loss – Level of service - deterministic, predictive, best effort gurantees – Qo. S management policy - adaptation and scaling capabilities – Cost of service - competitive, collaborative, individual/social good

Qo. S Architecture: Mechanisms • Provisioning – Initial flow establishment and end-to-end re-negotiation – Setting up configurations of resources and functionality in support of media transfer and processing – End-to-end time scale • Control – In-band handling of flows on time-scales close to media transfer • Management – Out-of-band reconfiguration of flows to ensure contracts are met – Time scales between media transfer and end-to-end

Qo. S Mechanisms: Provisioning • Qo. S mapping – Transforms application Qo. S requirements into policies for configuring and operating mechanisms in systems layers • Admission control – Determines whether new requirements can be met given current reservations and resources • Resource reservation – Once admission control has been performed, allocates (binds) resources to the new flow(s)

Qo. S Mechanisms: Control • Flow shaping – In-band traffic pacing, smoothing, pruning to improve overall good • Flow scheduling – Forwarding/processing based on per-flow or aggregate policy • Flow policing – Detects misbehaving flows • Flow control – Sender pacing or receiver feedback adaptation • Flow synchronization – Control of flows based on aggregate policies

Qo. S Mechanisms: Management • Qo. S monitoring – Each layer may track Qo. S achieved by other modules (feedback) • Qo. S maintenance – Tunes operations based on perceived/expected Qo. S • Qo. S degradation – Qo. S cannot be corrected by lower layers, must handle or pass up • Qo. S availability – Performance monitoring/assessment/signaling intervals • Qo. S scalability – Filtering and adaptation over heterogeneous resources/end-systems

Survey of Related Qo. S Architectures • • • IBM European Networking Center (Heidelberg) - Hei. RAT Columbia U - e. Xtended integrated Resource Model (XRM) U Pennsylvania - OMEGA IETF - Int. Serv Lancaster U, Alcatel - Qo. S-A OSI - Qo. S Framework UC Berkeley - Tenet Architecture TINA-C - Qo. S Framework U Pierre et Marie Curie - MASI End-to-End model Washington U (Gopal) - End System Qo. S Framework

Conclusions • Application-specific Qo. S requirements must be considered – Timeliness, correctness, liveness, security • Application-specific design forces combine with the inherent complexities of distribution – Asynchrony, failures, latency, etc. • System design principles articulate key idioms/forces • Design patterns and pattern languages appear helpful – Identifying and codifying trade-offs driven by design forces • End-to-end argument supports the case for middleware – Need flexibility to plug in application-specific high-level policies – Encapsulation of common mechanisms in middleware reduces accidental complexity for the application