Скачать презентацию System Design Amar Phanishayee System design n Скачать презентацию System Design Amar Phanishayee System design n

a01c77853720be13d728d0bcb866f0a6.ppt

  • Количество слайдов: 31

System Design Amar Phanishayee System Design Amar Phanishayee

System design n complex act n less precisely defined, changing requirements n Choices n System design n complex act n less precisely defined, changing requirements n Choices n Affects future choices n Affects system-wide performance n But how?

Butler Lampson - Background n Founding member of Xerox PARC (1970), DEC (1980 s), Butler Lampson - Background n Founding member of Xerox PARC (1970), DEC (1980 s), MSR (current) n ACM Turing Award (1992) n Laser printer design n PC n Two-phase commit protocols n Bravo, the first WYSIWYG text formatting program n Ethernet, the first high-speed local area network (LAN)

Some Projects & Collaborators n Charles Simonyi - Bravo: WYSIWYG editor n Bob Sproull Some Projects & Collaborators n Charles Simonyi - Bravo: WYSIWYG editor n Bob Sproull - Alto operating system, Dover: laser printer, Interpress: page description language n Mel Pirtle - 940 project, Berkeley Computer Corp. n Peter Deutsch - 940 operating system, QSPL: system programming language n Chuck Geschke, Jim Mitchell, Ed Satterthwaite - Mesa: system programming language

Some Projects & Collaborators (cont. ) n Roy Levin - Wildflower: Star workstation prototype, Some Projects & Collaborators (cont. ) n Roy Levin - Wildflower: Star workstation prototype, Vesta: software configuration n Andrew Birrell, Roger Needham, Mike Schroeder - Global name service and authentication n Eric Schmidt - System models: software configuration n Rod Burstall - Pebble: polymorphic typed language

Hints for Computer System Design Butler Lampson Hints for Computer System Design Butler Lampson

Functionality n Interface – Contract n separates implementation from client using abstraction n Eg: Functionality n Interface – Contract n separates implementation from client using abstraction n Eg: File (open, read, write, close) n Desirable properties n Simple n Complete n Admit small and fast impl.

Simplicity n Interfaces n Avoid generalizations n n too much = large, slow and Simplicity n Interfaces n Avoid generalizations n n too much = large, slow and complicated impl. Can penalize normal operations § PL/1 generic operations across data types n Should have predictable (reasonable) cost. n n eg: Find. Ith. Field [O(n)], Find. Namedfield [O(n^2)] Avoid features needed by only a few clients

Functionality Vs Assurance n As a system performs more (complex interface) assurance decreases. Functionality Vs Assurance n As a system performs more (complex interface) assurance decreases.

Example n Tenex System n reference to an unassigned page -> trap to user Example n Tenex System n reference to an unassigned page -> trap to user program n arguments to sys calls passed by reference n CONNECT(string passwd) -> if passwd wrong, fails after a 3 second delay n CONNECT for i : = 0 to Length(directory. Password) do if directory. Password[i] != password. Argument[i] then Wait three seconds; return Bad. Password end if end loop; connect to directory; return Success

Breaking CONNECT(string passwd) Unassigned Page Assigned Page B A Bad Passwd page Invalid Breaking CONNECT(string passwd) Unassigned Page Assigned Page B A Bad Passwd page Invalid

Breaking CONNECT(string passwd) Worst case Unassigned Page Assigned Page B A Z Bad Passwd Breaking CONNECT(string passwd) Worst case Unassigned Page Assigned Page B A Z Bad Passwd page Invalid 128*n tries as opposed to 128^n tries n = passwd length (bytes)

Functionality (cont. ) n basic (fast) operations rather than generic/powerful (slow) ones n n Functionality (cont. ) n basic (fast) operations rather than generic/powerful (slow) ones n n n Pay for what you want RISC Vs CISC Unix Pipe n grep –i 'spock' * | awk -F: '{print $1}' | sort | uniq | wc –l n Use timing tools (80% of the time in 20% of code) n Avoid premature optimization n May be useless and/or expensive n analyze usage and optimize heavily used I/Fs

n Avoid abstracting-out desirable properties n “don't hide power” n Eg: Feedback for page n Avoid abstracting-out desirable properties n “don't hide power” n Eg: Feedback for page replacement n How easy is it to identify desirable properties? n Procedure arguments n filter procedure instead of a complex language with patterns. n static analysis for optimization - DB query lang n failure handlers n trust?

Continuity n Interfaces n Changes should be infrequent n n Compatibility issues Backward compatibility Continuity n Interfaces n Changes should be infrequent n n Compatibility issues Backward compatibility on change n Implementation n Refactor to achieve “satisfactory” (small, fast, maintainable) results n Use prototyping

Implementation n Keep secrets n Impl. can change without changing contract n Client could Implementation n Keep secrets n Impl. can change without changing contract n Client could break if it uses Impl. details n But secrets can be used to improve performance n finding the balance an art? n Divide and conquer n Reuse a good idea in different settings n global replication using a transactional model n local replication for reliably storing transactional logs.

Completeness - handling all cases n Handle normal and worst case separately n normal Completeness - handling all cases n Handle normal and worst case separately n normal case – speed, worst case – progress n Examples n n caches incremental GC § trace-and-sweep (unreachable circular structures) n piece-table in the Bravo editor § Compaction either at fixed intervals or on heavy fragmentation n “emergency supply” helps in worst-case scenarios

Speed n Split resources in a fixed way n rather than share and multiplex Speed n Split resources in a fixed way n rather than share and multiplex n faster access, predictable allocation n Safety instead of optimality n over-provisioning ok, due to cheap hardware n Use static analysis where possible n dynamic analysis as a fallback option n Eg: sequential storage and pre-fetching based on prior knowledge of how data is accessed

Speed (cont. ) n Cache answers to expensive computations n x, f => f(x) Speed (cont. ) n Cache answers to expensive computations n x, f => f(x) n f is functional. n Use hints! n may not reflect the "truth" and so should have a quick correctness check. n Routing tables n Ethernet (CSMA/CD)

Speed (cont. ) n Brute force when in doubt n Prototype and test performance Speed (cont. ) n Brute force when in doubt n Prototype and test performance n Eg: linear search over a small search space n Beware of scalability! n Background processing (interactive settings) n GC n writing out dirty pages, preparing pages for replacement. n Shed load n Random Early Detection n Bob Morris' red button

Fault Tolerance n End-to-end argument n Error recovery at the app level essential n Fault Tolerance n End-to-end argument n Error recovery at the app level essential n Eg: File transfer n Log updates n Replay logs to recover from a crash n form 1: log n update proc must be functional n arguments must be values n form 2: log state changes. n idempotent (x = 10, instead of x++) n Make actions atomic n Aries algorithm - Atomicity and Durability

Conclusions n Remember these are “hints” from a Guru n Reuse good ideas, but Conclusions n Remember these are “hints” from a Guru n Reuse good ideas, but not blindly. n Your experiences

End-to-End arguments in System Design – J. H. Saltzer n Helps guide function placement End-to-End arguments in System Design – J. H. Saltzer n Helps guide function placement among modules of a distributed system n Argument n can the higher layer implement the functionality it needs? n if yes - implement it there are the app knows it's needs best n implement the functionality in the lower layer only if n A) a large number of higher layers / applications use this functionality and implementing it at the lower layer improves the performance of many of them AND n B) does not hurt the remaining applications

Example : File Transfer (A to B) 6. Route packet A 4. Pass msg/packet Example : File Transfer (A to B) 6. Route packet A 4. Pass msg/packet down the protocol stack 5. Send the packet over the network 1. 2. 3. 4. Read File Data blocks App buffers File Data Pass (copy) data to the network subsystem B

Example : File Transfer A 7. Receive packet and buffer msg. 8. Send data Example : File Transfer A 7. Receive packet and buffer msg. 8. Send data to the application 9. Store file data blocks B

Possible failures n Reading and writing to disk n Transient errors in the memory Possible failures n Reading and writing to disk n Transient errors in the memory chip while buffering and copying n n/w might drop packets, modify bits, deliver duplicates n OS buffer overflow at the sender or the receiver n Either of the hosts may crash

Solutions? n Make the network reliable n Packet checksums, sequence numbers, retry, duplicate elimination Solutions? n Make the network reliable n Packet checksums, sequence numbers, retry, duplicate elimination n Solves only the network problem. n What about the other problems listed? n Byte swapping problem while routing @ MIT n Introduce file checksums and verify once transfer completes – end-to-end check. n On failure – retransmit file.

Solutions? (cont. ) n n/w level reliability would improve performance. n But this may Solutions? (cont. ) n n/w level reliability would improve performance. n But this may not benefit all applications n n Huge overhead for say RT speech transmission Need for optional layers n Checksum parts of the file.

Formally stated Formally stated "The function in question can completely and correctly be implemented only with the knowledge and help of the application standing at the end points of the communication system. Therefore, providing that questioned function as a feature of the communication system itself is not possible. (Sometimes an incomplete version of the function provided by the communication system may be useful as a performance enhancement. )"

Other end-to-end requirements n Delivery guarantees n Application level ACKs n n n Deliver Other end-to-end requirements n Delivery guarantees n Application level ACKs n n n Deliver only if action guaranteed 2 phase commit NACKs n End-to-end authentication n Duplicate msg suppression n Application level retry results in new n/w level packet n cbcast, abcast, vsync

Panacea? n Routing n “source routing” Vs “transparent routing” n congestion control n TCP Panacea? n Routing n “source routing” Vs “transparent routing” n congestion control n TCP leaves it to the ends n Should the network trust the ends? § RED n In a wireless setting § packet loss != congestion n cheap test for success is needed n performance problems may appear in end-end systems under heavy load