DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW

DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S. TANENBAUM MAARTEN VAN STEEN Chapter 6/A Synchronization Modified by Dr. Gheith Abandah Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Overview • Clock Synchronization • • Physical Clocks Global Positioning System Clock Synchronization Algorithms Logical Clocks • • • Lamport’s Logical Clocks Vector Clocks Mutual Exclusion • • Overview Centralized Algorithm A Decentralized Algorithm A Distributed Algorithm A Token Ring Algorithm A Comparison of the Four Algorithms Global Positioning of Nodes Election Algorithms • • • Traditional Election Algorithms Elections in Wireless Environments Elections in Large-scale Systems Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Clock Synchronization Need for clock synchronization: Consider Unix make utility. When each machine has its own clock, an event that occurred after another event may nevertheless be assigned an earlier time. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Physical Clocks (1) The mean solar day is the time between to sun transits. 1 second = 1 solar day / (24*60*60). Sun transit: reaching highest apparent point. 300 millions years ago, the year was 400 days. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Physical Clocks (2) Problem: Sometimes we simply need the exact time, not just an ordering. Solution: Universal Coordinated Time (UTC): • Based on the number of transitions per second of the cesium 133 atom (pretty accurate), started in 1958. • At present, the real time is taken as the average of some 50 cesium-clocks around the world. • Introduces a leap second from time to compensate that days are getting longer. Note: UTC is broadcast through short wave radio (WWV) and geostationary environment operational satellite (GEOS). Satellites can give an accuracy of about ± 0. 5 ms. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Physical Clocks (3) International atomic time (TAI) seconds are of constant length, unlike solar seconds. Leap seconds are introduced when necessary to keep in phase with the sun. More than 30 seconds have been added since 1958. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Global Positioning System (1) You can get an accurate account of time as a side-effect of GPS. There are 29 satellites that broadcast their positions and time. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Global Positioning System (2) Real world facts that complicate GPS 1. It takes a while before data on a satellite’s position reaches the receiver. 2. The receiver’s clock is generally not in synch with that of a satellite. Therefore, need 4 satellites → 4 equations in 4 unknowns. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Clock Synchronization Algorithms (1) Problem: Suppose we have a distributed system with a UTC -receiver somewhere in it → we still have to distribute its time to each machine. Basic principle • Every machine has a timer that generates an interrupt H times per second. • There is a clock in machine p that ticks on each timer interrupt. Denote the value of that clock by Cp(t), where t is UTC time. • Ideally, we have that for each machine p, Cp(t) = t, or, in other words, d. C/dt = 1. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Clock Synchronization Algorithms (2) Goal: Never let two clocks in any system differ by more than δ time units → synchronize at least every δ /(2ρ) seconds. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Clock Synchronization Algorithms (3) Principle: Every machine asks a time server for the accurate time at least once every δ /(2ρ) seconds (Network Time Protocol). Note: Okay, but you need an accurate measure of round trip delay, including interrupt handling and processing incoming messages. Get time form a time server of a higher stratum level, like stratum-1 (has WWV receiver or atomic clock). Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Network Time Protocol Getting the current time from a time server. The NTP does 8 rounds and adopt the minimum time round. Offset = T 3 + ((T 2 -T 1)+(T 4 -T 3))/2 - T 4: -ve slowdown clock, +ve run faster, no jumbs. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

The Berkeley Algorithm (1) The time server is proactive. (a) The time daemon asks all the other machines for their clock values. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

The Berkeley Algorithm (2) (b) The machines answer. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

The Berkeley Algorithm (3) (c) The time daemon tells everyone how to adjust their clock. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Clock Synchronization in Wireless Networks (1) Need to minimize power consumption, e. g. , in WSN. (a) The usual critical path in determining network delays. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Clock Synchronization in Wireless Networks (2) (b) The critical path in the case of Reference Broadcast Synchronization (RBS) where only receivers synchronize their clocks. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Overview ü Clock Synchronization ü ü ü • Physical Clocks Global Positioning System Clock Synchronization Algorithms Logical Clocks • • • Lamport’s Logical Clocks Vector Clocks Mutual Exclusion • • Overview Centralized Algorithm A Decentralized Algorithm A Distributed Algorithm A Token Ring Algorithm A Comparison of the Four Algorithms Global Positioning of Nodes Election Algorithms • • • Traditional Election Algorithms Elections in Wireless Environments Elections in Large-scale Systems Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Lamport’s Logical Clocks (1) The notion of ordering The "happens-before" relation → can be observed directly in two situations: • If a and b are events in the same process, and a occurs before b, then a → b is true. • If a is the event of a message being sent by one process, and b is the event of the message being received by another process, then a → b • If a → b and b → c, then a → c Note: This introduces a partial ordering of events in a system with concurrently operating processes. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Lamport’s Logical Clocks (2) Each process Pi maintains a local counter Ci and adjusts this counter according to the following rules: 1. For any two successive events that take place within Pi , Ci is incremented by 1. 2. Each time a message m is sent by process Pi , the message receives a timestamp ts(m) = Ci. 3. Whenever a message m is received by a process Pj, Pj adjusts its local counter Cj to max{Cj , ts(m)}; then executes step 1 before passing m to the application. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Lamport’s Logical Clocks (3) (a) Three processes, each with its own clock. The clocks run at different rates. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Lamport’s Logical Clocks (4) (b) Lamport’s algorithm corrects the clocks. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Lamport’s Logical Clocks (5) The positioning of Lamport’s logical clocks in distributed systems. Adjustments take place in the middleware layer. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Example: Totally Ordered Multicasting (1) Updating a replicated database and leaving it in an inconsistent state. • P 1 adds $100 to an account (initial value: $1000) • P 2 increments account by 1% • There are two replicas Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Example: Totally Ordered Multicasting (2) Solution • Process Pi sends timestamped message msgi to all others. The message itself is put in a local queuei. • Any incoming message at Pj is queued in queuej, according to its timestamp, and acknowledged to every other process. Pj passes a message msgi to its application if: 1. msgi is at the head of queuej 2. for each process Pk, there is a message msgk in queuej with a larger timestamp. Note: We are assuming that communication is reliable and FIFO ordered. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Vector Clocks (1) Lamport’s clocks do not guarantee that if C(a) < C(b) that a causally preceded b Event a: m 1 is received at T = 16; Event b: m 2 is sent at T = 20. We cannot conclude that a causally precedes b. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Vector Clocks (2) Vector clocks are constructed by letting each process Pi maintain a vector VCi with the following two properties: 1. VCi [ i ] is the number of events that have occurred so far at Pi. In other words, VCi [ i ] is the local logical clock at process Pi. 2. If VCi [ j ] = k then Pi knows that k events have occurred at Pj. It is thus Pi’s knowledge of the local time at Pj. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Vector Clocks (3) Steps carried out to accomplish property 2 of previous slide: 1. Before executing an event Pi executes VCi [ i ] ← VCi [i ] + 1. 2. When process Pi sends a message m to Pj, it sets m’s (vector) timestamp ts (m) equal to VCi after having executed the previous step. 3. Upon the receipt of a message m, process Pj adjusts its own vector by setting VCj [k ] ← max{VCj [k ], ts (m)[k ]} for each k, after which it executes the first step and delivers the message to the application. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Vector Clocks (4) Observation We can now ensure that a message is delivered only if all causally preceding messages have already been delivered. Adjustment Pi increments VCi [i] only when sending a message, and Pj “adjusts” VCj when receiving a message (i. e. , effectively does not change VCj [j]). Pj postpones delivery of m until: 1. ts(m)[i] = VCj [i]+1 (next expected msg from Pi). 2. ts(m)[k] ≤ VCj [k] for all k ≠ i (Pj has seen all msgs seen by Pi). Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5

Enforcing Causal Communication Enforcing causal communication using vector clocks. m* delivery to the application at P 2 is delayed because ts(m)[0] is not ≤ VC 2 [0] Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2 e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0 -13 -239227 -5