Agreement Protocols

Examples

Agreeing whether to commit or to abort a transaction in a distributed database management system. Agreeing on a common clock value in a distributed system.

In the absence of failures or faulty processors, values (that is to be decided) can be exchanged. A vote can be taken and decision/agreement can be made based on: majority, minimum vote, mean, etc. Presence of failure: processors can fail or misbehave intentionally. Several rounds of message exchanges might be needed before agreement can be reached.

System Model

n processors in the system with at most m of them being faulty. Processors can exchange messages directly (no need to go through another processor). Receiver knows the identity of the sender. Communication medium is reliable: messages are delivered without errors.

Synchronous Computation

Processes run in lock step manner, or in rounds. In each step/round i, processes receive messages that were sent in the previous step/round i-1. Processes then do some computation and send out messages, that will be received in step/round i+1. Message delays or a slow processor slows down the whole system. Agreement protocols to be discussed assume synchronous computation. Asynchronous computation: A process can send, receive, and perform computation at any time. 3

Other Assumptions/Models

Processor can fail in 3 modes:

Type of messages:

Crash fault: functioning stops (timeouts can be used). Omission fault: a processor forgets or omits actions, e.g., misses to send a message to 2 out of 5 processors. Malicious fault: a processor can send fictitious messages or modify values in a message, intentionally. Also called Byzantine faults. Authenticated messages: a faulty processor cannot modify a message. Non-authenticated messages: messages can be modify.

Types of Agreement Problems

Byzantine Agreement Problem

A single value is to be agreed upon. Initial value to be proposed by an arbitrary processor: all non-faulty processors need to agree on that value. Every processor has its own initial value. All non-faulty processors agree on a common value.

Consensus Problem

Interactive Consistency Problem

Every processor has its own initial value. All non-faulty processors agree on a set of common values (or) a vector. Above three types are closely related. Byzantine agreement problem forms the basis of other 2 types. We focus on Byzantine agreement.

Byzantine Agreement Problem

An arbitrary source processor broadcasts its initial value to all others. If the source processor is faulty, other nonfaulty processor can agree on any common value. Faulty processors values and agreements do not matter. If faulty processors are in majority, then nonfaulty processors cannot reach an agreement. Number of faulty processors, m, cannot exceed: trunc[(n-1)/3]. This bound can be relaxed for systems using authenticated messages. 6

Impossible Scenario

Consider a system with 3 processors: p0, p1, p2. Two possibilities:

Case 1: p0 (source) is not faulty. p2 is faulty. p1 should agree upon 1 as the value. Not possible. Case 2: p0 is faulty. p1 may agree on 1 and p2 on 0. Case 1: p0 1 p1 1 0 1 p2 1 p1 Case 2: p2 0 0 1 p0

Lamports Algorithm

Referred to as Oral Message Algorithm: OM(m), m > 0. For 3m + 1 or more processors of which m are faulty. Recursive Algorithm:

Algorithm OM(0) Source processor sends its value to every processor. Each processor uses the value it receives from source. (If no value received, default value of 0 assumed).

Lamports Algorithm

Algorithm OM(m), m > 0 Source processor sends its value to every processor. For each processor, let vi be the value received by processor i (from the source). Processor i acts as new source. Initiates OM(m-1). Sends vi to each of other n-2 processors. Let vj be the value received by Pi from Pj in above step. (If no value received, vj is assumed to be 0). Pi
uses majority (v1, v2, ..vn-1).

Lamports Algorithm ...

Processors are successively divided into smaller groups in step 2 where OM(m-1) is executed. Step 3 is executed during the folding phases of recursion where a majority function selects the agreed value (among those received in step 2). Note: majority returns 0 if one does not exist. Number of messages:

OM(m) : n- 1 executions of OM(m-1) OM(m-1): n-2 executions of OM(m-2).... (n-1)(n-2)(n-3)... (n-k) executions of OM(m-k), k = 1,2,3,...,m+1. Message complexity: O(n power m).

10

Lamports Algorithm: Example 1

System with 4 processors: p0, p1, p2, p3. p0 is source, p2 is faulty. Assumption: possible values are only 1 and 0. Step 1: p0 initiates the initial value to be 1. (Algorithm OM(1), m =1). Step 2: OM(0). p1 sends 1 to {p2, p3}. p3 sends 1 to {p1,p2} p2 (the faulty one) sends 1 to p1 and 0 to p3. Step 3: majority function at p1 and p3 is 1, which is the desired result. (Not bothered about p2, the faulty one). p0 p0 1 1 1 1 1 1 1 0 p1 p2 p2 p1 p3 p3 1 1 1 1

11

Lamports Algorithm: Example 2

System with 4 processors: p0, p1, p2, p3. p0 is source, and is faulty. Assumption: possible values are only 1 and 0. Step 1: p0 initiates the initial value to be 1 for p1 and p3. For p2, it sends a 0(Algorithm OM(1), m =1). Step 2: OM(0). p1 sends 1 to {p2, p3}. p3 sends 1 to {p1,p2} p2 sends 0 to p1 and p3. Step 3: majority function at p1, p2, p3 is still the same (1), which is the desired result. p0 1 1 0 1 0 p1 p3 p2 1 0 1 1

12

Fault-tolerant Clock Sync

Synchronizing distributed clocks: At any time, values of clocks of all non-faulty processes must be approximately equal. There is a small bound on amount by which the clock of a non-faulty process is changed during resynchronization. Assumptions:

A1: All clocks are synchronized to approximately the same values. A2: A nonfaulty processs clock run at approximately the same rate. (No such assumptions about faulty clocks). A3: A nonfaulty process can read the clock value of another nonfaulty process with at most a small error .

13

Interactive Convergence

Each process reads the values of others clocks and sets its clock to the average of these values. If a clock value differs from its own clock by more than , it replaces that by its clock for taking average. (Takes care of faulty clocks). Example: Let 2 processes p and q, use Cpr and Cqr as clock values of a 3rd process r. If r is nonfaulty, then Cpr = Cqr. If r is faulty, then |Cpr Cqr| <= 3 . Why? Let Cq = Cp - ; Cpr = Cp + ; Cqr = Cq - . So |Cpr - Cqr| = Cp + - [(Cp - ) - ] = 3 . (Difference in clock values of any 2 processes is bounded by .) For n processes with m faulty ones: p and q use identical values of n-m nonfaulty processes and different values for m faulty processes that is bounded by 3 . So, Cp and Cq differ by at most (3m/n) . Since n > 3m, this difference [(3m/n) ] is always less than .

14

Interactive Convergence ...

Assumptions: All processes execute the algorithm simultaneously. Error in reading another processs clock is 0. This problem (with the assumptions) can be marginalized to a certain extent by: Rather than using absolute clock values, compute the average of difference in clock values. Increment local value by this average. Clock differences > , are replaced by 0. Added assumption: A nonfaulty process can read the difference between the clock value of another nonfaulty process, and its own with at most a small error of . If the clock reading error is , difference in clock values read by a process can be as large as + . Clock differences larger than + are replaced by 0, while computing the average increment.

15

Interactive Consistency

Improvements: Median of clock values rather than the mean. More strict conditions for clock reading: Any 2 processes obtain approximately the same value for a process ps clock. Every nonfaulty process obtains approximately the correct value of another nonfaulty processs clock. If majority of the processes are nonfaulty, median of all clock values is either approximately equal to a good clocks value (or) lies between the values of 2 good clocks.

16