You are on page 1of 1

reports which are old, so the failure was signaled by the majority of masters

within a window of time.


2. While every node detecting the FAIL condition will force that condition on other
nodes in the cluster using the FAIL message, there is no way to ensure the
message will reach all the nodes. For instance a node may detect
the FAIL condition and because of a partition will not be able to reach any other
node.
However the Redis Cluster failure detection has a liveness requirement: eventually all
the nodes should agree about the state of a given node. There are two cases that can
originate from split brain conditions. Either some minority of nodes believe the node is
in FAIL state, or a minority of nodes believe the node is not in FAIL state. In both the
cases eventually the cluster will have a single view of the state of a given node:
Case 1: If a majority of masters have flagged a node as FAIL, because of failure
detection and the chain effect it generates, every other node will eventually flag the
master as FAIL, since in the specified window of time enough failures will be reported.

Case 2: When only a minority of masters have flagged a node as FAIL, the slave
promotion will not happen (as it uses a more formal algorithm that makes sure
everybody knows about the promotion eventually) and every node will clear
the FAIL state as per the FAIL state clearing rules above (i.e. no promotion after N
times the NODE_TIMEOUT has elapsed).

The FAIL flag is only used as a trigger to run the safe part of the algorithm for the
slave promotion. In theory a slave may act independently and start a slave promotion
when its master is not reachable, and wait for the masters to refuse to provide the
acknowledgment if the master is actually reachable by the majority. However the added
complexity of the PFAIL -> FAIL state, the weak agreement, and the FAIL message
forcing the propagation of the state in the shortest amount of time in the reachable part
of the cluster, have practical advantages. Because of these mechanisms, usually all the
nodes will stop accepting writes at about the same time if the cluster is in an error state.
This is a desirable feature from the point of view of applications using Redis Cluster.
Also erroneous election attempts initiated by slaves that can't reach its master due to
local problems (the master is otherwise reachable by the majority of other master
nodes) are avoided.

Configuration handling, propagation,


and failovers

You might also like