Professional Documents
Culture Documents
Unit Ii
Unit Ii
-----------------------------------------------------------------------------------------------
Consensus Algorithms And Permissioned Blockchain: Block Propagation,
Consensus -Requirements for the consensus protocols, Scalability aspects of Block
chain consensus protocols, Consensus in Bitcoin, Solving Double Spending,
Attacks, Proof of Work (PoW), Proof of Stake (PoS), Proof of Burn, Proof of
Elapsed Time (PoET), Mining.
Permissioned Block chain: Basics, Smart Contracts, Consensus State Machine
Replication, Crowd Funding, distributed Consensus.
Consensus Algorithms: Requirements, PAXOS consensus Algorithm, RAFT
consensus Algorithm, Byzantine general problem, Practical Byzantine fault
tolerance Algorithm, and Three phases commit Protocol
-----------------------------------------------------------------------------------------------
BLOCK PROPAGATION
The P2P network for Bitcoin is composed of miner nodes that connect to one
another at random. These nodes disseminate transactions and blocks throughout
the network until all have received the information. Hops are required for a
transaction to spread a message throughout the network. The network dispersion
increases by a factor of 2n with each iteration as a set of two nodes are delivered
the message. After twelve to fifteen hops, the message is distributed over the whole
network, with the diffusion growing exponentially as the number of hops rises.
The throughput of a blockchain is determined by the number of transactions it can
support per second and is expressed as –
Where, Transactions Block is the factor of the current block capacity of the Bitcoin
and average transaction size.
With an average transaction size of about 540 bytes and the current Bitcoin block
capacity of 1MB for a block interval of ten minutes, the network now handles about
1,950 transactions each block, or about three transactions every second (TPS).
Either the transactions per block or the blocks per second can be increased to boost
throughput. It has already taken a lot of effort to use both on-chain and off-chain
techniques to increase the number of transactions per block. Teams have
experimented with expanding the block size by ten times in an effort to multiply
the capacity by a factor of ten. A higher block size typically maintains the hop count
at twelve to fifteen for the whole network, but necessitates delivering a bigger block
(one block of 10MB) through the Bitcoin connection.
Although the transaction/block size grows in this case, it also raises the issue of
network throttling, which exponentially lengthens propagation times and raises
the stakes in the argument over Bitcoin splits. One such attempt is the hard fork of
Bitcoin into Bitcoin Cash.
Double-spending
Double-spending is the outcome of spending some money more than once. Bitcoin
users secured themselves from double-spending cheating by waiting for
verification when receiving payments on the blockchain. There is a possibility that
a digital currency can be spent twice. Transactions in bitcoin are a digital file. It is
possible to duplicate transactions and spend the same Bitcoin twice.
Used by Etherium, Bitcoin, and other public blockchains, PoW is the most reliable
and secure consensus algorithm, but scalability concerns are plentiful.
Using Proof of Work (PoW) verification for every transaction helps the core
network of the blockchain prevent double-spending. In other words, if anyone tries
to duplicate a transaction in the blockchain network, it will be seen in the system
and will not be accepted. Hence, no one can change the transaction once it has been
verified and approved by every node participant.
Proof of Authority
The Proof of Authority consensus algorithm uses the value of identities, so the
block validators don’t stake coins but their reputations. As this consensus
algorithm depends on the limited number of validators, it makes the system highly
scalable.
In Proof of Authority, the rights to create new blocks are given to participant nodes
that have proven their authority.
Moreover, PoS saves both energy and time. But, to become a validator, participants
need to invest some stake (money). And, the miners also get the opportunity to
take their transaction fees as there are no rewards for them in the system.
In DPoS, there are 21-100 delegates charged periodically and assigned to deliver
their blocks. Having fewer delegates allows for an efficient organization to design
time slots for publishing blocks in the network. In case of, insufficient, invalid, or
missing block publishing, the miners vote them out to be replaced with other
selected delegates.
As DPoS works on the stake-weighted voting system, it has become one of the
fastest growing and adapted blockchain consensus models.
In LPoS, users lease crypto tokens to the node that wants to act as a block producer
for the network. A node with the maximum number of staked tokens is more likely
to be selected for the next block generation as well as receive rewards.
It also helps users with smaller tokens who might not have been eligible for
participating as the blockchain creator in the traditional proof of Stake process in
pooling their assets while enhancing their chances of receiving network transaction
fees’ share.
The leased proof of stake (LPoS) consensus algorithm is best for networks with
high high-technical requirements for operating full nodes capable of verifying and
validating transactions.
Proof of Importance (PoI) is also an advanced version of Proof of Stake (PoS) that
considers other factors than just weighing the level of influence of every node on
the network. It aims to take a comprehensive approach to evaluate contributions
from users rather than focusing on capital needs.
As the name suggests, DAG looks like a graph rather than a set of chains and is
being seen as the possible alternative for blockchain due to its effective storage and
processing of transactions. They are directed because they move in one direction
and acyclic because the vertices do not loop.
In the DAG consensus mechanism, each node prepares to be the miner and reduces
the associated costs as the transactions get validated by users themselves, making
the whole process fastest, lightweight, and secured.
So, Byzantine Fault Tolerance (BFT) is the blockchain system property that resists
the failures that might derive from this problem. Simply put, a BFT system can
continuously operate even when nodes act maliciously or fail.
BFT is used to fix the problem of unreliable nodes in the network. pBFT and dBFT
are two types of BFTs being used for determining unreliable chain code operations.
dBFT determines the validators based on real-time voting which improves the
algorithm and saves time and confirmation time for transactions. The selected
team of validators then uses the BFT algorithm to generate more blocks.
Needless to say, the PoI consensus algorithm helps in ensuring the integrity and
authenticity of the data, and therefore, is a great choice for being used in smart
cities.
The idea behind PoET is to equally distribute and expand bigger participants'
fractions by asking each of them to wait for a certain amount of time before
working on the next process. So, whenever a block is available and the node is
activated, that node gets the authority to share the information on the network
while maintaining decentralization and earning rewards.
Permissioned blockchains
Permissioned blockchains are blockchains that are closed or have an access control
layer. This additional layer of security only allows participants to perform the
actions that they are authorized to perform.
History: In 1994, Nick Szabo, a legal scholar, and a cryptographer recognized the
application of a decentralized ledger for smart contracts. He theorized that these
contracts could be written in code which can be stored and replicated on the
system and supervised by the network of computers that constitute the
blockchain. These smart contracts could also help in transferring digital assets
between the parties under certain conditions.
Features of Smart Contracts
The following are some essential characteristics of a smart contract:
1. Distributed: Everyone on the network is guaranteed to have a copy of all the
conditions of the smart contract and they cannot be changed by one of the
parties. A smart contract is replicated and distributed by all the nodes
connected to the network.
2. Deterministic: Smart contracts can only perform functions for which they are
designed only when the required conditions are met. The final outcome will not
vary, no matter who executes the smart contract.
3. Immutable: Once deployed smart contract cannot be changed, it can only be
removed as long as the functionality is implemented previously.
4. Autonomy: There is no third party involved. The contract is made by you and
shared between the parties. No intermediaries are involved which minimizes
bullying and grants full authority to the dealing parties. Also, the smart contract
is maintained and executed by all the nodes on the network, thus removing all
the controlling power from any one party’s hand.
5. Customizable: Smart contracts have the ability for modification or we can say
customization before being launched to do what the user wants it to do.
6. Transparent: Smart contracts are always stored on a public distributed ledger
called blockchain due to which the code is visible to everyone, whether or not
they are participants in the smart contract.
7. Trustless: These are not required by third parties to verify the integrity of the
process or to check whether the required conditions are met.
8. Self-verifying: These are self-verifying due to automated possibilities.
9. Self-enforcing: These are self-enforcing when the conditions and rules are met
at all stages.
Imagine a file, replicated across different servers around the world. An algorithm
that ensures that a change you make on that file is propagated to all of its replicas,
is called a consensus algorithm.
Paxos is a family of consensus protocols, that ensure replica consistency in a
distributed system of unreliable processors (that is servers can fail). The Paxos
protocol was introduced in 1989 by Leslie Lamport, named after a fictional
legislative consensus system used on the Paxos island in Greece
Assumptions
The following assumptions are made about the processors and the network on
which the protocol is implemented.
Processors
• Operate at arbitrary speed and may experience failures.
• Processors with stable storage may re-join the protocol after failures (following
a crash-recovery failure model).
• Do not collude, lie, or otherwise attempt to subvert the protocol. (Byzantine
failures aren’t considered)
Network
• Processors can send messages to any other processor.
• Messages are sent asynchronously and may take arbitrarily long to deliver.
• Messages may be lost, reordered, or duplicated.
• Messages are delivered without corruption.
Commit phase: If all the replicas accept the message, then the proposer sends
an accept request to all the replicas and the change is then asynchronously
propagated to all of them.
However, even if one of those replicas reject the proposer’s request, then the
proposer may abort the change or retry the commit with a higher number.
Raft Consensus Algorithm
It is named after Reliable, Replicated, Redundant, And Fault-Tolerant. The Raft
consensus algorithm mascot. Raft is not a Byzantine fault tolerant algorithm: the
nodes trust the elected leader.
Introduction
Raft protocol was developed by Diego Ongaro and John Ousterhout (Stanford
University) which won Diego his Ph.D in 2014(The link for the paper is in the
References section at the end of the article). Raft was designed for better
understandability of how Consensus(we will explain what consensus is, in a
moment) can be achieved considering that its predecessor, the Paxos Algorithm,
developed by Lesli Lamport is very difficult to understand and implement. Hence,
the title of the paper by Diego, ‘In Search of an Understandable Consensus
Algorithm’. Before Raft, Paxos was considered the holy grail in achieving
Consensus..
Consensus
So, to understand Raft, we shall first have a look at the problem which the Raft
protocol tries to solve and that is achieving Consensus. Consensus means multiple
servers agreeing on same information, something imperative to design fault-
tolerant distributed systems. Lets describe it with the help of couple visuals.
So, lets first define the process used when a client interacts with a server to clarify
the process.
Process : The client sends a message to the server and the server responds back
with a reply.
A consensus protocol tolerating failures must have the following features :
Now, there can be two types of systems assuming only one client(for the sake of
understandability):
• Single Server system: The client interacts with a system having only one
server with no backup. There is no problem in achieving consensus in such a
system.
• Multiple Server system: The client interacts with a system having multiple
servers. Such systems can be of two types:
• Symmetric :- Any of the multiple servers can respond to the client and all the
other servers are supposed to sync up with the server that responded to the
client’s request, and
• Asymmetric: - Only the elected leader server can respond to the client. All
other servers then sync up with the leader server.
Such a system in which all the servers replicate (or maintain) similar data (shared
state) across time can for now be referred to as, replicated state machine.
We shall now define some terms used to refer individual servers in a distributed
system.
• Leader – Only the server elected as leader can interact with the client. All other
servers sync up themselves with the leader. At any point of time, there can be at
most one leader(possibly 0, which we shall explain later)
• Follower – Follower servers sync up their copy of data with that of the leader’s
after every regular time intervals. When the leader server goes down(due to any
reason), one of the followers can contest an election and become the leader.
• Candidate – At the time of contesting an election to choose the leader server,
the servers can ask other servers for votes. Hence, they are called candidates
when they have requested votes. Initially, all servers are in the Candidate state.
So, the above system can now be labelled as in the following snap.
CAP theorem CAP Theorem is a concept that a distributed database system can
only have 2 of the 3:
• Consistency – The data is same in all the server nodes (leader or follower),
implying the system has nearly instantaneous sync capabilities
To begin with, Raft states that each node in a replicated state machine (server
cluster) can stay in any of the three states, namely, leader, candidate, follower. The
image below will provide the necessary visual aid.
Under normal conditions, a node can stay in any one of the above three states. Only
a leader can interact with the client; any request to the follower node is redirected
to the leader node. A candidate can ask for votes to become the leader. A follower
only responds to candidate(s) or the leader.
To maintain these server status(es), the Raft algorithm divides time into small
terms of arbitrary length. Each term is identified by a monotonically increasing
number, called term number.
Term number
Following tasks are executed by observing the term number of each node:
• Servers update their term number if their term number is less than the term
numbers of other servers in the cluster. This means that when a new term starts,
the term numbers are tallied with the leader or the candidate and are updated
to match with the latest one(Leader’s)
• Candidate or Leader demotes to the Follower state if their term number is out
of date(less than others). If at any point of time, any other server has a higher
term number, it can become the Leader immediately.
• As we said earlier that the term number of the servers are also communicated,
if a request is achieved with a stale term number, the said request is rejected.
This basically means that a server node will not accept requests from server with
lower term number
Raft algorithm uses two types of Remote Procedure Calls(RPCs) to carry out the
functions :
Leader election
In order to maintain authority as a Leader of the cluster, the Leader node sends
heartbeat to express dominion to other Follower nodes. A leader election takes
place when a Follower node times out while waiting for a heartbeat from the Leader
node. At this point of time, the timed out node changes it state to Candidate state,
votes for itself and issues RequestVotes RPC to establish majority and attempt to
become the Leader. The election can go the following three ways:
• The Candidate node becomes the Leader by receiving the majority of votes from
the cluster nodes. At this point of time, it updates its status to Leader and starts
sending heartbeats to notify other servers of the new Leader.
• The Candidate node fails to receive the majority of votes in the election and
hence the term ends with no Leader. The Candidate node returns to the
Follower state.
• If the term number of the Candidate node requesting the votes is less than other
Candidate nodes in the cluster, the AppendEntries RPC is rejected and other
nodes retain their Candidate status. If the term number is greater, the
Candidate node is elected as the new Leader.
Fig: Raft leader election
The following excerpt from the Raft paper(linked in the references below) explains
a significant aspect of server timeouts.
Raft uses randomized election timeouts to ensure that split votes are rare and that
they are resolved quickly. To prevent split votes in the first place, election timeouts
are chosen randomly from a fixed interval (e.g., 150–300ms). This spreads out the
servers so that in most cases only a single server will time out; it wins the election
and sends heartbeats before any other servers time out. The same mechanism is
used to handle split votes. Each candidate restarts its randomized election timeout
at the start of an election, and it waits for that timeout to elapse before starting the
next election; this reduces the likelihood of another split vote in the new election.
Log Replication
For the sake of simplicity while explaining to the beginner level audience, we will
restrict our scope to client making only write requests. Each request made by the
client is stored in the Logs of the Leader. This log is then replicated to other
nodes(Followers). Typically, a log entry contains the following three information :
• Command specified by the client to execute
• Index to identify the position of entry in the log of the node. The index is 1-
based(starts from 1).
There is a concept of entry commit in the algorithm. When the majority of the
servers in the cluster successfully copy the new entries in their logs, it is considered
committed. At this point, the Leader also commits the entry in its log to show that
it has been successfully replicated. All the previous entries in the log are also
considered committed due to obvious reasons. After the entry is committed, the
leader executes the entry and responds back with the result to the client.
It should be noted that these entries are executed in the order they are received.
If two entries in different logs(Leader’s and Followers’) have identical index and
term, they are guaranteed to store the same command and the logs are identical
upto that point(Index).
However, in case the Leader crashes, the logs may become inconsistent. Quoting
the Raft paper :
The Leader node will look for the last matched index number in the Leader and
Follower, it will then overwrite any extra entries further that point(index number)
with the new entries supplied by the Leader. This helps in Log matching the
Follower with the Leader. The AppendEntries RPC will iteratively send the RPCs
with reduced Index Numbers so that a match is found. When the match is found,
the RPC succeeds.
Safety
In order to maintain consistency and same set of server nodes, it is ensured by the
Raft consensus algorithm that the leader will have all the entries from the previous
terms committed in its log.
During a leader election, the RequestVote RPC also contains information about the
candidate’s log(like term number) to figure out which one is the latest. If the
candidate requesting the vote has less updated data than the Follower from which
it is requesting vote, the Follower simply doesn’t vote for the said candidate. The
following excerpt from the original Raft paper clears it in a similar and profound
way.
Raft determines which of two logs is more up-to-date by comparing the index and
term of the last entries in the logs. If the logs have last entries with different terms,
then the log with the later term is more up-to-date. If the logs end with the same
term, then whichever log is longer is more up-to-date.
The Raft protocol guarantees the following safety against consensus malfunction
by virtue of its design:
• Log Matching safety(If multiple logs have an entry with the same index and
term, then those logs are guaranteed to be identical in all entries up through to
the given index.
• Leader completeness – The log entries committed in a given term will always
appear in the logs of the leaders following the said term)
• State Machine safety – If a server has applied a particular log entry to its state
machine, then no other server in the server cluster can apply a different
command for the same log.
• Follower node crash – When the follower node crashes, all the requests sent to
the crashed node are ignored. Further, the crashed node can’t take part in the
leader election for obvious reasons. When the node restarts, it syncs up its log
with the leader node
• The Raft protocol has been decomposed into smaller subproblems which can
be tackled relatively independently for better understanding, implementation,
debugging, optimizing performance for a more specific use case
• The distributed system following the Raft consensus protocol will remain
operational even when minority of the servers fail. For example, if we have a 5
server node cluster, if 2 nodes fail, the system can still operate.
• The leader election mechanism employed in the Raft is so designed that one
node will always gain the majority of votes within a maximum of 2 terms.
• The Raft employs RPC(remote procedure calls) to request votes and sync up the
cluster(using AppendEntries). So, the load of the calls does not fall on the leader
node in the cluster.
• Raft was designed recently, so it employs modern concepts which were not yet
understood at the time of the formulation of the Paxos and similar protocols.
• Any node in the cluster can become the leader. So, it has a certain degree of
fairness.
• Many different open source implementations for different use cases are already
out there on GitHub and related places
Raft Alternatives
Limitations
• Raft is strictly single Leader protocol. Too much traffic can choke the system.
Some variants of Paxos algorithm exist that address this bottleneck.
• Cheap-paxos(a variant of Paxos), can work even when there is only one node
functioning in the server cluster. To generalise, K+1 replicated servers can
tolerate shutting down of/ fault in K servers.
In 1982, The Byzantine General’s Problem was invented by Leslie Lamport, Robert
Shostak, and Marshall Pease. Byzantine Generals Problem is an impossibility
result which means that the solution to this problem has not been found yet as well
as helps us to understand the importance of blockchain. It is basically a game
theory problem that provides a description of the extent to which decentralized
parties experience difficulties in reaching consensus without any trusted central
parties.
• The Byzantine army is divided into many battalions in this classic problem
called the Byzantine General’s problem, with each division led by a general.
• The generals connect via messenger in order to agree to a joint plan of action in
which all battalions coordinate and attack from all sides in order to achieve
success.
• As a result, the purpose of this challenge is for all of the faithful commanders to
reach an agreement without the imposters tampering with their plans.
Money is one such commodity whose value should be same throughout the society,
that is everyone should agree upon the value of a certain amount of money, despite
all the differences therefore in the initial times, precious metals and rare goods
were chosen as money because their value was seen equally throughout the society,
but in some cases such as precious metals the purity of the metals could not be
known for sure or checking the purity was an extremely tedious task which turned
out to be very inefficient for the daily transactions, therefore it was decided upon
to replace gold with a central party which would be highly trustable chosen by the
people in the society to establish and maintain the system of money. But with time
it was later realized that those central parties, how much-ever qualified were still
not completely trustworthy as it was so simple for them to manipulate the data.
• They forgo transparency in order to attain efficiency easily and prefer to avoid
dealing with the issue entirely.
• The fundamental issue of centralized systems, however, is that they are open to
corruption by the central authority, which implies that the data can be
manipulated by anyone who has control of the database itself because the
centralized system concentrates all power on one central decision maker.
Therefore, Bitcoin was invented to make the system of money decentralized using
blockchain to make money verifiable, counterfeit-resistant, trustless, and separate
from a central agency.
In the Byzantine Generals Problem, the untampered agreement that all the loyal
generals need to agree to is the blockchain. Blockchain is a public, distributed
ledger that contains the records of all transactions. If all users of the Bitcoin
network, known as nodes, could agree on which transactions occurred and in what
order, they could verify the ownership and create a functioning, trustless money
system without the need for a centralized authority. Due to its decentralized
nature, blockchain relies heavily on a consensus technique to validate transactions.
It is a peer-to-peer network that offers its users transparency as well as trust. Its
distributed ledger is what sets it apart from other systems. Blockchain technology
can be applied to any system that requires proper verification.
In order to address this issue, honest nodes (such as computers or other physical
devices) must be able to establish an agreement in the presence of dishonest nodes.
• The divisions can be viewed as computer nodes in the network, and the
commanders as programs running a ledger that records transactions and events
in the order that they occur. The ledgers are the same for all systems, and if any
of them is changed, the other ledgers are updated as well if the changes are
shown to be true, so all distributed ledgers should be in agreement.
A basic Byzantine fault is a digital signal that is stuck at “1/2,” i.e. a voltage that is
anywhere between the voltages for a valid logical “0” and a valid logical “1.”
Because these voltages are in the region of a gate’s transfer function’s maximum
gain, little quantities of noise on the gate’s input become enormous amounts of
noise on the gate’s output. This is due to the fact that “digital circuits are simply
analog circuitry driven to extremes.”
This problem is solvable because, with a dominating input, even a Byzantine input
has no output impact.
• If one of the inputs is “1/2” and the other two are both 0 or both 1, the result is
0 or 1 (due to masking within the voter).
• When one of the inputs is “1/2” and the other two are different values, the
output can be 0, “1/2,” or 1, depending on the precise gain and threshold
voltages of the voter gates and the properties of the “1/2” signal.
Instead of directly noting the commit decision in its persistent storage, the
coordinator first ensures that at least ‘k’ other sites know that it intended to commit
transaction.
In a situation where coordinator fails, remaining sites are bound to first select new
coordinator. This new coordinator checks status of the protocol from the
remaining sites. If the coordinator had decided to commit, at least one of other ‘k’
sites that it informed will be up and will ensure that commit decision is respected.
The new coordinator restarts third phase of protocol if any of rest sites knew that
old coordinator intended to commit transaction. Otherwise, new coordinator
aborts the transaction.
Dissemination Phase:
It is introduced above helps the user to deal with the cases as when either a
participant failure or both coordinator and participant node failure during the
commit phase.
The recovery coordinator when it takes over after coordinator failure during Phase-
2 of the previous Two-Phase Commit Protocol the new additional phase comes
handy as follows.
Note –On querying participants, if it learns that some nodes are in the commit
phase, it then assumes that previous coordinator before crashing has made
decision to commit.