2.

Amortized Analysis
Amortized Analysis refers to the complexity of a sequence of operations, that is, a sequence of
operations. It exists in order to obtain a more accurate complexity in certain scenarios. Because if
you want to perform a series of operations, some of the op complexity may be very high, and
sometimes the op complexity may be very low. In this way, if we use the most traditional worst-
case runtime complexity, it may give a very, very pessimistic estimate. We use Amortized
Analysis to give an (average) complexity estimate of some operations over a period of time. In
contrast, the average case complexity. The complexity of the average case is based on multiple
independent random inputs and then averaged; the complexity of amortized analysis is based on
a sequence.

2.1 Unbounded Array

The first is the introduction of Unbounded Array, which is an array of indefinite length. If you
have learned the data structure, you should understand that the array we applied is a continuous
storage space in the computer memory. ForInsert Operationally, if there is space, then the
complexity is O(1); if the space is full, you need to firstcopy Existing array, apply for a larger
memory, andcopy Is linear O(n), so this timeInsert Also O(n). So worst-caseInsert takes O(n).

Unbounded Array is actually such a thing. ByInsert to fulfill. So what is the amortized cost of
this op?
Before answering this question, we must solve another problem, that isInsert When full, a
mechanism to apply for more memory. We again assume that it is increase byk. (Linear increase,
examplek = 2) In this scenario, we insert a series of numbers. (As shown below)

It can be seen that all insertions, if it is an odd number of times, then O(1), even number of times
linear, because it is full to copy. Assuming n insertions, its amortized cost = n 2 + ∑ i = 1 n / 2 2
i = O ( n 2 ) \frac{n}{2}+\sum_{i=1}^{n/2}2i = O(n^2) 2n+∑i=1n/22i=O(n2), So for this
insertion, the amortized cost per operation is O(n).
There is a corollary here that if it is linear growth (increase by), then the insert of Unbounded
Array is O(n). And if you apply for new memory (multiply by) in proportion, then it is O(k), and
k is a constant. In addition, when using amortized cost analysis, O(k) is not written as O(1),
which needs a little attention.
If we are multiply by 2, then suppose to insert 2n+1 times, only n+1 copies, amortized cost = 2 n
− n + ∑ i = 0 n 2 i = O ( 3 ⋅ 2 n ) 2^n-n+\sum_{i=0}^{n}2^i = O(3·2^n) 2n−n+∑i=0n
2i=O(3⋅2n)，amortized cost per operation = 3 ⋅ 2 n 2 n + 1 = O ( 3 ) \frac{3·2^n}{2^n+1}= O(3)
2n+13⋅2n=O(3)

The above analysis method is calledAggregate Method.

Another analysis method is calledAccounting Method, Its core is to give each a certain amount
oftokenEach time we want to make an op, we obtain its corresponding token, and this operation
also requires the consumption of a certain existing token, and the final amortized cost of the op is
the minimum token that can make a series of ops successfully complete. .

The same is the example above (multiply by 2), adding we give insert 1 token. Then insert the
first number and get a token, but one token will be consumed in one assignment. When inserting
the second number, you get a token, but you need to apply for more memory first, copy the first
number first, and consume a token. At this time, the token balance is 0, and you cannot assign
the second number. So a token insert will not work. As shown below.

Correspondingly, increase the insertion token until it can be guaranteed to be feasible no matter
how it is inserted. At least 3 tokens are required here. The situation of 2 token and 3 token is
shown below.
Fail is because the balance is insufficient, the bank will not loan... For this example, it can be
seen that this method also obtains the amortized cost per operation of O(3).

2.2 Binary Counter

Binary Counter is another model different from Unbounded Array.

Given a binary number n n n, with log ⁡n \log n logn bits, stored as an array, where each entry
A[i] stores the i-th bit, the cost of incrementing a binary number is the number of bits flipped. We
use the standard way of incrementing the counter, which is to toggle the lowest order bit.
That is, given a number expressed in log n bits, we want to increase it and increase it by the least
significant bit +1, that is, binary addition. The cost is the number of digits changed every +1,
obviously the cost of increment is different each time. So what is the amortized cost of this
increment? (The cost of each flip is given here is the same, derived from cost is the number of
bits flipped)
Assuming n = 8, 3 bits, use Aggregate Method:

The lowest bit changes every time, the penultimate bit changes every two increments, the
penultimate bit changes every four times, and so on, up to the highest bit. Total Work = $
n\sum_{i=1}^{-1+\log n} \frac{1}{2^i} = O(n)$, amortized cost per operation = O(1).
The change of this type of model may be that the cost of each bit of flip is different, but the
analysis method is the same.

2.3 Amortized Dictionary

The amortized dictionary here is very similar to Binomial Heap. It is a nesting of linked list and
sorted array. Each node is a sorted array with a unique length, and the size of array k (binary) is
2k. For example, if 11 numbers are stored, then the 11 numbers will be split into 11 = 1 + 2 + 8,
which is B corresponding to Binomial Tree0, B1, B3。

For this dictionary, he has to provide insertion and search op. For insert, we need to maintain its
structure and ordering. When inserting new data, you may want to merge the existing array, then
it takes linear time to merge.
For example, insert a new element 1 at this time, then two Bs appear0，size 1，1，2，8    ⟹   
\implies ⟹ 2，2，8    ⟹    \implies ⟹ 4, 8. Among them, merge ordered array O(n),
respectively 2+4.

If we consider worst-case, then every insertion will necessarily involve merge, then if we have n
numbers, we need at most log n so many nodes. Then worst-case, we must merge from 1, 1-2-
4-...- log n, complexity = 2 ⋅ ∑ i = 1 log ⁡n i = O ( n ) 2·\sum_{i=1}^{\log n} i = O(n)
2⋅∑i=1logni=O(n), Insert once. n times is O(n2)... But it is obviously impossible to encounter this
situation every time you insert, so we look at its amortized cost to get the more practical (more
accurate) complexity of sequence insertion.

Suppose we insert n numbers consecutively. So this is actually the corresponding binary counter,
but the cost of each merge, the cost of each bit corresponding to flip is no longer O(1), but a
weight, depending on which is the number. The lowest bit of the binary filp cost 1, n times, the
penultimate cost 2, n/2 times, the penultimate 4, 4, n/4 times... so total cost = $ n·1+\frac{n}{2 }
·2+\frac{n}{4}·4+…+2·2^{-1+\log n} = n·\sum_{i=0}^{-1+\log n}2{-i}·2i = O(n\log n)$

2.4 Amortized Tree

In the last part of this article, about the Amortized Tree, the transformation of the binary search
tree-considering that the binary search tree does not guarantee balance, the worst becomes a
linear search. The amortized tree here is a variant-every time the search is completed, the search
tree is reconstructed, so that we let the number of the last search become the root node of the new
tree, called splay tree. The idea behind this approach is that our search for a period of time is
very likely to be directed to the same local area, similar to cache or swap in a block to save costs.
One of the more important operations isSplay(N), Put searchedN move to root. And we want to
ensure that the operation is in O ( log ⁡n ) O(\log n) O(logn) Completion within a period of time
involves 6 kinds of rotation operations-basically what the undergraduate data structure learns.
 1 Zig(Zag)-Right-handed (left-handed)
When the parent of N is root, turn right, zig. As shown below. N is P, right child is left-
handed.

 Zig-Zag (Zag-Zig)——Right-handed (left-handed), then left-handed (right-handed), as

shown below.
  Zig-Zig (Zag-Zag)-Right-handed (left-handed), then right-handed (left-handed), as
shown below.

Note that there is a seemingly feasible Zig-Zig, but it does not guarantee O(log n) rotation, but
linear, as shown below.

To illustrate that the above Zig-Zig rule that prefers rotate N is wrong, we assume the following
situation. Initially we have a sorted array, 1, 2, 3, ..., n. Left fork all the way to the end. As shown
In this way, if we search by Splay(1), Splay(2), Splay(3),..., Splay(n), then Splay(1) rotates n
times from the bottom, Splay(2) needs to be Zig n-1 times from the bottom... until the last
Splay(n) Zig 1 times. So total cost$ = n + n-1 + n-2… + 1 = O(n^2)$, amortized cost per
operation = O ( n ) O(n) O(n)。
for guarantee O ( log ⁡n ) O(\log n) O(logn)The amortized cost per operation can only use the
Zig-Zig (Zag-Zag) rotation rule. At this time, the complexity of performing m operations on a
Splay Tree with n nodes is O ( m log ⁡n ) O(m\log n) O(mlogn)。

proof.(Prove that it's super class...)

Algorithm Design This book does not find Amortized Analysis related content, all content comes
from the textbook edited by Victor himself. 570's lecture only covers 2.1 and 2.2

Discussion Problem: Using two stacks to implement a FIFO queue, stack has POP and PUSH
operations, cost of each is O(1). The FIFO should have ENQUEUE and DEQUEUE operation,
seeking amortized cost?

Solution: First of all, for sequence operation, we can only make it a series ofENQUEUE,If you
want to DEQUEUE It must be out of the queue. If it is implemented, all enqueue operations are
first pushed to stack A, then popped, then pushed to another stack B, and completed into the
queue. All dequeue pops one by one from stack B. Then use the accounting method analysis
here, consider the distributionENQUEUE and DEQUEUE The least feasible token.ENQUEUE
If token 1, then there is no balance after all pushes to A. Pop is completeENQUEUE.
IfENQUEUE Given 2 tokens, there is no balance to push A after n times, and there is no balance
to push B after pop A. IfENQUEUE Give 3 tokens, then you can ensure that each item can be
queued, 1 push A, 1 pop A, and 1 push B out of 3 tokens. andDEQUEUEYou only need to pop
B, O(1) directly, and then pop the stack in sequence, without a balance, 1 token is fine.