Professional Documents
Culture Documents
Sebastian Wild
27 January 2020
e. g. Java program
More precisely:
1. mechanically executable
� no “common sense” needed
2. finite description ≠ finite computation!
1
What is a data structure?
A data structure is
1. a rule for encoding data
(in computer memory), plus
2. algorithms to work with it
(queries, updates, etc.)
2
1.1 Algorithm analysis
Good algorithms
Our goal: Find good (best?) algorithms and data structures for a task.
3
Running time experiment
Why not simply run and time it?
4
Data Models
Algorithm analysis typically uses one of the following simple data models:
� worst-case performance:
consider the worst of all inputs as our cost metric
� best-case performance:
consider the best of all inputs as our cost metric
� average-case performance:
consider the average/expectation of a random input as our cost metric
5
1.2 The RAM Model
Machine models
The machine model decides
� what algorithms are possible
6
Random Access Machines
Random access machine (RAM) more detail in §2.2 of Sequential and Parallel Algorithms and Data Structures
by Sanders, Mehlhorn, Dietzfelbinger, Dementiev
� every memory cell MEM[𝑖] and register 𝑅 𝑖 stores a 𝑤-bit integer, i. e., a number in
[0..2𝑤 − 1]
𝑤 is the word width; typically 2𝑤 ≈ 𝑛
� Instructions:
� load & store: 𝑅 𝑖 := MEM[𝑅 𝑗 ] MEM[𝑅 𝑗 ] := 𝑅 𝑖
� operations on registers: 𝑅 𝑘 := 𝑅 𝑖 + 𝑅 𝑗 (arithmetic is modulo 2𝑤 !)
also 𝑅 𝑖 − 𝑅 𝑗 , 𝑅 𝑖 · 𝑅 𝑗 , 𝑅 𝑖 div 𝑅 𝑗 , 𝑅 𝑖 mod 𝑅 𝑗
C-style operations (bitwise and/or/xor, left/right shift)
� conditional and unconditional jumps
8
1.3 Asymptotics & Big-Oh
Why asymptotics?
Algorithm analysis focuses on (the limiting behavior for infinitely) large inputs.
� abstracts from unnecessary detail
� simplifies analysis
·104
2
1.5
0.5
0
0 10 20 30 40 50 60 70 80 90 100
9
Why asymptotics?
Algorithm analysis focuses on (the limiting behavior for infinitely) large inputs.
� abstracts from unnecessary detail
� simplifies analysis
·104
2
1.5
0.5
0
0 10 20 30 40 50 60 70 80 90 100
9
Asymptotic tools – Formal & definitive definition
if, and only if
𝑓 (𝑛)
� “Tilde Notation:” 𝑓 (𝑛) ∼ 𝑔(𝑛) iff = 1 lim
𝑔(𝑛) 𝑛→∞
„ 𝑓 and 𝑔 are asymptotically equivalent”
� �
�
also write ‘=’ instead
� � 𝑓 (𝑛) �
� �
� “Big-Oh Notation:” 𝑓 (𝑛) ∈ 𝑂 𝑔(𝑛) iff � � is bounded for 𝑛 ≥ 𝑛0
� 𝑔(𝑛) �
� �
� 𝑓 (𝑛) �
� �
need supremum since limit might not exist!
10
Asymptotic tools – Intuition
𝑐 𝑔(𝑛)
� 𝑓 (𝑛) = 𝑂(𝑔(𝑛)): 𝑓 (𝑛) is at most 𝑔(𝑛) 𝑐 𝑔 (𝑛 )
𝑓 (𝑛 ) ≤
up to constant factors and 𝑓 (𝑛)
𝑛
𝑛0
𝑐 2 𝑔(𝑛)
� 𝑓 (𝑛) = Θ(𝑔(𝑛)): 𝑓 (𝑛) is equal to 𝑔(𝑛) )
𝑓 (𝑛 ) ≤ 𝑐 2 𝑔 (𝑛
up to constant factors and 𝑐 1 𝑔 (𝑛 ) ≤
𝑓 (𝑛)
for sufficiently large 𝑛 𝑐 1 𝑔(𝑛)
𝑛
𝑛0
Plots can be misleading! Example
11
Clicker Question
A 𝑔(𝑛) = 𝑂( 𝑓 (𝑛))
B 𝑔(𝑛) = Ω( 𝑓 (𝑛))
C 𝑔(𝑛) = Θ( 𝑓 (𝑛))
pingo.upb.de/622222
12
Clicker Question
A 𝑔(𝑛) = 𝑂( 𝑓 (𝑛))
B 𝑔(𝑛) = Ω( 𝑓 (𝑛)) �
C 𝑔(𝑛) = Θ( 𝑓 (𝑛))
pingo.upb.de/622222
12
Clicker Question
A 𝑔(𝑛) = 𝑂( 𝑓 (𝑛))
C 𝑔(𝑛) = Θ( 𝑓 (𝑛))
pingo.upb.de/622222
12
Asymptotics – Example 1
Basic examples:
� 20𝑛 3 + 10𝑛 ln(𝑛) + 5 ∼ 20𝑛 3 = Θ(𝑛 3 )
� 10100 = 𝑂(1)
A Yes B No
pingo.upb.de/622222
14
Clicker Question
A Yes � B No
pingo.upb.de/622222
14
Asymptotics – Frequently used facts
� Rules:
� 𝑐 · 𝑓 (𝑛) = Θ( 𝑓 (𝑛)) for constant 𝑐 ≠ 0
� Θ( 𝑓 + 𝑔) = Θ(max{ 𝑓 , 𝑔}) largest summand determines order of growth
� linear Θ(𝑛)
� linearithmic Θ(𝑛 log 𝑛)
� quadratic Θ(𝑛 2 )
� polynomial 𝑂(𝑛 𝑐 ) for constant 𝑐
� exponential 𝑂(𝑐 𝑛 ) for constant 𝑐 Note: 𝑎 > 𝑏 > 0 constants � 𝑏 𝑛 = 𝑜(𝑎 𝑛 )
15
Asymptotics – Example 2
Square-and-multiply algorithm
for computing 𝑥 𝑚 with 𝑚 ∈ ℕ 1 double pow(double base, boolean[] exponentBits) {
2 double res = 1;
Inputs:
3 for (boolean bit : exponentBits) {
� 𝑚 as binary number (array of bits) 4 res ∗= res;
� 𝑛 = #bits in 𝑚 5 if (bit) res ∗= base;
6 }
� 𝑥 a floating-point number
7 return res;
8 }
� Cost: 𝐶 = # multiplications
� 𝐶 = 𝑛 (line 4) + #one-bits binary representation of 𝑚 (line 5)
� 𝑛 ≤ 𝐶 ≤ 2𝑛
16
Clicker Question
pingo.upb.de/622222
17
Asymptotics – Example 2 shift-and-multiply-exponentiation
Square-and-multiply algorithm 20
� Cost: 𝐶 = # multiplications 0
0 200 400 600 800 1000
18