Slides Lecture02

Advanced Programming and Algorithms I
Winter 2022/2023
Anja Rey
24th October 2021

Recap Last Week & Additional Details
Input ALGORITHM Output
computational problem model algorithm design software design programming
Algorithm in pseudocode: Implementation in Python:

some_function(input):
1 precise description of sequence of 1 def some \ _fuction ( input ) :
computational steps transforming 2 pass
the input into an output
2 return output
Extracts from notebook with first code overview: lecture01_basics.ipynb

Further details æ exercise groups
18/44 Advanced Programming and Algorithms I – Introduction Anja Rey


2 return output



2 return output



2 return output



2 return output


Insertion Sort
Computation problem:
Sorting
input: finite sequence of elements with a Æ relation, Èa1 , a2 , . . . , an Í
output: ordered sequence: permutation Èa1Õ , a2Õ , . . . , anÕ Í such that a1Õ Æ a2Õ Æ · · · Æ anÕ
7316452

Insertion Sort
Sorting
7316452
1234567
Insertion Sort
Sorting
7316452

Insertion Sort
Sorting
7316452

Insertion Sort
Sorting
7316452

Insertion Sort
Sorting
7316452

Insertion Sort
Sorting
7716452
3

Insertion Sort
Sorting
3716452

Insertion Sort
Sorting
3716452

Insertion Sort
Sorting
3716452

Insertion Sort
Sorting
3716452

Insertion Sort
Sorting
13376452
Insertion Sort
Sorting
1376452

Insertion Sort
Sorting
1376452

Insertion Sort
Sorting
1376452

Insertion Sort
Sorting
1376452

Insertion Sort
Sorting
1367452

Insertion Sort
Sorting
1367452

Insertion Sort
Sorting
1346752

Insertion Sort
Sorting
1346752

Insertion Sort
Sorting
1345672

Insertion Sort
Sorting
1345672

Insertion Sort
Sorting
1234567

Insertion Sort
Sorting
1234567

Insertion Sort
Sorting
Algorithm:
insertion_sort(A[0..n ≠ 1], n):
0 1 2 3 4 5 6
1 for j Ω 1 to n ≠ 1 do
2 key Ω A[j] 1376452
3 i Ωj ≠1
4 while i Æ 0 and A[i] > key do
5 A[i + 1] Ω A[i]
6 i Ωi ≠1
7 A[i + 1] Ω key

Insertion Sort
Sorting
Algorithm:
0 1 2 3 4 5 6
2 key Ω A[j] 1376452
3 i Ωj ≠1
5 A[i + 1] Ω A[i]
6 i Ωi ≠1
7 A[i + 1] Ω key

Insertion Sort
Sorting
Algorithm:
0 1 2 3 4 5 6
2 key Ω A[j] 1 376452
3 i Ωj ≠1
while i Æ 0 and A[i] > key do
4
5 A[i + 1] Ω A[i] They

6 i Ωi ≠1
7 A[i + 1] Ω key

Insertion Sort
Sorting
Algorithm:
insertion_sort(A[0..n ≠ 1], n): 24
0 1 3 4 5 6
2 key Ω A[j] 1376452
3 i Ωj ≠1
5 A[i + 1] Ω A[i] key 6
6 i Ωi ≠1
7 A[i + 1] Ω key

Insertion Sort
Sorting
Algorithm:
0 1
24 3 4 5 6
2 key Ω A[j] 1377452
3 i Ωj ≠1
5 A[i + 1] Ω A[i] key 6
6 i Ωi ≠1
7 A[i + 1] Ω key

Insertion Sort
Sorting
Algorithm:
0 1 2 3 4 5 6
2 key Ω A[j] 1377452
3 i Ωj ≠1
5 A[i + 1] Ω A[i] key 6
6 i Ωi ≠1
7 A[i + 1] Ω key

Insertion Sort
Sorting
Algorithm:
0 1 2 3 4 5 6
2 key Ω A[j] 1377452
3 i Ωj ≠1
5 A[i + 1] Ω A[i] key 6
6 i Ωi ≠1
7 A[i + 1] Ω key

Insertion Sort
Sorting
Algorithm:
0 1 2 3 4 5 6
2 key Ω A[j] 1367452
3 i Ωj ≠1
5 A[i + 1] Ω A[i] key 6
6 i Ωi ≠1
7 A[i + 1] Ω key

Insertion Sort
Sorting
Algorithm:
insertion_sort(A[0..n ≠ 1], n): 34 44
0 1 2 5 6
2 key Ω A[j] 1367452
3 i Ωj ≠1
5 A[i + 1] Ω A[i]
6 i Ωi ≠1
7 A[i + 1] Ω key

Insertion Sort
Sorting
Algorithm: Python implementation:

1 def insertio n_sort ( sequence ) :
2 for j in range (1 , len ( sequence ) ) :
2 key Ω A[j] 3 key = sequence [ j ]
3 i Ωj ≠1 4 i = j - 1
4 while i Æ 0 and A[i] > key do 5 while i >= 0 and sequence [ i ] > key :
6 sequence [ i + 1] = sequence [ i ]
5 A[i + 1] Ω A[i] 7 i = i - 1
6 i Ωi ≠1 8 sequence [ i + 1] = key
7 A[i + 1] Ω key

Efficiency
Running Time I
How efficient is an algorithm?
1 import time
2
3 start = time . perf_counter ()

4 pass
5 end = time . perf_counter ()
6 print ( end - start )
20/46 Advanced Programming and Algorithms I – Efficiency Anja Rey

Running Time I

Resources: Time, Space, Energy, . . .
1 import time
2

4 pass

Running Time I

Approach: Measure the time needed to run an implementation.
1 import time
2

4 pass

Running Time I

1 import time
2

4 pass
See notebook: lecture02_time.ipynb

Running Time I

1 import time
2

4 pass
See notebook: lecture02_time.ipynb

The actual running time (in seconds) is influenced by:
hardware: processor, cache, pipelining, . . .

software: operation system, programming language, compiler/interpreter, threading, . . .
Computation Model
Theoretical running time analysis: independent of hardware and software

Computation Model
Machine model:
Random Access Machine (RAM)
• one instruction in pseudocode takes one time step

Computation Model
Machine model:
• repeat instruction r times: r time steps

Computation Model
Machine model:
Approach: Count number of steps

Computation Model
Machine model:
factorial(n): instance n 4
1 x Ω1
2 for i Ω 2 to n do
3 x Ωx ·i
4 return x

Computation Model
Machine model:
1 x Ω1 1
3 x Ωx ·i
4 return x

Computation Model
Machine model:
1 x Ω1 1
3 x Ωx ·i
4 return x

Computation Model
Machine model:
1 x Ω1 1
3 x Ωx ·i 1
4 return x

Computation Model
Machine model:
1 x Ω1 1
for i Ω 2 to n do 3
1 h
2
i
3 x Ωx ·i
4 return x

Computation Model
Machine model:
1 x Ω1
2
4
for i Ω 2 to n do
x Ωx ·i
return x
i 4
I I k
Computation Model
Machine model:
1 x Ω1 1
for i Ω 2 to n do
1 1 11
2
i 5
3 x Ωx ·i
4 return x

Computation Model
Machine model:
1 x Ω1
Kid
3 x Ωx ·i
4 return x

Computation Model
Machine model:
1 x Ω1
for i Ω 2 to n do
Äh
2
3 x Ωx ·i
4 return x
21/44 Advanced Programming and Algorithms I – Efficiency

9 steps Anja Rey
Computation Model
Machine model:

general n 1
1 x Ω1
for i Ω 2 to n do 1 1 11 1
1 1
2
3 x Ωx ·i 4 1
return x
4
1 1

9 steps In 1 steps Anja Rey
Computation Model
Machine model:
factorial(n): insertions_sort([1, 2, 3], 3)
1 x Ω1 13 steps
insertions_sort([3, 2, 1], 3)
3 x Ωx ·i
return x
4
23 steps
Computation Model
Machine model:
factorial(n): insertions_sort([1, 2, 3], 3)
1 x Ω1
insertions_sort([3, 2, 1], 3)
3 x Ωx ·i
4 return x
How can we predict the number of steps in advance?

Running Time II
Observation: running time

• depends on input size (e.g., parameter n)
• depends on type of input (e.g., best case, worst case input)

Running Time II

Theoretical analysis:
• analyse time as a function T (n) of the input size n

Running Time II

• worst-case analysis: find upper bounds (guarantee for all inputs, max over all size n instances)

Running Time II


2 key Ω A[j]
3 i Ωj ≠1
5 A[i + 1] Ω A[i]
6 i Ωi ≠1
7 A[i + 1] Ω key
Running Time II


1 for j Ω 1 to n ≠ 1 do n 1 1 n
2 key Ω A[j]
3 i Ωj ≠1
5 A[i + 1] Ω A[i]
6 i Ωi ≠1
7 A[i + 1] Ω key
Running Time II


1 for j Ω 1 to n ≠ 1 do n 1 1 n
2 key Ω A[j]
3 i Ωj ≠1
4
5 A[i + 1] Ω A[i] factor In 1

6 i Ωi ≠1
7 A[i + 1] Ω key
Running Time II


1 for j Ω 1 to n ≠ 1 do n 1 1 n
2 key Ω A[j] 4 1
3 i Ωj ≠1 n 1
5 A[i + 1] Ω A[i]
6 i Ωi ≠1
A[i + 1] Ω key
7

n 1 Anja Rey
Running Time II


1 for j Ω 1 to n ≠ 1 do n 1 1 n
2 key Ω A[j] 4 1
3 i Ωj ≠1 n 1
5 A[i + 1] Ω A[i] depends on warst case
6 i Ωi ≠1
A[i + 1] Ω key
7

n d Anja Rey
Running Time II


1 for j Ω 1 to n ≠ 1 do n 1 1 n
2 key Ω A[j] 4 1
3 i Ωj ≠1 n d
5 A[i + 1] Ω A[i] depends on warst case
6 i Ωi ≠1
input sorted in reverse order
7 A[i + 1] Ω key
n
Running Time II

reverse order
1 for j Ω 1 to n ≠ 1 do n 1 d n
2 key Ω A[j] 4 1
3 i Ωj ≠1 n d
4
5
A[i + 1] Ω A[i] always true for ie 90 j d
6 i Ωi ≠1
A[i + 1] Ω key
7

n d Anja Rey
Running Time II

reverse order
1 for j Ω 1 to n ≠ 1 do n 1 1 n
2 key Ω A[j] 4 1
3 i Ωj ≠1
Für
c
5 A[i + 1] Ω A[i]
6 i Ωi ≠1
A[i + 1] Ω key
7

n 1 Anja Rey
Running Time II

reverse order
1 for j Ω 1 to n ≠ 1 do n 1 1 n
2 key Ω A[j] u 1
3 i Ωj ≠1
c
En
4
5 A[i + 1] Ω A[i]
6 i Ωi ≠1
A[i + 1] Ω key
7

1 Anja Rey
Running Time II

reverse order
1 for j Ω 1 to n ≠ 1 do n 1 1 n
2 key Ω A[j] 4 1
3 i Ωj ≠1 n 1
F2
ja
4
wie
5 A[i + 1] Ω A[i]
6
22/44
i Ωi ≠1
A[i + 1] Ω key
Advanced Programming and Algorithms I – Efficiency
n
we
d E Anja Rey
Running Time II

reverse order
1 for j Ω 1 to n ≠ 1 do n 1 1 n
2 key Ω A[j] 4 1
11
3 i Ωj ≠1 n d n n
YYFtu
3w.cn
2j n
4
wie
5 A[i + 1] Ω A[i]
nu
6
22/44
i Ωi ≠1
A[i + 1] Ω key
n 1
2
Eij Anja Rey
Running Time II

reverse order
1 for j Ω 1 to n ≠ 1 do n 1 1 n
2 key Ω A[j] 4 1
3
4
i Ωj ≠1
ü Eig 945min
5 A[i + 1] Ω A[i]
6
22/44
i Ωi ≠1
A[i + 1] Ω key
d
2
FIT
RE N
we
Anja Rey
Running Time II

• asymptotic analysis: ignore constants (may become twice as fast in the future or on another
machine, already ignore differences in operations)

Running Time II

Tu 2h43m 4

Running Time II

Tu 2h43m 4
time on Computer 1 2m13n 4 ca
time on Computer 2 2nd In 4 cz
n En 2 23 Anja Rey
Running Time II

• asymptotic growth: n æ infty

Running Time II

• asymptotic growth: n æ infty
MTU

O-Notation
Defintion: O-Notation
gut Let f , g : N æ R>0 .
O(f (n)) = {g(n) | ÷c > 0, n0 > 0 : ’n Ø n0 : g(n) Æ c · f (n)}.
In
Intuition: g(n) œ O(f (n)) means that g(n) grows at most as fast as f (n) for n æ Œ.
Representative functions and their hierarchy: Let c Ø 2, 0 < ‘ Æ 1/2.

Ô
O(1) ™ O(log n) ™ O(log2 n) ™ O(logc n) ™ O(log n‘ ) ™ O( n) ™ O(n)
O(n) ™ O(n2 ) ™ O(nc ) ™ O(2n ) = O(c n )
Insertion sort has an (asymptotic worst-case) running time in O(n2 ).

O-Notation
Let f , g : N æ R>0 .

Ô
O(n) ™ O(n2 ) ™ O(nc ) ™ O(2n ) = O(c n )

O-Notation
Let f , g : N æ R>0 .

Ô
O(n) ™ O(n2 ) ™ O(nc ) ™ O(2n ) = O(c n )

O-Notation
Let f , g : N æ R>0 .

Ô
O(n) ™ O(n2 ) ™ O(nc ) ™ O(2n ) = O(c n )

O-Notation
Let f , g : N æ R>0 .

Ô
O(n) ™ O(n2 ) ™ O(nc ) ™ O(2n ) = O(c n )

O-Notation
flu
gut Let f , g : N æ R>0 .
d
n

Ô
O(n) ™ O(n2 ) ™ O(nc ) ™ O(2n ) = O(c n )

O-Notation
Let f , g : N æ R>0 .
In Tu 2h43m 4 e 049
J

Ô
O(n) ™ O(n2 ) ™ O(nc ) ™ O(2n ) = O(c n )

Related Concepts
Average Running Time

• worst case might only occur rarely
• compute average running time over all possible inputs of one size

Related Concepts

Space Complexity
• memory needed to store information in the algorithm
• worst case: upper bound
• average case: all possible inputs of one size
æ see Exercise 2

Related Concepts

Space Complexity
• memory needed to store information in the algorithm
• worst case: upper bound
• average case: all possible inputs of one size
æ see Exercise 2
Energy Complexity
• energy consumed by the algorithm
• comparably new research field in theory
• Python is rather inefficient.. TODO

What’s next?
Within this week:

• Exercise groups start today:
• Mon 2;30 pm in seminar room 25.13.U1.23
• Fri 2:30 online via webex.
• Work on Exercise Sheet 2.
• Receive feedback on Sheet 1.
Next week:
• Mon 12:30 pm: Hand in exercise (via ILIAS).

• Lecture on Monday:
• How do we proof an algorithm works?
• How can we evaluate and improve code quality (I)?

Slides Lecture02

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Slides Lecture02

Uploaded by

Copyright:

Available Formats

Advanced Programming and Algorithms I

24th October 2021

Input ALGORITHM Output

computational problem model algorithm design software design programming

Algorithm in pseudocode: Implementation in Python:

Extracts from notebook with first code overview: lecture01_basics.ipynb

18/44 Advanced Programming and Algorithms I – Introduction Anja Rey

Input ALGORITHM Output

computational problem model algorithm design software design programming

Algorithm in pseudocode: Implementation in Python:

Extracts from notebook with first code overview: lecture01_basics.ipynb

18/44 Advanced Programming and Algorithms I – Introduction Anja Rey

Input ALGORITHM Output

computational problem model algorithm design software design programming

Algorithm in pseudocode: Implementation in Python:

Extracts from notebook with first code overview: lecture01_basics.ipynb

18/44 Advanced Programming and Algorithms I – Introduction Anja Rey

Input ALGORITHM Output

computational problem model algorithm design software design programming

Algorithm in pseudocode: Implementation in Python:

Extracts from notebook with first code overview: lecture01_basics.ipynb

18/44 Advanced Programming and Algorithms I – Introduction Anja Rey

Input ALGORITHM Output

computational problem model algorithm design software design programming

Algorithm in pseudocode: Implementation in Python:

Extracts from notebook with first code overview: lecture01_basics.ipynb

18/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

5 A[i + 1] Ω A[i] They

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

Algorithm: Python implementation:

19/44 Advanced Programming and Algorithms I – Introduction Anja Rey

How efficient is an algorithm?