Design and Analysis of Algorithms Chapter 1

Algorithm Historical Perspective

I Euclid’s algorithm for finding the greatest common divisor
I An algorithm is a sequence of unambiguous
instructions for solving a problem, i.e., for
I Muhammad ibn Musa al- al-Khwarizmi – 9th century
obtaining a required output for any legitimate mathematician
input in a finite amount of time. www.lib.virginia.edu/science/parshall/khwariz.html

Design and Analysis of Algorithms - Chapter 1 1 Design and Analysis of Algorithms - Chapter 1 2

Notion of algorithm Example of computational problem: sorting

I Statement of problem:
• Input: A sequence of n numbers <a1, a2, …, an>
problem
• Output: A reordering of the input sequence <a´1, a´2, …, a´n>
so that a´i ≤ a´j whenever i < j
algorithm
I Instance: The sequence <5, 3, 2, 8, 3>

input “computer” output I Algorithms:

• Selection sort
• Insertion sort
• Merge sort
• (many others)
Algorithmic solution
Design and Analysis of Algorithms - Chapter 1 3 Design and Analysis of Algorithms - Chapter 1 4

Selection Sort Some Well-

Well-known Computational Problems

I Input: array a[1],…,a[n

a[1],…,a[n]
] I Sorting
I Searching
I Output: array a sorted in non-
non-decreasing order I Shortest paths in a graph
I Minimum spanning tree
I Algorithm: I Primality testing
I Traveling salesman problem
I Knapsack problem
for i=1 to n I Chess
swap a[i] with smallest of a[i],…a[n] I Towers of Hanoi
I Program termination
• see also pseudocode, section 3.1
Design and Analysis of Algorithms - Chapter 1 5 Design and Analysis of Algorithms - Chapter 1 6

1
Design and Analysis of Algorithms Chapter 1

Basic Issues Related to Algorithms Algorithm design strategies

I How to design algorithms

I How to express algorithms I Brute force I Greedy approach

I Proving correctness I Divide and conquer I Dynamic programming

I Efficiency I Decrease and conquer I Backtracking and Branch and bound

• Theoretical analysis
I Transform and conquer I Space and time tradeoffs
• Empirical analysis

I Optimality

Design and Analysis of Algorithms - Chapter 1 7 Design and Analysis of Algorithms - Chapter 1 8

Analysis of Algorithms What is an algorithm?

I How good is the algorithm? I Recipe, process, method, technique, procedure, routine,…
• Correctness with following requirements:
• Time efficiency 1. Finiteness
• Space efficiency I terminates after a finite number of steps
2. Definiteness
I Does there exist a better algorithm? I rigorously and unambiguously specified
• Lower bounds 3. Input
• Optimality I valid inputs are clearly specified
4. Output
I can be proved to produce the correct output given a valid input
5. Effectiveness
I steps are sufficiently simple and basic

Design and Analysis of Algorithms - Chapter 1 9 Design and Analysis of Algorithms - Chapter 1 10

Why study algorithms?

I Theoretical importance

• the core of computer science

I Practical importance

• A practitioner’s toolkit of known algorithms

• Framework for designing and analyzing algorithms for new

problems

Design and Analysis of Algorithms - Chapter 1 11