Algdat

DatAlg
Maritta Heisel
Introduction
Alg. Design
Data Structures and Algorithms
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Prof. Dr. Maritta Heisel
Binary Trees Maritta.Heisel(AT)uni-due.de
Hashing
Graphs University Duisburg-Essen – Faculty of Engineering

Backtracking
Department of Computational and Applied Cognitive Science
Workgroup Software Engineering
Dynamic
programming
Greedy
Blockchain
Summary
1/ 550
Literature
DatAlg
Maritta Heisel
Introduction Les Goldschlager, Andrew Lister

Alg. Design Computer Science – A Modern Introduction – Second Edition
ADTs Prentice Hall International Series in Computer Science, 1987
Sorting ISBN 0-13-165945-6
Alg. Theory
Sorting (ct’d)
Robert Sedgewick
Binary Trees
Algorithmen
Hashing Pearson Studium / Addison-Wesley, 2002
Graphs ISBN 3-8273-7032-9
Backtracking
Both books are available in the library of the University
Dynamic
programming Duisburg-Essen in German and English.
Greedy
Blockchain
Summary
2/ 550
What will we learn in this course? I
DatAlg
Maritta Heisel
Notion of an algorithm
Introduction Syntax
Alg. Design Semantics
ADTs Specification
Sorting
Development of algorithms
Alg. Theory
Stepwise refinement
Sorting (ct’d)
Binary Trees
Theory of algorithms
Hashing Computability
Graphs Complexity
Backtracking Correctness
Dynamic Important Algorithms
programming
Searching
Greedy
Sorting
Blockchain
Summary
3/ 550
What will we learn in this course? II
DatAlg
Maritta Heisel
Concept of Abstract Data Types (ADT)
Specification of ADTs
Introduction
Alg. Design Implementation of ADTs

ADTs Algorithms on ADTs
Sorting
Important ADTs (plus corresponding algorithms):
Alg. Theory
linked lists
Sorting (ct’d)
binary trees
Binary Trees
balanced binary trees
Hashing
hash tables
Graphs
graphs
Backtracking
Important algorithm principles:
Dynamic
programming Divide and conquer
Greedy Greedy
Blockchain Backtracking
Summary
4/ 550
The computer revolution
DatAlg
Maritta Heisel Industrial revolution

Introduction
Augmentation of man’s physical powers, amplification of
Alg. Design
man’s muscle
ADTs Pressing a button can cause a large machine to stamp a
Sorting pattern into a metal sheet.
Alg. Theory Certain repetitive aspects of man’s physical activities were
Sorting (ct’d) replaced by machines.
Binary Trees
Computer revolution
Hashing
Augmentation of man’s mental powers, amplification of
Graphs
man’s brain
Backtracking
Pressing a button can cause a machine to make complex
Dynamic
programming decisions.
Greedy Certain repetitive aspects of man’s mental activities are
Blockchain being replaced by machines.
Summary
5/ 550
Computers and algorithms
DatAlg
Maritta Heisel
Introduction
A computer is a machine which can carry out routine
Alg. Design
mental tasks by performing simple operations at high
ADTs
speed.
Sorting
Alg. Theory
An algorithm
Sorting (ct’d) describes the method how a task is to be accomplished.
Binary Trees
consists of a finite sequence of steps.
Hashing
which if performed will result in a process being carried
Graphs
out.
Backtracking A processor (e.g. a person or a computer) carries out a
Dynamic process by executing the algorithm which describes it.
programming
Greedy
Blockchain
Summary
6/ 550
Examples
DatAlg
Maritta Heisel
Introduction
Process Algorithm Typical steps
Alg. Design
in algorithm
ADTs
Sorting
knitting a sweater knitting pattern knit one, purl one
Alg. Theory building a assembly instructions glue panel A
Sorting (ct’d) model plane to strut B
Binary Trees baking a cake recipe take three eggs,
Hashing beat until smooth
Graphs making a dress dress pattern sew up side seam
Backtracking a c a+c
calculating formula b + b = b
Dynamic
programming
Greedy
Blockchain
Summary
7/ 550
Major hardware components of a computer
DatAlg
Maritta Heisel
A computer is simply a particular kind of processor. Its major
Introduction components are:
Alg. Design
ADTs
Central processing unit (CPU): performs basic operations
Sorting Memory (MEM): algorithm and data
Alg. Theory
Input and output devices (I/O devices): communication
Sorting (ct’d)
Binary Trees
Hashing
Central
Graphs Input and processing
output devices Memory
Backtracking unit
(CPU)
Dynamic
programming
Greedy
Blockchain
Summary
8/ 550
Characteristics of a computer
DatAlg
Maritta Heisel
Speed
Introduction many millions of
Alg. Design instructions per second
ADTs (rate increases steadily)
Sorting
Reliability
Alg. Theory
mistakes are due to
Sorting (ct’d)
software not hardware
Binary Trees
Hashing
faults
Graphs Memory
Backtracking vast, growing steadily
Dynamic
programming Cost
Greedy decreases steadily
Blockchain
Summary
9/ 550
Significance of these characteristics
DatAlg
Example: airline reservation system
Maritta Heisel
Many offices, geographically separated by large distances
Introduction
Task: combine all seat bookings and cancellations for each
Alg. Design
flight, disseminate information on seat availability
ADTs
Sorting Performing that task manually would require a massive number

Alg. Theory of employees and office space. With a computer, however:
Sorting (ct’d) Speed one can continously maintain accurate status
Binary Trees
information for each flight
Hashing
Reliability passengers can have more confidence in the
Graphs
Backtracking
reservation system than in a manual one
Dynamic Memory large quantities of data can be handled
programming
Greedy
Cost cost of purchasing and running a computer for
Blockchain the reservation system is far less than employing
Summary a large number of clerical workers
10/ 550
Programs and programming languages
DatAlg
Maritta Heisel
An algorithm must be expressed in such a way that the
Introduction
processor can both understand and execute it (e.g.
Alg. Design
cook/recipe and musician/score).
ADTs
Sorting
If the processor is a computer then the algorithm must be
Alg. Theory
expressed in a form called program.
Sorting (ct’d) A program is written in a programming language. The
Binary Trees activity of writing a program is called programming.
Hashing
Each step of an algorithm is expressed by a statement in
Graphs
the program.
Backtracking
Dynamic =⇒ A program consists of a finite sequence of

programming
Greedy
statements, each of which specifies certain operations
Blockchain
that the computer is to perform.
Summary
11/ 550
Executing an algorithm on a computer
DatAlg
Maritta Heisel
Algorithm
Introduction
Machine languages:
Alg. Design Programming
statements can be directly
ADTs
Sorting
interpreted by the High level language
Alg. Theory
computer; programming is program
Sorting (ct’d) tedious

Translation
Binary Trees High level languages:
Hashing programs have to be
Machine language
Graphs
translated to machine program
Backtracking
language; e.g. Ada, C++,
Dynamic
programming Fortran, Java, Pascal Interpretation by CPU
Greedy
(Desired process is carried out)
Blockchain
Summary
12/ 550
The software-hardware hierarchy
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs Applications software

Sorting (e.g. packages)
Alg. Theory
Sorting (ct’d) System software

Binary Trees (e.g. operating system,
Hashing language translators)
Graphs
Backtracking Computer hardware

Dynamic (e.g. CPU, memory,
programming
I/O devices)
Greedy
Blockchain
Summary
13/ 550
The importance of algorithms
DatAlg
We have learnt that to carry out a process on a computer we
Maritta Heisel
must:
Introduction
1. Design an algorithm which describes how the process is to
Alg. Design
Syntax and semantics be performed
Stepwise refinement
Control structures 2. Express the algorithm as a program in a suitable
Imperative paradigm
Algorithm notation programming language
Example
development
Modularity
3. Get the computer to execute the program
Algorithm
specification =⇒ Without an algorithm there can be no program, and
Recursion
Data Structures without a program there is nothing to execute.
ADTs
Sorting Moreover algorithms are independent of the language in which

Alg. Theory they are expressed and the computer which executes them.
Sorting (ct’d)
=⇒ Algorithms can be devised and studied independently of
Binary Trees
the technology of the day!
Hashing
14/ 550
Algorithms and processes
DatAlg
Maritta Heisel
Introduction Recall: An algorithm is a description of how to carry out a

Alg. Design process.
Syntax and semantics
Stepwise refinement A process interacts with its environment (by accepting
Control structures
Imperative paradigm input and producing output).
Algorithm notation
Example
development
Example: the process of computing weekly wages requires
Modularity information about wage rates and hours worked; it
Algorithm
specification
Recursion
produces information about wages payable and taxes
Data Structures deducted.
ADTs
A process can terminate (e.g. the sweater is completed) or
Sorting
not (e.g. staying happy).
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
15/ 550
Nontermination
DatAlg
Maritta Heisel
Introduction
Alg. Design
Stepwise refinement
Even processes carried out by computers may never
Control structures
Imperative paradigm
terminate (e.g. controlling traffic signals).
Algorithm notation
Example
Under certain circumstances the process described may
development
Modularity not terminate, whereas it is always supposed to do so.
Algorithm
specification
Recursion Nontermination is a major source of error in the design of an
Data Structures
algorithm!
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
16/ 550
Programs and programming languages
DatAlg
Maritta Heisel
Recall: An algorithm to be executed by a computer is written
Introduction
in a programming language and is then called a program.
Alg. Design
Stepwise refinement Natural languages like English are not suitable for expressing
Control structures
Imperative paradigm algorithms:
Algorithm notation
Example
development
Enormous vocabulary and grammatical rules
Modularity
Algorithm Interpretation is complex and not well-defined (e.g.
specification
Recursion metaphors)
Data Structures
Example: “funny situation” (peculiar or humorous?)
ADTs
Sorting
Alg. Theory
=⇒ We must express an algorithm in a simpler form,
Sorting (ct’d)
using a programming language.
Binary Trees
Hashing
17/ 550
Why are there so many programming languages?
DatAlg
Maritta Heisel
Introduction Computer programming is a comparatively recent activity;

Alg. Design new programming paradigms are developed, e.g. object
Stepwise refinement
orientation or aspect orientation
Control structures
Imperative paradigm Special-purpose languages have been developed for
Algorithm notation
Example expressing algorithms in specific application areas.
development
Modularity
Algorithm
Tendency to reinvent the wheel (“Our own wheels are
specification
Recursion
better than anyone else’s”)
Data Structures
ADTs Example
Sorting COBOL MULTIPLY price BY quantity GIVING cost
Alg. Theory
Pascal cost := price ∗ quantity
Sorting (ct’d)
Binary Trees
Hashing
18/ 550
Syntax and Semantics
DatAlg
A processor must be able to interpret an algorithm to carry out
Maritta Heisel
the process the algorithm describes. That is, it must be able to
Introduction
1. understand the form in which the algorithm is expressed
Alg. Design
(e.g. a knitting pattern or a musical score).
Stepwise refinement
Control structures 2. carry out the corresponding actions.
Imperative paradigm
Algorithm notation To understand the expression of an algorithm the processor
Example
development
Modularity must know the vocabulary and grammar of the used
Algorithm
specification language,
Recursion
Data Structures e.g., “sleeve” is an English noun, “=” is a relation
ADTs between two numbers
Sorting
must be able to recognize and make sense of the symbols
Alg. Theory
in which the algorithm is expressed,
Sorting (ct’d)
e.g. “sleeve the seems” is a violation of the rules of
Binary Trees
English, a+ = b is an incorrect mathematical expression
Hashing
19/ 550
Definition: Syntax
DatAlg
Maritta Heisel
The set of grammatical rules which govern how the symbols in
Introduction a language may be legitimately used is called the syntax of the
Alg. Design language. For example, the syntax of English governs the use
Stepwise refinement of the word sleeve.
Control structures
Imperative paradigm A program which adheres to the syntax of the language in
Algorithm notation
Example
development
which it is written is called syntactically correct.
Modularity
Algorithm
A deviation from the syntax of the language is called a
specification
Recursion syntax error.
Data Structures
ADTs
Syntactic correctness is a necessary prerequisite for
Sorting
interpreting a computer program.
Alg. Theory Programming languages in contrast to natural languages
Sorting (ct’d) have a relatively simple syntax.
Binary Trees
Hashing
20/ 550
Meaning of Algorithms
DatAlg
Maritta Heisel
Second stage in understanding an algorithm: attach meaning
Introduction
Alg. Design
to each step, in terms of the operations the processor is
Syntax and semantics intended to carry out.
Stepwise refinement
Control structures
Imperative paradigm Example: the meaning of the Pascal expression
Algorithm notation
Example
development cost := price ∗ quantity
Modularity
Algorithm
specification
Recursion
is that two numbers called price and quantity are to be
Data Structures
multiplied together to give a third number called cost.
ADTs
Sorting
Remark When we express algorithms to be executed by a
Alg. Theory
computer, we will use a programming-language-like notation.
Sorting (ct’d)
Binary Trees
Hashing
21/ 550
Definition: Semantics
DatAlg
Maritta Heisel The meaning of particular forms of expression in a language is

Introduction called the semantics of the language.
Alg. Design Programming languages in contrast to natural languages
Stepwise refinement have relatively simple semantics
Control structures
Imperative paradigm
Algorithm notation
Example Examples:
development
Modularity the elephant ate the peanut meaningful sentence
Algorithm
specification the peanut ate the elephant syntactically correct, seman-
Recursion
Data Structures tically meaningless
ADTs write down the name of the meaningful step in an algo-
Sorting
1st month of the year rithm
Alg. Theory
write down the name of the syntactically correct, seman-
Sorting (ct’d)
13th month of the year tically meaningless
Binary Trees
Hashing
22/ 550
Semantic inconsistencies
DatAlg
Maritta Heisel
Detection of semantic inconsistencies relies on knowledge
Introduction of the objects being referred to, in particular their
Alg. Design attributes and the relations among them, e.g. relative
Stepwise refinement attributes of peanuts and elephants.
Control structures
Imperative paradigm Consider e.g. a processor faced with the command
Algorithm notation
Example
development
Write down the name of the 13th month of the year
Modularity If the processor knows that there are only 12 months in
Algorithm
specification
Recursion
the year it can detect the semantic inconsistency in this
Data Structures command before trying to execute it.
ADTs
Otherwise it will attempt to execute the command,
Sorting
possibly by looking up the name in a calendar. Then,
Alg. Theory
inconsistencies become apparent only during execution
Sorting (ct’d)
(run-time error).
Binary Trees
Hashing
23/ 550
More subtle semantic inconsistencies
DatAlg
Maritta Heisel
Introduction
When an inconsistency is a result of executing an earlier
Alg. Design
Syntax and semantics part of an algorithm there is in general no chance of
Stepwise refinement
Control structures detecting it beforehand.
Imperative paradigm
Algorithm notation Example:
Example
development
Modularity
Algorithm
think of a number from 1 to 13;
specification
Recursion
call this number N;
Data Structures
write down the name of the Nth month of the year
ADTs
Sorting contains a potential inconsistency

Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
24/ 550
Steps in interpreting an algorithm
DatAlg
Maritta Heisel
Introduction 1. Make sense of the symbols in which the step is expressed

Alg. Design 2. Attach meaning to the step in terms of operations to be
Stepwise refinement performed
Control structures
Imperative paradigm
Algorithm notation
3. Perform the appropriate operations
Example
development
Modularity
Algorithm Syntax errors can be detected in stage 1, and certain semantic
specification
Recursion errors can be detected in stage 2. Other semantic errors will
Data Structures
ADTs
not be detected until stage 3.
Sorting
Alg. Theory
When the processor is a computer, stages 1 and 2 are
Sorting (ct’d)
performed by a translator.
Binary Trees
Hashing
25/ 550
Logical errors
DatAlg
Maritta Heisel Third class of error in addition to syntactic and semantic errors.
Introduction A program may be syntactically correct and contain no
Alg. Design semantic inconsistencies, but may not properly describe the
Stepwise refinement desired process.
Control structures
Imperative paradigm
Algorithm notation
Example: compute the circumference of a circle by multiplying
Example
development
the radius by π
Modularity
Algorithm
specification
is syntactically and semantically correct, but produces the
Recursion wrong result.
Data Structures
ADTs Logical error: omission of the factor of 2.

Sorting
A processor cannot detect logical errors, since it has no idea
Alg. Theory
what process the algorithm is intended to describe.
Sorting (ct’d)
Binary Trees Therefore, algorithms have to be specified properly.

Hashing
26/ 550
Difficulty of designing algorithms
DatAlg
Maritta Heisel Design of nontrivial algorithms is very difficult.

Introduction Designers tend to overlook certain circumstances.
Alg. Design
Often there are unforeseen circumstances, such that the
Stepwise refinement
Control structures
algorithm usually results in the intended process being
Imperative paradigm
Algorithm notation
carried out but not always.
Example
development
Modularity
Algorithm
specification
Recursion
Data Structures
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
27/ 550
DatAlg

Alg. Design
Stepwise refinement
Control structures
Imperative paradigm
Algorithm notation
Example
development Example: calculation of flight time from an airline
Modularity
Algorithm
timetable:
specification
Recursion look up departure time and arrival time;
Data Structures
subtract departure time from arrival time
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
27/ 550
DatAlg

Alg. Design
Stepwise refinement
Control structures
Imperative paradigm
Algorithm notation
Example
development Example: calculation of flight time from an airline
Modularity
Algorithm
timetable:
specification
Recursion look up departure time and arrival time;
Data Structures
subtract departure time from arrival time
ADTs
Sorting This algorithm will fail if the departure point and the
Alg. Theory destination are in different time zones, or daylight saving
Sorting (ct’d) time is used only at one end of the route but not at the
Binary Trees other.
Hashing
27/ 550
Stepwise refinement of algorithms
DatAlg
Maritta Heisel
Introduction
Alg. Design The idea of stepwise refinement is to (repeatedly) break a

Stepwise refinement complex process into a number of steps, each of which can
Control structures
Imperative paradigm
be described by an algorithm which is smaller and simpler
Algorithm notation
Example
than that for the entire process.
development
Modularity Also known as top-down design or divide and conquer.
Algorithm
specification
Recursion
Designer must know what sort of step the processor can
Data Structures
interpret to know when a particular step in an algorithm is
ADTs
sufficiently primitive to need no further refinement.
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
28/ 550
Example: Stepwise refinement I
DatAlg
Maritta Heisel Initial version of an algorithm describing how to make a cup of

Introduction
instant coffee:
Alg. Design 1. boil water
Stepwise refinement 2. put coffee in cup
Control structures
Imperative paradigm
Algorithm notation
3. add water to cup
Example
development Detailed enough for a human servant but not for a robot.
Modularity
Algorithm
specification
Refinement of the step boil water into a sequence of simpler
Recursion steps:
Data Structures
ADTs
1. boil water
Sorting 1.1 fill kettle
Alg. Theory 1.2 switch on kettle
Sorting (ct’d) 1.3 wait until boiling
Binary Trees 1.4 switch off kettle
Hashing
29/ 550
Example: Stepwise refinement II
DatAlg
Maritta Heisel
Introduction Refinement of the step put coffee in cup into a sequence of

Alg. Design simpler steps:
Stepwise refinement 2. put coffee in cup
Control structures
Imperative paradigm 2.1 open coffee jar
Algorithm notation
Example 2.2 extract one spoonful of coffee
development
Modularity 2.3 tip spoonful into cup
Algorithm
specification 2.4 close coffee jar
Recursion
Data Structures Refinement of the step add water to cup:
ADTs
3. add water to cup
Sorting
3.1 pour water from kettle until cup is full
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
30/ 550
Example: Stepwise refinement III
DatAlg
Maritta Heisel
Introduction
Alg. Design Refinement of the step fill kettle into a sequence of simpler
Stepwise refinement steps:
Control structures
Imperative paradigm 1.1 fill kettle
Algorithm notation
Example 1.1.1 put kettle under tap
development
Modularity 1.1.2 turn on tap
Algorithm
specification 1.1.3 wait until kettle is full
Recursion
Data Structures 1.1.4 turn off tap
ADTs
and so on . . .
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
31/ 550
Final version of the coffee-making algorithm I
DatAlg
Maritta Heisel
Introduction
Alg. Design
{Algorithm for making a cup of coffee}
Syntax and semantics {first boil water}
Stepwise refinement
Control structures 1.1.1 put kettle under tap
Imperative paradigm
Algorithm notation 1.1.2 turn on tap
Example
development 1.1.3 wait until kettle is full
Modularity
Algorithm 1.1.4 turn off tap
specification
Recursion 1.2 switch on kettle
Data Structures
1.3.1 wait until kettle whistles
ADTs
Sorting
1.4 switch off kettle
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
32/ 550
Final version of the coffee-making algorithm II
DatAlg
Maritta Heisel
Introduction
Alg. Design
{put coffee in cup}
Syntax and semantics 2.1.1 take coffee jar from shelf
Stepwise refinement
Control structures 2.1.2 remove lid from jar
Imperative paradigm
Algorithm notation 2.2 extract one spoonful of coffee
Example
development 2.3 tip spoonful into cup
Modularity
Algorithm 2.4.1 put lid on coffee jar
specification
Recursion 2.4.2 replace coffee jar on shelf
Data Structures
{add water to cup}
ADTs
Sorting
3.1 pour water from kettle into cup until cup is full
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
33/ 550
Terminating the refinement
DatAlg
Maritta Heisel
The designer of an algorithm must know when a particular step
Introduction
Alg. Design
in the algorithm is sufficiently primitive to need no further
Syntax and semantics refinement.
Stepwise refinement
Control structures
Imperative paradigm
⇒ The designer must know what sort of step the processor can
Algorithm notation
Example
interpret.
development
Modularity
Algorithm
Example: switch on kettle is primitive, while fill kettle is not.
specification
Recursion
Data Structures
The designer of a computer algorithm refines the algorithm in
ADTs
such a way that its steps can be expressed in an appropriate
Sorting
programming language, and terminates the refinement when
Alg. Theory
every step is expressed in the language concerned.
Sorting (ct’d)
Binary Trees
Hashing
34/ 550
Sequence I
DatAlg
Maritta Heisel The simplest form of an algorithm is a sequence of statements,

meaning that:
Introduction
Alg. Design The statements are executed one at a time.

Stepwise refinement Each statement is executed exactly once; none is repeated
Control structures
Imperative paradigm
and none is omitted.
Algorithm notation
Example
development
The order in which the statements are executed is the
Modularity same as that in which they are written.
Algorithm
specification
Recursion
Successive statements are separated with “;”.
Data Structures
ADTs
Termination of the last statement implies termination of
Sorting
the algorithm.
Alg. Theory An algorithm which is solely a sequence of statements is
Sorting (ct’d) extremely inflexible, because its course of execution is
Binary Trees fixed and cannot be modified by circumstances.
Hashing
35/ 550
Sequence II
DatAlg
Maritta Heisel
Introduction
What happens in the coffee-making example if the coffee
Alg. Design
jar is empty?
Stepwise refinement
Control structures
The robot will probably either halt in confusion or serve a
Imperative paradigm
Algorithm notation
hot cup of water.
Example
development How could the robot handle a request for several cups of
Modularity
Algorithm coffee?
specification
Recursion Would it boil the water separately for each cup?
Data Structures
ADTs How could the sequence of statements be extended to

Sorting cater for optional milk and sugar?
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
36/ 550
Sequence III
DatAlg
Maritta Heisel
Introduction
Alg. Design
Another example:
Syntax and semantics catch Underground train to Heathrow Airport, London;
Stepwise refinement
Control structures fly by plane to Kennedy Airport, New York;
Imperative paradigm
Algorithm notation take a cab from Kennedy Airport to downtown New York
Example
development
Modularity
Algorithm
specification
Recursion
Data Structures
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
37/ 550
Sequence III
DatAlg
Maritta Heisel
Introduction
Alg. Design
Another example:
Syntax and semantics catch Underground train to Heathrow Airport, London;
Stepwise refinement
Control structures fly by plane to Kennedy Airport, New York;
Imperative paradigm
Algorithm notation take a cab from Kennedy Airport to downtown New York
Example
development
Modularity
No provision is taken for the not unlikely circumstance if
Algorithm
specification
there being either an Underground strike in London or a
Recursion
Data Structures
cab drivers’ strike in New York.
ADTs ⇒ The sequence is a very primitive structure for an
Sorting algorithm. We need less primitive structures.
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
37/ 550
Selection
DatAlg
The ability to execute a certain statement of an algorithm
Maritta Heisel
dependent on different cases is called selection.
Introduction General form: if condition then statement endif
Alg. Design condition specifies the circumstance under which
Stepwise refinement statement is to be executed.
Control structures
Imperative paradigm
The processor must be able to interpret the conditions in
Algorithm notation
Example
an algorithm just as it must be able to interpret the
development
Modularity statements.
Algorithm
specification ⇒ Conditions must be refined, too!
Recursion
Data Structures if, then, and endif are so-called keywords.
ADTs Example:
Sorting take coffee jar off shelf ;
Alg. Theory if jar is empty then get new jar from cupboard endif;
Sorting (ct’d) remove lid from jar
Binary Trees
The condition jar is empty will be evaluated to decide if
Hashing
38/ 550 the step get new jar from cupboard is executed.
Selection with two alternative steps
DatAlg
Maritta Heisel
An extension of the selection is a form which determines
which of two alternative statements is to be executed.
Introduction
Alg. Design
General form:
Syntax and semantics if condition then statement 1 else statement 2 endif
Stepwise refinement
Control structures
Imperative paradigm
if Underground on strike
Algorithm notation
Example
then catch airport bus
development
Modularity
else catch Underground train
Algorithm
specification
endif
Recursion
Data Structures The condition Underground on strike will be evaluated to
ADTs decide if the step catch airport bus or the step catch
Sorting Underground train is executed.
Alg. Theory
if condition then statement endif
Sorting (ct’d)
is equivalent to
Binary Trees
Hashing
if condition then statement else do nothing endif
39/ 550
Nesting
DatAlg We can design algorithms which contain nested

Maritta Heisel
occurrences of selection.
Consider a set of traffic signals:
Introduction
if no signal
Alg. Design
Syntax and semantics then proceed with great caution
Stepwise refinement
Control structures else if signal is red or signal is amber
Imperative paradigm
Algorithm notation
then stop
Example
development else proceed
Modularity
Algorithm
endif
specification
Recursion endif
Data Structures
ADTs
The second selection is nested inside the first and is
Sorting
executed only if the signal is working.
Alg. Theory Summary: The power of selection is that it allows a processor
Sorting (ct’d) to follow different paths through an algorithm according to
Binary Trees circumstance. Without selection it would be impossible to
Hashing
40/ 550
write algorithms of any significant practical use.
Indentation
DatAlg if no signal
Maritta Heisel then proceed with great caution
Introduction
else if signal is red or signal is amber
Alg. Design
then stop
Syntax and semantics else proceed
Stepwise refinement
Control structures
endif
Imperative paradigm
Algorithm notation
endif
Example
development is more readable than
Modularity
Algorithm
specification if no signal
Recursion
Data Structures
then proceed with great caution
ADTs else if signal is red or signal is amber
Sorting then stop
Alg. Theory else proceed
Sorting (ct’d) endif
Binary Trees endif
Hashing
41/ 550
⇒ Indentation is important for human readers!
Indentation/Example I
DatAlg
Develop an algorithm for determining the largest of three
Maritta Heisel
numbers x, y , z, given that the processor can compare only two
Introduction of them at a time.
Alg. Design
First version:
Stepwise refinement
Control structures if x > y
Imperative paradigm
Algorithm notation then choose between x and z
Example
development else choose between y and z
Modularity
Algorithm endif
specification
Recursion
Data Structures Refining the choice between x and z:
ADTs if x > z
Sorting
then choose x
Alg. Theory
else choose z
Sorting (ct’d)
endif
Binary Trees
Hashing The choice between y and z can be refined similarly.

42/ 550
Indentation/Example II
DatAlg
Maritta Heisel Final algorithm:

Introduction if x > y
Alg. Design
then if x > z
Stepwise refinement then choose x
Control structures
Imperative paradigm else choose z
Algorithm notation
Example endif
development
Modularity else if y > z
Algorithm
specification then choose y
Recursion
Data Structures else choose z
ADTs endif
Sorting endif
Alg. Theory
Sorting (ct’d) Note how indentation makes clear which statements of the
Binary Trees algorithm are part of each of the three selections.
Hashing
43/ 550
Iteration I
DatAlg
Consider the process of looking up a person’s address, given
Maritta Heisel
the name, from a list of names and addresses:
Introduction
Alg. Design
consider the first name in the list
Syntax and semantics if this name is the given name
Stepwise refinement
Control structures then extract the corresponding address
Imperative paradigm
Algorithm notation else consider the next name in the list
Example
development if this name is the given name
Modularity
Algorithm then extract the corresponding address
specification
Recursion else consider the next name in the list
Data Structures
if this name is the given name
ADTs
Sorting
then . . .
Alg. Theory Problem: When should we stop writing?
Sorting (ct’d)
Binary Trees
⇒ Sequence and selection are not sufficient to express
Hashing algorithms whose length varies according to circumstance.
44/ 550
Iteration II
DatAlg
Maritta Heisel
The ability of repeating certain steps of an algorithm an
Introduction arbitrary number of times is called iteration.
Alg. Design
General form:
Stepwise refinement
Control structures
repeat statement
| {z } until condition endrepeat
Imperative paradigm
Algorithm notation
body of loop
Example
development A repeat loop has its terminating condition at the end,
Modularity
Algorithm
after the until which follows the loop body. Loops of such
specification
Recursion kind are called post-tested loops.
Data Structures
ADTs
This means that the body is always executed at least once,
Sorting
even if the condition is initially true, since the terminating
Alg. Theory condition is evaluated only after the body has been
Sorting (ct’d) executed.
Binary Trees
Hashing
45/ 550
Iteration III
DatAlg
Maritta Heisel
Introduction Lookup algorithm:

Alg. Design consider the first name in the list
Stepwise refinement repeat
Control structures
Imperative paradigm
if this name is the given name
Algorithm notation
Example
then extract the corresponding address
development
Modularity
else consider the next name in the list
Algorithm
specification endif
Recursion
Data Structures
until given name is found or list is exhausted endrepeat
ADTs Use indentation to make the body of the loop stand out
Sorting
from the surrounding text.
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
46/ 550
Iteration IV
DatAlg
Maritta Heisel
Introduction
A while loop has its terminating condition at the start,
Alg. Design
before the do which precedes the loop body. Loops of
Stepwise refinement
Control structures
such kind are called pre-tested loops.
Imperative paradigm
Algorithm notation
Example
General form:
development
Modularity while condition do statement
| {z } endwhile
Algorithm
specification body of loop
Recursion
Data Structures
ADTs
Using a while loop means that the body of the loop is to
Sorting
be executed repeatedly as long as the condition is true.
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
47/ 550
Iteration V
DatAlg
Maritta Heisel Power of iteration: allows a process of indeterminate

Introduction
duration to be described by an algorithm of finite length.
Alg. Design However: possibility of non-termination arises.
Stepwise refinement Responsibility: ensure that the iteration does indeed
Control structures
Imperative paradigm terminate when (and if) intended.
Algorithm notation
Example
development
Example: the termination condition given name is found
Modularity
Algorithm
or list is exhausted eventually becomes true.
specification
Recursion However: leaving out the second part or list is exhausted
Data Structures
ADTs
is potentially disastrous. (If the name being sought is not
Sorting
present the processor will run off the end of the list.)
Alg. Theory
Sorting (ct’d) Failure to specify a terminating condition correctly is one of the

Binary Trees most common errors in algorithm design!
Hashing
48/ 550
Infinite loops and nested loops
DatAlg
Maritta Heisel
Introduction
Some processes are not intended to terminate, and an
Alg. Design
algorithm describing such a process must contain a loop
Stepwise refinement
Control structures
which is executed forever.
Imperative paradigm
Algorithm notation
General form:
Example
development
| {z } forever
repeat statement
Modularity
Algorithm body of loop
specification
Recursion
Data Structures
We can define so called nested loops: In this case an
ADTs
algorithm contains two loops, the inner loop nested inside
Sorting
Alg. Theory
the outer loop.
Sorting (ct’d)
Binary Trees
Hashing
49/ 550
Assignment / Imperative Paradigm I
DatAlg
Maritta Heisel
We have seen that (partial) algorithms can be combined
Introduction
by sequence, selection, and iteration. These are called
Alg. Design
control structures.
Stepwise refinement
Control structures But what are the basic statements of an algorithm?
Imperative paradigm
Algorithm notation In the imperative paradigm, the basic statements of
Example
development algorithms are assignments.
Modularity
Algorithm
specification
Imperative paradigm:
Recursion
Data Structures
Algorithms work on states, which they change.
ADTs
A state maps variables to values.
Sorting
Each variable has a type, which specifies the permissible
Alg. Theory
values a variable can take.
Sorting (ct’d)
Variables are “containers” for values; their “content” may
change.
Binary Trees
Hashing
50/ 550
Assignment / Imperative Paradigm II
DatAlg
Maritta Heisel
Assignment:
Introduction
Alg. Design
Changes the value of a variable.
Syntax and semantics Notation: x := t, where x is a variable and t is an
Stepwise refinement
Control structures
expression of the same type as x.
Imperative paradigm Example: x := x + 1 for an integer variable x.
Algorithm notation
Example The initial value of all variables should always be defined
development
Modularity by an algorithm (either as an input, or by an initialization
Algorithm
specification using an assignment).
Recursion
Data Structures Declaration:
ADTs We declare all variables using the keyword var.
Sorting Example: var x : Integer
Alg. Theory This introduces a variable x that can take integer values.
Sorting (ct’d)
Binary Trees
Hashing
51/ 550
Statement vs. Expression
DatAlg
Maritta Heisel
Statement Unit of an algorithm that can be executed by a
processor/computer. For example, assignments
Introduction
and statements composed by control structures
Alg. Design
Syntax and semantics are statements.
Stepwise refinement
Control structures Expression An entity composed of values and operators that
Imperative paradigm
Algorithm notation can be evaluated to produce a single value. An
Example
development expression cannot be executed in isolation, it is
Modularity
Algorithm always part of a statement. For example i + 3 is
specification
Recursion an expression, and i := i + 3 is a statement
Data Structures
ADTs
containing the expression.
Sorting
expression vs. statement
Alg. Theory
Sorting (ct’d) no side effect side effect

Binary Trees denotes a value of some type does not have a value
Hashing
52/ 550
Example
DatAlg For given inputs x, y , z of type integer , compute their

Maritta Heisel maximum and store it in the variable w .
Introduction var x, y , z, w : Integer ;
Alg. Design
Syntax and semantics if x > y
Stepwise refinement
Control structures then if x > z
Imperative paradigm
Algorithm notation
then w:= x
Example
development else w:= z
Modularity
Algorithm endif
specification
Recursion else if y > z
Data Structures
then w:= y
ADTs
else w:= z
Sorting
Alg. Theory
endif
Sorting (ct’d)
endif
Binary Trees (Later on, we will learn how to specify algorithms more
Hashing
53/ 550
precisely.)
Algorithm notation (inductive definition)
DatAlg
We will use the following statements to express algorithms:
Maritta Heisel
Assignment: if x is a variable and t is an expression of the
Introduction
same type as x then x := t is a statement.
Alg. Design
Syntax and semantics Composition/sequence: if S1 and S2 are statements, then
Stepwise refinement
Control structures S1 ; S2 is a statement.
Imperative paradigm
Algorithm notation
Example
Conditional/selection: if b is a boolean expression and S1 ,
development
Modularity
S2 are statements, then
Algorithm
specification
if b then S1 else S2 endif and
Recursion
Data Structures
if b then S1 endif are statements.
ADTs While loop: if b is a boolean expression and S is a
Sorting statement, then while b do S endwhile is a statement.
Alg. Theory
Repeat loop: if b is a boolean expression and S is a
Sorting (ct’d)
statement, then repeat S until b endrepeat is a
Binary Trees
Hashing
statement.
54/ 550
Algorithm notation
(Grammar in Backus-Naur form) I
DatAlg
Maritta Heisel
statement ::= assignment | sequence | conditional |
iteration
Introduction
assignment ::= name := expression
Alg. Design
Syntax and semantics sequence ::= statement ; statement
Stepwise refinement
Control structures
conditional ::= if booleanexpr then statement endif |
Imperative paradigm
Algorithm notation
if booleanexpr then statement
Example
development else statement endif
Modularity
Algorithm
iteration ::= repeatloop | whileloop
specification
Recursion repeatloop ::= repeat statement until
Data Structures
booleanexpr endrepeat |
ADTs
repeat arithmeticexpr times
Sorting
Alg. Theory
statement endrepeat
Sorting (ct’d)
whileloop ::= while booleanexpr do
Binary Trees
statement endwhile
Hashing
55/ 550
Algorithm notation
(Grammar in Backus-Naur form) II
DatAlg
Maritta Heisel
Introduction
expression ::= arithmeticexpr | booleanexpr
Alg. Design
arithmeticexpr ::= operand | (arithmeticexpr) |
Syntax and semantics arithmeticexpr arithmeticop arithmeticexpr
Stepwise refinement
Control structures operand ::= name | signednumber
Imperative paradigm
Algorithm notation arithmeticop ::= + | - | ∗ | / | ...
Example
development booleanexpr ::= name | (booleanexpr) | ¬ booleanexpr |
Modularity
Algorithm arithmeticexpr comparison arithmeticexpr |
specification
Recursion booleanexpr logicalop booleanexpr |
Data Structures
ADTs
true | false
Sorting
comparison ::= = | 6= | ≤ | ≥ | < | > | . . .
Alg. Theory
logicalop ::= ∧|∨|⇒|⇔
Sorting (ct’d)
Binary Trees
Hashing
56/ 550
Algorithm notation
(Grammar in Backus-Naur form) III
DatAlg
Maritta Heisel digit ::= 0 | 1 | ...| 9

Introduction
number ::= digit | digit number
Alg. Design signednumber ::= number | -number
Stepwise refinement
letter ::= A | ...| Z | a | ...| z
Control structures name ::= letter | letter name
Imperative paradigm
Algorithm notation declarations ::= var declarationlist
Example
development declarationlist ::= namelist : type |
Modularity
Algorithm
specification
namelist : type ; declarationlist
Recursion
Data Structures
namelist ::= name | name , namelist
ADTs
type ::= Bool | Integer | . . .
Sorting algorithm ::= declarations; statement
Alg. Theory
Sorting (ct’d)
algorithm is the start symbol of the grammar.
Binary Trees
Hashing
57/ 550
Example algorithm development: prime number I
DatAlg
Maritta Heisel
Introduction Task: calculate the first prime number which is greater than a
Alg. Design given “starting” number x (which is assumed to be a positive
Stepwise refinement
integer).
Control structures
Imperative paradigm
Algorithm notation
Example
var x: Integer ;
development
Modularity
repeat
Algorithm
specification
x := x + 1 ;
Recursion
Data Structures
test x for primeness
ADTs until x is prime endrepeat
Sorting
Alg. Theory Why does this algorithm terminate?

Sorting (ct’d)
Binary Trees
Hashing
58/ 550
Example algorithm development: prime number II
DatAlg
The step test x for primeness must be refined, because it is
Maritta Heisel
unlikely that the processor can execute it directly.
Introduction
Alg. Design Test if a number x has any nontrivial factors.

Stepwise refinement
Control structures
Imperative paradigm
divide x by every number between 1 and itself ;
Algorithm notation if no division is exact
Example
development
Modularity
then x is prime
Algorithm
specification
else x is not prime
Recursion
Data Structures
endif
ADTs
Sorting
The first step clearly involves iteration, in which the potential
Alg. Theory prime x is divided by successive possible factors starting with 2.
Sorting (ct’d)
Binary Trees Remark: For x = 1 and x = 2 no possible factors exist. Hence,

Hashing these cases have to be considered in the algorithm explicitly.
59/ 550
Example algorithm development: prime number III
DatAlg
Maritta Heisel
Introduction var x, factor , y : Integer ;

Alg. Design factor := 2 ;
Stepwise refinement repeat
Control structures
Imperative paradigm y := x mod factor ;
Algorithm notation
factor := factor + 1
Example
development √
Modularity until y = 0 ∨ factor > x endrepeat
Algorithm
specification
Recursion
We can stop looking for possible factors when
Data Structures
√
ADTs division is exact or factor > x.
Sorting
Alg. Theory
This saves processing time!
Sorting (ct’d)
Binary Trees
Hashing
60/ 550
Example algorithm development: prime number IV
DatAlg
Maritta Heisel
Overall algorithm for testing for primeness:
Introduction
Alg. Design
Stepwise refinement
var x, factor, y : Integer; prime : Bool;
Control structures
Imperative paradigm
factor := 2;
Algorithm notation
Example
repeat
development
Modularity
y := x mod factor;
Algorithm
specification
√
Recursion
Data Structures
until y = 0 ∨ factor > x endrepeat;
ADTs if (y 6= 0 ∧ x 6= 1) ∨ x = 2
Sorting then prime := true
Alg. Theory else prime := false
Binary Trees
Hashing
61/ 550
Example algorithm development: prime number V
DatAlg
Maritta Heisel
Introduction
Alg. Design
Stepwise refinement
Control structures
p
Imperative paradigm Here, the operations mod and (. . . ) have been used. It
Algorithm notation
Example
could be the case that these are not basic. Then, further
development
Modularity refinement steps would be necessary.
Algorithm
specification
Recursion
Data Structures
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
62/ 550
Example algorithm development: prime number VI
DatAlg
Maritta Heisel
Final algorithm:
Introduction var x, factor, y : Integer; prime : Bool;
Alg. Design repeat
Stepwise refinement x := x + 1;
Control structures
Imperative paradigm {Test x for primeness}
Algorithm notation
Example
factor := 2;
development
Modularity repeat
Algorithm
specification y := x mod factor;
Recursion
Data Structures
√
ADTs until y = 0 ∨ factor > x endrepeat;
Sorting if (y 6= 0 ∧ x 6= 1) ∨ x = 2
Alg. Theory then prime := true
Sorting (ct’d)
else prime := false
Binary Trees
endif
Hashing
63/ 550 until prime = true endrepeat
Post-tested loops vs. pre-tested loops I
DatAlg
Maritta Heisel
Introduction Algorithm development task: determine largest of a list of

Alg. Design numbers.
Stepwise refinement
Control structures
Algorithm skeleton:
Imperative paradigm
Algorithm notation
Example
set largest so far to first number in list;
development
Modularity
repeat
Algorithm
specification
consider next number in list;
Recursion
Data Structures
if this number > largest so far
ADTs then set largest so far to this number
Sorting endif
Alg. Theory until list is exhausted endrepeat
Sorting (ct’d)
Binary Trees
Hashing
64/ 550
Post-tested loops vs. pre-tested loops II
DatAlg
Maritta Heisel
Introduction Preparation for detailed algorithm:

Alg. Design
Syntax and semantics Lists are an abstract data type which will be discussed
Stepwise refinement
Control structures later in this course.
Imperative paradigm
Algorithm notation The following operations on lists are needed:
Example
development
Modularity
head selects the first element of a nonempty list
Algorithm
specification
tail removes the first element of a nonempty list
Recursion nil denotes the empty list.
Data Structures
ADTs We need variables l for the list, the largest number so far
Sorting (max), and the number currently considered (n).
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
65/ 550
Post-tested loops vs. pre-tested loops III
DatAlg
Maritta Heisel
Introduction var l : LIST[Integer]; max, n : Integer;

Alg. Design
Syntax and semantics max := head(l);
Stepwise refinement
Control structures l := tail(l);
Imperative paradigm
Algorithm notation
repeat
Example
development n := head(l);
Modularity
Algorithm
l := tail(l);
specification
Recursion if n > max
Data Structures
then max := n
ADTs
endif
Sorting
Alg. Theory
until l = nil endrepeat
Sorting (ct’d)
Binary Trees
Hashing
66/ 550
Post-tested loops vs. pre-tested loops IV
DatAlg
Maritta Heisel
The above algorithm seems correct, but it contains a serious
error which becomes apparent if the list has only one number in
Introduction
it. We must put the terminating condition at the start of the
Alg. Design
Syntax and semantics loop rather than at the end:
Stepwise refinement
Control structures
Imperative paradigm
var l : LIST[Integer] ; max, n : Integer;
Algorithm notation
Example max := head(l);
development
Modularity l := tail(l);
Algorithm
specification while l 6= nil do
Recursion
Data Structures n := head(l);
ADTs l := tail(l);
Sorting if n > max
Alg. Theory
then max := n
Sorting (ct’d)
endif
Binary Trees
endwhile
Hashing
67/ 550
Post-tested loops vs. pre-tested loops V
DatAlg
Maritta Heisel
Introduction Remember: The body of a post-tested loop is always executed

Alg. Design at least once, whereas the body of a pre-tested loop may not
Stepwise refinement be executed at all.
Control structures
Imperative paradigm
Algorithm notation
Example
This observation is very important because the wrong usage of
development
Modularity post- or pre-tested loops can generate serious errors in
Algorithm
specification designing an algorithm!
Recursion
Data Structures
ADTs
However: Even the second algorithm does not work if the list
Sorting
Alg. Theory
is empty!
Sorting (ct’d)
Binary Trees
Hashing
68/ 550
Example: Greatest common divisor (GCD)
DatAlg Mathematical laws:

Maritta Heisel
GCD(x, y ) = GCD(y , x mod y ) if y > 0
Introduction GCD(x, y ) = x if y = 0
Alg. Design
where x and y are any nonnegative integers.
Stepwise refinement
Control structures Example:
Imperative paradigm
Algorithm notation GCD(24, 9) = GCD(9, 6) = GCD(6, 3) = GCD(3, 0) = 3
Example
development
Modularity
Algorithm:
Algorithm
specification var x, y , r : Integer ;
Recursion
Data Structures
while y 6= 0 do
ADTs r := x mod y ;
Sorting x := y ;
Alg. Theory y := r
Sorting (ct’d) endwhile;
Binary Trees r := x
Hashing
69/ 550 A post-tested loop could result in division by 0!
Definite iteration
DatAlg
Particularly simple form
Maritta Heisel
Number of repetitions is known before the loop is executed
Introduction
Alg. Design
General form:
Stepwise refinement repeat N times
Control structures
Imperative paradigm body of loop
Algorithm notation
Example endrepeat
development
Modularity
Algorithm where N is an arbitrary positive integer
specification
Recursion
Data Structures
Example: calculate x N
ADTs
var x, N, p : Integer ;
Sorting
p := 1 ;
Alg. Theory
Sorting (ct’d)
repeat N times
Binary Trees
p := p ∗ x
Hashing endrepeat
70/ 550
Everyday examples
DatAlg
Maritta Heisel
Introduction Definite Indefinite

Alg. Design Play it again, Sam Play it until you get it right
Stepwise refinement
Control structures Keep walking for two miles Keep right on to the end of
Imperative paradigm
Algorithm notation the road
Example
development Wait here for five minutes Wait here until I return
Modularity
Algorithm
specification
Recursion
Bake for 20 minutes Bake until brown
Data Structures
ADTs
Sorting Definite iteration is safer than indefinite iteration, because it is

Alg. Theory guaranteed to terminate, but it is by no means as powerful.
Sorting (ct’d)
Binary Trees
Hashing
71/ 550
Alternative notation
DatAlg
Maritta Heisel repeat for each instance of an object

Introduction body of loop
Alg. Design endrepeat
Stepwise refinement
Control structures
Examples
Imperative paradigm
Algorithm notation
repeat for each wheel
Example
development
check tyre pressure
Modularity endrepeat
Algorithm
specification
Recursion
Data Structures
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
72/ 550
Alternative notation
DatAlg
Maritta Heisel repeat for each instance of an object

Introduction body of loop
Alg. Design endrepeat
Stepwise refinement
Control structures
Examples
Imperative paradigm
Algorithm notation
repeat for each wheel
Example
development
check tyre pressure
Modularity endrepeat
Algorithm
specification
Recursion
calculate factorial of N
Data Structures
ADTs var N, p, i : Integer ;

Sorting p := 1;
Alg. Theory repeat for each i from 1 to N
Sorting (ct’d) p := p ∗ i
Binary Trees endrepeat
Hashing
72/ 550
Modularity: motivation
DatAlg
Maritta Heisel
Introduction
Stepwise refinement divides algorithms into smaller
Alg. Design
Syntax and semantics components.
Stepwise refinement
Control structures These are often independent of the context in which they
Imperative paradigm
Algorithm notation are to be used.
Example
development
Modularity
“Plug-in” components, called modules (in programming
Algorithm
specification languages: procedure, routine, function, method).
Recursion
Data Structures A module is a self-contained algorithm.
ADTs
An algorithm that uses a module is said to call the
Sorting
module.
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
73/ 550
Example: Picture-drawing robot I
DatAlg
Maritta Heisel
A robot is equipped with wheels which allow it to move
Introduction
around over a sheet of paper.
Alg. Design
Syntax and semantics It has a pen which it can lower onto the paper when it
Stepwise refinement
Control structures wants to draw a line.
Imperative paradigm
Algorithm notation
Example
It can interpret and execute commands of the form
development
Modularity
Algorithm move(x) move forward x cm
specification
Recursion left(x) rotate x degrees to the left
Data Structures
right(x) rotate x degrees to the right
ADTs
Sorting
raise pen lift pen off paper
Alg. Theory
lower pen put pen down to paper
Sorting (ct’d) where x is a real number.
Binary Trees
Hashing
74/ 550
Example: Picture-drawing robot II
DatAlg B
Maritta Heisel
move to point A; A
Introduction draw a square of side 10 cm;

Alg. Design move to point B; 20 cm 10 cm X
Stepwise refinement draw a square of side 20 cm
Control structures
Imperative paradigm
Algorithm notation
Example
development
Modularity
Algorithm
specification
Recursion
Drawing squares is a self-contained process quite
Data Structures
independent of the rest of the algorithm.
ADTs
Sorting
Hence, we can design a square-drawing algorithm without
Alg. Theory
reference to the larger algorithm in which it is to be used.
Sorting (ct’d) Once that algorithm is designed we can simply “plug it in”
Binary Trees at the appropriate points of the above algorithm.
Hashing
75/ 550
Formal and actual parameters
DatAlg
Maritta Heisel
Introduction For a square drawing module to be generally useful, it

Alg. Design must be capable of drawing a square of any size (formal
Stepwise refinement parameter).
Control structures
Imperative paradigm When the module is called, the specific size required must
Algorithm notation
Example
development
be indicated (actual parameter).
Modularity
Algorithm
Example: drawsquare(10) and drawsquare(20)
specification
Recursion Formal parameters represent the information the module
Data Structures
ADTs
needs when it is called.
Sorting Actual parameters are the pieces of information supplied
Alg. Theory for a particular call.
Sorting (ct’d)
Binary Trees
Hashing
76/ 550
Module for drawing squares
DatAlg module drawsquare(size : Real)

Maritta Heisel {Draws a square of size cm. The square is drawn
Introduction counterclockwise, starting from the current position of the
Alg. Design robot. The first edge is drawn according to the robot’s current
Stepwise refinement
orientation. The robot returns to its initial position and
Control structures
Imperative paradigm
orientation, with the pen raised.
Algorithm notation Assumption: Initially the pen is raised.}
Example
development
Modularity lower pen;
Algorithm
specification repeat 4 times
Recursion
Data Structures move(size);
ADTs left(90)
Sorting
endrepeat;
Alg. Theory
raise pen
Sorting (ct’d)
endmodule
Binary Trees
Hashing
77/ 550
Note: Comments (and specifications) are important!
Syntax for defining and calling modules
DatAlg
Definition
Maritta Heisel
Procedure: is called for the effects it causes
Introduction module modulename(formal parameters)
Alg. Design . . . body of module . . .
endmodule
Stepwise refinement
Control structures Function: is called to obtain the function value
Imperative paradigm
Algorithm notation
module modulename(formal parameters): type
Example
development
. . . body of module . . . return . . .
Modularity endmodule
Algorithm
specification
Recursion
Call
Data Structures Procedure: modulename(actual parameters)
ADTs
Function: x := modulename(actual parameters)
Sorting
Alg. Theory
Body of module is executed, with the formal parameters
Sorting (ct’d)
replaced by the actual parameters
Binary Trees Number and types of formal and actual parameters must
Hashing be the same!
78/ 550
Updating the grammar
DatAlg
algorithm ::= declarations; statement | moduledecl
Maritta Heisel
moduledecl ::= module moduleheader
Introduction declarations; statement endmodule
Alg. Design statement ::= assignment | sequence | conditional |
Stepwise refinement iteration | modulecall
Control structures
Imperative paradigm moduleheader ::= name(declarationlist) |
Algorithm notation
Example name(declarationlist) : type
development
Modularity assignment ::= name := expression |
Algorithm
specification name := modulecall
Recursion
Data Structures modulecall ::= name(namelist)
ADTs
Exercise:
Sorting
Alg. Theory Add the return statement to the grammar.

Sorting (ct’d) Enhance the grammar such that module calls can be used
Binary Trees inside expressions.
Hashing
79/ 550
Refinement of example algorithm
DatAlg
Maritta Heisel
Introduction
Alg. Design
{Assumption: robot is initially at point X }
Syntax and semantics left(45);
√
Stepwise refinement
Control structures move( 50); {move to point A }
Imperative paradigm
Algorithm notation left(135);
Example
development drawsquare(10); {draw inner square }
Modularity
Algorithm right(135);
√
specification
Recursion move( 50); {move to point B }
Data Structures
ADTs
left(135);
Sorting
drawsquare(20) {draw outer square }
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
80/ 550
Abstraction of the square-drawing algorithm
DatAlg Modules can be nested, too.

Maritta Heisel A square is simply a special case of a polygon.
Introduction module drawpolygon(size : Real; N : Integer)
Alg. Design {Draws an N-sided regular polygon with sides size cm long.
Stepwise refinement The polygon is drawn counterclockwise, starting from the
Control structures
Imperative paradigm current position of the robot. The first edge is drawn according
Algorithm notation
Example to the robot’s current orientation. The robot returns to its
development
Modularity initial position and orientation, with the pen raised.
Algorithm
specification Assumption: Initially the pen is raised.}
Recursion
Data Structures lower pen;
ADTs repeat N times;
Sorting move(size);
Alg. Theory
left(360/N)
Sorting (ct’d)
endrepeat;
Binary Trees
raise pen
Hashing
81/ 550 endmodule
Using drawpolygon
DatAlg
Maritta Heisel module drawsquare(size : Real)

Introduction
{Draws a square of size cm. . . . }
Alg. Design
drawpolygon(size, 4)
Syntax and semantics endmodule
Stepwise refinement
Control structures
Imperative paradigm
Algorithm notation module drawtriangle(size : Real)
Example
development {Draws a triangle of size cm. . . . }
Modularity
Algorithm drawpolygon(size, 3)
specification
Recursion endmodule
Data Structures
ADTs Note:
Sorting
The new definition of drawsquare does not require any
Alg. Theory
alterations of the algorithms in which it is called!
Sorting (ct’d)
Binary Trees
The input parameters must be positive numbers!
Hashing
82/ 550
Advantages of using modules
DatAlg
Maritta Heisel
Modules fit naturally into stepwise refinement, giving a
Introduction top-down design.
Alg. Design
Syntax and semantics A module is a self-contained component of any larger
Stepwise refinement
Control structures
algorithm which calls it. The design of the module and of
Imperative paradigm
Algorithm notation
the calling algorithm can be considered separately.
Example
development To incorporate a module in an algorithm it is necessary to
Modularity
Algorithm know only what the algorithm does, but not how it does it.
specification
Recursion
Data Structures
Modules not only simplify the design, but also the
ADTs understanding of algorithms.
Sorting Once a module has been designed it can be incorporated
Alg. Theory in any algorithm which needs it. It is therefore possible to
Sorting (ct’d)
build up a “library” of modules.
Binary Trees
Hashing
83/ 550
Summary: basic concepts
DatAlg
Maritta Heisel
The concept of an algorithm is central to computer
Introduction
Alg. Design
science.
Stepwise refinement
Algorithms are timeless and do not depend on the
Control structures
Imperative paradigm
technology of the day.
Algorithm notation
Example
An algorithm describes a process to be carried out by a
development
Modularity processor.
Algorithm
specification
Recursion
To execute an algorithm on a computer, it must be
Data Structures transformed into a program, expressed in some
ADTs
programming language.
Sorting
Alg. Theory
The description of an algorithm must be free of syntactic
Sorting (ct’d)
errors, semantic inconsistencies, and logical errors.
Binary Trees
Hashing
84/ 550
Summary: expressing algorithms
DatAlg
Maritta Heisel
Control structures for algorithms are sequence, selection,
Introduction
Alg. Design
and iteration. In combination with basic steps
Syntax and semantics (assignments, evaluation of expressions), they suffice to
Stepwise refinement
Control structures express all conceivable algorithms.
Imperative paradigm
Algorithm notation To re-use algorithms, we use modules.
Example
development
Modularity Modules have formal parameters, and they are called with
Algorithm
specification actual parameters.
Recursion
Data Structures For each algorithm, it must be shown that it terminates.
ADTs
Sorting
Alg. Theory For syntax definitions, we can use grammars or inductive

Sorting (ct’d) definitions.
Binary Trees
Hashing
85/ 550
Summary: developing algorithms by stepwise
refinement
DatAlg
Maritta Heisel
Algorithms can be developed by stepwise refinement.
Introduction
Alg. Design In stepwise refinement, we start out with a coarse-grained

Stepwise refinement
algorithm and gradually replace non-basic steps by further
Control structures
Imperative paradigm
algorithms.
Algorithm notation
Example
This approach corresponds to a top-down problem
development
Modularity decomposition.
Algorithm
specification
Recursion
Stepwise refinement terminates when all steps of the
Data Structures algorithm are basic, i.e., can be carried out by the
ADTs
envisaged processor.
Sorting
Alg. Theory
Therefore, it is necessary to know what steps are basic for
Sorting (ct’d)
a given processor.
Binary Trees
Hashing
86/ 550
Specification of algorithms
DatAlg
Maritta Heisel
Introduction
Alg. Design For each algorithm, we must write down the intended
Stepwise refinement process that algorithm is expected to perform; otherwise
Control structures
Imperative paradigm
we cannot know if the algorithm is correct.
Algorithm notation
Example This description is called the specification of the algorithm.
development
Modularity
Algorithm
Such a specification consists of a precondition and a
specification
Recursion
postcondition.
Data Structures
ADTs
They form a contract between a module/algorithm and its
Sorting
callers.
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
87/ 550
Design by Contract
DatAlg
Maritta Heisel
Introduction
Alg. Design
Source:
Stepwise refinement
Bertrand Meyer
Control structures Object-Oriented Software Construction
Imperative paradigm
Algorithm notation Prentice Hall 1988 (first edition), 1997 (second edition)
Example
development
Modularity
Algorithm online see:
specification
Recursion https://www.eiffel.com/values/design-by-
Data Structures
contract/introduction/
ADTs
Sorting
(last visited February 20, 2017)
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
88/ 550
Contracts in daily life
DatAlg
Maritta Heisel
Introduction
Alg. Design
Stepwise refinement
Control structures Contractual partners are clients and sellers or service
Imperative paradigm
Algorithm notation providers.
Example
development
Modularity
Both expect advantages from the contract and are willing
Algorithm
specification to make a commitment.
Recursion
Data Structures
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
89/ 550
Example: traveling
DatAlg
I want to travel from Berlin to Duisburg.
Maritta Heisel
Introduction
Commitments Advantages
Alg. Design Passenger Pay ticket Getting to Duisburg
Stepwise refinement
Be there at depar-
Control structures
Imperative paradigm
ture time
Algorithm notation Must keep precondi- Has advantages from the
Example
development
Modularity
tion postcondition
Algorithm
specification
Traffic Must take the pas- Receives the price for the
Recursion
Data Structures
provider senger to Duisburg ticket; does not have to
ADTs take passengers who have
Sorting not paid or did not arrive
Alg. Theory in time
Sorting (ct’d) Must guarantee Can assume precondition
Binary Trees postcondition
Hashing
90/ 550
Example: tailor services
DatAlg
Maritta Heisel
I want to get my trousers shortened.
Introduction Commitments Advantages
Alg. Design
Client Bring trousers Get shortened trousers
Stepwise refinement Pay
Control structures
Imperative paradigm Must keep precon- Has advantages from the
Algorithm notation
Example dition postcondition
development
Modularity Tailor Must shorten my Receives payment; does not
Algorithm
specification
Recursion
trousers have to shorten trousers that
Data Structures are not taken to the work-
ADTs shop or have not been paid
Sorting
for
Alg. Theory
Must guarantee Can assume precondition
Sorting (ct’d)
postcondition
Binary Trees
Hashing
91/ 550
Advantages of explicit contracts
DatAlg
Maritta Heisel
Introduction
Alg. Design
Stepwise refinement Meyer:
Control structures
Imperative paradigm A contract document protects both the client, by spec-
Algorithm notation
Example
development
ifying how much should be done, and the supplier, by
Modularity stating that the supplier is not liable for failing to carry
Algorithm
specification
Recursion
out tasks outside of the specified scope.
Data Structures
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
92/ 550
Application to software
DatAlg
A contract is a formal agreement between a module/algorithm
Maritta Heisel
and its callers. It specifies the rights and duties for both sides:
Introduction
Alg. Design caller must ensure precondition of module/algorithm

Stepwise refinement can assume that postcondition holds after the
Control structures
Imperative paradigm module/algorithm has terminated
Algorithm notation
Example
development
module/algorithm can assume precondition
Modularity
Algorithm
must ensure postcondition
specification
Recursion
Data Structures
ADTs
If the caller calls the module/algorithm with the precondition
Sorting
not satisfied, the module/algorithm has no obligations
Alg. Theory
whatsoever!
Sorting (ct’d)
Binary Trees
It is not the module/algorithm that has to test if the
Hashing
precondition holds, but the caller of the module!
93/ 550
More precise specification notation
DatAlg
Maritta Heisel
Introduction In order to develop correct algorithms (i.e., algorithms that

Alg. Design indeed describe the intended process), the algorithm
Stepwise refinement
development task must be stated unambiguously by describing
Control structures
Imperative paradigm
the precondition and the postcondition of the algorithm.
Algorithm notation
Example In the following, we will not use verbal descriptions any
development
Modularity more, neither for specifications nor for algorithms.
Algorithm
specification
Recursion
For specifications, we will use formulas.
Data Structures
We will furthermore classify the variables used in the
ADTs
Sorting
formulas expressing the pre- and postcondition of an
Alg. Theory
algorithm (instead of read- or write-statements).
Sorting (ct’d)
Binary Trees
Hashing
94/ 550
Stating algorithm development problems precisely
DatAlg We specify algorithms as follows:

Maritta Heisel
var declaration of variables that are used in the speci-
fication
Introduction
pre precondition, expressed as a formula
Alg. Design
post postcondition, expressed as a formula
Stepwise refinement
Control structures
reads variables that are allowed to be read by the algo-
Imperative paradigm
Algorithm notation
rithm, but not changed
Example
development
changes variables that are allowed to be changed (and read)
Modularity
Algorithm
by the algorithm
specification
Recursion
mem variables that are not allowed to occur in the algo-
Data Structures
rithm; they serve to memorize the state in which
ADTs
the algorithm is called.
Sorting
where
Alg. Theory
Sorting (ct’d)
var declares all free variables occurring in pre and post
Binary Trees reads, changes and mem are disjoint and contain all free
Hashing variables occurring in pre and post
95/ 550
Syntax of formulas
DatAlg
Maritta Heisel
formula ::= booleanexpr | (formula) | ¬ formula |
Introduction
formula logicalop formula |
Alg. Design
∀ declarationlist • formula |
Stepwise refinement
Control structures
∃ declarationlist • formula
Imperative paradigm
Algorithm notation Example:
Example
development ∀ x : Integer • ∃ y : Integer • prime(y ) ∧ y > x
Modularity
Algorithm
specification Precedence rules: ¬ , ∧, ∨, ⇒, ⇔
Recursion
Data Structures ∀, ∃: scope of quantifiers as large as possible
ADTs
Sorting
The variables occurring in the declarationlist of a quantified
Alg. Theory
formula (∀ and ∃ are called quantifiers) are bound, the others
Sorting (ct’d)
are free.
Binary Trees
Hashing
96/ 550
Semantics of formulas
DatAlg
Maritta Heisel
Formulas can be either true or false.
Introduction
The truth of formulas with free variables depends on the
Alg. Design
Syntax and semantics value of these variables, i.e., on the state.
Stepwise refinement
Control structures
Example: x > y .
Imperative paradigm
Algorithm notation A universally quantified formula is true iff for all possible
Example
development values of the quantified variables, the matrix (i.e., the
Modularity
Algorithm formula without quantifier and declaration list) is true.
specification
Recursion
Data Structures
An existentially quantified formula is true iff there are
ADTs possible values of the quantified variables such that the
Sorting matrix is true.
Alg. Theory The truth of formulas built using logical operators is
Sorting (ct’d)
defined by truth tables.
Binary Trees
Hashing
97/ 550
Truth tables
DatAlg
Maritta Heisel
x ¬x
true false
Introduction
false true
Alg. Design
Stepwise refinement x y x∧y x y x∨y
Control structures
Imperative paradigm true true true true true true
Algorithm notation
Example
development
true false false true false true
Modularity false true false false true true
Algorithm
specification
Recursion
false false false false false false
Data Structures
ADTs x y x⇒y x y x⇔y

Sorting true true true true true true
Alg. Theory true false false true false false
Sorting (ct’d)
false true true false true false
Binary Trees
false false true false false true
Hashing
98/ 550
Correctness of algorithms
DatAlg
Maritta Heisel
Introduction
An algorithm is correct with respect to a specification iff
Alg. Design the algorithm, started in a state where pre holds,
Stepwise refinement
terminates in a state where post holds (total correctness,
Control structures
Imperative paradigm
as opposed to partial correctness, where the termination is
Algorithm notation
Example
not shown) and
development
Modularity the algorithm does not contain any variables from mem
Algorithm
specification and does not change any variables from reads.
Recursion
Data Structures
ADTs We can introduce new variables during algorithm design that

Sorting
are not contained in reads, changes and mem. There is no
Alg. Theory
restriction on the use of the newly introduced variables.
Sorting (ct’d)
Binary Trees
Hashing
99/ 550
Summary: algorithm specifications
DatAlg
Input Output
Maritta Heisel
Algorithm
Introduction
Alg. Design
Stepwise refinement
Control structures
Imperative paradigm State
Algorithm notation
Pre−state Post−state
Example
development
Modularity
Algorithm
specification
Recursion Precondition describes input (reads, possibly changes-clause)
Data Structures
ADTs
and pre-state; characterizes the conditions under
Sorting
which the algorithm may be called (pre-clause)
Alg. Theory Postcondition states relation between input/pre-state
Sorting (ct’d) and output/post-state (changes, possibly mem
Binary Trees and post-clauses); describes what the algorithm
Hashing does, i.e., its effect
100/ 550
If precondition is not satisfied
DatAlg
Maritta Heisel
Input ???
Introduction Algorithm
Alg. Design
Stepwise refinement
Control structures
Imperative paradigm
Algorithm notation
State
Example
development Pre−state ???
Modularity
Algorithm
specification
Recursion
Data Structures If we call an algorithm with the precondition not satisfied
ADTs
we do not know if there is any output and – if so – how it
Sorting
looks like
Alg. Theory
Sorting (ct’d)
we do not know if the module will terminate and – if so –
Binary Trees how the post-state will look like
Hashing
101/ 550
Example: GCD I
DatAlg
Maritta Heisel
Introduction
Alg. Design Specification:

Stepwise refinement
Control structures
Imperative paradigm
var x, y , r : Integer
Algorithm notation
Example
pre x ≥0 ∧ y ≥0
development
Modularity post r = GCD(x, y )
Algorithm
specification reads -
Recursion
Data Structures changes r , x, y
ADTs mem -
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
102/ 550
Example: GCD II
DatAlg
Maritta Heisel
Introduction Algorithm (using a procedure):

Alg. Design
Syntax and semantics module gcd(x, y : Integer)
Stepwise refinement
Control structures var r : Integer ;
Imperative paradigm
Algorithm notation while y 6= 0 do
Example
development r := x mod y ;
Modularity
Algorithm x := y ;
specification
Recursion y := r
Data Structures
ADTs
endwhile;
Sorting
r := x
Alg. Theory
endmodule
Sorting (ct’d)
Binary Trees
Hashing
103/ 550
Rule for using memory variables
DatAlg
Maritta Heisel Problem: the algorithm

Introduction
Alg. Design x := 1; y := 1; r := 1
Stepwise refinement
Control structures
would also be correct with respect to the above specification,
Imperative paradigm
Algorithm notation
because the postcondition refers to the values of the variables
Example
development
after termination of the algorithm.
Modularity
Algorithm
To prevent “cheating”, we use the following
specification
Recursion
Data Structures
Specification rule for algorithms where the input may be
ADTs
changed:
Sorting
If the input of an algorithm is allowed to be changed, introduce
Alg. Theory
memory variables (using equations in the precondition) and
Sorting (ct’d)
express the postcondition using the memory variables.
Binary Trees
Hashing
104/ 550
Corrected specification for GCD problem
DatAlg
Maritta Heisel
Introduction
Alg. Design
Stepwise refinement
var x, y , x 0 , y 0 , r : Integer
Control structures
Imperative paradigm
pre x ≥ 0 ∧ y ≥ 0 ∧ x = x0 ∧ y = y0
Algorithm notation post r = GCD(x 0 , y 0 )
Example
development
Modularity
reads -
Algorithm
specification
changes r , x, y
Recursion
Data Structures
mem x 0, y 0
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
105/ 550
Commitments and advantages
DatAlg
Maritta Heisel
Introduction Commitments Advantages

Alg. Design Caller Call gcd(x, y ) only Get result r with r =
Stepwise refinement
with integer numbers GCD(x, y ).
Control structures
Imperative paradigm
x ≥ 0 and y ≥ 0
Algorithm notation
Example
Must keep precondition Has advantages from post-
development
Modularity
condition
Algorithm
specification Module Make sure that r really Unnecessary to treat the
Recursion
Data Structures
is the GCD of x and y case where x or y is not a
ADTs non-negative integer num-
Sorting ber
Alg. Theory Must guarantee post- Can assume precondition
Sorting (ct’d) condition
Binary Trees
Hashing
106/ 550
Example: function for GCD I
DatAlg
Maritta Heisel
Introduction
Alg. Design Specification:

Stepwise refinement
Control structures
Imperative paradigm
var x, y , x 0 , y 0 : Integer
Algorithm notation pre x ≥ 0 ∧ y ≥ 0 ∧ x = x0 ∧ y = y0
Example
development
Modularity
post gcd fct(x 0 , y 0 ) = GCD(x 0 , y 0 )
Algorithm
specification
reads -
Recursion
Data Structures
changes x, y
ADTs mem x 0, y 0
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
107/ 550
Example: function for GCD II
DatAlg
Maritta Heisel
Introduction Algorithm (using a function):

Alg. Design
Syntax and semantics module gcd fct(x, y : Integer) : Integer
Stepwise refinement
Control structures var tmp : Integer ;
Imperative paradigm
Algorithm notation while y 6= 0 do
Example
development tmp := x mod y ;
Modularity
Algorithm x := y;
specification
Recursion y := tmp
Data Structures
ADTs
endwhile;
Sorting
return x
Alg. Theory
endmodule
Sorting (ct’d)
Binary Trees
Hashing
108/ 550
Recursive algorithms
DatAlg
Maritta Heisel
Introduction
A recursive algorithm is one which calls itself (maybe
Alg. Design
indirectly).
Stepwise refinement
Control structures In the section on definite loops, we developed an algorithm for
Imperative paradigm
Algorithm notation
calculating the factorial of n.
Example
development
Modularity
Algorithm
Alternative solution: use the fact that for any n > 1 the
specification
Recursion
formula
Data Structures
n! = n × (n − 1)!
ADTs
Sorting applies. Moreover,

Alg. Theory 1! = 1
Sorting (ct’d)
Binary Trees
Hashing
109/ 550
Factorial: specification and recursive algorithm
DatAlg
Specification:
Maritta Heisel
Introduction var n : Integer

Alg. Design pre n>0
Stepwise refinement
post factorial(n) = n!
Control structures
Imperative paradigm
reads n
Algorithm notation
Example
changes -
development
Modularity
mem -
Algorithm
specification Hence, a possible algorithm for calculating the factorial of n is
Recursion
Data Structures
module factorial(n : Integer) : Integer
ADTs
Sorting
if n = 1
Alg. Theory
then return 1
Sorting (ct’d) else return n * factorial(n - 1)
Binary Trees endif
Hashing endmodule
110/ 550
Calculating factorial(3)
DatAlg
Maritta Heisel
Introduction
Alg. Design if 3 = 1
Syntax and semantics then... 6
Stepwise refinement else return 3 * factorial(2)
Control structures
Imperative paradigm
Algorithm notation
Example
if 2 = 1
2
development
Modularity then...
Algorithm
specification
else return 2 * factorial(1)
Recursion
Data Structures
ADTs if 1 = 1
Sorting then return 1 1
Alg. Theory
else...
...
Sorting (ct’d)
Binary Trees
Hashing
111/ 550
Termination of recursive algorithms
DatAlg
Maritta Heisel
Introduction
Alg. Design
Syntax and semantics The input for recursive calls must be in some sense
Stepwise refinement
Control structures
“simpler” than the original input.
Imperative paradigm
Algorithm notation There must be a limiting case, in which the input is so
Example
development “simple” that the process can be carried out without
Modularity
Algorithm
specification
calling itself further.
Recursion
Data Structures
Factorial-example: input becomes smaller and eventually
ADTs reaches 1.
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
112/ 550
Reversing the order of letters in a word I
DatAlg
Maritta Heisel
Introduction For example, transform “star” into “rats”

Alg. Design
Stepwise refinement
Specification:
Control structures var seq : LIST [Char ]
Imperative paradigm
Algorithm notation pre seq 6= nil
Example
development post reverse alg (seq) = reverse(seq)
Modularity
Algorithm
specification
reads seq
Recursion changes -
Data Structures
ADTs
mem -
Sorting Idea: remove first letter of word, reverse rest of the word,
Alg. Theory append the letter which was removed.
Sorting (ct’d)
Binary Trees
Hashing
113/ 550
Reversing the order of letters in a word II
DatAlg
Maritta Heisel Algorithm:

Introduction module reverse alg(seq : LIST[Char]) : LIST[Char]
Alg. Design if length(seq) = 1
Stepwise refinement then return seq
Control structures
Imperative paradigm
else return append(reverse alg(tail(seq)),
Algorithm notation
Example
cons(head(seq),nil))
development
Modularity
endif
Algorithm
specification endmodule
Recursion
Data Structures
where the function cons concatenates an element to a list, the
ADTs
function append concatenates two lists and the function length
Sorting
counts the elements of a list.
Alg. Theory
Sorting (ct’d) Termination: length of sequence is reduced by one, eventually

Binary Trees reaching 1.
Hashing
114/ 550
Reversing “star”
DatAlg
if length(’star’) = 1
Maritta Heisel then...
else return append(reverse_alg(tail(’star’)),cons(head(’star’),nil))
Introduction
Alg. Design
Syntax and semantics if length(’tar’) = 1
Stepwise refinement then...
Control structures else return append(reverse_alg(tail(’tar’)),
Imperative paradigm cons(head(’tar’),nil))
Algorithm notation
Example
development
if length(’ar’) = 1
Modularity then...
Algorithm else return append(reverse_alg(
specification
tail(’ar’)),cons(head(’ar’),nil))
Recursion
Data Structures
if length(’r’) = 1 r
ADTs then return ’r’
Sorting else...
Alg. Theory append(’r’,’a’) ra

Sorting (ct’d) rat
append(’ra’,’t’)
Binary Trees
append(’rat’,’s’) rats
Hashing
115/ 550
Recursion vs. iteration
DatAlg
Maritta Heisel
Introduction
Alg. Design
Both the factorial and the reversion algorithms can be
Stepwise refinement
Control structures
replaced by iterative algorithms.
Imperative paradigm
Algorithm notation In general, for any algorithm which uses recursion there is
Example
development an equivalent algorithm which uses iteration.
Modularity
Algorithm
specification
However, a recursive algorithm for describing a particular
Recursion
Data Structures
process is often far more concise than an iterative one, and
ADTs in many cases it is much easier to derive and understand.
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
116/ 550
Recursive version of GCD
DatAlg
Maritta Heisel The equations

Introduction
GCD(x, y ) = GCD(y , MOD(x, y )) if y > 0
Alg. Design
GCD(x, y ) = x if y = 0
Stepwise refinement
Control structures can directly be translated into a recursive algorithm:
Imperative paradigm
Algorithm notation
Example
development module gcd(x : Integer, y : Integer) : Integer
Modularity
Algorithm if y = 0
specification
Recursion then return x
Data Structures
else return gcd(y, x mod y )
ADTs
Sorting
endif
Alg. Theory
endmodule
Sorting (ct’d)
Binary Trees
Termination?
Hashing
117/ 550
Calculating GCD(24,9)
DatAlg
if 9 = 0 3
Maritta Heisel
then...
Introduction else return gcd(9,6)
Alg. Design
Syntax and semantics if 6 = 0 3
Stepwise refinement
then...
else return gcd(6,3)
Control structures
Imperative paradigm
Algorithm notation
Example
3
if 3 = 0
development
Modularity
Algorithm
specification
then...
Recursion else return gcd(3,0)
Data Structures
ADTs
if 0 = 0 3
Sorting
then return 3
Alg. Theory else...
Sorting (ct’d)
Binary Trees
Hashing
118/ 550
The Towers of Hanoi
DatAlg
Maritta Heisel
According to legend an order of Buddhist monks in Hanoi has
Introduction been engaged for many years in the following task:
Alg. Design
The monks have three vertical rods and a set of 64
Stepwise refinement
Control structures
circular disks of different sizes
Imperative paradigm
Algorithm notation
Each disk has a hole in the center which allows it to be
Example
development slid onto any of the rods
Modularity
Algorithm
specification
Initially the disks were all placed on a single rod, one on
Recursion
Data Structures
the top of the other in order of decreasing size
ADTs The task the monks have set themselves is to transfer this
Sorting original tower to one of the other rods, moving only one
Alg. Theory disk at a time and ensuring that no disk ever rests on top
Sorting (ct’d) of a smaller one
Binary Trees
Hashing
119/ 550
The Towers of Hanoi
DatAlg
Maritta Heisel
Introduction
Alg. Design
Stepwise refinement
Control structures 64 disks
Imperative paradigm in all
Algorithm notation
Example
development
Modularity
Algorithm
specification
Recursion
Data Structures
ADTs
The labor involved in carrying out this task is said to be
Sorting
spiritually uplifting, and the monks believe that its completion
Alg. Theory
will herald the end of the world.
Sorting (ct’d)
Binary Trees
Hashing
120/ 550
Transfer the bottom disk to another rod
DatAlg
Maritta Heisel
Introduction Suppose that the rod supporting the original tower is

Alg. Design called a, the rod which the tower is to be moved is called
Stepwise refinement
b, and the other rod is called c
Control structures
Imperative paradigm The only way in which the bottom disk of the original
Algorithm notation
Example tower can be moved to form the base of the new tower is
development
Modularity to first of all get the 63 disks on top of it out of the way
Algorithm
specification
Recursion
The only place to put these 63 disks is on the third rod
Data Structures
ADTs transfer the top 63 disks from rod a to rod c;

Sorting
move the bottom disk from rod a to rod b;
Alg. Theory
transfer the 63 disks from rod c to rod b
Sorting (ct’d)
Binary Trees
Hashing
121/ 550
Transfer N disks from source to target
DatAlg
Maritta Heisel
Introduction
Similarly, the task of transferring 63 disks between two
Alg. Design
rods can be described in terms of transferring 62 between
Stepwise refinement
Control structures
two rods, and so on
Imperative paradigm
Algorithm notation
Generalizing an algorithm for transferring N disks from a
Example
development rod called source to a rod called target (given a third rod
Modularity
Algorithm
called workspace) is:
specification
Recursion
Data Structures
transfer N - 1 disks from source to workspace;
ADTs
move 1 disk from source to target;
Sorting
transfer N-1 disks from workspace to target
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
122/ 550
Module for transferring N disks from source to
target
DatAlg
Maritta Heisel
module movetower(N: Integer; source, target, workspace: Rod)
{Moves a number of N ≥ 1 discs from source
Introduction
to target using workspace where necessary}
Alg. Design
if N = 1
Stepwise refinement
Control structures
then {Output: move the disc from source to target}
Imperative paradigm
Algorithm notation
else {Move all discs except the bottom disc to workspace
Example
development
using target}
Modularity
Algorithm
movetower(N-1, source, workspace, target);
specification
Recursion
{Output: move the bottommost disc from source
Data Structures
to target}
ADTs
{Move the discs which are currently on workspace
Sorting
to target using source}
Alg. Theory
movetower(N-1, workspace, target, source)
Sorting (ct’d)
endif
Binary Trees
Hashing
endmodule
123/ 550
Execution of the module movetower I
DatAlg
if 3 = 1 a b c
Maritta Heisel
then ...
else transfer 2 disks from a to c
Introduction
if 2 = 1 transfer 3 disks from a to b
Alg. Design
then ...
else transfer 1 disk from a to b
a b c
Stepwise refinement
Control structures
Imperative paradigm if 1 = 1
Algorithm notation
then move disk from a to b
Example
development else ... a b c
Modularity
Algorithm move disk from a to c
specification transfer 1 disk from b to c
Recursion
a b c
Data Structures
if 1 = 1
ADTs
then move disk from b to c
Sorting else ... a b c
Alg. Theory
move disk from a to b
Sorting (ct’d)
transfer 2 disks from c to b
Binary Trees
Hashing
124/ 550
Execution of the module movetower II
DatAlg
Maritta Heisel
Introduction
if 2 = 1
Alg. Design then ...
Syntax and semantics else transfer 1 disk from c to a
Stepwise refinement a b c
Control structures if 1 = 1
Imperative paradigm
Algorithm notation
then move disk from c to a
Example else ... a b c
development
Modularity move disk from c to b
Algorithm transfer 1 disk from a to b
specification
Recursion if 1 = 1 a b c
Data Structures
then move disk from a to b
ADTs else ...
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
125/ 550
Execution time of the algorithm
DatAlg
Maritta Heisel
Execution of the algorithm with N = 64 results in two
Introduction
Alg. Design
executions with N = 63, each of which results in two
Syntax and semantics executions with N = 62, and so on.
Stepwise refinement
Control structures
Imperative paradigm
Transfer of the complete tower requires 264 − 1 executions
Algorithm notation of the algorithm.
Example
development
Modularity If the monks can move one disk in a second, and never
Algorithm
specification make a mistake, the time taken to transfer the whole
Recursion
Data Structures tower will be about 600 000 000 000 years!
ADTs
Furthermore, it can be proved there is no faster algorithm.
Sorting
Alg. Theory
Finally, an iterative algorithm would be less concise and
Sorting (ct’d)
comprehensible.
Binary Trees
Hashing
126/ 550
Strategy for devising recursive algorithms
DatAlg
Maritta Heisel
Introduction
1. Determine how the process to be carried out can be
Alg. Design
expressed recursively:
Stepwise refinement
Control structures
that is, how the process can be expressed in terms of
Imperative paradigm similar processes which differ from the original only by
Algorithm notation
Example
development
having an input which is in some sense simpler (e.g. a
Modularity smaller number, or a shorter sequence)
Algorithm
specification
Recursion 2. Ensure that there is a limiting case in which the input is so
Data Structures
simple that the process can be executed without recourse
ADTs
to further recursive calls; the limiting case then acts as an
Sorting
Alg. Theory
escape route from recursion
Sorting (ct’d)
Binary Trees
Hashing
127/ 550
Summary: specification notation, recursion
DatAlg
Maritta Heisel
Algorithms must be specified, using preconditions,
Introduction
postconditions and a classification of variables.
Alg. Design
Syntax and semantics Specifications express what is to be done; an algorithm
Stepwise refinement
Control structures expresses how it is done. Hence, different algorithms may
Imperative paradigm
Algorithm notation satisfy the same specification.
Example
development
Modularity
Specifications are necessary to apply the
Algorithm
specification
design-by-contract principle. The caller of a module is
Recursion
Data Structures
responsible for guaranteeing the precondition, the module
ADTs is responsible for establishing the postcondition.
Sorting Using recursion often leads to intuitive, comprehensible
Alg. Theory algorithms. However, it must be demonstrated that the
Sorting (ct’d) recursion terminates.
Binary Trees
Hashing
128/ 550
Control vs. data structures
DatAlg
Maritta Heisel Control structures

Introduction
Constructs used to control the order and circumstances in
Alg. Design which the individual steps of an algorithm are executed
Stepwise refinement The choice of appropriate control structures is an essential
Control structures
Imperative paradigm
part of the development of an algorithm
Algorithm notation
Example However, the choice of appropriate control structures
development
Modularity cannot be performed without considering the information,
Algorithm
specification
Recursion
or data, which the algorithm manipulates
Data Structures
Data structures
ADTs
Sorting
In general, the data manipulated by an algorithm are not
Alg. Theory
merely an arbitrary collection of unrelated items, but
Sorting (ct’d) rather consist of pieces of information which are in some
Binary Trees way related to each other
Hashing
129/ 550
Example
DatAlg
Unstructured data
Maritta Heisel
Personnel officer
Freda Bloggs 27
Introduction
Alg. Design 56
John Brown
47, Main Street
Stepwise refinement
23
Control structures
Shift foreman
Imperative paradigm
Joe Cool
Algorithm notation
Example
development
Modularity
Algorithm
specification
Name, address, age, position describe the same person.
Recursion
Data Structures
ADTs
John Brown 47, Main Street 27 Shift foreman
Sorting
Freda Bloggs 14, City Road 56 Personnel officer
Alg. Theory
Sorting (ct’d) Joe Cool 98, High Street 23 Lathe operator
Binary Trees
Hashing
130/ 550
Data structures represent relationships
DatAlg
Maritta Heisel
Introduction
Alg. Design Each record of the previous figure contains all the
Stepwise refinement information about a particular employee.
Control structures
Imperative paradigm These records may themselves have logical relationships
Algorithm notation
Example between them reflecting the logical relationships between
development
Modularity the people they describe.
Algorithm
specification
Recursion
For example, particular employees (and hence the records
Data Structures
describing them) may be related by working in the same
ADTs
section of the company.
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
131/ 550
Example: supervisor relationship
DatAlg
Maritta Heisel
Introduction
Alg. Design
Syntax and semantics Jane Smith ............
Stepwise refinement
Control structures
Imperative paradigm
Algorithm notation
Example Fred Jones ............ John Brown ..........
development
Modularity
Algorithm
specification
Recursion
Joe Cool ............ Mary Green ............. Sam White .............. Bill Bowes ............. Kate Miles .............
Data Structures
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
132/ 550
Definition of data structures
DatAlg
Maritta Heisel
Introduction
Alg. Design Data that are organized to reflect the logical relationships
Stepwise refinement
between their elements are said to be structured.
Control structures
Imperative paradigm The data elements and their interrelationships together
Algorithm notation
Example
form a data structure.
development
Modularity
Algorithm
specification
Recursion
Data Structures We will define data structures in a mathematical way that is
ADTs independent from specific programming languages, namely as
Sorting abstract data types.
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
133/ 550
Concept of ADTs
DatAlg
Maritta Heisel
Introduction Specify data structures independently of their later

Alg. Design implementation in a concrete programming language
ADTs
Basics
Abstract from unnecessary details
Container types
Yield re-usable modules
Sorting
Alg. Theory
Clients rely only on the interface of the ADT (i.e., the
Sorting (ct’d) corresponding contract)
Binary Trees The implementation can be changed without effecting the
Hashing clients
Graphs
Can be implemented in (object-oriented) programming
Backtracking
Dynamic
languages
programming
Greedy
Blockchain
134/ 550
Learning goals
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
At the end of this course, you should be able to
Basics
Container types
Sorting
Specify ADTs
Alg. Theory Implement ADTs
Sorting (ct’d)
Develop, implement, and analyze algorithms working on
Binary Trees
ADTs
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
135/ 550
Specifications of ADTs
DatAlg
Type Name of the ADT, possibly including param-
Maritta Heisel eters
Introduction Functions Signature of functions defined on the ADT.
Alg. Design For each function:
ADTs Name
Basics
Container types
Types and order of input parameters
Sorting Result type
Alg. Theory Total function: →
Sorting (ct’d) Partial function: →
7
Binary Trees
Preconditions For each partial function: condition that
Hashing
specifies when the function is defined, ex-
Graphs
pressed as a formula
Backtracking
Axioms Describe properties of the introduced func-
Dynamic
programming tions, expressed as formulas. An implemen-
Greedy tation must guarantee that the axioms are
Blockchain
136/ 550
fulfilled.
Example of a simple ADT I
DatAlg
Maritta Heisel
Type Point
Introduction
Alg. Design
Functions
ADTs
Basics create : Real × Real → Point
Container types
Sorting get x : Point → Real

Alg. Theory get y : Point → Real
Sorting (ct’d)
Binary Trees
is origin : Point → Bool
Hashing translate : Point × Real × Real → Point
Graphs
scale : Point × Real → Point
Backtracking
Dynamic distance : Point × Point → Real

programming
Greedy
Blockchain
137/ 550
Example of a simple ADT II
DatAlg
Preconditions —
Maritta Heisel
Axioms
Introduction
Alg. Design
∀ x, y , a, b, z, w : Real •
ADTs get x(create(x, y )) = x
Basics
Container types
get y (create(x, y )) = y
Sorting
Alg. Theory is origin(create(x, y )) = true ⇔ x = 0 ∧ y = 0

Sorting (ct’d)
translate(create(x, y ), a, b) = create(x + a, y + b)
Binary Trees
Hashing
scale(create(x, y ), a) = create(x ∗ a, y ∗ a)
Graphs distance(create(x, y ), create(z, w )) =
Backtracking p
((x − z)2 + (y − w )2 )
Dynamic
programming
Greedy (New line means “∧”, i.e. all conditions must hold)
Blockchain
138/ 550
Difference to algorithm specifications
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs ADTs have no state, they just contain data elements and
Basics
Container types
the functions defined on them
Sorting Axioms do not only give pre- and postconditions for single
Alg. Theory functions, but relate different functions
Sorting (ct’d)
Implementations of ADTs may contain an internal state
Binary Trees
(private attributes/ variables)
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
139/ 550
Different kinds of ADT functions I
DatAlg
Maritta Heisel
Introduction Constructor functions

Alg. Design
ADTs Used to construct the elements of the ADT

Basics
Container types Each element of an ADT can be constructed using only
Sorting constructor functions
Alg. Theory
Often, constructor functions have no associated axioms
Sorting (ct’d)
Binary Trees
(then, different constructor terms denote different
Hashing
elements of the ADT)
Graphs Create is a (the only) constructor function of the ADT
Backtracking Point
Dynamic
programming
Greedy
Blockchain
140/ 550
Different kinds of ADT functions II
DatAlg
Maritta Heisel
Introduction Selector functions

Alg. Design
Are “inverse” functions of constructor functions
ADTs
Basics Decompose ADT elements into their constituting parts
Container types
Sorting For Point: get x and get y

Alg. Theory
Sorting (ct’d)
Predicates
Binary Trees Have Bool as result type
Hashing
For Point: is origin
Graphs
Backtracking
The rest of the ADT functions are “other functions”.
Dynamic
programming
Greedy
Blockchain
141/ 550
ADT: natural numbers I
DatAlg
Maritta Heisel
Type Nat
Introduction Functions
Alg. Design
ADTs zero : Nat constructor

Basics
Container types succ : Nat → Nat constructor
Sorting
pred : Nat →
7 Nat selector
Alg. Theory
Sorting (ct’d) less : Nat × Nat → Bool predicate

Binary Trees
add : Nat × Nat → Nat
Hashing
Graphs mult : Nat × Nat → Nat

Backtracking
Dynamic Preconditions
programming
Greedy ∀ i : Nat • pre(pred(i)) ⇔ less(zero, i)

Blockchain
142/ 550
ADT: natural numbers II
DatAlg
Maritta Heisel Axioms

Introduction
∀ i, j : Nat •
Alg. Design
ADTs
pred(succ(i)) = i
Basics
Container types less(zero, succ(i)) = true
Sorting
less(i, zero) = false
Alg. Theory
Sorting (ct’d) less(succ(i), succ(j)) = less(i, j)

Binary Trees add(zero, i) = i
Hashing
Graphs
add(succ(i), j) = succ(add(i, j))
Backtracking mult(zero, i) = zero
Dynamic
programming mult(succ(i), j) = add(j, mult(i, j))
Greedy
Blockchain
143/ 550
Heuristics for setting up axioms
DatAlg
Maritta Heisel
Introduction
Case distinctions for all functions:
Alg. Design
According to constructor functions
ADTs
Basics
Base case: non-recursive constructor functions (e.g., zero)
Container types Recursive cases: recursive constructor functions
Sorting (e.g., succ(i))
Alg. Theory
Sorting (ct’d) For predicates:

Binary Trees Define base case where predicate is true
Hashing Define base case where predicate is false
Graphs Define recursive cases
Backtracking
Are usually total
Dynamic
programming
Greedy
Blockchain
144/ 550
Container types
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Frequently used kind of ADT
Basics
Container types
Are used to store other data
Sorting Examples: arrays, lists, sets, trees, etc.
Alg. Theory
Behavior independent of type of stored data
Sorting (ct’d)
Binary Trees
Hence, stored data are parameters for container types
Hashing Generic data types, type generators
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
145/ 550
Container type array I
DatAlg
Maritta Heisel
Type ARRAY [T ]
Introduction
Alg. Design
Functions
ADTs
Basics
Container types create : Integer × Integer → ARRAY [T ] constructor
Sorting
put : ARRAY [T ] × Integer × T →
7 ARRAY [T ] constructor
Alg. Theory
Sorting (ct’d)
lower : ARRAY [T ] → Integer selector
Binary Trees upper : ARRAY [T ] → Integer selector
Hashing
Graphs
get : ARRAY [T ] × Integer →
7 T selector
Backtracking empty : ARRAY [T ] → Bool predicate
Dynamic
programming
Greedy
Blockchain
146/ 550
Container type array II
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs Preconditions
Basics
Container types
∀ a : ARRAY [T ]; i : Integer ; x : T •
Sorting
Alg. Theory pre(put(a, i, x)) ⇔ lower (a) ≤ i ≤ upper (a)

Sorting (ct’d)
pre(get(a, i)) ⇔ lower (a) ≤ i ≤ upper (a)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
147/ 550
Container type array III
DatAlg

Introduction
∀ i, j, k : Integer ; a : ARRAY [T ]; x : T •
Alg. Design
ADTs
lower (create(i, j)) = i
Basics
Container types lower (put(a, k, x)) = lower (a)
Sorting
upper (create(i, j)) = j
Alg. Theory
Sorting (ct’d) upper (put(a, k, x) = upper (a)

Binary Trees k = j ⇒ get(put(a, k, x), j) = x
Hashing
Graphs
k 6= j ⇒ get(put(a, k, x), j) = get(a, j)
Backtracking empty (create(i, j)) = true
Dynamic
programming empty (put(a, k, x)) = false
Greedy
Blockchain
148/ 550
Points for discussion
DatAlg
Maritta Heisel
What happens if you call create(i, j) with numbers i > j?
Introduction
Alg. Design
ADTs
Basics
Container types
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
149/ 550
DatAlg
Maritta Heisel
Introduction
What happens when get(a, i) is called before put(a, i, x)
Alg. Design
has been called?
ADTs
Basics
Container types
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
149/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design
has been called?
ADTs
Basics What happens when we overwrite an array element?
Container types
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
149/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design
has been called?
ADTs
Container types
get yields the correct value, but we cannot show that
Sorting
Alg. Theory
put(put(a, i, x), i, y ) = put(a, i, y )
Sorting (ct’d)
Binary Trees holds.

Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
149/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design
has been called?
ADTs
Container types
get yields the correct value, but we cannot show that
Sorting
Alg. Theory
put(put(a, i, x), i, y ) = put(a, i, y )
Sorting (ct’d)
Binary Trees holds.

Hashing If we want that equation to hold, we have to add the
Graphs following axioms:
Backtracking i 6= j ⇒ put(put(a, i, x), j, y ) = put(put(a, j, y ), i, x)
Dynamic
programming i = j ⇒ put(put(a, i, x), j, y ) = put(a, j, y )
Greedy
Blockchain
149/ 550
Container type finite sets I
DatAlg
Maritta Heisel
Introduction Type SET [T ]

Alg. Design
ADTs Functions
Basics
Container types
Sorting
mt set : SET [T ] constructor
Alg. Theory add : T × SET [T ] → SET [T ] constructor
Sorting (ct’d)
is in : T × SET [T ] → Bool predicate
Binary Trees
Hashing delete : T × SET [T ] → SET [T ]

Graphs
Backtracking Preconditions —
Dynamic
programming
Greedy
Blockchain
150/ 550
Container type finite sets II
DatAlg

Introduction
∀ s : SET [T ]; x, y : T •
Alg. Design
ADTs
x = y ⇒ add(y , add(x, s)) = add(y , s)
Basics
Container types x 6= y ⇒ add(y , add(x, s)) = add(x, add(y , s))
Sorting
is in(x, mt set) = false
Alg. Theory
Sorting (ct’d) x = y ⇒ is in(x, add(y , s)) = true

Binary Trees x 6= y ⇒ is in(x, add(y , s)) = is in(x, s)
Hashing
Graphs
delete(x, mt set) = mt set
Backtracking x = y ⇒ delete(x, add(y , s)) = delete(x, s)
Dynamic
programming x 6= y ⇒ delete(x, add(y , s)) = add(y , delete(x, s))
Greedy
Blockchain
151/ 550
Container type stack I
DatAlg
Type STACK [T ]
Maritta Heisel
Introduction
Functions
Alg. Design
mt stack : STACK [T ] constructor
ADTs
Basics
Container types
push : T × STACK [T ] → STACK [T ] constructor
Sorting pop : STACK [T ] →
7 STACK [T ] selector
Alg. Theory
top : STACK [T ] →
7 T selector
Sorting (ct’d)
Binary Trees empty : STACK [T ] → Bool predicate

Hashing
Graphs
Preconditions
Backtracking ∀ s : STACK [T ] •
Dynamic
programming pre(pop(s)) ⇔ empty (s) = false
Greedy
pre(top(s)) ⇔ empty (s) = false
Blockchain
152/ 550
Container type stack II
DatAlg
Maritta Heisel
Introduction Axioms
Alg. Design
ADTs
∀ x : T ; s : STACK [T ] •
Basics
Container types pop(push(x, s)) = s
Sorting
top(push(x, s)) = x
Alg. Theory
Sorting (ct’d) empty (mt stack) = true

Binary Trees
empty (push(x, s)) = false
Hashing
Graphs
Backtracking Remark: Such stacks can contain arbitrarily many elements.

Dynamic
programming
Greedy
Blockchain
153/ 550
Container type finite stack I
DatAlg
Maritta Heisel
Type FSTACK [T ]
Alg. Design
ADTs mt stack : Nat → FSTACK [T ] constructor

Basics
Container types push : T × FSTACK [T ] →
7 FSTACK [T ] constructor
Sorting
Alg. Theory
pop : FSTACK [T ] →
7 FSTACK [T ] selector
Sorting (ct’d) top : FSTACK [T ] →
7 T selector
Binary Trees
capacity : FSTACK [T ] → Nat selector
Hashing
Graphs push count : FSTACK [T ] → Nat

Backtracking
empty : FSTACK [T ] → Bool
Dynamic
programming full : FSTACK [T ] → Bool
Greedy
Blockchain
154/ 550
Container type finite stack II
DatAlg
Maritta Heisel
Introduction
Alg. Design
Preconditions
ADTs
Basics
Container types ∀ x : T ; s : FSTACK [T ] •
Sorting
pre(push(x, s)) ⇔ ¬ full(s)
Alg. Theory
Sorting (ct’d) pre(pop(s)) ⇔ ¬ empty (s)

Binary Trees pre(top(s)) ⇔ ¬ empty (s)
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
155/ 550
Container type finite stack III
DatAlg

Introduction
∀ x : T ; s : FSTACK [T ]; k : Nat •
Alg. Design
ADTs
¬ full(s) ⇒ pop(push(x, s)) = s
Basics
Container types ¬ full(s) ⇒ top(push(x, s)) = x
Sorting
capacity (mt stack(k)) = k
Alg. Theory
Sorting (ct’d) capacity (push(x, s)) = capacity (s)

Binary Trees push count(mt stack(k)) = zero
Hashing
Graphs
push count(push(x, s)) = succ(push count(s))
Backtracking empty (s) ⇔ push count(s) = zero
Dynamic
programming full(s) ⇔ push count(s) ≥ capacity (s)
Greedy
Blockchain
156/ 550
Container type priority queue I
DatAlg
Type PQ[T ]
Maritta Heisel
Introduction
Functions
Alg. Design
priority : T → Nat
ADTs
Basics
Container types
new : PQ[T ] constructor
Sorting insert : T × PQ[T ] → PQ[T ] constructor
Alg. Theory
delete min : PQ[T ] →
7 PQ[T ] selector
Sorting (ct’d)
Binary Trees min : PQ[T ] →

7 T selector
Hashing
Graphs
Preconditions
Backtracking ∀ pq : PQ[T ] •
Dynamic
programming pre(delete min(pq)) ⇔ pq 6= new
Greedy
pre(min(pq)) ⇔ pq 6= new
Blockchain
157/ 550
Container type priority queue II
DatAlg
Axioms
Maritta Heisel
∀ x, y : T ; pq : PQ[T ] •
Introduction
Alg. Design
min(insert(x, new )) = x
ADTs delete min(insert(x, new )) = new
Basics
Container types
priority (x) < priority (min(insert(y , pq))) ⇒
Sorting
Alg. Theory (min(insert(x, insert(y , pq))) = x

Sorting (ct’d) ∧ delete min(insert(x, insert(y , pq))) = insert(y , pq))
Binary Trees
Hashing
priority (x) ≥ priority (min(insert(y , pq))) ⇒
Graphs (min(insert(x, insert(y , pq))) = min(insert(y , pq))
Backtracking
∧ delete min(insert(x, insert(y , pq)))
Dynamic
programming
= insert(x, delete min(insert(y , pq))))
Greedy
Blockchain Remark: axioms taken from www.nist.gov/dads/

158/ 550
Container type list I
DatAlg Type LIST [T ]

Maritta Heisel
Functions
Introduction
Alg. Design
nil : LIST [T ]
ADTs cons : T × LIST [T ] → LIST [T ]
Basics
Container types
head : LIST [T ] →
7 T
Sorting
Alg. Theory tail : LIST [T ] →

7 LIST [T ]
Sorting (ct’d)
length : LIST [T ] → Nat
Binary Trees
Hashing
insert : T × LIST [T ] × Nat →
7 LIST [T ]
Graphs append : LIST [T ] × LIST [T ] → LIST [T ]
Backtracking
n th : LIST [T ] × Nat →
7 T
Dynamic
programming
Greedy Q: What are constructors, selectors, predicates?

Blockchain
159/ 550
Container type list II
DatAlg
Maritta Heisel
Introduction
Alg. Design Preconditions

ADTs
Basics ∀ l : LIST [T ]; x : T ; i : Nat •
Container types
Sorting pre(head(l)) ⇔ length(l) > 0

Alg. Theory pre(tail(l) ⇔ length(l) > 0
Sorting (ct’d)
Binary Trees
pre(n th(l, i)) ⇔ 1 ≤ i ≤ length(l)
Hashing pre(insert(x, l, i)) ⇔ 1 ≤ i ≤ length(l) + 1
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
160/ 550
Container type list III
DatAlg
Axioms
Maritta Heisel
∀ l, l1 : LIST [T ]; x, y : T ; i : Nat •
Introduction
head(cons(x, l)) = x
Alg. Design
ADTs
tail(cons(x, l)) = l
Basics
Container types length(nil) = 0
Sorting
length(cons(x, l)) = length(l) + 1
Alg. Theory
Sorting (ct’d) insert(x, l, 1) = cons(x, l)

Binary Trees
i > 0 ⇒ insert(x, cons(y , l), i + 1) = cons(y , insert(x, l, i))
Hashing
Graphs
append(nil, l) = l
Backtracking append(cons(x, l1), l) = cons(x, append((l1, l)))
Dynamic
programming n th(cons(x, l), 1) = x
Greedy
i > 0 ⇒ n th(cons(y , cons(x, l)), i + 1) = n th(cons(x, l), i)
Blockchain
161/ 550
Container type list IV
DatAlg
Points for discussion:
Maritta Heisel
Why don’t we have an equation
Introduction
insert(x, nil, i + 1) = . . . ?
Alg. Design
ADTs
Could we define append by recursion on its second
Basics argument?
Container types
Sorting Remarks
Alg. Theory In the exercises, we will define more functions on lists, e.g.
Sorting (ct’d)
delete, search, reverse, change value, last.
Binary Trees
Hashing
In an implementation, if we first want to search for an
Graphs
element in a list and then delete that element, the list
Backtracking
must be traversed twice.
Dynamic To eliminate that inefficiency, we introduce cursor lists,
programming
where a cursor points to the element that was accessed
Greedy
most recently.
Blockchain
162/ 550
Container type cursor list
DatAlg
Maritta Heisel
Introduction
Alg. Design offleft offright
1111
0000 1111
0000 1111
0000 00000
11111 00000
11111
000011111
00000
ADTs
0000
1111 0000
1111 1111 00000 11111
11111 00000
00000 11111
00000
Basics
0000
1111 0000
1111 0000
1111 00000
11111 11111 00000
11111
Container types
value
Sorting
Alg. Theory
0000
1111 0000
1111 00000 11111
000011111
1111 00000 11111
00000
Sorting (ct’d)
head back forth
Binary Trees
Hashing cursor
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
163/ 550
DatAlg
Maritta Heisel Type CURSOR LIST [T ]

Introduction
Alg. Design
Functions
ADTs nil : CURSOR LIST [T ]
Basics
Container types
cons : T × CURSOR LIST [T ] → CURSOR LIST [T ]
Sorting
Alg. Theory go : CURSOR LIST [T ] × Nat →

7 CURSOR LIST [T ]
Sorting (ct’d)
head : CURSOR LIST [T ] →
7 T
Binary Trees
Hashing
tail : CURSOR LIST [T ] →
7 CURSOR LIST [T ]
Graphs cursor : CURSOR LIST [T ] → Nat
Backtracking
empty : CURSOR LIST [T ] → Bool
Dynamic
programming
length : CURSOR LIST [T ] → Nat
Greedy
Blockchain
164/ 550
DatAlg
Maritta Heisel
Introduction back : CURSOR LIST [T ] →

7 CURSOR LIST [T ]
Alg. Design
forth : CURSOR LIST [T ] →
7 CURSOR LIST [T ]
ADTs
Basics
Container types
offleft : CURSOR LIST [T ] → Bool
Sorting offright : CURSOR LIST [T ] → Bool
Alg. Theory
value : CURSOR LIST [T ] →
7 T
Sorting (ct’d)
Binary Trees insert right : T × CURSOR LIST [T ] →

7 CURSOR LIST [T ]
Hashing
insert left : T × CURSOR LIST [T ] →
7 CURSOR LIST [T ]
Graphs
Backtracking delete : CURSOR LIST [T ] →

7 CURSOR LIST [T ]
Dynamic
programming
change value : T × CURSOR LIST [T ] →
7 CURSOR LIST [T ]
Greedy search : T × CURSOR LIST [T ] → CURSOR LIST [T ]
Blockchain
165/ 550
DatAlg
Maritta Heisel
Hidden Functions
Introduction
Alg. Design
ADTs n th : CURSOR LIST [T ] × Nat →

7 T
Basics
Container types
hidden insert : T × CURSOR LIST [T ] × Nat
Sorting
Alg. Theory →CURSOR

7 LIST [T ]
Sorting (ct’d)
hidden delete : CURSOR LIST [T ] × Nat →
7 CURSOR LIST [T ]
Binary Trees
Hashing hidden change value : T × CURSOR LIST [T ] × Nat

Graphs
Backtracking
→CURSOR
7 LIST [T ]
Dynamic
programming
hidden search : T × CURSOR LIST [T ] → Nat
Greedy
Blockchain
166/ 550
DatAlg
Maritta Heisel Preconditions

Introduction
Alg. Design ∀ l : CURSOR LIST [T ]; x : T ; i : Nat •

ADTs
Basics pre(head(l)) ⇔ ¬ empty (l)
Container types
Sorting pre(tail(l)) ⇔ ¬ empty (l)

Alg. Theory
pre(go(l, i)) ⇔ 0 ≤ i ≤ length(l) + 1
Sorting (ct’d)
Binary Trees
pre(back(l)) ⇔ ¬ offleft(l)
Hashing pre(forth(l)) ⇔ ¬ offright(l)
Graphs
Backtracking
pre(value(l)) ⇔ ¬ offleft(l) ∧ ¬ offright(l)
Dynamic pre(insert right(l)) ⇔ empty (l) ∨ ¬ offright(l)
programming
Greedy pre(insert left(l)) ⇔ ¬ offleft(l)

Blockchain
167/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs pre(delete(l)) ⇔ ¬ offleft(l) ∧ ¬ offright(l)

Basics
Container types pre(change value(x, l)) ⇔ ¬ offleft(l) ∧ ¬ offright(l)
Sorting
Alg. Theory
pre(n th(l, i)) ⇔ 1 ≤ i ≤ length(l)
Sorting (ct’d)
Binary Trees pre(hidden insert(x, i, l)) ⇔ 1 ≤ i ≤ length(l) + 1

Hashing
pre(hidden delete(l, i)) ⇔ 1 ≤ i ≤ length(l)
Graphs
Backtracking pre(hidden change value(x, l, i)) ⇔ 1 ≤ i ≤ length(l)

Dynamic
programming
Greedy
Blockchain
168/ 550
DatAlg

Introduction ∀ l : CURSOR LIST [T ]; x, y : T ; i, k : Nat •
Alg. Design
ADTs
Basics head(go(l, i)) = head(l)
Container types
Sorting tail(cons(x, l)) = l

Alg. Theory i > 0 ⇒ tail(go(l, i)) = go(tail(l), i − 1)
Sorting (ct’d)
tail(go(l, 0)) = go(tail(l), 0)
Binary Trees
Hashing
cursor (nil) = 0
Graphs offleft(l) ⇒ cursor (cons(x, l)) = cursor (l)
Backtracking
¬offleft(l) ⇒ cursor (cons(x, l)) = cursor (l) + 1
Dynamic
programming cursor (go(l, i)) = i
Greedy
Blockchain
169/ 550
DatAlg

Alg. Design
ADTs
Container types

Sorting (ct’d)
Binary Trees
Hashing
cursor (nil) = 0
Backtracking
Dynamic
Greedy
Blockchain
169/ 550
DatAlg

Alg. Design
ADTs
Container types

Sorting (ct’d)
Binary Trees
Hashing
cursor (nil) = 0
Backtracking
Dynamic
Greedy
Blockchain
169/ 550
DatAlg

Alg. Design
ADTs
Container types

Sorting (ct’d)
Binary Trees
Hashing
cursor (nil) = 0
Backtracking
Dynamic
Greedy
Blockchain
169/ 550
DatAlg

Alg. Design
ADTs
Container types

Sorting (ct’d)
Binary Trees
Hashing
cursor (nil) = 0
Backtracking
Dynamic
Greedy
Blockchain
169/ 550
DatAlg

Alg. Design
ADTs
Container types

Sorting (ct’d)
Binary Trees
Hashing
cursor (nil) = 0
Backtracking
Dynamic
Greedy
Blockchain
169/ 550
DatAlg

Alg. Design
ADTs
Container types

Sorting (ct’d)
Binary Trees
Hashing
cursor (nil) = 0
Backtracking
Dynamic
Greedy
Blockchain
169/ 550
DatAlg

Alg. Design
ADTs
Container types

Sorting (ct’d)
Binary Trees
Hashing
cursor (nil) = 0
Backtracking
Dynamic
Greedy
Blockchain
169/ 550
DatAlg

Alg. Design
ADTs
Container types

Sorting (ct’d)
Binary Trees
Hashing
cursor (nil) = 0
Backtracking
Dynamic
Greedy
Blockchain
169/ 550
DatAlg

Alg. Design
ADTs
Container types

Sorting (ct’d)
Binary Trees
Hashing
cursor (nil) = 0
Backtracking
Dynamic
Greedy
Blockchain
169/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design
empty (l) ⇔ length(l) = 0
ADTs length(nil) = 0
Basics
Container types length(cons(x, l)) = length(l) + 1
Sorting
length(go(l, i)) = length(l)
Alg. Theory
Sorting (ct’d)
back(l) = go(l, cursor (l) − 1)
Binary Trees forth(l) = go(l, cursor (l) + 1)
Hashing
offleft(l) ⇔ cursor (l) = 0
Graphs
Backtracking
offright(l) ⇔ empty (l) ∨ cursor (l) = length(l) + 1
Dynamic value(l) = n th(l, cursor (l))
programming
Greedy
Blockchain
170/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design
Basics
Sorting
Alg. Theory
Sorting (ct’d)
Hashing
Graphs
Backtracking
programming
Greedy
Blockchain
170/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design
Basics
Sorting
Alg. Theory
Sorting (ct’d)
Hashing
Graphs
Backtracking
programming
Greedy
Blockchain
170/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design
Basics
Sorting
Alg. Theory
Sorting (ct’d)
Hashing
Graphs
Backtracking
programming
Greedy
Blockchain
170/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design
Basics
Sorting
Alg. Theory
Sorting (ct’d)
Hashing
Graphs
Backtracking
programming
Greedy
Blockchain
170/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design
Basics
Sorting
Alg. Theory
Sorting (ct’d)
Hashing
Graphs
Backtracking
programming
Greedy
Blockchain
170/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design
Basics
Sorting
Alg. Theory
Sorting (ct’d)
Hashing
Graphs
Backtracking
programming
Greedy
Blockchain
170/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design
Basics
Sorting
Alg. Theory
Sorting (ct’d)
Hashing
Graphs
Backtracking
programming
Greedy
Blockchain
170/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design
Basics
Sorting
Alg. Theory
Sorting (ct’d)
Hashing
Graphs
Backtracking
programming
Greedy
Blockchain
170/ 550
DatAlg
Maritta Heisel
Introduction insert right(x, l) =

Alg. Design go(hidden insert(x, l, cursor (l) + 1), cursor (l))
ADTs
Basics
insert left(x, l) =
Container types go(hidden insert(x, l, cursor (l)), cursor (l) + 1)
Sorting
length(l) > 1 ⇒
Alg. Theory
delete(l) = go(hidden delete(l, cursor (l)), cursor (l))
Sorting (ct’d)
Binary Trees length(l) = 1 ⇒ delete(l) = nil

Hashing change value(x, l) =
Graphs
go(hidden change value(x, l, cursor (l)), cursor (l))
Backtracking
search(x, l) = go(l, hidden search(x, l))
Dynamic
programming
Greedy
Blockchain
171/ 550
DatAlg
Maritta Heisel

ADTs
Basics
insert left(x, l) =
Sorting
length(l) > 1 ⇒
Alg. Theory
Sorting (ct’d)

Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
171/ 550
DatAlg
Maritta Heisel

ADTs
Basics
insert left(x, l) =
Sorting
length(l) > 1 ⇒
Alg. Theory
Sorting (ct’d)

Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
171/ 550
DatAlg
Maritta Heisel

ADTs
Basics
insert left(x, l) =
Sorting
length(l) > 1 ⇒
Alg. Theory
Sorting (ct’d)

Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
171/ 550
DatAlg
Maritta Heisel

ADTs
Basics
insert left(x, l) =
Sorting
length(l) > 1 ⇒
Alg. Theory
Sorting (ct’d)

Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
171/ 550
DatAlg
Maritta Heisel

ADTs
Basics
insert left(x, l) =
Sorting
length(l) > 1 ⇒
Alg. Theory
Sorting (ct’d)

Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
171/ 550
DatAlg
Maritta Heisel
n th(cons(x, l), 1) = x
Introduction
i > 0 ⇒ n th(cons(x, l), i + 1) = n th(l, i)
Alg. Design
ADTs n th(go(l, i), k) = n th(l, k)

Basics
Container types hidden insert(x, l, 1) = cons(x, l)
Sorting i > 1 ⇒ hidden insert(x, cons(y , l), i)
Alg. Theory
= cons(y , hidden insert(x, l, i − 1))
Sorting (ct’d)
Binary Trees
hidden insert(x, go(l, k), i) = hidden insert(x, l, i)
Hashing hidden delete(cons(x, l), 1) = l
Graphs
i > 1 ⇒ hidden delete(cons(x, l), i)
Backtracking
= cons(x, hidden delete(l, i − 1))
Dynamic
programming hidden delete(go(l, k), i) = hidden delete(l, i)
Greedy
Blockchain
172/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design

Basics
Alg. Theory
Sorting (ct’d)
Binary Trees
Graphs
Backtracking
Dynamic
Greedy
Blockchain
172/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design

Basics
Alg. Theory
Sorting (ct’d)
Binary Trees
Graphs
Backtracking
Dynamic
Greedy
Blockchain
172/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design

Basics
Alg. Theory
Sorting (ct’d)
Binary Trees
Graphs
Backtracking
Dynamic
Greedy
Blockchain
172/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design

Basics
Alg. Theory
Sorting (ct’d)
Binary Trees
Graphs
Backtracking
Dynamic
Greedy
Blockchain
172/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design

Basics
Alg. Theory
Sorting (ct’d)
Binary Trees
Graphs
Backtracking
Dynamic
Greedy
Blockchain
172/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design

Basics
Alg. Theory
Sorting (ct’d)
Binary Trees
Graphs
Backtracking
Dynamic
Greedy
Blockchain
172/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design

Basics
Alg. Theory
Sorting (ct’d)
Binary Trees
Graphs
Backtracking
Dynamic
Greedy
Blockchain
172/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design

Basics
Alg. Theory
Sorting (ct’d)
Binary Trees
Graphs
Backtracking
Dynamic
Greedy
Blockchain
172/ 550
DatAlg
Maritta Heisel
Introduction hidden change value(y , cons(x, l), 1) = cons(y , l)

Alg. Design i > 1 ⇒ hidden change value(y , cons(x, l), i)
ADTs = cons(x, hidden change value(y , l, i − 1))
Basics
Container types
hidden change value(y , go(l, k), i) =
Sorting
hidden change value(y , l, i)
Alg. Theory
Sorting (ct’d)
hidden search(y , nil) = 1
Binary Trees x = y ⇒ hidden search(y , cons(x, l)) = 1
Hashing
x 6= y ⇒ hidden search(y , cons(x, l)) =
Graphs
1 + hidden search(y , l)
Backtracking
Dynamic
hidden search(x, go(l, i)) = hidden search(x, l))
programming
Greedy
Blockchain
173/ 550
DatAlg
Maritta Heisel

Basics
Container types
Sorting
Alg. Theory
Sorting (ct’d)
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
173/ 550
DatAlg
Maritta Heisel

Basics
Container types
Sorting
Alg. Theory
Sorting (ct’d)
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
173/ 550
DatAlg
Maritta Heisel

Basics
Container types
Sorting
Alg. Theory
Sorting (ct’d)
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
173/ 550
DatAlg
Maritta Heisel

Basics
Container types
Sorting
Alg. Theory
Sorting (ct’d)
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
173/ 550
DatAlg
Maritta Heisel

Basics
Container types
Sorting
Alg. Theory
Sorting (ct’d)
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
173/ 550
DatAlg
Maritta Heisel

Basics
Container types
Sorting
Alg. Theory
Sorting (ct’d)
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
173/ 550
What have we learnt on abstract data types?
DatAlg
Maritta Heisel
Introduction
You should now be able to
Alg. Design
ADTs Explain what an abstract data type is

Basics
Container types Give examples of ADTs and their characteristic properties
Sorting
Define ADTs giving function signatures, preconditions,
Alg. Theory
and axioms
Sorting (ct’d)
Binary Trees
Explain the roles of constructors, selectors, and predicates
Hashing Implement ADTs in an object-oriented programming
Graphs language
Backtracking
Dynamic
programming
Greedy
Blockchain
174/ 550
Sorting algorithms
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Work on a linear data structure (arrays)
Sorting
Bubble sort
(Later, we will consider non-linear data structures and
Insertion sort
Selection sort
algorithms working on them)
Alg. Theory Are very important for many applications
Sorting (ct’d)
Exist in several versions, distinguished by their complexity
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
175/ 550
Blockchain
Algorithm schema for processing linear data
structures
DatAlg
Maritta Heisel
Introduction
Alg. Design
start at the beginning of the data structure;
ADTs
while end of of data structure not reached do
Sorting
Bubble sort
process next item
Insertion sort
Selection sort
endwhile
Alg. Theory
Sorting (ct’d) If the data structure is known to contain at least one item, the
Binary Trees while loop can be replaced by a repeat loop.
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
176/ 550
Blockchain
Concrete syntax for arrays
DatAlg
Maritta Heisel
Introduction
Alg. Design
concrete abstract
ADTs
Sorting
Bubble sort
a : ARRAY (i..j)[T ] a : ARRAY [T ]
Insertion sort
Selection sort
a:= create(i,j)
Alg. Theory
Sorting (ct’d)
a[i] get(a, i)
Binary Trees
a[i] := x a := put(a, i, x)
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
177/ 550
Blockchain
Specification of sorting
DatAlg
Maritta Heisel
Introduction Let a be an array with lower index 1 and upper index n.

Alg. Design
ADTs
var n : Integer ; a, a0 : ARRAY (1..n)[T ]
Sorting
pre n ≥ 1 ∧ a = a0
Bubble sort
Insertion sort
post ordered(a) ∧ permutation(a, a0 )
Selection sort reads n
Alg. Theory changes a
Sorting (ct’d)
mem a0
Binary Trees
Hashing In general, we can sort all data for which a total ordering
Graphs relation such as ≤ or < is defined.
Backtracking
Dynamic
programming
Greedy
178/ 550
Blockchain
Ordering relations
DatAlg
Maritta Heisel An ordering relation ≤ is a binary relation on some type T that

Introduction fulfills the following properties:
Alg. Design
ADTs ∀ x, y , z : T •
Sorting
Bubble sort
reflexivity x ≤x
Insertion sort
Selection sort antisymmetry x ≤ y ∧ y ≤ x ⇒ x = y
Alg. Theory
transitivity x ≤y ∧y ≤z ⇒x ≤z
Sorting (ct’d)
Binary Trees
An ordering relation is total if the following holds:
Hashing
Graphs totality x ≤y ∨y ≤x
Backtracking
Dynamic
programming Strict order: x < y ⇔ x ≤ y ∧ ¬ y ≤ x
Greedy
179/ 550
Blockchain
Predicates ordered and permutation
DatAlg
Maritta Heisel
∀ a : ARRAY [T ] •
Introduction
Alg. Design ordered(a) ⇔

ADTs
∀ i : Integer • lower (a) ≤ i < upper (a) ⇒ a[i] ≤ a[i + 1]
Sorting
Bubble sort
Insertion sort
Selection sort
Alg. Theory
∀ a, b : ARRAY (i..j)[T ] •
Sorting (ct’d)
permutation(a, b) ⇔
Binary Trees
Hashing ∀ k : Integer • i ≤ k ≤ j ⇒
Graphs
#{h : Integer | i ≤ h ≤ j ∧ a[k] = a[h]}
Backtracking
Dynamic
= #{h : Integer | i ≤ h ≤ j ∧ a[k] = b[h]}
programming
Greedy
180/ 550
Blockchain
Sorting: commitments and advantages
DatAlg
Maritta Heisel Commitments Advantages

Introduction
Caller Call sorting algorithm Get a sorted array
Alg. Design
with a nonempty ar-
ADTs ray a, instatiated with
Sorting some type for which a
Bubble sort
Insertion sort
total ordering relation
Selection sort
is defined
Alg. Theory
Must keep precondition Has advantages from post-
Sorting (ct’d)
condition
Binary Trees
Module Make sure that the in- Unnecessary to treat the
Hashing
Graphs
put array is sorted cor- case where the input is not
Backtracking
rectly a nonempty array or no to-
Dynamic
tal ordering relation is given
programming
Must guarantee post- Can assume precondition
Greedy
condition
181/ 550
Blockchain
Bubble sort
DatAlg
Maritta Heisel
Introduction
Task (for example):
Alg. Design
Sort names into an alphabetical order
ADTs
Sorting
Bubble sort
Basic idea:
Insertion sort
Selection sort
Pass through the array of names, compare each name with its
Alg. Theory successor; interchange them if they are out of order;
Sorting (ct’d) if at the end of the pass all the names are in the correct order,
Binary Trees we are done;
Hashing if not, another pass is made through the array; the passes must
Graphs be repeated as long as any names are out of order.
Backtracking
Dynamic
programming
Greedy
182/ 550
Blockchain
Actions of bubble sort
DatAlg
Maritta Heisel
original list after pass 1 after pass 2 after pass 3
Introduction
4 exch. 3 exch. 2 exch.
Alg. Design
ADTs
John John Fred Bill
Sorting
Kate Fred Bill Fred
Bubble sort
Insertion sort
Fred Bill John Jill
Selection sort Bill Kate Jill John
Alg. Theory Sam Jill Kate Kate
Sorting (ct’d)
Jill Mary Mary Mary
Binary Trees
Mary Sam Sam Sam
Hashing
Graphs
Names bubble upward or sink downward to their correct
Backtracking
position in the list.
Dynamic
programming
Greedy
183/ 550
Blockchain
Bubble sort – Initial version
DatAlg
Maritta Heisel
Introduction
We use a pre-tested loop since the processor may be fortunate
Alg. Design
enough to find that the array is already in order and there is
ADTs
nothing to do:
Sorting
Bubble sort
Insertion sort
Selection sort module bubblesort(a : ARRAY(1..n)[T])
Alg. Theory while a is not ordered do
Sorting (ct’d) make a pass through a, exchanging
Binary Trees adjacent names as necessary
Hashing endwhile
Graphs
endmodule
Backtracking
Dynamic
programming
Greedy
184/ 550
Blockchain
Bubble sort – First refinement step
DatAlg
Maritta Heisel
The body of the while-loop can be refined into:
Introduction
var i : Integer ; auxvar : T ;
Alg. Design
if n > 1 then
ADTs
i := 1;
Sorting
Bubble sort repeat n - 1 times
Insertion sort
Selection sort if a[i] > a[i + 1] then
Alg. Theory auxvar := a[i];
Sorting (ct’d) a[i] := a[i + 1];
Binary Trees a[i + 1] := auxvar
Hashing endif;
Graphs
i := i + 1
Backtracking
endrepeat
Dynamic
programming endif
Greedy
185/ 550
Blockchain
Bubble sort – second refinement step I
DatAlg
Maritta Heisel
Introduction Since the processor is incapable of telling whether or not the

Alg. Design array is ordered, the terminating condition in line 1 needs
ADTs further refinement.
Sorting
Bubble sort
Insertion sort One way of telling whether the array is ordered is to remember
Selection sort
whether any names needed exchanging during the last pass
Alg. Theory
through it.
Sorting (ct’d)
Binary Trees
At least one pass through the array is required to see whether
Hashing
it is ordered to start with. Thus, we should replace the
Graphs
pre-tested loop by a post-tested loop.
Backtracking
Dynamic
programming
Greedy
186/ 550
Blockchain
Bubble sort – second refinement step II
DatAlg var i : Integer ; auxvar : T ; exchanged : Bool;

Maritta Heisel repeat
exchanged := false;
Introduction
Alg. Design
if n > 1 then
ADTs
i := 1;
Sorting repeat n - 1 times
Bubble sort
Insertion sort
if a[i] > a[i + 1] then
Selection sort auxvar := a[i];
Alg. Theory
a[i] := a[i + 1];
Sorting (ct’d)
a[i + 1] := auxvar;
Binary Trees
exchanged := true
Hashing
endif;
Graphs
i := i + 1
Backtracking
Dynamic
endrepeat
programming endif
Greedy until exchanged = false endrepeat
187/ 550
Blockchain
Complexity
DatAlg
Maritta Heisel Bubble sort is a particularly simple sorting algorithm, but

Introduction
it is not always the fastest
Alg. Design Best case: array is (almost) ordered
ADTs
Worst case: last name in array is alphabetically first; then
Sorting
Bubble sort
n − 1 passes are needed
Insertion sort
Selection sort Average: number of passes roughly proportional to n
Alg. Theory
Since each pass involves n − 1 executions of the inner
Sorting (ct’d)
loop, the average number of steps executed is roughly
Binary Trees
proportional to n2
Hashing
Graphs
The number of steps involved in the execution of the
Backtracking algorithm, and hence the time needed to execute it, are
Dynamic attributes of the algorithm which fall under the general
programming
term complexity
Greedy
188/ 550
Blockchain
Insertion sort
DatAlg
Maritta Heisel
Introduction
Alg. Design Basic idea:

ADTs
Sorting
1. An array of length 1 is ordered
Bubble sort
Insertion sort
2. If we have sorted the first i elements of the array, we
Selection sort
obtain an ordered (partial) array of length i + 1 by
Alg. Theory
inserting a[i + 1] at the appropriate position
Sorting (ct’d)
j, 1 ≤ j ≤ i + 1 and by shifting all elements
Binary Trees
a[k], j ≤ k ≤ i one position to the right.
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
189/ 550
Blockchain
Example for insertion sort
DatAlg
Maritta Heisel
Introduction
The first i elements of the array are ordered.
Alg. Design
ADTs i=1 44 55 12 42 6 66 41
Sorting
Bubble sort
i=2 44 55 12 42 6 66 41
Insertion sort
Selection sort i=3 12 44 55 42 6 66 41
Alg. Theory
i=4 12 42 44 55 6 66 41
Sorting (ct’d)
Binary Trees i=5 6 12 42 44 55 66 41

Hashing
i=6 6 12 42 44 55 66 41
Graphs
Backtracking i=7 6 12 41 42 44 55 66
Dynamic
programming
Greedy
190/ 550
Blockchain
Algorithm skeleton for insertion sort
DatAlg
Maritta Heisel
Introduction
Alg. Design
var i : Integer ; a : ARRAY (1..n)[T ];
ADTs
Sorting
i := 1;
Bubble sort
Insertion sort while i 6= n do
Selection sort
Alg. Theory i := i + 1;
Sorting (ct’d) “insert a[i] at the appropriate position”
Binary Trees
Hashing
endwhile
Graphs
Backtracking
Dynamic
programming
Greedy
191/ 550
Blockchain
Inserting a[i]: specification
DatAlg
Maritta Heisel
Introduction
Alg. Design
var n, i : Integer ; a, a00 : ARRAY (1..n)[T ]
ADTs
pre n ≥ 1 ∧ a = a00 ∧ 1 < i ≤ n ∧
Sorting ordered(a[1..i − 1])
Bubble sort
Insertion sort
post ordered(a[1..i]) ∧
Selection sort permutation(a[1..i], a00 [1..i]) ∧
Alg. Theory
a[i + 1..n] = a00 [i + 1..n]
Sorting (ct’d)
reads n, i
Binary Trees
changes a
Hashing
mem a00
Graphs
Backtracking
Dynamic
programming
Greedy
192/ 550
Blockchain
Inserting a[i]: algorithm
DatAlg Now the idea is to move a[i] to the left until it is in the
Maritta Heisel appropriate position.
Introduction
var i, j : Integer ; x : T ; a : ARRAY (1..n)[T ];
Alg. Design
ADTs j := i;
Sorting
Bubble sort x := a[i];
Insertion sort
Selection sort
while j > 1 ∧ x < a[j − 1] do
Alg. Theory
Sorting (ct’d)
a[j] := a[j − 1];
Binary Trees j := j − 1
Hashing
Graphs
endwhile;
Backtracking a[j] := x
Dynamic
programming
Greedy
193/ 550
Blockchain
Inserting a[i]: algorithm
DatAlg Now the idea is to move a[i] to the left until it is in the
Maritta Heisel appropriate position.
Introduction
var i, j : Integer ; x : T ; a : ARRAY (1..n)[T ];
Alg. Design
ADTs j := i;
Sorting
Bubble sort x := a[i];
Insertion sort
Selection sort
while j > 1 ∧ x < a[j − 1] do
Alg. Theory
Sorting (ct’d)
a[j] := a[j − 1];
Binary Trees j := j − 1
Hashing
Graphs
endwhile;
Backtracking a[j] := x
Dynamic
programming
Greedy Question: What is wrong with this algorithm?

193/ 550
Blockchain
“Safe” connectives
DatAlg
Maritta Heisel The connectives “∧” and “∨” are commutative, i.e., their
Introduction meaning is independent of the order of their arguments.
Alg. Design
ADTs
Sorting
Bubble sort
Insertion sort
Selection sort
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
194/ 550
Blockchain
DatAlg
Alg. Design We cannot assume any order in which the arguments are
ADTs evaluated.
Sorting
Bubble sort
Insertion sort
Selection sort
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
194/ 550
Blockchain
DatAlg
ADTs evaluated.
Sorting
Bubble sort When the processor evaluates j > 1 ∧ x < a[j − 1] for
Insertion sort
Selection sort
j = 1, then it is possible that it tries to evaluate x < a[0],
Alg. Theory which is not defined!
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
194/ 550
Blockchain
DatAlg
ADTs evaluated.
Sorting
Insertion sort
Selection sort
Sorting (ct’d)
Hence, we must make sure that before evaluating
Binary Trees
x < a[j − 1], the processor evaluates j > 1. If this
Hashing
condition is false, it should not evaluate x < a[j − 1], but
Graphs
give the whole conjunction the value false.
Backtracking
Dynamic
programming
Greedy
194/ 550
Blockchain
DatAlg
ADTs evaluated.
Sorting
Insertion sort
Selection sort
Sorting (ct’d)
Hence, we must make sure that before evaluating
Binary Trees
x < a[j − 1], the processor evaluates j > 1. If this
Hashing
condition is false, it should not evaluate x < a[j − 1], but
Graphs
give the whole conjunction the value false.
Backtracking
Dynamic To achieve that behavior, we introduce a new logical

programming
connective cand. Similar: connective cor.
Greedy
194/ 550
Blockchain
cand and cor
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Bubble sort b cand c =
b if b then c else false
Insertion sort
Selection sort
b cor c =
b if b then true else c
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
195/ 550
Blockchain
Complete algorithm for insertion sort
DatAlg var i, j, n : Integer ; x : T ; a : ARRAY (1..n)[T ];

Maritta Heisel i := 1;
Introduction
while i 6= n do
Alg. Design
ADTs
i := i + 1;
Sorting j := i;
Bubble sort
Insertion sort
Selection sort
x := a[i];
Alg. Theory while j > 1 cand x < a[j − 1] do
Sorting (ct’d)
Binary Trees
a[j] := a[j − 1];
Hashing j := j − 1
Graphs
endwhile;
Backtracking
Dynamic a[j] := x
programming
Greedy
endwhile
196/ 550
Blockchain
Complete algorithm for insertion sort without cand
DatAlg var i, j, n : Integer ; stop : Bool; x : T ; a : ARRAY (1..n)[T ];

Maritta Heisel i := 1; stop := false;
Introduction while i 6= n do
Alg. Design i := i + 1; j := i; x := a[i];
ADTs
while j > 1 ∧ ¬ stop do
Sorting
Bubble sort
Insertion sort
if x < a[j − 1] then
Selection sort
Alg. Theory
a[j] := a[j − 1]; j := j − 1
Sorting (ct’d) else stop := true
Binary Trees
endif;
Hashing
Graphs endwhile;
Backtracking
a[j] := x; stop := false
Dynamic
programming endwhile
Greedy
197/ 550
Blockchain
Analysis of insertion sort
DatAlg
Maritta Heisel
Introduction We are interested in

Alg. Design
ADTs
the number of assignments the algorithm requires
Sorting
Bubble sort the number of comparisons the algorithm requires
Insertion sort
Selection sort each of them for the minimal, the maximal, and the
Alg. Theory average case
Sorting (ct’d)
Binary Trees
The outer loop is executed exactly n − 1 times, the inner loop
Hashing
does not need be executed at all. This is the case, if the array
Graphs
Backtracking
is already ordered.
Dynamic
programming
Greedy
198/ 550
Blockchain
Minimal number of assignments
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Bubble sort
Insertion sort
1 + 4 · (n − 1) = 4n − 3
Selection sort
initialization minimal number of | {z }
Alg. Theory
assignments per loop cycle number of loop cycles
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
199/ 550
Blockchain
Minimal number of comparisons
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Bubble sort
Insertion sort
1 + 3 · (n − 1) = 3n − 2
Selection sort
abortion of loop comparisons when | {z }
Alg. Theory loop body is executed number of loop cycles
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
200/ 550
Blockchain
Maximal number of assignments
DatAlg
Maritta Heisel
Introduction
Alg. Design n
X
ADTs 1 + ( 4 +
initialization assignments in
Sorting i=2 outer loop
Bubble sort number of cycles
Insertion sort of outer loop
Selection sort
Alg. Theory
i
X
Sorting (ct’d)
2 )
Binary Trees assignments per
j=2 loop cycle
Hashing
maximum number of
Graphs cycles of inner loop
Backtracking
Dynamic
programming
Greedy
201/ 550
Blockchain
Evaluating the maximal number of assignments
DatAlg n
X i
X
Maritta Heisel 1+ (4 + 2)
i=2 j=2
Introduction
n
Alg. Design X
ADTs
= 1+ (4 + 2(i − 1))
i=2
Sorting
Bubble sort
n
X
Insertion sort
Selection sort
= 1 + 4(n − 1) + 2 i −1
Alg. Theory i=2
Sorting (ct’d)
n−1
X
Binary Trees = 1 + 4(n − 1) + 2 i
Hashing i=1
Graphs n(n − 1)
= 1 + 4(n − 1) + 2
Backtracking 2
2
Dynamic
programming
= 1 + 4n − 4 + n − n
Greedy = n2 + 3n − 3
202/ 550
Blockchain
Maximal number of comparisons
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Bubble sort
n
X i
X
Insertion sort
Selection sort
1 + ( 1 + 2+ 1 )
loop abortion comparisons in since j≯1, x<a[j−1]
Alg. Theory (i=n) i=2 outer loop j=2 is not evaluated
(i6=n) (j>1; x<a[j−1])
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
203/ 550
Blockchain
Evaluating the maximal number of comparisons
DatAlg
Maritta Heisel
n
X i
X
Introduction 1+ (1 + 2 + 1)
Alg. Design
i=2 j=2
ADTs n
X
Sorting
Bubble sort
= 1+ (2 + 2(i − 1))
Insertion sort i=2
Selection sort
n
X
Alg. Theory
= 1 + 2(n − 1) + 2 i −1
Sorting (ct’d)
i=2
Binary Trees
n−1
X
Hashing
= 1 + 2(n − 1) + 2 i
Graphs
i=1
Backtracking
Dynamic
= 1 + 2n − 2 + n2 − n
programming
= n2 + n − 1
Greedy
204/ 550
Blockchain
Average number of assignments
DatAlg We assume that all permutations have the same probability.

Maritta Heisel The inner loop for each i is no longer executed i − 1 times, but
Introduction i div 2 times. This results in the following approximation
Alg. Design formula:
ADTs X n X i
Sorting 1+ (4 + 2)
Bubble sort
Insertion sort
i=2 j=i div 2
Selection sort
Xn
Alg. Theory = 1 + 4(n − 1) + i
Sorting (ct’d)
i=2
Binary Trees
n(n + 1)
Hashing = 1 + 4(n − 1) + −1
2
Graphs
8n − 8 + n2 + n
Backtracking =
Dynamic
2
programming 2
n + 9n − 8
Greedy =
2
205/ 550
Blockchain
Average number of comparisons
DatAlg
Maritta Heisel
Introduction
Alg. Design n i
X X
ADTs 1+ (1 + 2)
Sorting
i=2 j=i div 2
Bubble sort
Insertion sort
n(n + 1)
Selection sort
=1 + (n − 1) + −1
Alg. Theory 2
Sorting (ct’d) 2n − 2 + n2 + n
=
Binary Trees 2
Hashing n2 + 3n − 2
Graphs
=
2
Backtracking
Dynamic
programming
Greedy
206/ 550
Blockchain
Asymptotic behavior
DatAlg
Maritta Heisel As n grows larger, some term in the function which

Introduction
expresses the amount of resources used may begin to
Alg. Design dominate the other terms.
ADTs For example, if the execution time is 3n2 + 5n, then as n
Sorting
Bubble sort
grows larger, 3n2 grows very much larger than 5n, and
Insertion sort therefore 5n becomes increasingly less significant.
Selection sort
Alg. Theory The dominating term 3n2 (or even n2 ) is called the
Sorting (ct’d) asymptotic behavior of the algorithm.
Binary Trees
Usually, we only consider the asymptotic behavior of
Hashing
algorithms. That behavior ultimately governs whether or
Graphs
not a particular algorithm is feasible.
Backtracking
Dynamic
This subject will be discussed in more detail in the chapter
programming “Theory of algorithms”.
Greedy
207/ 550
Blockchain
Overall complexity of insertion sort
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting The maximal and average number of assignments and of

Bubble sort
Insertion sort
comparisons are all functions in n2 .
Selection sort
Alg. Theory
Hence, insertion sort is of quadratic time complexity.
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
208/ 550
Blockchain
Natural and stable sorting algorithms
DatAlg
Maritta Heisel The minimal effort of insertion sort occurs if the array is
Introduction already ordered; the maximum effort, if the array is
Alg. Design ordered in reverse order. Sorting algorithms with this
ADTs property are called natural sorting algorithms.
Sorting
Bubble sort
Insertion sort
In case several elements are equal according to the
Selection sort ordering relation, then their order remains unchanged;
Alg. Theory insertion sort is thus stable.
Sorting (ct’d)
That property can be important if several keys exist
Binary Trees
according to which the sorting can take place. If e.g. an
Hashing
originally alphabetically ordered array of students should
Graphs
now be sorted according to grades, then a stable sorting
Backtracking
Dynamic
method leaves the students with the same grades in
programming alphabetical order.
Greedy
209/ 550
Blockchain
Selection sort
DatAlg
Maritta Heisel
Introduction
Alg. Design
Basic idea:
ADTs
Sorting
Bubble sort
The first element of the ordered array is a minimum, the
Insertion sort
Selection sort
second is the second smallest element etc.
Alg. Theory Thus if the array a[1..n] is ordered to position i, search the
Sorting (ct’d) minimum of a[i + 1..n] and exchange it with a[i + 1] to
Binary Trees obtain an array that is ordered to position i + 1.
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
210/ 550
Blockchain
Example for selection sort
DatAlg
Maritta Heisel
Introduction
Alg. Design
i=0 44 55 12 42 6 66 41
ADTs
i=1 6 55 12 42 44 66 41
Sorting
Bubble sort i=2 6 12 55 42 44 66 41
Insertion sort
Selection sort
i=3 6 12 41 42 44 66 55
Alg. Theory i=4 6 12 41 42 44 66 55
Sorting (ct’d) i=5 6 12 41 42 44 66 55
Binary Trees i=6 6 12 41 42 44 55 66
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
211/ 550
Blockchain
Algorithm skeleton for selection sort
DatAlg
Maritta Heisel
var i, n : Integer ; a : ARRAY (1..n)[T ];
Introduction
Alg. Design i := 1;
ADTs
while i 6= n do
Sorting
Bubble sort
Insertion sort
“exchange a[i] with the minimum of a[i..n]”;
Selection sort
i := i + 1
Alg. Theory
Sorting (ct’d) endwhile

Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
212/ 550
Blockchain
Algorithm skeleton for selection sort
DatAlg
Maritta Heisel
var i, n : Integer ; a : ARRAY (1..n)[T ];
Introduction
Alg. Design i := 1;
ADTs
while i 6= n do
Sorting
Bubble sort
Insertion sort
“exchange a[i] with the minimum of a[i..n]”;
Selection sort
i := i + 1
Alg. Theory
Sorting (ct’d) endwhile

Binary Trees
Hashing
Graphs
Question: Why do we increase i at the end of the loop body
Backtracking
Dynamic
and not at the beginning, as in insertion sort?
programming
Greedy
212/ 550
Blockchain
Searching the minimum: specification
DatAlg
Maritta Heisel
Introduction
Alg. Design
var i, n, k : Integer ; a : ARRAY (1..n)[T ]
ADTs pre 1≤i <n
Sorting post 1≤i ≤k ≤n ∧
Bubble sort
Insertion sort
∀ x : Integer • (i ≤ x ≤ n ⇒ a[k] ≤ a[x])
Selection sort
reads i, n, a
Alg. Theory
changes k
Sorting (ct’d)
mem –
Binary Trees
Hashing
Thus, k is an index of the minimal element of a[i..n].
Graphs
Backtracking
Dynamic
programming
Greedy
213/ 550
Blockchain
Searching the minimum: algorithm
DatAlg
The following algorithm yields the first occurrence of the
Maritta Heisel
minimum, so that the sorting algorithm becomes stable.
Introduction
var i, j, k : Integer ; a : ARRAY (1..n)[T ];
Alg. Design
ADTs
k := i;
Sorting
Bubble sort
Insertion sort
j := i;
Selection sort
Alg. Theory
while j 6= n do
Sorting (ct’d) j := j + 1;
Binary Trees
if a[k] > a[j] then
Hashing
Graphs k := j
Backtracking
endif
Dynamic
Greedy
214/ 550
Blockchain
Complete algorithm for selection sort
DatAlg var i, j, k, n : Integer ; h : T ; a : ARRAY (1..n)[T ];

Maritta Heisel
Introduction
i := 1;
Alg. Design while i 6= n do
ADTs
k := i;
Sorting
Bubble sort
Insertion sort
j := i
Selection sort
while j 6= n do
Alg. Theory
Sorting (ct’d) j := j + 1;
Binary Trees
if a[k] > a[j] then k := j endif
Hashing
Graphs endwhile;
Backtracking h := a[i]; a[i] := a[k]; a[k] := h;
Dynamic
programming i := i + 1
Greedy
215/ 550
endwhile
Blockchain
Analysis of selection sort
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs The number of comparisons for selection sort is

Sorting independent of a.
Bubble sort
Insertion sort The minimal as well as the maximal time complexity is
Selection sort
Alg. Theory
quadratic. This can be shown in a similar way as for
Sorting (ct’d)
insertion sort.
Binary Trees Selection sort is a natural and stable sorting algorithm.
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
216/ 550
Blockchain
The theory of algorithms
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs Interesting properties of algorithms in general

Sorting
For which sorts of problems do algorithms exist at all?
Alg. Theory
Computability
Complexity
Which algorithms are feasible in terms of their usage of
Correctness physical resources?
Sorting (ct’d)
How to increase confidence in an algorithm’s correctness
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
217/ 550
Blockchain
Computability
DatAlg
Maritta Heisel Question:

Introduction
Is there a job which a computer cannot do – a job for
Alg. Design which no algorithm exists?
ADTs
Sorting
Alg. Theory
Computability
Complexity
Correctness
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
218/ 550
Blockchain
Computability
DatAlg

Introduction
ADTs
Answer:
Sorting
Yes, there is. In fact, computers cannot do most things!
Alg. Theory
Computability
Complexity
Correctness
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
218/ 550
Blockchain
Computability
DatAlg

Introduction
ADTs
Answer:
Sorting
Yes, there is. In fact, computers cannot do most things!
Alg. Theory
Computability Historically, philosophers (Leibniz) and mathematicians
Complexity
Correctness (Hilbert) believed that the truth of philosophical
Sorting (ct’d) arguments and mathematical theorems could be decided
Binary Trees
by computation.
Hashing
Graphs
In 1931, Gödel proved his famous incompleteness theorem,
Backtracking
which refuted that belief.
Dynamic That result was obtained before the first computers were
programming
built.
Greedy
218/ 550
Blockchain
The Church-Turing thesis
DatAlg
Maritta Heisel Before we can prove that no algorithm exists for a

Introduction
particular task, we must exactly know what we mean by
Alg. Design an algorithm.
ADTs We gave a definition earlier in this course, and Gödel,
Sorting Church and Turing gave other definitions in the 1930s.
Alg. Theory
Computability Fact: All reasonable definitions of “algorithm” which are
Complexity
Correctness
known so far are equivalent.
Sorting (ct’d) Conjecture: Any reasonable definition of “algorithm”
Binary Trees
which anyone will ever make will turn out be equivalent to
Hashing
the definitions we know.
Graphs
Backtracking
Universality: For any programming language there exists
Dynamic
an interpreter which, given the description of any program
programming in the language, can simulate that program.
Greedy
219/ 550
Blockchain
The halting problem
DatAlg
Maritta Heisel
The statement that no algorithm exists for solving a given
Introduction
Alg. Design
problem is a very strong statement.
ADTs It is much deeper than saying that we don’t know an
Sorting algorithm for solving the problem.
Alg. Theory
Computability
We say that we will never know such an algorithm!
Complexity
Correctness
The most famous non-computable problem relevant for
Sorting (ct’d) computer science is the halting problem.
Binary Trees
Its solution would be an algorithm which, given an
Hashing
arbitrary program P and its input data D can tell us
Graphs
whether or not P would eventually halt when executed
Backtracking
with input data D.
Dynamic
programming
Greedy
220/ 550
Blockchain
Proof of non-computability of halting problem I
DatAlg
Maritta Heisel Assume that there is an algorithm for solving the halting
Introduction
problem. Call that algorithm halttester .
Alg. Design Given P and D, halttester (P, D) prints “OK” if program
ADTs P would terminate when executed with input data D, and
Sorting it prints “BAD” otherwise.
Alg. Theory
Computability P D
Complexity
Correctness
Sorting (ct’d)
Binary Trees
Does P(D) halt?
Hashing
Graphs
Yes No
Backtracking
Dynamic
programming
output "OK" output "BAD"
Greedy and halt and halt
221/ 550
Blockchain
Proof of non-computability of halting problem II
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs More limited problem: Does P(P) halt?

Sorting
Since a program is just a sequence of characters, it can be
Alg. Theory
Computability
used as an input for some other program, or the same
Complexity
Correctness
program.
Sorting (ct’d) Call that algorithm newhalttester .
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
222/ 550
Blockchain
Proof of non-computability of halting problem III
DatAlg Specification:
Maritta Heisel
var P : LIST [Char ]
Introduction pre P 6= hi
Alg. Design
post (P(P) halts ⇒ newhalttester (P) =“OK”) ∧
ADTs
(P(P) does not halt ⇒ newhalttester (P) =“BAD”)
Sorting
reads P
Alg. Theory
Computability
changes -
Complexity
mem - P
Correctness
Sorting (ct’d)
Binary Trees
Hashing Does P(P) halt?

Graphs
Backtracking Yes No
Dynamic
programming
Greedy output "OK" output "BAD"

223/ 550
Blockchain
and halt and halt
Proof of non-computability of halting problem IV
DatAlg
Maritta Heisel If we assume that an algorithm halttester exists, and that

therefore newhalttester exists, we may construct the following
Introduction
algorithm funny , which has just one input P.
Alg. Design
ADTs P
Sorting
Alg. Theory
Computability module funny (P : LIST [Char ])
Complexity
Correctness if newhalttester(P) = “BAD” Does P(P) halt?
Sorting (ct’d) then “halt”
Binary Trees else “loop forever“ Yes No
Hashing endif
Graphs
endmodule halt
Backtracking
Dynamic
programming loop forever
Greedy
224/ 550
Blockchain
Proof of non-computability of halting problem V
DatAlg
Maritta Heisel Finally, we consider what happens during execution of

Introduction funny (funny ).
Alg. Design funny
ADTs
Sorting
Alg. Theory
Does funny(funny) halt?
Computability
Complexity
Correctness
Yes No
Sorting (ct’d)
Binary Trees
Hashing
loop forever halt
Graphs
This is clearly a contradiction, since it shows on the one hand
Backtracking
that if it halts then it loops forever, but on the other hand that
Dynamic
programming if it loops forever it halts.
Greedy
225/ 550
Blockchain
Proof of non-computability of halting problem VI
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Alg. Theory
The contradiction can only be resolved by admitting that the
Computability algorithm funny , and hence halttester , cannot exist.
Complexity
Correctness
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
226/ 550
Blockchain
Summary of proof
DatAlg
Maritta Heisel 1. Assume that we can write a program halttester .

Introduction 2. Use it to construct another program funny (via an
Alg. Design intermediate program newhalttester ).
ADTs
3. Show that funny has some impossible property (it can
Sorting
neither halt nor loop forever).
Alg. Theory
Computability
Complexity
4. Conclude that the assumption in step 1 must be wrong.
Correctness
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
227/ 550
Blockchain
Summary of proof
DatAlg
Maritta Heisel 1. Assume that we can write a program halttester .

Introduction 2. Use it to construct another program funny (via an
Alg. Design intermediate program newhalttester ).
ADTs
3. Show that funny has some impossible property (it can
Sorting
neither halt nor loop forever).
Alg. Theory
Computability
Complexity
4. Conclude that the assumption in step 1 must be wrong.
Correctness
Sorting (ct’d) Hence, there is no purely mechanical method which is

Binary Trees guaranteed to solve the halting problem in every case,
Hashing
even though for concrete algorithms, we can (and must)
Graphs
show that they terminate.
Backtracking
Dynamic
There is a difference between a routine algorithm and
programming human creativity in the solution of a problem!
Greedy
227/ 550
Blockchain
More noncomputable problems
DatAlg
Maritta Heisel
To show non-computability for other problems, we show that if
Introduction the problem at hand were computable then so would the
Alg. Design halting problem.
ADTs
Sorting
Totality problem: Will an arbitrary program P halt on all
Alg. Theory
inputs? We say that P is total if P(D) halts for all input data
Computability D.
Complexity
Correctness
If the totality problem were computable, we could define a
Sorting (ct’d)
Binary Trees
module funnypd:
Hashing module funnypd(I : Arbitrary type)
Graphs
{I is the input, but the module does not use it}
Backtracking
”simulate program P on input data D“
Dynamic
programming endmodule
Greedy
228/ 550
Blockchain
Totality problem
DatAlg
Now, asking if funnypd is total is the same as asking whether
Maritta Heisel
P(D) halts. So, an algorithm for the totality problem would
Introduction
give us the following algorithm for the halting problem:
Alg. Design
ADTs module halttester (P : LIST [Char ]; D : Arbitrary type)

Sorting ”construct the text of the program funnypd”;
Alg. Theory if total(funnypd)
Computability
Complexity then “output “OK” and halt”
Correctness
else “output “BAD” and halt”
Sorting (ct’d)
endif
Binary Trees
endmodule
Hashing
Graphs However, we know that no algorithm for the halting problem

Backtracking exists. Hence, the assumption that there exists an algorithm for
Dynamic
programming
the totality problem is false. We have therefore proved that the
Greedy totality problem is not computable.
229/ 550
Blockchain
Equivalence problem I
DatAlg
Maritta Heisel
The problem to determine if two programs perform the same
Introduction job (i.e. for the same input they produce the same output), is
Alg. Design not computable.
ADTs
Sorting Otherwise, given any P, we could construct a program that

Alg. Theory simulates P and outputs “13” if P eventually halts:
Computability
Complexity
Correctness
module funnyp(D : Arbitrary type)
Sorting (ct’d) {Output is “13” if P(D) ever halts}
Binary Trees “simulate program P on input data D”;
Hashing {The previous step may loop for ever, but if it doesn’t then
Graphs ...}
Backtracking output(13)
Dynamic
programming
endmodule
Greedy
230/ 550
Blockchain
Equivalence problem II
DatAlg
Maritta Heisel Now, consider a simple-minded program that always outputs

Introduction “13”:
Alg. Design
ADTs
module simple(D : Arbitrary type)
Sorting {D is the input, but the module ignores it and outputs “13”
Alg. Theory regardless}
Computability
Complexity
output(13)
Correctness
endmodule
Sorting (ct’d)
Binary Trees Now, asking whether funnyp is equivalent to simple is the same
Hashing as asking whether P is total, because, if P halts on every input,
Graphs
then funnyp will output “13” on every input. However, if there
Backtracking
is some input on which P does not halt, then funnyp will not
Dynamic
programming halt and will not output “13” on that input.
Greedy
231/ 550
Blockchain
Equivalence problem III
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Exercise: construct a module total(P) for the totality problem
Alg. Theory
Computability
on the assumption that there exists an algorithm solving the
Complexity
Correctness
equivalence problem.
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
232/ 550
Blockchain
Partial computability
DatAlg
Maritta Heisel
Some noncomputable problems are less computable than
others.
Introduction
Alg. Design
The halting problem is partially computable, because there
ADTs
exists an algorithm which outputs “Yes” if P(D) halts,
Sorting
though it loops for ever if P(D) does not halt.
Alg. Theory Input Input
Computability
Complexity
Correctness
Sorting (ct’d)
Binary Trees Algorithm Algorithm

Hashing
Graphs
Backtracking
Dynamic Yes No Yes loop forever

programming
Greedy
computable problem partially computable problem
233/ 550
Blockchain
Partially computable vs. not partially computable
problems
DatAlg
Maritta Heisel
The totality and equivalence problems are not partially
Introduction computable.
Alg. Design
Problems are partially computable if they have a proof
ADTs
Sorting
system, i.e., a set of axioms together with some rules of
Alg. Theory
inference.
Computability
Complexity
A proof is just a sequence of lines, the early ones being
Correctness
axioms and the remainder following from previous ones by
Sorting (ct’d)
using the rules of inference.
Binary Trees
Hashing
In particular, if a program does halt on some input data, it
Graphs
is possible to write down a convincing proof of that fact.
Backtracking In summary, problems can be computable, partially
Dynamic
programming
computable, or not even partially computable.
Greedy
234/ 550
Blockchain
What have we learnt about computability?
DatAlg
Maritta Heisel
Introduction
Not all problems are solvable by an algorithm.
Alg. Design
ADTs
A famous non-computable problem is the halting problem.
Sorting Proving that a problem is non-computable is done by
Alg. Theory contradiction.
Computability
Complexity
Correctness
Other non-computable problems can be reduced to the
Sorting (ct’d)
halting problem or other problems known to be
Binary Trees non-computable.
Hashing There are different degrees of non-computability: some
Graphs problems are partially computable, others not even that.
Backtracking
Dynamic
programming
Greedy
235/ 550
Blockchain
Computability vs. complexity
DatAlg
Maritta Heisel
Introduction Computability leads to an understanding of which problems

Alg. Design
admit algorithmic solution, and which do not.
ADTs
Sorting
Complexity: Of those problems for which there do exist
Alg. Theory
algorithms, it is of interest to know how much
Computability computer resources are required for their
Complexity
Correctness execution.
Sorting (ct’d) Only those algorithms which use a feasible
Binary Trees
amount of resources are useful in practice.
Hashing
Complexity theory deals with questions about computational
Graphs
Backtracking
resources.
Dynamic
programming
Greedy
236/ 550
Blockchain
The universe of problems
DatAlg
All problems
Maritta Heisel
Introduction
Computable problems
Alg. Design
ADTs
Sorting
Feasibly computable

problems

Alg. Theory
Computability
Complexity

Correctness
Sorting (ct’d)
Binary Trees
Hashing
Although the set of feasibly computable problems is only a
Graphs
small portion of all computable problems (which in turn are
Backtracking only a small portion of all problems) that set is so large that
Dynamic computer science has become an interesting, practical, and
programming
flourishing science.
Greedy
237/ 550
Blockchain
Computer resources
DatAlg
Maritta Heisel
Introduction
Alg. Design
Time elapsed period from start to finish of algorithm
ADTs
execution
Sorting Memory amount of storage required by algorithm
Alg. Theory
Computability
Hardware amount of physical mechanism required for
Complexity
Correctness
execution of algorithm (e.g., number of
Sorting (ct’d)
processors for parallel algorithms)
Binary Trees
Hashing
The amount of any resource used by an algorithm may vary
Graphs
with the size of the input data.
Backtracking
Dynamic
programming
Greedy
238/ 550
Blockchain
Multiplication algorithms I
DatAlg
Maritta Heisel
If the “junior school algorithm” is pre-
Introduction
sented with two n-digit numbers, it adds
Alg. Design
together n rows, each containing n (or
ADTs 1984
n + 1) digits.
Sorting x 6713
Each row can be computed in n steps, and
Alg. Theory
Computability there are n separate rows. 5952
Complexity
Correctness
Thus all rows can be computed in n × n 1984
Sorting (ct’d) (or n2 ) units of time. 13888
11904
Binary Trees Adding the rows also takes n2 units of
13318592
Hashing time.
Graphs Hence the execution time of the whole al-
Backtracking
gorithm is proportional to n2 .
Dynamic
programming
Greedy
239/ 550
Blockchain
Multiplication algorithms II
DatAlg
Maritta Heisel
A faster algorithm:
Introduction A B C D
Alg. Design 19 84 x 67 13
ADTs
Sorting
Alg. Theory AC = 19 x 67 = 1273

Computability (A + B)(C + D) − AC − BD = (103 x 80) − 1273 − 1092 = 5875
Complexity
Correctness BD = 83 x 13 = 1092
Sorting (ct’d) 13318592
Binary Trees
Hashing This algorithm takes time proportional to n1.59

Graphs
Backtracking The fastest known algorithm for a sequential machine has

Dynamic execution time proportional to n log n log log n (log n meaning
programming
Greedy
the logarithm of n using the base 2).
240/ 550
Blockchain
Important points
DatAlg
A problem can be solved by quite different algorithms
Maritta Heisel
which perhaps use different amounts of resources.
Introduction
It is interesting to find the best algorithm, which uses the
Alg. Design
ADTs
least resources.
Sorting Often, as one attempts to reduce some resource, some
Alg. Theory other resource must necessarily be increased (trade-off).
Computability
Complexity
Correctness
Moreover:
Sorting (ct’d) The amount of resource used depends on the size of the
Binary Trees input data.
Hashing
The more digits there are in the numbers to be multiplied,
Graphs
the longer it will take to perform the multiplication.
Backtracking
Dynamic
In general, if there are n characters of input data, we can
programming express the amount of resource used as a function of n,
Greedy
such as 3n2 + 5n, or 2n log n + n + 17.
241/ 550
Blockchain
Asymptotic behavior
DatAlg
Maritta Heisel
Introduction
As n grows larger, some term in the function which
Alg. Design
expresses the amount of resource used may begin to
ADTs dominate the other terms.
Sorting For example, if the execution time is 3n2 + 5n, then as n
Alg. Theory grows larger, 3n2 grows very much larger than 5n, and
Computability
Complexity therefore 5n becomes increasingly less significant.
Correctness
Sorting (ct’d) The dominating term 3n2 (or even n2 ) is called the
Binary Trees asymptotic behavior of the algorithm.
Hashing Usually, we only consider the asymptotic behavior of
Graphs
algorithms. That behavior ultimately governs whether or
Backtracking
not a particular algorithm is feasible.
Dynamic
programming
Greedy
242/ 550
Blockchain
Examples of execution times
DatAlg Assumption: one step takes one microsecond. Current

Maritta Heisel processors (e.g., AMD Athlon64 2.8 GHz) execute 8400 billion
Introduction instructions per second.
Alg. Design
ADTs
Input size n log2 n n n2 2n
Sorting
10 0.000003 s 0.00001 s 0.0001 s 0.001 s
Alg. Theory 100 0.000007 s 0.0001 s 0.01 s 1014 centuries
Computability
Complexity
1000 0.00001 s 0.001 s 1s astronomical
Correctness 10000 0.000013 s 0.01 s 1.7 min astronomical
Sorting (ct’d)
100000 0.000017 s 0.1 s 2.8 h astronomical
Binary Trees
Hashing
The 2n algorithm is infeasible even for relatively small sizes
Graphs
of input data.
Backtracking
The n2 algorithm is much better, but will also use too
Dynamic much time for large input data.
programming
The complexity can hardly be less than linear, because an
Greedy
algorithm must at least look through the input data.
243/ 550
Blockchain
Big O notation
DatAlg
Maritta Heisel
Approach to analyze an algorithm’s usage of computational
Introduction resources:
Alg. Design
Worst case analysis ignoring constant factors.
ADTs
Sorting
Allows to prove that the runtime behavior of an algorithm
Alg. Theory is proportional to a certain function.
Computability
Complexity To express the notion proportional to, we make use of the
Correctness
big O Notation:
Sorting (ct’d)
O(f (n)) = {g : N → N | ∃ n0 , c : N • n0 > 0 ∧ c > 0 ∧
Binary Trees
(∀ n : N • n ≥ n0 ⇒ g (n) ≤ c ∗ f (n))}
Hashing
Graphs In words: express the asymptotic upper bound of an

Backtracking algorithm’s runtime behavior in terms of another (simpler)
Dynamic function.
programming
Greedy
244/ 550
Blockchain
Laws for big O notation
DatAlg
Maritta Heisel
Introduction
Let h ∈ O(f ) and g arbitrary functions with g (n) ≥ 0 for
Alg. Design
n ≥ n0 . Let c ≥ 0 be a constant. Then
ADTs
h+g ∈ O(f + g )
Sorting c ±h ∈ O(f )
h∗g ∈ O(f ∗ g )
Alg. Theory c ∗h ∈ O(f )
Computability h/g ∈ O(f /g )
Complexity
Correctness
Sorting (ct’d)
Furthermore:
Binary Trees
g ∈ O(f + h) ⇒ g ∈ O(max{f , h})
Hashing
Graphs Transitivity: (f ∈ O(g ) ∧ g ∈ O(h)) ⇒ f ∈ O(h)

Backtracking Reflexivity: f ∈ O(f )
Dynamic
programming
Greedy
245/ 550
Blockchain
Complexity classes
DatAlg
Maritta Heisel
Introduction
Alg. Design
Algorithms whose asymptotic behavior is 2n , or more
ADTs general c n for some constant c, are called exponential
Sorting algorithms.
Alg. Theory Exponential algorithms are infeasible for all but the
Computability
Complexity
smallest input data sizes.
Algorithms whose asymptotic behavior is nc for some
Correctness
Sorting (ct’d)
Binary Trees
constant c are called polynomial algorithms.
Hashing
Many polynomial algorithms are feasible for practical input
Graphs data sizes, but, unfortunately, many are not.
Backtracking
Dynamic
programming
Greedy
246/ 550
Blockchain
Important complexity classes
DatAlg
Maritta Heisel
Introduction
Alg. Design O(1) constant some hashing-based searching

ADTs algorithms for tables
Sorting
O(log n) logarithmic binary search
Alg. Theory
Computability
O(n) linear linear search
Complexity
Correctness
O(n log n) fast sorting algorithms
Sorting (ct’d)
O(n2 ) quadratic slow sorting algorithms
Binary Trees O(n3 ) cubic matrix multiplication
Hashing O(2n ) exponential many optimization problems
Graphs
Backtracking
Dynamic
programming
Greedy
247/ 550
Blockchain
Different kinds of complexity
DatAlg
Maritta Heisel
For a given input size, there are many different inputs, and it is
not unusual that an algorithm uses different amounts of
Introduction
resources on different input data, even when the different inputs
Alg. Design
have the same size (see for example the bubble sort algorithm).
ADTs
Sorting Worst-case complexity: maximum amount of resource required

Alg. Theory by the algorithm for a given input size.
Computability
Complexity
Correctness
Average-case complexity: average amount of resource used
Sorting (ct’d) over all the inputs of a given size.
Binary Trees
For some algorithms, the worst-case complexity is decisive for
Hashing
their feasibility (e.g., control algorithms).
Graphs
Backtracking For others, an acceptable average-case complexity suffices

Dynamic (e.g., Quicksort, a sorting algorithm with worst-case complexity
programming
n2 and average-case complexity n log n).
Greedy
248/ 550
Blockchain
Complexity of problems
DatAlg
Maritta Heisel So far: complexity of algorithms.

Introduction Complexity of a problem: complexity of the best algorithm
Alg. Design that solves the problem.
ADTs
The complexity of many problems is unknown, because we
Sorting
only know the complexity of the best algorithm found so
Alg. Theory
Computability
far.
Complexity
Correctness That complexity constitutes an upper bound on the
Sorting (ct’d) complexity of the problem.
Binary Trees
Lower bound: least amount of resource needed by every
Hashing
algorithm solving the problem (often n).
Graphs
Backtracking For integer multiplication, a lower bound is n and an

Dynamic upper bound is n log n log log n. Hence, the exact
programming
complexity of the problem lies between those two.
Greedy
249/ 550
Blockchain
Complexity analysis
DatAlg
Maritta Heisel
Introduction
Alg. Design
Recursive procedure based on the syntactical structure of
ADTs
an algorithm.
Sorting
Alg. Theory TA (n) is the maximum complexity of an algorithm A for

Computability
Complexity
inputs of size n.
Correctness
Assignments using basic data types (e.g., not creating a
Sorting (ct’d)
copy of an array) and comparisons have a complexity of
Binary Trees
Hashing
O(1).
Graphs
Backtracking
Dynamic
programming
Greedy
250/ 550
Blockchain
Complexity of sequences and conditionals
DatAlg
Maritta Heisel
Introduction
Alg. Design Ts1 ;s2 (n) = Ts1 (n1 ) + Ts2 (n2 )

ADTs
Sorting
where n1 and n2 are the size of the input for s1 and s2 ,
Alg. Theory
respectively.
Computability
Complexity Upper bound: Ts1 (n) + Ts2 (n)
Correctness
Sorting (ct’d)
Binary Trees
Hashing Tif c then s1 else s2 endif (n) = Tc (n) + max{Ts1 (n), Ts2 (n)}
Graphs
Backtracking
Dynamic
programming
Greedy
251/ 550
Blockchain
Complexity of loops I
DatAlg
Maritta Heisel
Introduction
Let while c do s endwhile be a pre-tested loop.
Alg. Design
ADTs
Twhile...endwhile (n) = Tc (n) + Ts (n) + Tc (n1 ) + Ts (n1 )
Sorting
Alg. Theory
+ · · · + Tc (nj−1 ) + Ts (nj−1 ) + Tc (nj )
Computability
Pj−1
Complexity
Correctness
= Tc (nj ) + i=0 (Tc (ni ) + Ts (ni )),
Sorting (ct’d)
Binary Trees where n0 = n and ni−1 results from ni by executing the loop
Hashing body once and for the case that for nj the loop condition is not
Graphs fulfilled for the first time.
Backtracking
Dynamic
programming
Greedy
252/ 550
Blockchain
Complexity of loops II
DatAlg
Maritta Heisel
Introduction More coarse-grained analysis:

Alg. Design
ADTs
Tc (sz(n)) = max{Tc (ni ) | 0 ≤ i ≤ j}
Sorting
Ts (sz(n)) = max{Ts (ni ) | 0 ≤ i ≤ j − 1}
Alg. Theory
Computability
Complexity
Twhile...endwhile (n) ≤ (nb(n) + 1)Tc (sz(n)) + nb(n)Ts (sz(n))
Correctness
Sorting (ct’d)
where nb(n) is the maximum number of loop executions for
Binary Trees inputs of size n and sz(n) is the maximum size of intermediate
Hashing results. Then
Graphs
Twhile...endwhile (n) ∈ O(nb(n)(Tc (sz(n)) + Ts (sz(n))))
Backtracking
Dynamic
programming
Greedy
253/ 550
Blockchain
Recall: insertion sort algorithm
DatAlg var i, j, n : Integer ; x : T ; a : ARRAY (1..n)[T ];

Maritta Heisel i := 1;
Introduction
while i 6= n do
Alg. Design
ADTs
i := i + 1;
Sorting j := i;
Alg. Theory
Computability x := a[i];
Complexity
Correctness while j > 1 cand x < a[j − 1] do
Sorting (ct’d)
Binary Trees
a[j] := a[j − 1];
Hashing j := j − 1
Graphs
endwhile;
Backtracking
Dynamic a[j] := x
programming
Greedy
endwhile
254/ 550
Blockchain
Example: complexity analysis of insertion sort I
DatAlg
Maritta Heisel Complexity of the outer loop:

Introduction
nb1 (n) ∈ O(n), since the loop is executed “about” n times.
Alg. Design
sz1 (n) ∈ O(n), since the maximum value of i in the loop body
ADTs equals n.
Sorting
Tbody1 ∈ O(Tinner loop (n))
Alg. Theory
Computability
Complexity
Complexity of the inner loop:
Correctness
depends on i, where the maximum of i equals n.
Sorting (ct’d)
Binary Trees nb2 (n) ∈ O(n) and sz2 (n) ∈ O(n)

Hashing Tcond2 ∈ O(1)
Graphs Tbody2 ∈ O(1)
Backtracking
Tinner loop (n) ∈ O(n(k1 + k2 )) = O(n),
Dynamic
programming where k1 = Tcond2 (n) and k2 = Tbody2 (n).
Greedy
255/ 550
Blockchain
Example: complexity analysis of insertion sort II
DatAlg
Maritta Heisel
Introduction
Alg. Design
Tinsertion sort (n) ∈ O(n ∗ n) = O(n2 ),
ADTs
Sorting where n = nb1 (n) = sz1 (n).

Alg. Theory
Computability
Complexity
Correctness
General rule: if an algorithm consists of two nested loops which
Sorting (ct’d)
Binary Trees
are passed through linearly, then the algorithm’s complexity is
Hashing
quadratic.
Graphs
Backtracking
Dynamic
programming
Greedy
256/ 550
Blockchain
Divide and conquer
DatAlg
Maritta Heisel Common and elegant method of devising an efficient algorithm:

Introduction
divide the problem into smaller pieces, thus leaving only smaller
Alg. Design problems to be solved.
ADTs
Example: Mergesort
Sorting
Alg. Theory module sort(a : ARRAY(1..n)[T])

Computability
Complexity
{Sorts given array a of n data items}
Correctness
if n > 1
Sorting (ct’d)
then merge(sort(left(a)), sort(right(a)))
Binary Trees
endif
Hashing
endmodule
Graphs
Backtracking
What is the time complexity of this algorithm? (I.e., how long
Dynamic
programming does the algorithm take to sort n data items?)
Greedy
257/ 550
Blockchain
Recurrence relations
DatAlg
Let T (n) be the time taken by the algorithm to sort n
Maritta Heisel
data items.
Introduction
Alg. Design
ADTs
Sorting
Alg. Theory
Computability
Complexity
Correctness
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
258/ 550
Blockchain
DatAlg
Maritta Heisel
data items.
Introduction
The algorithm begins by sorting the first half of the data
Alg. Design
items. This will take T (n/2).
ADTs
Sorting
Alg. Theory
Computability
Complexity
Correctness
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
258/ 550
Blockchain
DatAlg
Maritta Heisel
data items.
Introduction
Alg. Design
ADTs
Sorting Merging the two halves together can be carried out in time
Alg. Theory proportional to n (say cn for some constant c)(Exercise!).
Computability
Complexity
Correctness
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
258/ 550
Blockchain
DatAlg
Maritta Heisel
data items.
Introduction
Alg. Design
ADTs
Computability
Complexity Thus, the total time taken is T (n) = 2T (n/2) + cn
Correctness
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
258/ 550
Blockchain
DatAlg
Maritta Heisel
data items.
Introduction
Alg. Design
ADTs
Computability
Correctness
Sorting (ct’d)
Also T (1) = k for some constant k, since when n = 1 only
Binary Trees
the test (n > 1) need be carried out.
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
258/ 550
Blockchain
DatAlg
Maritta Heisel
data items.
Introduction
Alg. Design
ADTs
Computability
Correctness
Sorting (ct’d)
Binary Trees
Hashing Such equations are called recurrence relations.
Graphs
Backtracking
Dynamic
programming
Greedy
258/ 550
Blockchain
DatAlg
Maritta Heisel
data items.
Introduction
Alg. Design
ADTs
Computability
Correctness
Sorting (ct’d)
Binary Trees
Graphs
The above recurrence relation has the solution
Backtracking
T (n) = cn log n + kn.
Dynamic
programming
Greedy
258/ 550
Blockchain
DatAlg
Maritta Heisel
data items.
Introduction
Alg. Design
ADTs
Computability
Correctness
Sorting (ct’d)
Binary Trees
Graphs
The above recurrence relation has the solution
Backtracking
T (n) = cn log n + kn.
Dynamic
programming Hence, the asymptotic execution time of the merge sort
Greedy algorithm is proportional to n log n.
258/ 550
Blockchain
More recurrence relations
DatAlg
Maritta Heisel Consider the multiplication algorithm of slide 240.

Introduction This algorithm uses a divide and conquer approach, since
Alg. Design in order to multiply two numbers it splits each into halves
ADTs (A, B and C , D), and then performs three multiplications
Sorting
of numbers only half the original length: AB, BC and
Alg. Theory
Computability
(A + B)(C + D).
Complexity
Correctness
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
259/ 550
Blockchain
DatAlg

Sorting
Alg. Theory
Computability
(A + B)(C + D).
Complexity
Correctness If T (n) is the time taken by the algorithm to multiply two
Sorting (ct’d) n digit numbers, then T (n) = 3T (n/2) + cn because
Binary Trees the time needed to add and subtract the pieces is only cn.
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
259/ 550
Blockchain
DatAlg

Sorting
Alg. Theory
Computability
(A + B)(C + D).
Complexity
Hashing
This recurrence relation has the
Graphs
solution T (n) = (2c + k)nlog 3 − 2cn.
Backtracking
Dynamic
programming
Greedy
259/ 550
Blockchain
DatAlg

Sorting
Alg. Theory
Computability
(A + B)(C + D).
Complexity
Hashing
This recurrence relation has the
Graphs
solution T (n) = (2c + k)nlog 3 − 2cn.
Backtracking
Dynamic Thus the asymptotic behavior is proportional to nlog 3 ,

programming
which is about n1.59
Greedy
259/ 550
Blockchain
Advantages of divide and conquer algorithms
DatAlg
Maritta Heisel
Introduction
Alg. Design
Divide and conquer can sometimes be used to derive
ADTs
interesting, elegant and efficient algorithms.
Sorting
Alg. Theory Recurrence relations express the resource usage of such

Computability
Complexity
algorithms, and the solution of the recurrence relation
Correctness
gives the resource usage in a clearer form.
Sorting (ct’d)
The algorithms themselves are best expressed using the
Binary Trees
Hashing
recursion feature of a modern programming language.
Graphs
Backtracking
Dynamic
programming
Greedy
260/ 550
Blockchain
Feasible vs. infeasible algorithms
DatAlg
Maritta Heisel
Aim of complexity theory: improve classification of
Introduction
algorithms, thereby improving our understanding of the
Alg. Design
difference between feasible and infeasible problems.
ADTs
Sorting Polynomial algorithms tend to be feasible for reasonable

Alg. Theory sizes of input data, even though many of them are
Computability
Complexity
infeasible, such as one which takes 5n1000 hours to solve a
Correctness
problem.
Sorting (ct’d)
Conversely, exponential algorithms tend to exceed the
Binary Trees
Hashing
available resources even for small amounts of input data.
Graphs
Again, there are exceptions. An algorithm which uses
Backtracking 20.0001n seconds may be quite reasonable for values of n
Dynamic up to 10000.
programming
Greedy
261/ 550
Blockchain
Polynomial algorithms as feasible class I
DatAlg
Maritta Heisel
Introduction First approximation to distinguish between feasible and

Alg. Design infeasible algorithms: say that algorithms executing in a
ADTs
polynomial amount of time are feasible, and all others are
Sorting
infeasible.
Alg. Theory
Computability Attractive approximation because polynomial algorithms
Complexity
Correctness
have a number of closure properties:
Sorting (ct’d) Sequential composition of two feasible algorithms yields a
Binary Trees feasible algorithm.
Hashing If a step in a feasible algorithm is replaced by a call to a
Graphs module representing a second feasible algorithm, the new
Backtracking
algorithm is also feasible.
Dynamic
programming
Greedy
262/ 550
Blockchain
Polynomial algorithms as feasible class II
DatAlg
Maritta Heisel
Moreover, the concept of a feasible algorithm should not
Introduction depend on the details of the computer on which the algorithm
Alg. Design will be executed.
ADTs
For the polynomial algorithms, this is the case, because
Sorting
Alg. Theory
Not only can all reasonable sequential computers simulate
Computability each other, but the time losses associated with the
Complexity
Correctness simulations are not excessive.
Sorting (ct’d)
Any algorithm which executes in polynomial time on one
Binary Trees
computer can be run in polynomial time on any other
Hashing
computer.
Graphs
Backtracking Hence, a theory of feasible algorithms based on polynomial

Dynamic time is machine independent.
programming
Greedy
263/ 550
Blockchain
Sequential computation thesis
DatAlg
Maritta Heisel
The sequential computation thesis says that all reasonable
Introduction
sequential computers which will ever be dreamed of have
Alg. Design
polynomially related execution times.
ADTs
Sorting
This is stronger than the Church-Turing thesis, because it
Alg. Theory
claims not only that the computable problems are the
Computability
Complexity
same for all computers, but also that the feasibly
Correctness computable problems are the same for all computers.
Sorting (ct’d)
Binary Trees
In summary, the best definition of to date is that the feasibly
Hashing computable problems are those which have polynomial time
Graphs (sequential) algorithms.
Backtracking
This definition matches practical experience quite well, and
Dynamic
programming seems to be machine independent.
Greedy
264/ 550
Blockchain
Infeasible problems I
DatAlg
Maritta Heisel
Introduction
Following the previous discussion, it is of great interest to
Alg. Design
know which problems are computable in polynomial time.
ADTs Many interesting and practical problems are in this
Sorting category, including most of the problems discussed in the
Alg. Theory chapter “Design of Algorithms”.
Computability
Complexity
Correctness
Some problems have proved not to be polynomial time
Sorting (ct’d) computable. Such proofs are usually very difficult, as it is
Binary Trees necessary to show that no algorithm that runs in
Hashing polynomial time can solve the problem.
Graphs
An example of such a problem is a generalization of the
Backtracking
game of chess to an n × n board.
Dynamic
programming
Greedy
265/ 550
Blockchain
Infeasible problems II
DatAlg
Maritta Heisel
Introduction There are many problems for which the existence of a fast
Alg. Design (i.e. polynomial time) algorithm remains an open
ADTs question. No fast algorithms have been discovered for
Sorting such problems, but neither has anyone yet been able to
Alg. Theory prove that no such algorithm exists.
Computability
Complexity
Correctness
Many of these problems have practical applications, as
Sorting (ct’d) well as presenting an interesting challenge to computer
Binary Trees scientists.
Hashing An example of such a problem arises in connection with
Graphs
trucking companies.
Backtracking
Dynamic
programming
Greedy
266/ 550
Blockchain
Bin-packing problem I
DatAlg
Maritta Heisel
Suppose a trucking company has a number N of crates of
Introduction
different weights.
Alg. Design
ADTs
The problem is to determine if T trucks, each of which
Sorting can carry a load W , are sufficient to transport the crates.
Alg. Theory
Computability 3 3
Complexity
tonnes tonnes
Correctness
Sorting (ct’d) 4
tonnes 5 tonnes
Binary Trees
Hashing
Graphs 8 tonnes 5 tonnes

Backtracking
T = 2 trucks
Dynamic N = 6 crates
programming maximum load W = 14 tonnes
Greedy
267/ 550
Blockchain
Bin-packing problem II
DatAlg
Clearly, there are also many other applications in which
Maritta Heisel
the bin-packing problem arises.
Introduction Unfortunately, every known algorithm for this problem uses
Alg. Design
exponential time.
ADTs
One might at first think of repeatedly loading the largest
Sorting
crate which still fits on to a truck until no more creates fit,
Alg. Theory
Computability and then proceeding to the next truck. The figure
Complexity
Correctness
illustrates that this algorithm is not guaranteed to work.
Sorting (ct’d)
Binary Trees 5 tonnes
Hashing 3 3
tonnes tonnes 8 tonnes
Graphs
Backtracking 4
tonnes 5 tonnes
Dynamic
programming
Greedy
268/ 550
Blockchain
Traveling salesperson problem
DatAlg
This is another problem which has puzzled operations
Maritta Heisel
research experts and computer scientists for years.
Introduction
Given a roadmap of N cities, is it possible for a
Alg. Design
salesperson to complete a round trip within a given
ADTs
Sorting
mileage allowance, visiting each city exactly once?
Alg. Theory A related problem is to determine whether a round trip
Computability
Complexity
which visits each city exactly once is possible on the given
Correctness
map.
Sorting (ct’d)
Binary Trees
For both problems, no poly-
Hashing
nomial time algorithm has
Graphs
ever been found, nor has
Backtracking
anyone been able to prove
Dynamic
programming that no such algorithm ex-
Dots represent cities
Greedy ists.
Lines represent roads
269/ 550
Blockchain
Timetabling problem
DatAlg
Maritta Heisel
Introduction
Alg. Design
This is also a problem of great practical interest for which
ADTs
Sorting
no polynomial time algorithm is currently known.
Alg. Theory In this problem, one is given a list of subjects and the
Computability
Complexity
students enrolled in them, as well as the number of time
Correctness
slots available.
Sorting (ct’d)
Binary Trees
The problem is to timetable the subjects so that no
Hashing
student has a clash.
Graphs
Backtracking
Dynamic
programming
Greedy
270/ 550
Blockchain
Dealing with infeasible problems I
DatAlg
Maritta Heisel
What should computer scientists do when a computer
Introduction
solution to any of these problems is required?
Alg. Design
ADTs
It is necessary to lower our sights in one of the following
Sorting
ways:
Alg. Theory Rather than to seek an algorithm which provides an exact
Computability solution, it will sometimes be adequate to provide an
Complexity
Correctness approximate solution only.
Sorting (ct’d) For example, in the timetabling problem, a solution with
Binary Trees only a few clashes may be better than no solution at all.
Hashing For the traveling salesperson problem, a fast algorithm
Graphs that produces a reasonably short trip is better than using
Backtracking
an infeasible amount of computer time to calculate the
Dynamic
absolutely minimal mileage round trip.
programming
Greedy
271/ 550
Blockchain
Dealing with infeasible problems II
DatAlg
Maritta Heisel
Introduction Other ways of lowering one’s sight:

Alg. Design Try to produce an algorithm which executes quickly on the
ADTs average input data, but which nevertheless exhibits an
Sorting exponential behavior in the worst case.
Alg. Theory The hope is that the algorithm will terminate reasonably
Computability
Complexity
quickly on most of the data which it encounters in practice.
Correctness
Relax the condition that the algorithm must be correct.
Sorting (ct’d) This may seem bizarre, but it may be useful in certain
Binary Trees circumstances to try to produce a fast algorithm which is
Hashing known to contain an error.
Graphs Cute example: primality problem.
Backtracking
Dynamic
programming
Greedy
272/ 550
Blockchain
Primality problem I
DatAlg
Maritta Heisel
Introduction
Given an n-digit number as input, the problem is to
Alg. Design
determine whether or not the number is prime.
ADTs An obvious algorithm is to divide the input number by
Sorting every smaller number and check if the remainder is ever
Alg. Theory zero.
Computability
Complexity
Correctness
This is an exponential algorithm and becomes quite
Sorting (ct’d) infeasible even for moderate values of n, such as 20 or 30
Binary Trees digits.
Hashing
The algorithm that results from dividing only by prime
Graphs
numbers, and then only up to the square root of the input
Backtracking
number is slightly faster but still infeasible.
Dynamic
programming
Greedy
273/ 550
Blockchain
Primality problem II
DatAlg
Maritta Heisel
Introduction Consider the following mathematical fact concerning any odd

Alg. Design number x. If x is prime, then every number y between 1 and
ADTs x − 1 satisfies the following condition:
Sorting
Alg. Theory
GCD(x, y ) = 1 and y (x−1)/2 mod x = J(y , x)
Computability
Complexity where GCD is the greatest common divisor and J is called the
Correctness
Jacobi function. That function can be computed quickly.
Sorting (ct’d)
Binary Trees It turns out that if x is not prime, then at least half the
Hashing possible values of y will fail to satisfy the condition.
Graphs
Backtracking Thus the following algorithm can be used to check if x is prime.

Dynamic
programming
Greedy
274/ 550
Blockchain
Primality tester
DatAlg If x is prime then “PRIME” is output. If x is not prime then

Maritta Heisel “NOT PRIME” will probably be output, but there is a very
Introduction small chance of the algorithm outputting “PRIME” erroneously.
Alg. Design module prime(x : Integer )
ADTs
var e, y : Integer ;
Sorting
repeat e times
Alg. Theory
Computability
“randomly pick a number y between 1 and x-1”;
Complexity
Correctness
if gcd(x,y) 6= 1
Sorting (ct’d) then {output “NOT PRIME” and halt}
Binary Trees else if y (x−1)/2 mod x 6= J(y , x)
Hashing then {output “NOT PRIME” and halt}
Graphs endif
Backtracking endif
Dynamic
programming
endrepeat
Greedy
{output “PRIME”}
275/ 550
Blockchain endmodule
Analysis of prime(x)
DatAlg
If x is prime then, assuming the mathematical fact given
Maritta Heisel
above, it is easy to see that the algorithm outputs
Introduction
“PRIME”.
Alg. Design
e is a constant such as 20 or 30.
ADTs
Sorting If x is not prime, then each time around the loop there is
Alg. Theory at least a one in two chance that the randomly chosen y
Computability
Complexity
either satisfies GCD(x, y ) 6= 1 or y (x−1)/2 mod x 6= J(y , x).
Correctness
Therefore, after e times around the loop, the chance of
Sorting (ct’d)
Binary Trees
not finding an appropriate y is less than one in 2e .
Hashing If e is 20, the chance is less than one in a million.
Graphs If e is 30, the chance is less than one in a billion.
Backtracking
Therefore, if x is not prime then the algorithm will
Dynamic
programming probably output “NOT PRIME”, the chance of making an
Greedy error being less than one in 2e .
276/ 550
Blockchain
Probabilistic algorithms
DatAlg
Maritta Heisel
Introduction
Alg. Design
This type of algorithm – which may make an error, but
ADTs
very infrequently – is called a probabilistic algorithm.
Sorting
Alg. Theory These algorithms are quite practical, because the constant
Computability
Complexity
e can be adjusted to make the probability of an error as
Correctness
small as desired.
Sorting (ct’d)
If e is around 40, then the algorithm is probably more
Binary Trees
Hashing
reliable than the computer on which it is being executed.
Graphs
Backtracking
Dynamic
programming
Greedy
277/ 550
Blockchain
NP-problems
DatAlg
For the problems described in the previous section, such as
Maritta Heisel
the bin-packing problem, the traveling salesperson
Introduction problem, and the timetabling problem, no polynomial
Alg. Design algorithms are currently known.
ADTs Given arbitrary values of the inputs, or an arbitrary
Sorting instance of the problem, there seems to be no fast method
Alg. Theory
Computability
of finding the solution.
Complexity However, these problems have a very interesting property
Correctness
Sorting (ct’d)
in common:
Binary Trees
Once a solution is found for a particular instance of a
Hashing problem (e.g. a particular timetable which works is
Graphs discovered), it is easy to verify that the solution is correct.
Backtracking More precisely, for each of these problems there exists an
Dynamic algorithm which, given a particular instance of the problem
programming
Greedy
and a proposed solution, can verify in polynomial time
278/ 550
Blockchain
whether or not the solution is correct.
Examples of correct solutions
DatAlg
3 3
Maritta Heisel
Introduction tonnes tonnes

Alg. Design
ADTs 8 tonnes
Sorting
Alg. Theory
Computability
Complexity
Correctness
4 tonnes
Sorting (ct’d)
Binary Trees
Hashing 5 5
Graphs tonnes tonnes
Backtracking
Dynamic
programming
Greedy
279/ 550
Blockchain
NP problems
DatAlg
Maritta Heisel
Introduction
Alg. Design
The set of problems which have a fast verification
ADTs
algorithm as described above is called NP
Sorting
Alg. Theory
(nondeterministic polynomial).
Computability
Complexity
All feasible problems are in NP.
Correctness
NP contains also many of the open problems which
Sorting (ct’d)
different disciplines have been trying to solve for many
Binary Trees
Hashing
years.
Graphs
Backtracking
Dynamic
programming
Greedy
280/ 550
Blockchain
NP-complete problems
DatAlg
Maritta Heisel
NP-complete problems are the hardest problems in NP.
Introduction
If an algorithm which runs in polynomial time were ever
Alg. Design
found for any one of these problems, then there would be
ADTs
Sorting
a polynomial time algorithm for every problem in NP.
Alg. Theory The bin-packing, the traveling salesperson, Hamiltonian
Computability
Complexity
cycle, timetabling and many other problems are each
Correctness
NP-complete.
Sorting (ct’d)
Binary Trees
There are literally thousands of well-known, naturally
Hashing
occurring and practically interesting problems which have
Graphs
been proved to be NP-complete.
Backtracking It follows from the definition of NP-completeness that all
Dynamic
programming
NP-complete problems are computationally equivalent.
Greedy
281/ 550
Blockchain
Infeasibility of NP-complete problems
DatAlg
Maritta Heisel
Introduction
It is widely believed that NP-complete problems are
Alg. Design
infeasible.
ADTs One reason to believe this is that many different people
Sorting have tried to find polynomial algorithms, but all have
Alg. Theory failed.
Computability
Complexity
Correctness
Moreover, we experience the same dichotomy in real life
Sorting (ct’d) between problems which are easy to solve and those which
Binary Trees are hard to solve but whose solution appears obvious once
Hashing it is found.
Graphs
Example: Given green leaves and a plastic sheet, how do
Backtracking
you get water?
Dynamic
programming
Greedy
282/ 550
Blockchain
Old scouting trick for collecting water by
condensation
DatAlg
Maritta Heisel
Introduction Plastic sheet
Alg. Design Condensation about to drip

ADTs
Sorting
Alg. Theory
Computability
Complexity
Hole in ground
Correctness
Sorting (ct’d)
Binary Trees
Hashing
Leaves
Graphs Container
Backtracking
Dynamic
programming
Greedy
283/ 550
Blockchain
Practical benefit of NP-completeness
DatAlg
Maritta Heisel
Introduction
Alg. Design
If one is having difficulty finding a polynomial time
ADTs
algorithm for a problem, then one can alternatively check
Sorting
if the problem is NP-complete.
Alg. Theory In this case, one should seek a method of lowering one’s
Computability
Complexity sights.
Correctness
Sorting (ct’d)
Unfortunately, there are problems for which no fast
Binary Trees algorithm is known, but neither does the problem appear
Hashing so difficult that it can be proved NP-complete.
Graphs An example was the primality problem.
Backtracking
Dynamic
programming
Greedy
284/ 550
Blockchain
Summary
DatAlg
Maritta Heisel
Introduction
Alg. Design
Feasible problems have a fast algorithm for finding their
ADTs
solutions.
Sorting
Alg. Theory Problems in NP may or may not have such an algorithm,

Computability
Complexity
but at least proposed solutions are easy to verify.
Correctness
NP-complete problems are the hardest problems in NP,
Sorting (ct’d)
and it is widely believed that they do not have fast
Binary Trees
Hashing
algorithms.
Graphs
Backtracking
Dynamic
programming
Greedy
285/ 550
Blockchain
What have we learnt about complexity? I
DatAlg
Maritta Heisel Many problems that are computable are nevertheless

Introduction
infeasible, because the algorithms solving the problems use
Alg. Design
too much resources.
ADTs Possible resources are time, memory and hardware.
Sorting
The complexity of an algorithm is the amount of resources
Alg. Theory
Computability
used.
Complexity
Correctness The resources used are expressed in terms of a function on
Sorting (ct’d) the size of the input of an algorithm.
Binary Trees
The asymptotic behavior of an algorithm results from the
Hashing
dominating term of the complexity function. It is often
Graphs
expressed using the big O notation.
Backtracking
Dynamic Polynomial algorithms are often feasible, while exponential

programming
ones are clearly infeasible.
Greedy
286/ 550
Blockchain
What have we learnt about complexity? II
DatAlg
Maritta Heisel
Introduction
Alg. Design We distinguish between worst-case and average-case

ADTs complexity of an algorithm.
Sorting
The complexity of a problem is the complexity of the best
Alg. Theory
Computability algorithm solving the problem.
Complexity
Correctness That algorithm may be unknown.
Sorting (ct’d)
Hence, we consider lower and upper bounds for the
Binary Trees
complexity of a problem.
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
287/ 550
Blockchain
What have we learnt about complexity? III
DatAlg
Maritta Heisel
Introduction
A complexity analysis can be performed by a recursive
Alg. Design procedure considering the syntactic constructs that make
ADTs up an algorithm.
Sorting The divide-and-conquer principle often leads to efficient
Alg. Theory
Computability
algorithms.
Complexity
Correctness The complexity of recursive algorithms is determined by
Sorting (ct’d) solving recurrence relations.
Binary Trees
The sequential computation thesis supports the hypothesis
Hashing
that the class of feasible algorithms are the polynomial
Graphs
algorithms.
Backtracking
Dynamic
programming
Greedy
288/ 550
Blockchain
What have we learnt about complexity? IV
DatAlg
Maritta Heisel Many practically relevant problems are judged to be

Introduction
infeasible.
Alg. Design Among them are the traveling salesperson problem, the
ADTs bin-packing problem, and the timetabling problem.
Sorting
In order to deal with infeasible problems, one can look for
Alg. Theory
Computability approximate solutions, an algorithm which executes quickly
Complexity
Correctness
on the average input data, or a probabilistic algorithm.
Sorting (ct’d)
Binary Trees
The problems in NP are characterized by the fact that a
Hashing proposed solution can be verified in polynomial time.
Graphs NP-complete problems are the hardest ones in NP. If for
Backtracking one of them, a polynomial algorithm is found, then P =
Dynamic
programming
NP.
Greedy
289/ 550
Blockchain
Defects
DatAlg
Maritta Heisel
Introduction
Alg. Design
Most computer programs in current use contain defects.
ADTs
Sorting A significant amount of programmer effort is deployed in

Alg. Theory fixing defects in programs. The process of finding and
Computability
Complexity
correcting defects is referred to as debugging.
Correctness
Methods of producing programs without defects (or with
Sorting (ct’d)
fewer defects) can be grouped into two categories: testing
Binary Trees
Hashing
and proving.
Graphs
Backtracking
Dynamic
programming
Greedy
290/ 550
Blockchain
Testing
DatAlg
Maritta Heisel
Introduction
Alg. Design
Testing a program consists of executing it on a particular
ADTs
set of test data.
Sorting
Alg. Theory The effect of a program is discovered only for the

Computability
Complexity
particular chosen set of test data.
Correctness
Testing has been the debugging technique most
Sorting (ct’d)
emphasized in the past, which partly explains the
Binary Trees
Hashing
proliferation of defects in commercially available programs.
Graphs
Backtracking
Dynamic
programming
Greedy
291/ 550
Blockchain
Proving I
DatAlg
Maritta Heisel
Proving a program correct means certifying its correctness
Introduction
on all permissible input data.
Alg. Design
ADTs Proving correctness involves saying something about a

Sorting program’s overall behavior, rather than about its behavior
Alg. Theory on a limited set of inputs.
Computability
Complexity Even proving the correctness of a program does not leave
Correctness
Sorting (ct’d)
one fully confident that no defects are present, since it is
Binary Trees
possible to make a mistake in the proof.
Hashing Understanding is essential when designing an algorithm,
Graphs and a proof of correctness is just a representation of that
Backtracking understanding. Hence, an algorithm design and a
Dynamic
programming
correctness proof proceed hand in hand.
Greedy
292/ 550
Blockchain
Proving II
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs The confidence in a program’s correctness increases with

Sorting the formality of the correctness proof.
Alg. Theory
Computability An extremely formal proof may be important in
Complexity
Correctness
particularly critical applications involving human life or
Sorting (ct’d) health, e.g., monitoring patients in an intensive care unit.
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
293/ 550
Blockchain
Definitions of correctness
DatAlg
Maritta Heisel
Introduction
Alg. Design
Partial correctness The produced result of an algorithm is the
ADTs one expected whenever the algorithm terminates
Sorting (or performs an output operation). Note that if
Alg. Theory an algorithm is partially correct, it may not
Computability
Complexity
terminate or produce a result in every
Correctness
circumstance. The only guarantee is that if ever a
Sorting (ct’d)
result is produced, that result is correct.
Binary Trees
Hashing
Total correctness An algorithm which always terminates, and
Graphs which is partially correct.
Backtracking
Dynamic
programming
Greedy
294/ 550
Blockchain
Example: proof by induction I
DatAlg
Consider the following algorithm (assuming x ≥ 0):
Maritta Heisel
Introduction
module exponentiate(x : Integer) : Integer
Alg. Design
var sum : Integer ;
ADTs sum := 1 ;
Sorting repeat x times
Alg. Theory sum := sum + sum
Computability
Complexity endrepeat;
Correctness
return sum
Sorting (ct’d)
Binary Trees
endmodule
Hashing In this simple case, it is sufficient to realize that after the nth
Graphs
execution of the loop body, sum will equal 2n (the condition
Backtracking
sum = 2n is called a loop invariant). It is then straightforward
Dynamic
programming to see that, provided the return statement is reached, 2x will be
Greedy the output.
295/ 550
Blockchain
Example: proof by induction II
DatAlg
The previous text is an adequate informal proof of the
Maritta Heisel
algorithm’s partial correctness, and it is trivial to see that the
Introduction
algorithm also always terminates.
Alg. Design
Documentation should be added to help readers of the
ADTs
algorithm as follows:
Sorting
module exponentiate(x : Integer) : Integer
Alg. Theory
Computability {pre x ≥ 0; post exponentiate(x) = 2x }
Complexity
Correctness
var sum : Integer ;
Sorting (ct’d) sum := 1 ;
Binary Trees repeat x times
Hashing
sum := sum + sum
Graphs
{The nth time this point is reached, sum = 2n holds}
Backtracking
endrepeat;
Dynamic
programming return sum
Greedy endmodule
296/ 550
Blockchain
Example: proof by induction III
DatAlg
Maritta Heisel
Introduction
Proof by induction:
Alg. Design 1. Before the loop is executed for the first time, sum = 1
ADTs (which equals 20 ).
Sorting
Alg. Theory
2. The first time the loop body is executed, sum = 2 (which
Computability
Complexity
equals 21 ).
Correctness
Sorting (ct’d)
3. Assume that the nth time the loop body is executed,
Binary Trees
sum = 2n . Then the (n + 1)th time,
Hashing sum = 2n + 2n = 2n+1 .
Graphs 4. Therefore, for every n, the nth time the loop body is
Backtracking
executed, sum = 2n .
Dynamic
programming
Greedy
297/ 550
Blockchain
Example: proof by induction IV
DatAlg
Maritta Heisel
Introduction
Alg. Design
5. If the return statement is ever reached, there are two
ADTs
Sorting
possibilities. Either the loop was never executed, in which
Alg. Theory
case x = 0, and so the initial value of sum was never
Computability changed. Therefore, sum = 1 (which equals 20 ). Or the
Complexity
Correctness loop was executed, in which case the loop body was
Sorting (ct’d) executed x times. So sum = 2x .
Binary Trees
6. In either case, if the output statement is reached, 2x will
Hashing
Graphs
be the output.
Backtracking
Dynamic
programming
Greedy
298/ 550
Blockchain
Induction I
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs Induction is useful when it is desired to prove that some

Sorting
statement is true in every case under consideration.
Alg. Theory
Computability The idea is to prove the statement for a small case, and
Complexity
Correctness then show that whenever the statement is true for a
Sorting (ct’d) particular case, it is also true for the next larger case. It
Binary Trees follows that the statement is true in every case.
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
299/ 550
Blockchain
Induction II
DatAlg
Maritta Heisel
Introduction
Alg. Design The statement to be proved (step 4 in our example) is

ADTs called the inductive hypothesis.
Sorting
The first step (steps 1 or 2 in our example) is called the
Alg. Theory
Computability basis, because it proves the base case.
Complexity
Correctness
The second step (step 3 in our example) is called the
Sorting (ct’d)
inductive step, which allows one to step forward from the
Binary Trees
basis to progressively larger cases.
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
300/ 550
Blockchain
Induction III
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
General format for a proof by induction:
Sorting
Inductive hypothesis The nth time the loop body is executed,
Alg. Theory
Computability
sum = 2n .
Basis When n = 1, sum is set to 1 + 1 which equals 21 .
Complexity
Correctness
Sorting (ct’d) or
Binary Trees When n = 0, sum is set to 1 which equals 20 .
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
301/ 550
Blockchain
Induction IV
DatAlg
Inductive step Assume the inductive hypothesis is true for a
Maritta Heisel
particular value of n. The (n + 1)th the loop
Introduction body is executed, sum is set equal to the previous
Alg. Design
value of sum added to itself. By the inductive
ADTs
hypothesis, the previous value of sum is 2n .
Sorting
Therefore the new value is 2n + 2n = 2n+1 . So
Alg. Theory
Computability the inductive hypothesis also holds for n + 1.
Complexity
Correctness Conclusion From the basis and the inductive step it follows
Sorting (ct’d) that the inductive hypothesis holds for every
Binary Trees value of n ≥ 1 or n ≥ 0.
Hashing
Graphs
Note: One has to choose an appropriate base case. For the
Backtracking
natural numbers, this is usually n = 0. In our example, n = 1 is
Dynamic
programming also possible. Then, the case that the loop is not executed at
Greedy all must be treated separately (see step 5 of the example).
302/ 550
Blockchain
Proof by induction: gcd algorithm I
DatAlg
Maritta Heisel
Recall: gcd algorithm
Introduction
Alg. Design module gcd(x, y : Integer)

ADTs {pre x ≥ 0 ∧ y ≥ 0 ∧ x = x 0 ∧ y = y 0 }
Sorting {post r = GCD(x 0 , y 0 )}
Alg. Theory var r : Integer ;
Computability
Complexity while y 6= 0 do
Correctness
r := x mod y ;
Sorting (ct’d)
x := y ;
Binary Trees
Hashing
y := r
Graphs
endwhile;
Backtracking r := x
Dynamic endmodule
programming
Greedy
303/ 550
Blockchain
Proof by induction: gcd algorithm II
DatAlg
Maritta Heisel
Introduction
Alg. Design In order to prove the correctness of the gcd algorithm, we also
ADTs have to consider the corresponding specification:
Sorting
var x, y , x 0 , y 0 , r : Integer
Alg. Theory
Computability pre x ≥ 0 ∧ y ≥ 0 ∧ x = x0 ∧ y = y0
r = GCD(x 0 , y 0 )
Complexity
Correctness post
Sorting (ct’d) reads -
Binary Trees changes r , x, y
Hashing mem x 0, y 0
Graphs
Backtracking
Dynamic
programming
Greedy
304/ 550
Blockchain
Proof by induction: gcd algorithm III
DatAlg
Maritta Heisel
Introduction
Alg. Design
We can now prove the correctness of the gcd algorithm with
ADTs respect to its specification using induction:
Sorting
Alg. Theory Inductive hypothesis:

Computability
Complexity
After n executions of the loop body, GCD(x, y ) = GCD(x 0 , y 0 ).
Correctness
Sorting (ct’d)
Basis: n = 0
After 0 executions of the loop body, x = x 0 ∧ y = y 0 still
Binary Trees
Hashing
holds, and thus the inductive hypothesis holds.
Graphs
Backtracking
Dynamic
programming
Greedy
305/ 550
Blockchain
Proof by induction: gcd algorithm IV
DatAlg
Maritta Heisel
Inductive step:
Assume g = GCD(x 0 , y 0 ) = GCD(x, y ) holds after n executions
Introduction
Alg. Design
of the loop body.
ADTs
Sorting For any two numbers x and y , there are two numbers x̄ and ȳ
Alg. Theory that have no common factors such that x = ḡ x̄ and y = ḡ ȳ for
Computability
Complexity ḡ = GCD(x, y ). By the inductive hypothesis, we can conclude
Correctness
x = g x̄ and y = g ȳ .
Sorting (ct’d)
Binary Trees Now suppose x = my + r , so that r is the remainder when x is

Hashing divided by y .
Graphs Then g x̄ = mg ȳ + r , so r = g (x̄ − mȳ ), and thus r must have
Backtracking g as a factor.
Dynamic
programming
Greedy
306/ 550
Blockchain
Proof by induction: gcd algorithm V
DatAlg
Maritta Heisel
The equation g x̄ = mg ȳ + r also implies that r /g cannot
Introduction have any factor in common with ȳ , or else x̄ would have the
Alg. Design same factor, which would contradict the fact that x̄ and ȳ have
ADTs no common factors.
Sorting
Thus y and r have g as their greatest common factor. Since x
Alg. Theory
Computability is set to y and y is set to r , the new values of x and y have g
Complexity
Correctness as their greatest common factor. Therefore,
Sorting (ct’d) GCD(x 0 , y 0 ) = GCD(x, y ) holds after n + 1 executions of the
Binary Trees loop body.
Hashing
Graphs Conclusion
Backtracking The inductive hypothesis holds for every value for n ≥ 0, i.e.
Dynamic
programming
before and after every loop body execution.
Greedy
307/ 550
Blockchain
Proof by induction: gcd algorithm VI
DatAlg
Maritta Heisel
Introduction
Alg. Design
Completion of the proof
ADTs
Sorting After the last execution of the loop body, y = 0 holds. Because
Alg. Theory of GCD(x, 0) = x, we know that x = g , such that after the
Computability
Complexity
assignment r := x the postcondition holds.
Correctness
Sorting (ct’d)
This proof demonstrates the partial correctness of the gcd
Binary Trees
algorithm.
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
308/ 550
Blockchain
Proof by induction: 91 algorithm I
DatAlg
Consider McCarthy’s 91 algorithm:
Maritta Heisel
Introduction
var x : Integer
Alg. Design
pre x ≥ 0
ADTs
Sorting
post (x > 100 ⇒ f(x) = x−10) ∧ (x ≤ 100 ⇒ f(x) = 91)
Alg. Theory
reads x
Computability
Complexity
changes −
Correctness mem −
Sorting (ct’d)
Binary Trees
module f(x : Integer) : Integer
Hashing
if x > 100
Graphs
then return x−10
Backtracking
else return f(f(x+11))
Dynamic
programming endif
Greedy endmodule
309/ 550
Blockchain
Proof by induction: 91 algorithm II
DatAlg We can now prove the correctness of the 91 algorithm with

Maritta Heisel respect to its specification using induction:
Introduction
Inductive hypothesis If x > 100, the algorithm returns x − 10,
Alg. Design
otherwise 91.
ADTs
Sorting
Basis If x > 100, clearly the answer is x − 10. If
Alg. Theory
x = 100, the answer is f (f (111)) = f (101) = 91.
Computability
Complexity
Inductive step Suppose x < 100 and assume the inductive
Correctness
hypothesis holds for all numbers greater than x.
Sorting (ct’d)
There are two cases:
Binary Trees
Hashing
1. if x + 11 > 100 then
Graphs
f (x) = f (f (x + 11)) = f (x + 1) which
Backtracking equals 91 by the inductive hypothesis.
Dynamic (Recall that we are assuming the inductive
programming
hypothesis holds for all numbers greater than
Greedy
310/ 550
x, and x + 1 > x.)
Blockchain
Proof by induction: 91 algorithm III
DatAlg
Maritta Heisel
2. if x + 11 ≤ 100 then
Introduction f (x) = f (f (x + 11)) = f (91) by the
Alg. Design inductive hypothesis (which applies because
ADTs
x + 11 > x). Since 91 > x, we can apply the
Sorting
inductive hypothesis again, giving
Alg. Theory
Computability
f (91) = 91.
Complexity
Correctness Therefore the inductive hypothesis holds for x as
Sorting (ct’d) well.
Binary Trees
Conclusion The inductive hypothesis holds for any
Hashing
permissible value of x.
Graphs
Backtracking This proof demonstrates the partial correctness of the 91

Dynamic algorithm.
programming
Greedy
311/ 550
Blockchain
Termination I
DatAlg
Maritta Heisel
Introduction
Alg. Design For a proof of total correctness, we must prove

ADTs termination.
Sorting
We know from the section about computability that there
Alg. Theory
Computability is no mechanical method of deciding this issue.
Complexity
Correctness A fundamental technique for understanding why a loop
Sorting (ct’d) terminates is to observe some value which decreases (or
Binary Trees
increases) significantly every time the loop is executed and
Hashing
is bounded from below (or above).
Graphs
Backtracking
Dynamic
programming
Greedy
312/ 550
Blockchain
Termination II
DatAlg
Maritta Heisel
Recall the loop contained in the gcd algorithm:
Introduction
Alg. Design while y 6= 0 do

ADTs r := x mod y ;
Sorting x := y ;
Alg. Theory y := r
Computability
Complexity endwhile
Correctness
Sorting (ct’d)
In fact, y decreases by at least 1 each time around the loop.
Binary Trees
This is true because y is set to the remainder of something
Hashing
Graphs
divided by y . The remainder is a number between 0 and y − 1.
Backtracking
Thus y is set equal to a value between 0 and y − 1, and
Dynamic
therefore y eventually becomes 0 and the loop terminates.
programming
Greedy
313/ 550
Blockchain
Termination: Ackermann function I
DatAlg
Maritta Heisel Consider this algorithm calculating the Ackermann function:

Introduction
module A(x, y : Integer) : Integer
Alg. Design
{pre x ≥ 0 ∧ y ≥ 0}
ADTs
if x = 0
Sorting
Alg. Theory
then return y + 1
Computability else if y = 0
Complexity
Correctness then return A(x − 1, 1)
Sorting (ct’d) else return A(x − 1, A(x, y − 1))
Binary Trees endif
Hashing endif
Graphs
endmodule
Backtracking
Dynamic
programming What is the result of A(2, 2) and A(3, 1)?
Greedy
314/ 550
Blockchain
Termination: Ackermann function II
DatAlg
Maritta Heisel
Our aim is to find something that is steadily decreasing,
Introduction because when x is 0, no recursive call is made.
Alg. Design
Note that on two of the recursive calls, x is decreased by
ADTs
1, so progress is being made.
Sorting
Alg. Theory
On the remaining recursive call, x is unchanged but y is
Computability decreased by 1. This represents progress, too, albeit slow
Complexity
Correctness progress, because when y eventually reaches 0 the
Sorting (ct’d) recursive call A(x − 1, 1) causes x to be decreased.
Binary Trees
Thus all three recursive calls either immediately decrease
Hashing
Graphs
x, or cause x to be decreased eventually.
Backtracking Either way, the algorithm steadily grinds towards its
Dynamic terminating condition, x = 0.
programming
Greedy
315/ 550
Blockchain
What have we learnt about correctness?
DatAlg
Maritta Heisel A correctness proof provides more confidence in the

Introduction
correctness of an algorithm than testing. It embodies the
Alg. Design comprehension of how the algorithm solves its problem.
ADTs Total correctness consists of partial correctness and
Sorting termination.
Alg. Theory
Computability Iteration and recursion are the difficult constructs in
Complexity
Correctness
proving correctness. They are treated by induction.
Sorting (ct’d) Induction needs an inductive hypothesis, a base case, and
Binary Trees an inductive step.
Hashing
For loops, the inductive hypothesis is called loop invariant.
Graphs
Backtracking The termination of an algorithm is proved by showing that

Dynamic some expression that is bounded from below strictly
programming
decreases with each iteration or recursive call.
Greedy
316/ 550
Blockchain
Fast sorting algorithms
DatAlg
Maritta Heisel
Introduction The sorting algorithms discussed so far (namely, bubble

Alg. Design
sort, insertion sort and selection sort) are of quadratic
ADTs
time complexity.
Sorting
Alg. Theory
This is too slow for large quantities of data to be sorted.
Sorting (ct’d) We will discuss three faster sorting algorithms, namely
Mergesort
Quicksort
Mergesort
Heapsort Quicksort
Binary Trees Heapsort
Hashing
Mergesort and Quicksort are divide-and-conquer
Graphs
algorithms.
Backtracking
Dynamic
programming
Greedy
317/ 550
Blockchain
Divide-and-conquer algorithms: basic idea
DatAlg
Maritta Heisel
Introduction Problem solving proceeds as follows:

Alg. Design
ADTs
1. Divide the problem into several subproblems of similar type
Sorting 2. Solve the subproblems independently from each other
Alg. Theory
3. Compose the solutions of the subproblems to the solution
Sorting (ct’d)
Mergesort
of the overall problem.
Quicksort
Heapsort
Binary Trees Often the problem is divided into two subproblems, which are
Hashing
solved using the same algorithm as the original problem.
Graphs
Hence, divide-and-conquer algorithms are inherently recursive.
Backtracking
Dynamic
programming
Greedy
318/ 550
Blockchain
Divide-and-conquer algorithms: algorithm schema
DatAlg
Maritta Heisel module d&c(p)

Introduction
if primitive(p) then
Alg. Design
ADTs
directly solve(p)
Sorting else divide(p);
Alg. Theory
d&c(p1 ); . . . ; d&c(pn );
Sorting (ct’d)
Mergesort
Quicksort
compose(p1 , . . . , pn )
Heapsort
Binary Trees
endif
Hashing endmodule
Graphs
Backtracking Often, one can choose a simple decomposition algorithm or a

Dynamic
programming
simple composition algorithm. The specification of the other
Greedy algorithm is determined by the choice of the simple algorithm.
319/ 550
Blockchain
Divide-and-conquer schema for sorting algorithms
DatAlg
Maritta Heisel
Introduction
Alg. Design
1. Divide the given array into two parts
ADTs
Sorting 2. Sort the two parts separately

Alg. Theory 3. Compose the sorted parts in such a way that the
Sorting (ct’d)
Mergesort
composed array is sorted
Quicksort
Heapsort
Binary Trees Mergesort uses a simple decomposition algorithm, Quicksort

Hashing uses a simple composition algorithm.
Graphs
Backtracking
Dynamic
programming
Greedy
320/ 550
Blockchain
Mergesort: basic idea
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Simple decomposition algorithm: cut the array in two
Alg. Theory halves of (roughly) equal length.
Sorting (ct’d) Sort the two halves.
Mergesort
Quicksort
Heapsort
Merge the sorted halves together.
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
321/ 550
Blockchain
Mergesort: algorithm
DatAlg
Maritta Heisel
module mergesort(a : ARRAY (1..n)[T ]; l, r : Integer )
Introduction
var m : Integer ;
Alg. Design
ADTs if l < r then

Sorting
m := (l + r ) div 2;
Alg. Theory
Sorting (ct’d) mergesort(a, l, m);

Mergesort
Quicksort mergesort(a, m + 1, r );
Heapsort
Binary Trees merge(a, l, m, r )

Hashing
endif
Graphs
Backtracking
endmodule;
Dynamic
programming
Initial call: mergesort(a, 1, n).
Greedy
322/ 550
Blockchain
Merge: specification
DatAlg
Maritta Heisel
Introduction
var l, m, r : Integer ; a, a1 : ARRAY (1..n)[T ]
Alg. Design
ADTs
pre l ≤ m < r ∧ a = a1 ∧
Sorting
Alg. Theory
ordered(a[l..m]) ∧ ordered(a[m + 1..r ])
Sorting (ct’d) post ordered(a[l..r ]) ∧ permutation(a, a1)
Mergesort
Quicksort
Heapsort
reads l, m, r
Binary Trees
changes a
Hashing
Graphs mem a1
Backtracking
Dynamic
programming
Greedy
323/ 550
Blockchain
Merge: basic idea
DatAlg
Maritta Heisel
Introduction
1. Use an auxiliary array b to temporarily store the sorted
Alg. Design
array.
ADTs
Sorting 2. Use a pointer i, that runs through a from the beginning

Alg. Theory and a pointer j, that runs through a from the middle + 1.
Sorting (ct’d) If a[i] ≤ a[j], copy a[i] to b, otherwise a[j].
Mergesort
Quicksort 3. Stop, if i > m or j > r .
Heapsort
Binary Trees 4. Fill b with remaining elements of second half that has not
Hashing been completely traversed.
Graphs 5. Copy array b to a.
Backtracking
Dynamic
programming
Greedy
324/ 550
Blockchain
Merge: algorithm [OW02]
DatAlg
module merge(a : ARRAY (1..n)[T ]; l, m, r : Integer )
Maritta Heisel
var i, j, k, h : Integer ; b : ARRAY (1..n)[T ];
Introduction
Alg. Design
i := l; j := m + 1; k := l;
ADTs while i ≤ m ∧ j ≤ r do
Sorting
if a[i] <= a[j] then
Alg. Theory
Sorting (ct’d) b[k] := a[i]; i := i + 1

Mergesort
Quicksort else
Heapsort
Binary Trees b[k] := a[j]; j := j + 1

Hashing
endif;
Graphs
Backtracking k := k + 1
Dynamic
programming
endwhile;
Greedy
325/ 550
Blockchain
Merge: algorithm (ct’d)
DatAlg
Maritta Heisel
if i > m then
Introduction
while j <= r do
Alg. Design
ADTs b[k] := a[j]; j := j + 1; k := k + 1

Sorting endwhile
Alg. Theory
Sorting (ct’d)
else
Mergesort
Quicksort
while i <= m do
Heapsort
Binary Trees
b[k] := a[i]; i := i + 1; k := k + 1
Hashing endwhile
Graphs
endif
Backtracking
Dynamic
programming
Greedy
326/ 550
Blockchain
Merge: algorithm (ct’d)
DatAlg
Maritta Heisel
Introduction
Alg. Design h := l;
ADTs
Sorting
while h ≤ r do
Alg. Theory a[h] := b[h]; h := h + 1
Sorting (ct’d)
Mergesort
endwhile
Quicksort
Heapsort endmodule
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
327/ 550
Blockchain
Properties of Mergesort
DatAlg
Maritta Heisel
Introduction
Alg. Design Merge has linear complexity.

ADTs
Since for each recursive call, the size of the input is
Sorting
divided by two, the entire complexity of Mergesort is of
Alg. Theory
order n log n (log n meaning the logarithm of n using the
Sorting (ct’d)
Mergesort
base 2).
Quicksort
Heapsort Mergesort is stable.
Binary Trees
Mergesort has the same complexity in the minimal, the
Hashing
average, and the worst case. Hence, it is not really natural.
Graphs
Backtracking
Dynamic
programming
Greedy
328/ 550
Blockchain
Quicksort
DatAlg
Maritta Heisel
Introduction
Alg. Design
Invented by Tony Hoare.
ADTs
Sorting
Simple composition algorithm: identity.
Alg. Theory Decomposition algorithm: the array must be divided in
Sorting (ct’d) such a way that after sorting both parts the entire array is
Mergesort
Quicksort sorted.
Heapsort
Binary Trees
This is the case if all elements of the first part are smaller
Hashing than or equal to all elements of the second part.
Graphs
Backtracking
Dynamic
programming
Greedy
329/ 550
Blockchain
Quicksort: algorithm
DatAlg
Maritta Heisel
Introduction
module quicksort(a : ARRAY (1..n)[T ]; i, j : Integer )
Alg. Design var p : Integer ;
ADTs
if i < j then
Sorting
Alg. Theory partition(a, i, j, p); {sets p}

Sorting (ct’d)
Mergesort
quicksort(a, i, p − 1);
Quicksort
Heapsort quicksort(a, p + 1, j);
Binary Trees
endif
Hashing
Graphs endmodule
Backtracking
Dynamic Initial call: quicksort(a, 1, n).

programming
Greedy
330/ 550
Blockchain
Partition: basic idea
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs 1. choose any element of a (e.g. the last one a[n])

Sorting
2. partition a in parts, where the first partial array only
Alg. Theory
contains elements of a that are smaller than a[n]. The
Sorting (ct’d)
Mergesort
second partial array only contains elements that are larger
Quicksort
Heapsort
than or equal to a[n], and a[n] is shifted to its final
Binary Trees position.
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
331/ 550
Blockchain
Partition: specification
DatAlg
Maritta Heisel
Introduction
The array a[i..j] is partitioned with respect to a[j].
Alg. Design
ADTs
var i, j, n, p : Integer ; a, a1 : ARRAY (1..n)[T ]
Sorting
pre 1 ≤ i < j ≤ n ∧ a = a1
Alg. Theory
post i ≤ p ≤ j ∧ permutation(a, a1) ∧
Sorting (ct’d)
Mergesort a[p] = a1[j] ∧
Quicksort
Heapsort
a[i . . . p − 1] < a[p] ∧ a[p] ≤ a[p + 1 . . . j]
Binary Trees reads n, i, j
Hashing changes a, p
Graphs mem a1
Backtracking
Dynamic
programming
Greedy
332/ 550
Blockchain
Partition: algorithm
DatAlg
module partition(a : ARRAY (1..n)[T ]; i, j, p : Integer )
Maritta Heisel var x, h : T ; q : Integer ;
Introduction x := a[j]; p := i; q := j − 1;
Alg. Design
while p ≤ q do
ADTs
Sorting if a[p] < x then p := p + 1

Alg. Theory
else if a[q] ≥ x
Sorting (ct’d)
Mergesort then q := q − 1
Quicksort
Heapsort
else h := a[p]; a[p] := a[q]; a[q] := h;
Binary Trees
Hashing p := p + 1; q := q − 1 endif
Graphs endif
Backtracking
Dynamic
endwhile;
programming
a[j] := a[p]; a[p] := x
Greedy
333/ 550
Blockchain endmodule
Execution example for partition
DatAlg
Maritta Heisel state a[i] a[i + 1] a[i + 2] a[i + 3] a[i + 4] a[i + 5] comment
= a[j]
Introduction Init 44 55 12 42 94 18
↑ ↑ ↑ a[p] ≮ x ∧ a[q] ≥ x
Alg. Design p q x
1st cyc. ↑ a[p] ≮ x ∧ a[q] ≥ x
ADTs
q
Sorting 2nd cyc. ↑ a[p] ≮ x ∧ a[q] ≥ x
q
Alg. Theory 3rd cyc. 12 55 44 42 94 18 a[p] ≮ x ∧ a[q] x
↑
Sorting (ct’d) p, q
Mergesort 4th cyc. ↑ ↑ a[p] ≮ x ∧ a[q] ≥ x
Quicksort q p
Heapsort term. 12 18 44 42 94 55
< 18 ↑ ≥ 18
Binary Trees p
Hashing
Graphs
Note that the two parts generated by partition need not have
Backtracking
Dynamic
the same size!
programming
Greedy
334/ 550
Blockchain
Properties of Quicksort
DatAlg
Maritta Heisel The complexity of partition is linear.

Introduction The complexity of quicksort hence depends on the number
Alg. Design of recursive calls that are necessary.
ADTs
The worst case occurs when the array is already sorted.
Sorting
Then, there are n − 1 recursive calls, and the overall
Alg. Theory
complexity is quadratic.
Sorting (ct’d)
Mergesort
Quicksort
The minimal complexity occurs, when partition always
Heapsort cuts the array in two equal halves. Then the complexity is
Binary Trees
of order n log n.
Hashing
Graphs
The same holds for the average case. Therefore, Quicksort
Backtracking
is considered to be a fast sorting algorithm, even if its
Dynamic
complexity is quadratic in the worst case.
programming
Greedy
Quicksort is not natural. Question: is it stable? Exercise!
335/ 550
Blockchain
Heapsort
DatAlg
Maritta Heisel
Introduction
Alg. Design Non-recursive algorithm

ADTs
Follows the principle of selection sort, but selecting the
Sorting
maximal element takes less than linear time
Alg. Theory
Sorting (ct’d)
Uses a special data structure, called heap
Mergesort
Quicksort
A heap is a binary tree (i.e., a data structure where each
Heapsort
element has at most two successors of the same type),
Binary Trees
where each node is greater than or equal to both of its
Hashing
successor nodes, which is repesented as an array.
Graphs
Backtracking
Dynamic
programming
Greedy
336/ 550
Blockchain
Heap: example
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Mergesort
Quicksort
Heapsort
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
linear representation in an array:
programming Values 8 6 7 3 4 5 2 1
Greedy Position a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8]
337/ 550
Blockchain
Heap: definition
DatAlg
Maritta Heisel
An array a[1. . . n] is thus a heap if and only if:
Introduction
Alg. Design
a[i] ≤ a[i div 2] for 2 ≤ i ≤ n
ADTs
Sorting or equivalently
Alg. Theory
Sorting (ct’d) a[i] ≥ a[2i] ∧ a[i] ≥ a[2i + 1]

Mergesort
Quicksort
Heapsort for 2i ≤ n or 2i + 1 ≤ n, respectively, because the successors of
Binary Trees the element stored in a[i] (as far as these exist) are a[2i] and
Hashing a[2i + 1].
Graphs
Backtracking
If an array is a heap then the maximum element is always the
Dynamic
first. It will have the last position in a sorted array.
programming
Greedy
338/ 550
Blockchain
Using a heap for sorting
DatAlg
Problem: How can the next smaller element be found?
Maritta Heisel
Idea: Remove a[1] from heap, restore heap property
Introduction
In our example we had:
Alg. Design
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Mergesort
Quicksort
Heapsort
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
339/ 550
Blockchain
Now only two partial heaps remain.
Restoring the heap property
DatAlg
First write the element with the highest index into the root.
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Mergesort
Quicksort
Heapsort
Binary Trees
Hashing
Graphs
Backtracking
This generally destroys the heap property. In order to restore it,
Dynamic let a[1] sift in the heap, which means to exchange the element
programming
with its larger successor until both sucessors are smaller or
Greedy
none remains.
340/ 550
Blockchain
Sifting: example
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Mergesort
Quicksort
Heapsort
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
341/ 550
Blockchain
Heap predicate: definition
DatAlg
Maritta Heisel
Introduction
Alg. Design heap(a, l, r ) ⇔

ADTs
∀ k : Integer •
Sorting
Alg. Theory ((l ≤ k ∧ 2k ≤ r ⇒ a[k] ≥ a[2k])

Sorting (ct’d)
Mergesort
∧
Quicksort
Heapsort (l ≤ k ∧ 2k + 1 ≤ r ⇒ a[k] ≥ a[2k + 1]))
Binary Trees
Hashing The heap property for a must therefore hold between the
Graphs boundaries l and r .
Backtracking
Dynamic
programming
Greedy
342/ 550
Blockchain
Procedure sift: specification
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs var l, r , n : Integer ; a, a1 : ARRAY (1..n)[T ]

Sorting pre 1 ≤ l ∧ heap(a, l + 1, r ) ∧ r ≤ n ∧ a = a1
Alg. Theory post heap(a, l, r ) ∧ permutation(a, a1)
Sorting (ct’d) reads l, r , n
Mergesort
Quicksort changes a
Heapsort
mem a1
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
343/ 550
Blockchain
Procedure sift: algorithm
DatAlg module sift(a : ARRAY (1..n)[T ]; l, r : Integer )

Maritta Heisel var k, j : Integer ; h : T ;
Introduction k := l;
Alg. Design
while ¬ ((2k > r cor a[k] ≥ a[2k]) ∧
ADTs
Sorting (2k + 1 > r cor a[k] ≥ a[2k + 1])) do

Alg. Theory
j := 2 ∗ k; {index of left successor}
Sorting (ct’d)
Mergesort if j < r cand a[j] < a[j + 1] then
Quicksort
Heapsort
j := j + 1 {determine larger successor}
Binary Trees
Hashing endif;
Graphs
h := a[k]; a[k] := a[j]; a[j] := h; k := j
Backtracking
Dynamic
endwhile
programming
endmodule
Greedy
344/ 550
Blockchain
Using sift for sorting
DatAlg
Maritta Heisel
Introduction How can we use the procedure sift for sorting?

Alg. Design
1. Transform a into a heap.
ADTs
Note that heap(a, n div 2 + 1, n) holds trivially, because in
Sorting
the second half of the array only the leaves of the tree are
Alg. Theory
contained.
Sorting (ct’d)
Mergesort Hence, let a[m], a[m − 1], . . . , a[1] (where m = n div 2)
Quicksort
Heapsort consecutively sift into the heap.
Binary Trees 2. Since the first element of the heap is the maximum,
Hashing
exchange this with the last element of the not yet sorted
Graphs
partial array, and let that element sift.
Backtracking
Dynamic
programming
Greedy
345/ 550
Blockchain
Heapsort: algorithm
DatAlg
var i, n : Integer ; h : T ; a : ARRAY (1..n)[T ];
Maritta Heisel i := n div 2;
Introduction while i ≥ 1 do
Alg. Design
sift(a, i, n);
ADTs
Sorting i := i − 1
Alg. Theory
endwhile;
Sorting (ct’d)
Mergesort {a is a heap}
Quicksort
Heapsort
i := n;
Binary Trees
Hashing while i 6= 1 do
Graphs h := a[i]; a[i] := a[1]; a[1] := h;
Backtracking
Dynamic
i := i − 1;
programming
sift(a, 1, i)
Greedy
346/ 550
Blockchain endwhile
Properties of Heapsort
DatAlg
Maritta Heisel
Procedure sift has constant complexity in the minimal case
Introduction (when the sifting element is the maximum of a[l . . . r ]).
Alg. Design
In the maximal case, the complexity is of order log n,
ADTs
because in each cycle, the loop lets the sifting element
Sorting
move one level deeper in the tree, and a balanced binary
Alg. Theory
tree of n elements has log n levels (this will be shown in
Sorting (ct’d)
Mergesort more detail when we discuss balanced trees).
Quicksort
Heapsort The maximal complexity of heapsort is thus proportional
Binary Trees to n log n, and only a constant amount of additional
Hashing
storage is used. Heapsort is however not natural, since
Graphs
large elements are first moved to the left and then to the
Backtracking
right, so that heapsort needs the least resources when the
Dynamic
programming array is sorted in reverse order.
Greedy
347/ 550
Blockchain
What have we learnt on sorting algorithms?
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs You should now be able to

Sorting
Name and explain the sorting algorithms we have discussed
Alg. Theory
Apply the different sorting algorithms to an example
Sorting (ct’d)
Mergesort
Quicksort
Explain the complexity of the different algorithms
Heapsort
Explain the divide-and-conquer principle
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
348/ 550
Blockchain
Trees: basic notions I
DatAlg
Maritta Heisel We now introduce another important data type, namely trees.
Introduction A tree consists of a set of nodes having a finite set of
Alg. Design successors.
ADTs
A list is a degenerated tree: all nodes have at most one
Sorting
successor.
Alg. Theory
Sorting (ct’d) There is exactly one node in a tree having no predecessor

Binary Trees (if x is successor of y , then y is predecessor of x). That
Searchtrees
Balanced trees
node is called the root of the tree.
Hashing Nodes having no successor are called leaves.
Graphs
All other nodes are called inner nodes.
Backtracking
Dynamic A sequence of nodes k1 . . . kn , where each ki+1 is

programming
successor of ki , is called path (of length n).
Greedy
Blockchain
349/ 550
Trees: basic notions II
DatAlg
Maritta Heisel
Introduction Each node is connected to the root by exactly one path.

Alg. Design The length of that path is called the depth of the node in
ADTs
the tree.
Sorting
Alg. Theory
The length of the longest path is the height of the tree.
Sorting (ct’d) A subtree of a tree is the partial structure that is obtained
Binary Trees by choosing some node of the tree as the root and adding
Searchtrees
Balanced trees all direct and indirect successors of that node.
Hashing
A tree is balanced if for each node the heights of its
Graphs
subtrees differ by at most one.
Backtracking
Dynamic
programming
Greedy
Blockchain
350/ 550
Trees: basic notions III
DatAlg
Maritta Heisel
The degree of a node is the number of its direct
Introduction
successors. The degree of a tree is the maximum of the
Alg. Design
degrees of its nodes.
ADTs
Sorting Trees of degree two are especially important: they are

Alg. Theory called binary trees. All other trees (i.e. of degree greater
Sorting (ct’d) than two) are called Multi-way trees (M-way trees).
Binary Trees
Searchtrees
Trees where the order of their subtrees is of importance
Balanced trees
are called ordered. We will only consider ordered trees.
Hashing
Graphs Convention: trees are drawn in such a way that the root is at
Backtracking the top and the leaves are at the bottom (see heapsort
Dynamic examples).
programming
Greedy
Blockchain
351/ 550
ADT specification of binary trees I
DatAlg
Maritta Heisel
Types BINTREE [T ]
Alg. Design mt tree : BINTREE [T ]
ADTs make : BINTREE [T ] × T × BINTREE [T ] → BINTREE [T ]
Sorting
Alg. Theory left : BINTREE [T ] 9 BINTREE [T ]

Sorting (ct’d) right : BINTREE [T ] 9 BINTREE [T ]
Binary Trees
Searchtrees
data : BINTREE [T ] 9 T
Balanced trees
is mt : BINTREE [T ] → Bool
Hashing
Graphs
is in : T × BINTREE [T ] → Bool
Backtracking
height : BINTREE [T ] → Integer
Dynamic
balanced : BINTREE [T ] → Bool
programming
subtrees : BINTREE [T ] → SET [BINTREE [T ]]
Greedy
Blockchain
352/ 550
ADT specification of binary trees II
DatAlg
Maritta Heisel
Introduction preorder traversal : BINTREE [T ] → LIST [T ]

Alg. Design
inorder traversal : BINTREE [T ] → LIST [T ]
ADTs
postorder traversal : BINTREE [T ] → LIST [T ]
Sorting
Alg. Theory
Preconditions ∀ t : BINTREE [T ] •
Sorting (ct’d)
Binary Trees
pre(left(t)) ⇔ ¬ is mt(t)
Searchtrees pre(right(t)) ⇔ ¬ is mt(t)
Balanced trees
Hashing
pre(data(t)) ⇔ ¬ is mt(t)
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
353/ 550
ADT specification of binary trees III
DatAlg
Maritta Heisel
Axioms ∀ t1 , t2 : BINTREE [T ]; x, y : T •
Introduction left(make(t1 , x, t2 )) = t1
Alg. Design right(make(t1 , x, t2 )) = t2
ADTs data(make(t1 , x, t2 )) = x
Sorting
is mt(mt tree) = true
Alg. Theory
is mt(make(t1 , x, t2 )) = false
Sorting (ct’d)
Binary Trees is in(x, mt tree) = false

Searchtrees
Balanced trees
x = y ⇒ is in(x, make(t1 , y , t2 )) = true
Hashing x 6= y ⇒ (is in(x, make(t1 , y , t2 )) ⇔ is in(x, t1 ) ∨ is in(x, t2 ))
Graphs
height(mt tree) = 0
Backtracking
height(make(t1 , x, t2 )) = max{height(t1 ), height(t2 )} + 1
Dynamic
programming
Greedy
Blockchain
354/ 550
ADT specification of binary trees IV
DatAlg
Maritta Heisel
Introduction balanced(mt tree) = true

Alg. Design
balanced(make(t1 , x, t2 )) ⇔
ADTs
| height(t1 ) − height(t2 ) | ≤ 1 ∧
Sorting
balanced(t1 ) ∧ balanced(t2 )
Alg. Theory
Sorting (ct’d)
Binary Trees
subtrees(mt tree) = {mt tree}
Searchtrees subtrees(make(t1 , x, t2 )) =
Balanced trees
{make(t1 , x, t2 )} ∪ subtrees(t1 ) ∪ subtrees(t2 )
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
355/ 550
ADT specification of binary trees V
DatAlg
Maritta Heisel preorder traversal(mt tree) = nil

Introduction preorder traversal(make(t1 , x, t2 )) =
Alg. Design hxi a preorder traversal(t1 ) a preorder traversal(t2 )
ADTs
inorder traversal(mt tree) = nil
Sorting
Alg. Theory
inorder traversal(make(t1 , x, t2 )) =
Sorting (ct’d) inorder traversal(t1 ) a hxi a inorder traversal(t2 )
Binary Trees
Searchtrees
postorder traversal(mt tree) = nil
Balanced trees postorder traversal(make(t1 , x, t2 )) =
Hashing
postorder traversal(t1 ) a postorder traversal(t2 ) a hxi
Graphs
Backtracking
Dynamic where a is an infix notation for append and hxi is an

programming
abbreviation for cons(x, nil).
Greedy
Blockchain
356/ 550
Example of a binary tree
DatAlg
Maritta Heisel
Introduction
Alg. Design
This tree is balanced.
ADTs
5 Its height is 3.
Sorting Preorder traversal:
Alg. Theory
7 1 578219
Sorting (ct’d)
Binary Trees
Inorder traversal:
Searchtrees
8 2 9 872519
Balanced trees
Hashing Postorder traversal:

Graphs 827915
Backtracking
Dynamic
programming
Greedy
Blockchain
357/ 550
Searchtrees
DatAlg
Maritta Heisel
Introduction Searchtrees are used to store data that are ordered according to
Alg. Design some key. They are characterized by the following property:
ADTs
Sorting For each inner node of the tree with data element x:
Alg. Theory
for each data element y contained in the left successor
Sorting (ct’d)
subtree of the node: y ≤ x
Binary Trees
Searchtrees for each data element y contained in the right successor
Balanced trees
subtree of the node: y > x
Hashing
Graphs
where x ≤ y ⇔ (x < y ∨ x = y )
Backtracking
and x >y ⇔¬x ≤y
Dynamic
programming
Greedy
Blockchain
358/ 550
Searchtrees: formal definition
DatAlg
Maritta Heisel
Introduction
We add the predicate searchtree : BINTREE [T ] → Bool to the
Alg. Design ADT BINTREE [T ], with the following axiom:
ADTs
Sorting
∀ t, t 0 : BINTREE [T ]; x : T •
Alg. Theory searchtree(t) ⇔
Sorting (ct’d)
Binary Trees
(t 0 6= mt tree ∧ t 0 ∈ subtrees(t) ⇒
Searchtrees
Balanced trees
(is in(x, left(t 0 )) ⇒ x ≤ data(t 0 ))
Hashing
∧
Graphs
Backtracking
(is in(x, right(t 0 )) ⇒ x > data(t 0 )))
Dynamic
programming
Greedy
Blockchain
359/ 550
Property of searchtrees
DatAlg
Maritta Heisel
The inorder traversal of a searchtree yields an ordered list:
Introduction
Alg. Design ∀ t : BINTREE [T ] •

ADTs
searchtree(t) ⇒ ordered(inorder traversal(t))
Sorting
Alg. Theory
Example:
Sorting (ct’d)
5
Binary Trees
Searchtrees
Balanced trees
Hashing 3 7 Inorder traversal:

Graphs 235578
Backtracking
2 5 8
Dynamic
programming
Greedy
Blockchain
360/ 550
Proof by induction: recursive algorithm to output
all nodes of a search tree I
DatAlg
Maritta Heisel Specification:

Introduction
var t : BINTREE [T ]
Alg. Design pre searchtree(t)
ADTs post ordered(inorder (t))
Sorting reads t
Alg. Theory changes -
Sorting (ct’d) mem -
Binary Trees
Searchtrees
Balanced trees module inorder(t : BINTREE[T]) : LIST[T]
Hashing if is mt(t)
Graphs then return nil
Backtracking else return inorder(left(t))ahdata(t)iainorder(right(t))
Dynamic
programming
endif
Greedy endmodule
Blockchain
361/ 550
all nodes of a search tree II
DatAlg
Maritta Heisel
Introduction
Alg. Design
We can now prove the correctness of the inorder algorithm
ADTs
with respect to its specification using induction:
Sorting
Inductive hypothesis The algorithm outputs all the nodes in
Alg. Theory
the tree t in an ordered way.
Sorting (ct’d)
Binary Trees
Searchtrees
Basis If t contains zero nodes, then it is empty and the
Balanced trees algorithm returns the empty list. It is true to say
Hashing that all the nodes in the output are in the correct
Graphs
order, since there aren’t any.
Backtracking
Dynamic
programming
Greedy
Blockchain
362/ 550
all nodes of a search tree III
DatAlg
Maritta Heisel Inductive step Suppose t has n nodes and that the inductive
Introduction hypothesis holds for all trees with fewer than n
Alg. Design nodes.
ADTs
According to the inductive hypothesis, the nodes
Sorting
of the left subtree are output in an ordered way
Alg. Theory
(the inductive hypothesis can be applied because
Sorting (ct’d)
the left subtree contains less than n nodes since it
Binary Trees
Searchtrees does not contain the root).
Balanced trees
Hashing The algorithm then outputs the data element in

Graphs the root. Since t is a search tree, data(t) is
Backtracking greater than or equal to all data elements in
Dynamic
programming
left(t). Therefore, inorder (left(t)) a hdata(t)i is
Greedy
ordered.
Blockchain
363/ 550
all nodes of a search tree IV
DatAlg
Maritta Heisel
Finally, applying the inductive hypothesis again,

Introduction
Alg. Design
all the nodes of the right subtree are output in an
ADTs
ordered way.
Sorting Because t is a searchtree, all data elements in
Alg. Theory right(t) are greater than all data elements in
inorder (left(t)) a hdata(t)i. Therefore,
Sorting (ct’d)
Binary Trees
Searchtrees inorder (left(t)) a hdata(t)i a inorder (right(t)) =
Balanced trees
inorder (t) is ordered.
Hashing
Graphs
Conclusion The inductive hypothesis holds for any binary
Backtracking
tree.
Dynamic
programming
Greedy
Blockchain
364/ 550
Searching in searchtrees: specification
DatAlg
Maritta Heisel
x : T ; t, t 0 : BINTREE [T ]
Introduction
var
Alg. Design
ADTs pre searchtree(t)

Sorting
post t 0 ∈ subtrees(t) ∧
(is in(x, t) ⇒ ¬ is mt(t 0 ) ∧ data(t 0 ) = x) ∧
Alg. Theory
Sorting (ct’d)
(¬ is in(x, t) ⇒ is mt(t 0 ))
Binary Trees
Searchtrees
Balanced trees
reads x, t
Hashing
changes t0
Graphs
mem —
Backtracking
Dynamic
programming
Greedy
Blockchain
365/ 550
Searching in searchtrees: recursive algorithm
DatAlg
Maritta Heisel
module search(t : BINTREE [T ]; x : T )
Introduction
var t 0 : BINTREE [T ];
Alg. Design
ADTs if is mt(t) cor data(t) = x

Sorting
then t 0 := t
Alg. Theory
Sorting (ct’d)
else if x ≤ data(t)
Binary Trees then search(left(t), x)
Searchtrees
Balanced trees
else search(right(t), x)
Hashing
Graphs endif
Backtracking
endif
Dynamic
Greedy
Blockchain
366/ 550
Searching in searchtrees: iterative algorithm
DatAlg
Maritta Heisel
Introduction var x : T ; t, t 0 : BINTREE [T ];

Alg. Design
ADTs
t 0 := t;
Sorting while (¬ is mt(t 0 )) cand data(t 0 ) 6= x do
Alg. Theory
if x ≤ data(t 0 )
Sorting (ct’d)
Binary Trees then t 0 := left(t 0 )

Searchtrees
Balanced trees else t 0 := right(t 0 )
Hashing
endif
Graphs
Backtracking endwhile
Dynamic
programming
Greedy
Blockchain
367/ 550
Maximum search: specification
DatAlg
Maritta Heisel
Introduction
Alg. Design
var t, t 0 : BINTREE [T ]
ADTs pre searchtree(t) ∧ (¬ is mt(t))
Sorting
post t 0 = maximum search(t) ⇒
Alg. Theory
Sorting (ct’d)
t 0 ∈ subtrees(t) ∧ (¬ is mt(t 0 )) ∧
Binary Trees
data(t 0 ) = max{x : T | is in(x, t)}
Searchtrees
Balanced trees reads t
Hashing changes t0
Graphs mem —
Backtracking
Dynamic
programming
Greedy
Blockchain
368/ 550
Maximum search: algorithm
DatAlg
Maritta Heisel module maximum search(t : BINTREE [T ]) : BINTREE [T ]

Introduction var t 0 : BINTREE [T ];
Alg. Design
t 0 := t;
ADTs
Sorting while ¬ is mt(right(t 0 )) do

Alg. Theory
t 0 := right(t 0 )
Sorting (ct’d)
Binary Trees endwhile;

return t 0
Searchtrees
Balanced trees
Hashing
endmodule
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
369/ 550
Maximum search: algorithm
DatAlg
Maritta Heisel module maximum search(t : BINTREE [T ]) : BINTREE [T ]

Introduction var t 0 : BINTREE [T ];
Alg. Design
t 0 := t;
ADTs
Sorting while ¬ is mt(right(t 0 )) do

Alg. Theory
t 0 := right(t 0 )
Sorting (ct’d)
Binary Trees endwhile;

return t 0
Searchtrees
Balanced trees
Hashing
endmodule
Graphs
Backtracking
Dynamic
programming
The time complexity of both search algorithms is of order
Greedy
height(t), because each loop can be executed at most height(t)
Blockchain times.
369/ 550
Inserting an element into a searchtree: specification
DatAlg
var x : T ; t, t1 : BINTREE [T ]
Maritta Heisel
pre searchtree(t) ∧ t = t1
Introduction
Alg. Design post searchtree(t) ∧ is in(x, t) ∧

ADTs permutation(inorder traversal(t),
Sorting inorder traversal(t1) a hxi)
Alg. Theory
Sorting (ct’d)
reads x
Binary Trees
changes t
Searchtrees
Balanced trees
mem t1
Hashing
Graphs
The postcondition expresses that the input tree is changed, and
Backtracking
that it is still a searchtree containing x as well as exactly all the
Dynamic
other elements it contained before.
programming
Greedy
Alternatively, we could use bags to express that the element x
Blockchain is inserted into the tree.
370/ 550
Inserting an element into a searchtree: algorithm
DatAlg module insert(t : BINTREE [T ]; x : T )

Maritta Heisel
if is mt(t)
Introduction
then t := make(mt tree, x, mt tree)
Alg. Design
ADTs else if x ≤ data(t)

Sorting
then insert(left(t), x)
Alg. Theory
Sorting (ct’d)
else insert(right(t), x)
Binary Trees endif
Searchtrees
Balanced trees
endif
Hashing
Graphs endmodule
Backtracking
Dynamic
programming
Greedy
Blockchain
371/ 550

Maritta Heisel
if is mt(t)
Introduction
Alg. Design

Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees endif
Searchtrees
Balanced trees
endif
Hashing
Graphs endmodule
Backtracking Remark: We assume that a tree is represented by a pointer, as in
Dynamic
programming
object-oriented programming languages. Hence, if a subtree of a
Greedy tree is changed by an assignment, the whole tree is changed.
Blockchain
371/ 550

Maritta Heisel
if is mt(t)
Introduction
Alg. Design

Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees endif
Searchtrees
Balanced trees
endif
Hashing
Graphs endmodule
Backtracking Remark: We assume that a tree is represented by a pointer, as in
Dynamic
programming
object-oriented programming languages. Hence, if a subtree of a
Greedy tree is changed by an assignment, the whole tree is changed.
Blockchain
371/ 550
Complexity: O(height(t)) (at most height(t) recursive calls)
Inserting an element into a searchtree: example
DatAlg
Maritta Heisel
Inserting 15:
Introduction
Alg. Design 15
ADTs
Sorting 6 18
Alg. Theory
Sorting (ct’d)
Binary Trees
Searchtrees
3 7 17 20
Balanced trees
Hashing
Graphs 2 4 13
Backtracking
Dynamic
programming
9 15
Greedy
Blockchain
372/ 550
Deleting an element x from a searchtree t I
DatAlg
Maritta Heisel
The following cases have to be considered:
Introduction
x is not contained in t. Then, t is not changed.
Alg. Design
ADTs
x is contained in a leaf of t. Then, the corresponding node
Sorting
can just be deleted.
Alg. Theory
Example: delete 13
Sorting (ct’d)
10
Binary Trees
Searchtrees 5 15
Balanced trees
Hashing
Graphs 3 8 13 18
Backtracking
Dynamic
programming 7
Greedy
Blockchain
373/ 550
Deleting an element x from a searchtree t II
DatAlg
Maritta Heisel
Introduction x is stored in a node with only one successor. Then, the

Alg. Design predecessor of x can be linked to the (only) successor of x.
ADTs Example: delete 15
Sorting
Alg. Theory 10 10
Sorting (ct’d)
Binary Trees
5 15 5 18
Searchtrees
Balanced trees
Hashing 3 8 18 3 8
Graphs
Backtracking 7 7
Dynamic
programming
Greedy
Blockchain
374/ 550
Deleting an element x from a searchtree t III
DatAlg
Maritta Heisel
x is stored in a node with two successors.
Idea: replace x by the maximum of its left subtree. This
Introduction
element has no right subtree, and the searchtree property
Alg. Design
is preserved. Then, the maximum can be deleted
ADTs
according to the second or the third case above.
Sorting
Example: delete 10
Alg. Theory
Sorting (ct’d)
10
Binary Trees
Searchtrees 8
Balanced trees
5 18
Hashing
Graphs
5 18
Backtracking 3 8
Dynamic
programming
3 7
Greedy 7
Blockchain
375/ 550
Deleting an element from a searchtree:
specification
DatAlg
Maritta Heisel
var x : T ; t : BINTREE [T ]
Introduction
Alg. Design pre searchtree(t)

ADTs
post searchtree(t) ∧
Sorting
“if t contains x, then one occurrence of x
Alg. Theory
is deleted from t, else t remains unchanged”
Sorting (ct’d)
Binary Trees reads x

Searchtrees
Balanced trees
changes t
Hashing mem ...
Graphs
Backtracking Remark. In a fully formal specification, a memory variable

Dynamic would be needed, because the input is changed.
programming
Greedy
Blockchain
376/ 550
Deleting an element from a searchtree: algorithm
DatAlg module delete(t : BINTREE [T ]; x : T )

Maritta Heisel
if ¬ is mt(t)
Introduction
then if x = data(t)
Alg. Design
ADTs then delete aux(t)

Sorting else if x < data(t)
Alg. Theory
Sorting (ct’d)
then delete(left(t), x)
Binary Trees else delete(right(t), x)
Searchtrees
Balanced trees
endif
Hashing
Graphs
endif
Backtracking endif
Dynamic
Greedy
Complexity: O(height(t)) (at most height(t) recursive calls).
Blockchain
377/ 550
Auxiliary delete operation: specification
DatAlg
Maritta Heisel
Introduction
Alg. Design var t : BINTREE [T ]

ADTs
pre searchtree(t) ∧ ¬ is mt(t)
Sorting
Alg. Theory post searchtree(t) ∧

Sorting (ct’d) “the root element is deleted from t”
Binary Trees
Searchtrees reads —
Balanced trees
changes t
Hashing
mem ...
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
378/ 550
Auxiliary delete operation: algorithm
DatAlg
Maritta Heisel
module delete aux(t : BINTREE [T ])
Introduction
var th : BINTREE [T ];
Alg. Design if is mt(left(t))
ADTs
Sorting
then t := right(t)
Alg. Theory else if is mt(right(t))
Sorting (ct’d)
then t := left(t)
Binary Trees
Searchtrees
Balanced trees
else th := maximum search(left(t))
Hashing t := make(left(t), data(th)), right(t));
Graphs
delete aux(th)
Backtracking
Dynamic endif
programming
Greedy endif
Blockchain
379/ 550 endmodule
Complexity of searchtree operations
DatAlg
The complexity of searching, inserting and deleting
Maritta Heisel
elements are each of order height(t).
Introduction
We only know how good that is when we can say
Alg. Design
something about the average height of a tree with n nodes.
ADTs
Sorting The height lies between log n + 1 (for a full and balanced
Alg. Theory tree) and n (for a degenerated tree, i.e., a list).
Sorting (ct’d) Empirical results: When a searchtree is generated by
Binary Trees randomly inserting n different elements, then
Searchtrees
Balanced trees the average height of the resulting searchtree is of order
Hashing log n
Graphs the average improvement of the length of the search path
Backtracking when using balanced trees is 38.6%
Dynamic These numbers say nothing about how search trees
programming
Greedy
develop when they are not only constructed by inserting,
Blockchain
but also by deleting elements.
380/ 550
Balanced Trees (AVL-Trees)
DatAlg
Maritta Heisel When reading access to a tree is much more frequent than
Introduction changing the tree, it may be advantageous to keep the tree
Alg. Design balanced when inserting and deleting elements.
ADTs
AVL-trees:
Sorting
Alg. Theory
Named after their inventors Adelson–Velsky and Landis.
Sorting (ct’d) Are balanced binary search trees.
Binary Trees
Searchtrees
Inserting and deleting elements must preserve that
Balanced trees
property.
Hashing
Graphs
Since trees which allow the repeated occurrence of the
Backtracking
same key cannot always be balanced (a tree, containing
Dynamic
the same key n times, degenerates into a list), we require
programming
all keys in an AVL-tree to be different.
Greedy
Blockchain
381/ 550
Balancing a tree after inserting or deleting elements
DatAlg
Maritta Heisel
Inserting and deleting are more complicated than with
Introduction
Alg. Design
ordinary search trees.
ADTs To recover the balance of a tree after inserting or deleting
Sorting a node, rotation operations are applied to the tree.
Alg. Theory
Two types of rotations exist: the left and the right
Sorting (ct’d)
rotation.
Binary Trees
Searchtrees These operations (when implemented with pointers) only
Balanced trees
Hashing
need a constant number of pointers being changed.
Graphs Therefore, their complexity is constant.
Backtracking The rotation operations can be applied to (almost) any
Dynamic
programming
node of the tree.
Greedy
Blockchain
382/ 550
Rotation operations
DatAlg
x rotate_left y
Maritta Heisel
> <
Introduction rotate_right >y
<x y x
Alg. Design A C
ADTs
>x >y <x >x
Sorting <y B C A <y B
Alg. Theory
Sorting (ct’d) After left rotation, the root of the right subtree becomes
Binary Trees
Searchtrees
the new root of the tree. The old root becomes the root
Balanced trees of the left subtree and receives as its right subtree the
Hashing
previous left subtree of the new root.
Graphs
The right rotation is the reverse of the left rotation.
Backtracking
Precondition for left-/right rotation: the right/left subtree
Dynamic
programming is not empty.
Greedy The rotation operations preserve the searchtree property.
Blockchain
383/ 550
Formal definition of rotation operations
DatAlg
Maritta Heisel Functions

Introduction
rotate left : BINTREE [T ] →
7 BINTREE [T ]
Alg. Design rotate right : BINTREE [T ] →7 BINTREE [T ]
ADTs
Sorting Preconditions ∀ A : BINTREE [T ] •

Alg. Theory pre(rotate left(A)) ⇔ ¬ is mt(A) ∧ ¬ is mt(right(A))
Sorting (ct’d) pre(rotate right(A)) ⇔ ¬ is mt(A) ∧ ¬ is mt(left(A))
Binary Trees
Searchtrees
Balanced trees Axioms ∀ A, B, C : BINTREE [T ]; x, y : T •
Hashing rotate left(make(A, x, make(B, y , C )))
Graphs
= make(make(A, x, B), y , C )
Backtracking
rotate right(make(make(A, x, B), y , C ))
Dynamic
programming = make(A, x, make(B, y , C ))
Greedy
Blockchain
384/ 550
Almost balanced trees
DatAlg
Maritta Heisel
Introduction We will apply the rotation operations to turn an

Alg. Design almost-AVL-tree into an AVL-tree.
ADTs
An almost-AVL-tree is balanced at all points except for the
Sorting
Alg. Theory
root. Here the height difference is exactly two.
Sorting (ct’d)
almost balanced(make(A, x, B)) ⇔
Binary Trees | height(A) − height(B) | = 2 ∧
Searchtrees
Balanced trees
balanced(A) ∧ balanced(B)
Hashing
Graphs
Backtracking
In the following considerations we focus on the case where the
Dynamic
left subtree is the larger one. The opposite case is analog.
programming
Greedy
Blockchain
385/ 550
Transforming almost-AVL-trees into AVL-trees
DatAlg
Maritta Heisel Let x, y be nodes, A, B, C be AVL-trees.

Introduction Subtree C has the height h − 1.
Alg. Design
The tree with y as a root is an almost-AVL-tree. This
ADTs
means the subtree with x as a root must have height h + 1.
Sorting
Alg. Theory
Thus, A or B must have the height h.
Sorting (ct’d) We distinguish two cases: whether A has height h or not.
Binary Trees
Searchtrees rotate_right
Balanced trees y x
Hashing
Graphs
x h−1 h y
Backtracking C A
Dynamic
programming
h A B h−1 or h−1 or B C h−1
Greedy h h
Blockchain
386/ 550
Case 1: A has height h (“left-left leaning tree”)
rotate_right
DatAlg y x
Maritta Heisel
Introduction x h−1 h y
C A
Alg. Design
ADTs
h A B h−1 or h−1 or B C h−1
Sorting
h h
Alg. Theory
Sorting (ct’d) Then tree B has height h or h − 1.

Binary Trees Right rotation at node y results in the tree represented on
Searchtrees
Balanced trees
the right-hand side of the figure.
Hashing This tree now fulfills the AVL-conditions at every node.
Graphs The new tree has x as a root, and its subtrees differ in
Backtracking height by at most 1.
Dynamic
programming
The original tree had height h + 2, while the new tree has
Greedy height h + 1 or h + 2. Hence, through the transformation
Blockchain the height of the tree remains the same or decreases by 1.
387/ 550
Case 2: A has height h − 1 (“left-right leaning
tree”) I
DatAlg This means that B must have the height h, where h > 1.
Maritta Heisel Let z be the root of B, and B 0 and B 00 be the left and
right subtrees.
Introduction
Alg. Design
Both of the subtrees have the height h − 1 or h − 2, where
ADTs
one of them needs to have the height h − 1.
Sorting
Left rotation at the node x results in the tree at the
Alg. Theory right-hand side.
rotate_left(x)
Sorting (ct’d) y y
Binary Trees
Searchtrees
Balanced trees
x h−1 z h−1
C C
Hashing
Graphs
z x
Backtracking h−1 A h−2 or
B’’ h−1
Dynamic
programming
Greedy h−1 or B’
‘ B’’ h−1 or h−1 A B’ h−1 or
h−2 h−2 h−2
Blockchain
388/ 550
Case 2: A has height h − 1 (“left-right leaning
tree”) II
DatAlg
Maritta Heisel After applying a right-rotation in the node y , we obtain the

Introduction tree on the right-hand side.
Alg. Design
ADTs y
Sorting rotate_right(y) Z
Alg. Theory z h−1
C
Sorting (ct’d) x y
x
Binary Trees h−2 or
B’’ h−1
Searchtrees
Balanced trees
h−1 B’ h−1 or B’’ h−2 or
A h−2 h−1 C h−1
h−1 A B’ h−1 or
Hashing h−2
Graphs
Backtracking
Dynamic This tree is an AVL-tree. Note that its height is always h + 1,

programming
i.e., the double rotation decreases the height of the tree by one.
Greedy
Blockchain
389/ 550
Summary of cases
DatAlg
Maritta Heisel
Introduction
Alg. Design
Case Tree Rotation
ADTs
1 left-left leaning right rotation on root
Sorting
2 left-right leaning left rotation on left subtree
Alg. Theory
right rotation on root
Sorting (ct’d)
Binary Trees
3 right-right leaning left rotation on root
Searchtrees 4 right-left leaning right rotation on right subtree
Balanced trees
Hashing
left rotation on root
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
390/ 550
Formal definition of balancing I
DatAlg
The function balance transforms an almost balanced (or
Maritta Heisel
already balanced) tree into a balanced tree.
Introduction
Alg. Design
Functions
ADTs balance : BINTREE [T ] → BINTREE [T ]
Sorting Preconditions -
Alg. Theory Axioms ∀ A, B, C : BINTREE [T ]; x, y : T •
Sorting (ct’d) balance(mt tree) = mt tree
Binary Trees
Searchtrees | height(A) − height(B) | ≤ 1
Balanced trees
⇒ balance(make(A, x, B)) = make(A, x, B)
Hashing
Graphs height(A) > height(B) + 1 ∧

Backtracking height(left(A)) ≥ height(right(A)) ⇒
Dynamic balance(make(A, x, B)) = rotate right(make(A, x, B))
programming
{case 1 above}
Greedy
Blockchain
391/ 550
Formal definition of balancing II
DatAlg
height(A) > height(B) + 1 ∧
Maritta Heisel
height(left(A)) < height(right(A))
Introduction ⇒ balance(make(A, x, B))
Alg. Design
= rotate right(make(rotate left(A)), x, B)
ADTs
{case 2 above}
Sorting
height(B) > height(A) + 1 ∧
Alg. Theory
height(right(B)) ≥ height(left(B)) ⇒
Sorting (ct’d)
Binary Trees
balance(make(A, x, B)) = rotate left(make(A, x, B))
Searchtrees {case 3 above}
Balanced trees
height(B) > height(A) + 1 ∧
Hashing
Graphs
height(right(B)) < height(left(B))
Backtracking
⇒ balance(make(A, x, B))
Dynamic = rotate left(make(A, x, rotate right(B)))
programming
{case 4 above}
Greedy
Blockchain
392/ 550
Complexity of balance
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Under the condition that the height of the subtrees can be
Alg. Theory
compared in constant time, the function balance has constant
Sorting (ct’d)
complexity, as well.
Binary Trees
Searchtrees
Balanced trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
393/ 550
Inserting elements into AVL trees
DatAlg
Using the function balance, we can define the function
Maritta Heisel avl insert as follows:
Introduction
Alg. Design
Functions
ADTs
avl insert : T × BINTREE [T ] → BINTREE [T ]
Sorting
Preconditions -
Alg. Theory Axioms ∀ A, B, C : BINTREE [T ]; x, y : T •
Sorting (ct’d)
avl insert(x, mt tree) = make(mt tree, x, mt tree)
Binary Trees
Searchtrees x = y ⇒ avl insert(x, make(A, y , B)) = make(A, y , B)
Balanced trees
Hashing x < y ⇒ avl insert(x, make(A, y , B))

Graphs = balance(make(avl insert(x, A), y , B))
Backtracking
x > y ⇒ avl insert(x, make(A, y , B))
Dynamic
programming = balance(make(A, y , avl insert(x, B)))
Greedy
Blockchain
394/ 550
Example for insertion I
DatAlg
4
Maritta Heisel
starting point:
Introduction 5
Alg. Design
ADTs 4
5
Sorting
Alg. Theory rotate left

inserting 7: 5
Sorting (ct’d) (balance, 4 7
case 3)
Binary Trees
Searchtrees 7
Balanced trees
Hashing
5
Graphs
Backtracking inserting 2: is balanced

Dynamic 4 7
programming
Greedy
2
Blockchain
395/ 550
Example for insertion II
DatAlg
5 5
Maritta Heisel
balance (4)
inserting 1: (case 1)
Introduction 4 7 2 7
rotate_right
Alg. Design
2 1 4
ADTs
Sorting
1
Alg. Theory
Sorting (ct’d)
Binary Trees
Searchtrees
Balanced trees inserting 3: 5 5 4
Hashing
balance (5)
Graphs 2 7 (case 2) 4 7 rotate_right (5) 2 5
Backtracking rotate_left(2)
1 4 2 1 3 7
Dynamic
programming
Greedy 3 1 3
Blockchain
396/ 550
Example for insertion III
DatAlg
Maritta Heisel
Introduction
inserting 6:
Alg. Design
ADTs
4 4 4
Sorting
Alg. Theory
balance (5) rotate_left (5)
Sorting (ct’d) 2 5 (case 4) 2 5 2 6
Binary Trees
rotate_right(7)
Searchtrees
1 3 7 1 3 6 1 3 5 7
Balanced trees
Hashing
Graphs
6 7
Backtracking
Dynamic
programming
Greedy
Blockchain
397/ 550
Deleting elements from AVL trees
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
For deletion the same cases are distinguished as for deletion in
Sorting
a searchtree, where the function balance is again used.
Alg. Theory
Sorting (ct’d) This works, because on insertion and deletion, the AVL-features
Binary Trees either remain or an almost-AVL-tree emerges, which can be
Searchtrees
Balanced trees balanced with balance.
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
398/ 550
Formal definition of deletion I
DatAlg
Maritta Heisel Functions

Introduction
avl max : BINTREE [T ] →
7 T
Alg. Design
avl delete : T × BINTREE [T ] → BINTREE [T ]
ADTs Preconditions ∀ A : BINTREE [T ] •
Sorting
pre(avl max(A)) ⇔ searchtree(A) ∧ ¬is mt(A)
Alg. Theory
Sorting (ct’d)
Axioms ∀ A, B, C : BINTREE [T ]; x, y : T •
Binary Trees is mt(right(A)) ⇒ avl max(A) = data(A)
Searchtrees
Balanced trees
¬is mt(right(A)) ⇒ avl max(A) = avl max(right(A))
Hashing
avl delete(x, mt tree) = mt tree
Graphs
Backtracking
x < y ⇒ avl delete(x, make(A, y , B))
Dynamic
= balance(make(avl delete(x, A), y , B))
programming
x > y ⇒ avl delete(x, make(A, y , B))
Greedy
= balance(make(A, y , avl delete(x, B)))
Blockchain
399/ 550
Formal definition of deletion II
DatAlg
Maritta Heisel
Introduction
Alg. Design avl delete(x, make(A, x, mt tree)) = A

ADTs avl delete(x, make(mt tree, x, B)) = B
Sorting
¬ is mt(A) ∧ ¬ is mt(B)
Alg. Theory
⇒ avl delete(x, make(A, x, B))
Sorting (ct’d)
Binary Trees
= balance(make(avl delete(avl max(A), A),
Searchtrees avl max(A),
Balanced trees
Hashing
B))
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
400/ 550
Example for deletion I
DatAlg
5 deleting 4: 5
Maritta Heisel Starting point:
Introduction 3 8 3 8
Alg. Design
ADTs 2 4 7 10 2 7 10
Sorting
Alg. Theory 1 6 9 11 1 6 9 11
Sorting (ct’d)
5 deleting 8: 5
Binary Trees
balance(3)
Searchtrees
(case 1)
Balanced trees 2 8 2 7
rotate_right(3)
Hashing
Graphs 1 3 7 10 1 3 6 10
Backtracking
Dynamic 6 9 11 9 11
programming
is balanced
Greedy
Blockchain
401/ 550
Example for deletion II
DatAlg
5 5
deleting 6:
Maritta Heisel balance(7)
2 7 (case 3) 2 10
Introduction
Alg. Design
rotate_left(7)
x=7
ADTs 1 3 10 y=10 1 3 7 11
Sorting
Alg. Theory
9 11 9
Sorting (ct’d)
Binary Trees
deleting 5: 3
Searchtrees
Balanced trees
Hashing
2 10
Graphs is balanced
Backtracking
Dynamic 1 7 11
programming
Greedy
Blockchain
9
402/ 550
Complexity of avl insert and avl delete
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
On condition that balance has constant complexity, the
Alg. Theory
complexity of avl insert and avl delete is of order log n, since
Sorting (ct’d)
the height of the tree amounts to log n in the worst case.
Binary Trees
Searchtrees
Balanced trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
403/ 550
Comparing tree heights in constant time
DatAlg
Maritta Heisel
Introduction
In order to achieve that the height comparisons required for
Alg. Design
balance are possible in constant time, an implementation must
ADTs store in each node (apart from the node data) information
Sorting about the height difference of the subtrees.
Alg. Theory
Sorting (ct’d) Two alternatives exist:

Binary Trees
Searchtrees
store the height of both subtrees
Balanced trees
store height(right(A)) − height(left(A)) ∈ {−1, 0, 1}
Hashing
Graphs
Backtracking
This information of course needs to be updated after each
Dynamic
insertion and deletion operation.
programming
Greedy
Blockchain
404/ 550
What have we learnt on trees?
DatAlg
Maritta Heisel
Introduction
Alg. Design
explain what a tree is, especially a binary tree
ADTs
explain the different functions defined on trees, e.g.,
Sorting
height, preorder traversal, etc.
Alg. Theory give the definition of search trees, and give and apply the
Sorting (ct’d) algorithms for searching, inserting, and deleting elements
Binary Trees in search trees
Searchtrees
Balanced trees
explain AVL trees as well as explain and apply the
Hashing algorithms for searching, inserting, and deleting elements
Graphs in AVL trees
Backtracking give the motivation of using AVL trees instead of search
Dynamic
programming
trees
Greedy
Blockchain
405/ 550
Hashing: basic idea I
DatAlg
Maritta Heisel
So far we have dealt with the following problem:
Introduction
A given set M is characterized by key values, i.e., an
Alg. Design
ordering relation is defined on M.
ADTs How should M be stored so that an element with given
Sorting key can be recovered with least possible effort?
Alg. Theory
Every element needs to be stored under an address in the
Sorting (ct’d)
storage space.
Binary Trees
Hashing Hence, the above problem can be considered as the problem to

Graphs find a suitable mapping
Backtracking
Dynamic
programming
h:K →A
Greedy
where K is the set of keys of the set M and A is the set of
Blockchain
addresses.
Summary
406/ 550
Hashing: basic idea II
DatAlg
Maritta Heisel
Introduction We try to find the stored data by some computation.

Alg. Design
The data are stored in a linear array with indices
ADTs
0 . . . m − 1.
Sorting
Alg. Theory Such a data structure is called a hash table.

Sorting (ct’d) The word ” hashing” stands for ”chopping”. The naming
Binary Trees
has its origin in the fact that the hash function should
Hashing
distribute the data sets evenly over the table.
Graphs
Backtracking
The hash function assigns to each key k an index h(k), its
Dynamic hash address.
programming
Greedy
Blockchain
Summary
407/ 550
Problems with hashing
DatAlg
Maritta Heisel
The number of actually occurring keys (e.g. names) is
Introduction
Alg. Design
generally much less than the number of possible keys (e.g.
ADTs
strings).
Sorting Therefore, a hash function cannot be injective, since too
Alg. Theory much storage space would be wasted.
Sorting (ct’d)
The question is how to deal with address collisions (i.e.
Binary Trees
two different keys are mapped to the same hash address).
Hashing
Graphs Thus, we have identified two problems:

Backtracking
1. How should the hash function h be chosen, so that as few
Dynamic
programming address collisions as possible occur?
Greedy 2. How should collisions be dealt with?
Blockchain
Summary
408/ 550
Choice of hash function I
DatAlg
Maritta Heisel
A good hash function should be easily and quickly
Introduction
computable and distribute the data evenly on the storage
Alg. Design
space in order to avoid address collisions.
ADTs
Sorting
This should also be the case if the keys themselves are not
Alg. Theory
evenly distributed (a name such as Meyer is more likely
Sorting (ct’d) than Qzwrtx).
Binary Trees Since the hash function eventually needs to apply
Hashing arithmetics for computing hash addresses, the keys are
Graphs
first converted into natural numbers.
Backtracking
Dynamic
A bit-representation of a key can for example be
programming interpreted as a number, or a letter can be identified by its
Greedy position in the alphabet.
Blockchain
Summary
409/ 550
Choice of hash function II
DatAlg
Maritta Heisel
Introduction
When the key is converted into a natural number k, a
Alg. Design suitable hash function is h(k) = k mod m , where m is
ADTs the size of the hash table.
Sorting In order to obtain an even distribution, m must be chosen
Alg. Theory
with care.
Sorting (ct’d)
Binary Trees
If m is e.g. even, then h(k) is odd if k is odd.
Hashing In case the key is represented as binary number, whose
Graphs last digit represents a fact ( e.g. 0 = male, 1 = female),
Backtracking then a uniform distribution of the key states is not possible
Dynamic
programming
(data on women is only stored in odd storage spaces).
Greedy
Blockchain
Summary
410/ 550
Choice of hash function III
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
A good choice for m is a prime number, because then it is
Sorting
guaranteed that all parts of a key are considered in the
Alg. Theory computation of the hash function.
Sorting (ct’d) Other methods for key conversion exist. We restrict
Binary Trees ourselves to the division/remainder-method, since it
Hashing
provides the best results.
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
411/ 550
Dealing with Collisions
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
In general two possibilities of dealing with collisions exist:
Sorting 1. place colliding elements in a separate storage space (not in
Alg. Theory the hash table itself)
Sorting (ct’d)
2. store colliding elements in a free space inside the hash
Binary Trees
table, which is computed from h(k).
Hashing
Graphs Both possibilities are discussed in the following.

Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
412/ 550
Hashing with chaining of collisions
DatAlg
Here also two possibilities exist.
Maritta Heisel
Separate chaining of collisions:
Introduction
Alg. Design entry in hash table consists of elements as well as collision

ADTs chain
Sorting example: K = {0, 1, ..., 500}, m = 7, h(k) = k mod m
Alg. Theory
insertion of the keys 12, 53, 5, 15, 2, 19, 43 results in
Sorting (ct’d)
Binary Trees
0 1 2 3 4 5 6
Hashing t: 15 2 53 12
Graphs
Backtracking
Dynamic 43 5
programming
Greedy
Blockchain 19
Summary
413/ 550
Separate chaining of collisions (cont’d)
DatAlg
Maritta Heisel
Introduction
Alg. Design
Search first compare elements, then search in collision
ADTs
chain
Sorting
Alg. Theory Insert search element; insert at location where search is

Sorting (ct’d) unsuccessful
Binary Trees Delete from collision chain: clear
Hashing
from table: transmit first element of the collision
Graphs
chain into table, if existing
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
414/ 550
Direct chaining of collisions
DatAlg
Maritta Heisel
each entry in the hash table consists of a list
Introduction
previous example becomes
0 1 2 3 4 5 6
Alg. Design
t:
ADTs
Sorting
15 2 53 12
Alg. Theory
Sorting (ct’d)
Binary Trees 43 5
Hashing
Graphs
19
Backtracking
Dynamic
programming requires less storage space for large data sets than
Greedy separate chaining
Blockchain
search, insert, delete: same as in lists
Summary
415/ 550
Chaining of collisions: complexity analysis I
DatAlg
Maritta Heisel
In the worst case all n elements are stored in one list;
Introduction
results in search complexity proportional to n.
Alg. Design
ADTs
average search complexity for unsuccessful search:
n
Sorting average length of the list is m = α (occupation or load
Alg. Theory factor)
Sorting (ct’d) needs to be traversed completely.
α
Binary Trees
average search complexity for successful search: ≈ 1 + 2
Hashing
Graphs
The unsuccessful search can be improved by keeping the chains
Backtracking
sorted.
Dynamic
programming
The successful search can be improved if the chains are sorted
Greedy
according to the access frequency.
Blockchain
Summary
416/ 550
Chaining of collisions: complexity analysis II
DatAlg
Maritta Heisel
complexity of insertion = complexity of unsuccessful search
Introduction
Alg. Design complexity of deletion = complexity of successful search

ADTs
numerical example: number of search steps for
Sorting
Alg. Theory
Sorting (ct’d) separate chaining direct chaining

Binary Trees α successful unsuccessful successful unsuccessful
Hashing 0.5 1.25 1.11 1.25 0.5
Graphs 0.9 1.45 1.307 1.45 0.9
Backtracking 0.95 1.475 1.337 1.475 0.95
Dynamic
programming
1.0 1.5 1.368 1.5 1.0
Greedy
Blockchain
Summary
417/ 550
Advantages of collision chaining
DatAlg
Maritta Heisel
Introduction
Alg. Design
Occupation factor of > 1 possible
ADTs
Sorting
(can occur if amount of data increases faster than
Alg. Theory
expected)
Sorting (ct’d) Deletion of data causes no problems
Binary Trees
Usage of external storage possible
Hashing
Graphs
Backtracking Disadvantage: additional storage space needed for collisions

Dynamic
programming
Greedy
Blockchain
Summary
418/ 550
Hashing with probing
DatAlg
Idea: try to place collisions in hash table itself.
Maritta Heisel
Each key k is assigned an (arithmetically computed)
Introduction
probing sequence a0 (k), a1 (k), a2 (k), . . . of hash
Alg. Design
addresses.
ADTs
Sorting
insert Store key in the first free space of the probing
Alg. Theory
sequence.
Sorting (ct’d)
Binary Trees search Along the probing sequence, until key is found
Hashing (success) or until the first free space (failure).
Graphs delete Simple release of an address can disrupt the
Backtracking
probing sequence and lead to errors in searching.
Dynamic
programming Hence: no release, but marking as deleted.
Greedy Locations marked as deleted are considered as
Blockchain free during insertion and as occupied during
Summary search.
419/ 550
Linear probing
DatAlg
Maritta Heisel
ai (k) = (a0 (k) + c ∗ i) mod m , c constant
Introduction
Alg. Design
Example: m = 7, c = 1, a0 (k) = k mod m
ADTs
insertion of 12, 53
Sorting
Alg. Theory 0 1 2 3 4 5 6
Sorting (ct’d) t: 53 12
Binary Trees
Hashing
insertion of 5: a0 (5) = 5 occupied
Graphs
Backtracking
a1 (5) = 6 mod 7 = 6
Dynamic
0 1 2 3 4 5 6
programming
t: 53 12 5
Greedy
Blockchain
Summary
420/ 550
Linear probing (example)
DatAlg
Maritta Heisel
insertion of 15: position 1 free
insertion of 11: probing sequence 4-5-6-0
Introduction
Alg. Design 0 1 2 3 4 5 6
ADTs
t: 11 15 53 12 5
Sorting
Alg. Theory
Sorting (ct’d)
This method leads to a so-called clustering; i.e. cohesive
Binary Trees
occupied areas are formed, such that a uniform distribution of
Hashing
the keys no longer exists in the hash-table.
Graphs
Backtracking
This effect is lessened by quadratic probing:
Dynamic ai (k) = (a0 (k) + c ∗ i 2 ) mod m, c constant
programming
Greedy
Moreover, it should be assured that the probing sequence
Blockchain
provides a permutation of all hash addresses.
Summary
421/ 550
Hashing with probing: properties I
DatAlg
Maritta Heisel
Another disadvantage of linear and quadratic probing is
Introduction
that all synonyms (k, k 0 with k 6= k 0 , but a0 (k) = a0 (k 0 ))
Alg. Design
traverse the same probing sequence.
ADTs
n
Sorting Efficiency of probing (α = m)
Alg. Theory
Sorting (ct’d)
Method successful search unsuccessful search
Binary Trees
Hashing
linear probing ≈ 21 (1 + (1−α)
1
) ≈ 12 (1 + (1−α)
1
2)
1 α 1 1
Graphs quadratic probing ≈ 1 + ln( 1−α ) − 2 ≈ 1−α − α + ln( 1−α )
Backtracking
Dynamic Source:
programming
T. Ottmann/P. Widmayer: Algorithmen und Datenstrukturen,
Greedy
4. Auflage, Spektrum Verlag, 2002.
Blockchain
Summary
422/ 550
Hashing with probing: properties II
DatAlg
Maritta Heisel
Numerical example:
Introduction
linear probing quadratic probing
Alg. Design
α successful unsuccessful successful unsuccessful
ADTs
0.5 1.5 2.5 1.44 2.19
Sorting
0.9 5.5 50.5 2.85 11.40
Alg. Theory
Sorting (ct’d)
0.95 10.5 200.5 3.52 22.05
Binary Trees
1.0 – – – –
Hashing
Graphs
This method is significantly inferior to the method of chaining
Backtracking collisions.
Dynamic
programming An improvement can be reached through double hashing:
Greedy
ai (k) = (a0 (k) + h0 (k) ∗ i) mod m, where h0 is a second hash
Blockchain
function.
Summary
423/ 550
What have we learnt on hashing?
DatAlg
Maritta Heisel
Introduction You should now be able to

Alg. Design
explain the ideas underlying hashing
ADTs
Sorting
give properties of good hash functions
Alg. Theory explain the differences between linking of collisions and
Sorting (ct’d) probing
Binary Trees
explain how searching, inserting and deleting work for the
Hashing
different hashing procedures
Graphs
Backtracking know the approximate number of search steps for the

Dynamic different hashing procedures
programming
Greedy
Blockchain
Summary
424/ 550
Graphs: motivation (Sedgewick)
DatAlg
Maritta Heisel
A great many problems are naturally formulated in terms
of objects and connections between them, e.g., route maps
Introduction
or electronic circuits.
Alg. Design
ADTs
A graph is a mathematical structure that accurately
Sorting models such situations.
Alg. Theory Graph theory is a major branch of combinatorial
Sorting (ct’d) mathematics and has been studied intensively for hundreds
Binary Trees of years.
Hashing
Here we can only scratch the surface of what is known
Graphs
Topological sorting about graphs.
Transitive closure
Graph traversal Even trivial graph algorithms lead to interesting computer
Shortest paths
Backtracking
programs, and the nontrivial algorithms are among the
Dynamic
most elegant and interesting (though difficult to
programming
understand) algorithms known.
Greedy
425/ 550
Graphs: definition I
DatAlg
A graph is a set of
Maritta Heisel
vertices (simple objects that can have names and other
Introduction properties)
Alg. Design
edges that are connections between two vertices.
ADTs
Sorting
Example:
Alg. Theory 1 2
Sorting (ct’d)
Binary Trees 3 4 5
Hashing
Graphs
6 7
Topological sorting ‘
Transitive closure
Graph traversal
Shortest paths
Backtracking set of vertices: V = {1, . . . , 7}

Dynamic
programming
set of edges:
Greedy
E = {(1, 2), (1, 3), (1, 4), (2, 4), (2, 5), . . . , (7, 6)}
426/ 550
Graphs: definition II
DatAlg
Maritta Heisel
Formal definition:
A directed graph G is an ordered pair (V , E ), where
Introduction
Alg. Design
V is a finite set of vertices, and
ADTs E ⊆ (V × V ) where for (a, b) ∈ E it holds a 6= b.
Sorting
Alg. Theory
This definition can be generalized in several ways:
Sorting (ct’d)
Binary Trees undirected graphs: edges have no direction (no

Hashing arrowhead), thus edges are unordered pairs
Graphs
Topological sorting
Transitive closure i
Graph traversal graphs with loops: , i.e., (i, i) ∈ E possible
Shortest paths
Backtracking graphs with several edges between a and b

Dynamic graphs with an infinite set of vertices
programming
Greedy
427/ 550
Graphs: terminology
DatAlg
Maritta Heisel
For (a, b) ∈ E ,
Introduction
Alg. Design a is a predecessor of b and

ADTs b is a successor of a
Sorting
Alg. Theory
The input degree of a vertex is equal to the number of its
Sorting (ct’d)
predecessors:
Binary Trees
indeg (a) = #{b : V | (b, a) ∈ E }
Hashing
Graphs
Topological sorting
Transitive closure
The output degree of a vertex is equal to the number of its
Graph traversal
Shortest paths
successors:
Backtracking
outdeg (a) = #{b : V | (a, b) ∈ E }
Dynamic
programming
Greedy
428/ 550
ADT specification of a graph
DatAlg
Maritta Heisel
Type GRAPH
Introduction
Functions
Alg. Design
make null : GRAPH
ADTs insert vertex : Vertex × GRAPH → GRAPH
Sorting
delete vertex : Vertex × GRAPH →
7 GRAPH
Alg. Theory
Sorting (ct’d) connect : Vertex × Vertex × GRAPH → GRAPH

Binary Trees
disconnect : Vertex × Vertex × GRAPH → GRAPH
Hashing
Graphs
member vertex : Vertex × GRAPH → Bool
Topological sorting
Transitive closure member edge : Vertex × Vertex × GRAPH → Bool
Graph traversal
Shortest paths
...
Backtracking
Dynamic
However, term representations of graphs are not useful for
programming performing computations on graphs.
Greedy
429/ 550
Representing graphs: standard list
DatAlg In the following, G is a graph with n vertices V = {1, . . . , n}

Maritta Heisel and m edges e1 , . . . , em .
Introduction For each ei = (vi1 , vi2 ) with 1 ≤ i ≤ m different representation
Alg. Design
options exist:
ADTs
Sorting
Standard list:
Alg. Theory
The graph is represented by a list of 2 + 2m numbers:
Sorting (ct’d)
(n, m, v11 , v12 , . . . , vm1 , vm2 )
Binary Trees
Note that the list need not be ordered.
Hashing
Graphs Example:
Topological sorting
Transitive closure
1 2
Graph traversal
Shortest paths
3 4 5
Backtracking
Dynamic 6 7
programming
Greedy
430/ 550
(7, 12, 1, 2, 1, 3, 1, 4, . . . , 7, 6)
Representing graphs: vertex-oriented list
DatAlg
For each vertex, its successors are given in a list. To know
Maritta Heisel
when all successors of a vertex are given, the output degree of
Introduction each vertex must be stored.
Alg. Design
ADTs Let ui1 , ui2 , . . . , uioutdeg (i) be the successors of vertex vi . Then,
Sorting the graph is represented by the list:
Alg. Theory
(n, m, outdeg (1), u11 , . . . , u1outdeg (1) , . . . , outdeg (n), un1 , . . . , unoutdeg (n) )
Sorting (ct’d)
Binary Trees Pn
The length of this list is 2 + n + i=1 outdeg (i) = 2 + n + m.
Hashing
Graphs
Topological sorting
For most graphs with m > n (that is a graph with more edges
Transitive closure
Graph traversal
than vertices), the vertex-oriented list requires less space than
Shortest paths the standard list.
Backtracking
Dynamic
Example: (7, 12, 3, 2, 3, 4, 2, 4, 5, . . . , 1, 6)
programming
| {z }
vertex 1
Greedy
431/ 550
Representing graphs: adjacency matrix I
DatAlg
Maritta Heisel The graph is represented by a two-dimensional array

Introduction
Alg. Design A : ARRAY (1..n)[ARRAY (1..n)[Integer ]]

ADTs
Sorting
with
Alg. Theory
1, if (i, j) ∈ E
Sorting (ct’d)
A[i, j] = A[i][j] =
Binary Trees 0, otherwise
Hashing
Graphs This requires storage of O(n2 ).

Topological sorting
Transitive closure
Graph traversal
A graph without loops consists of at maximum n2 − n edges.
Shortest paths
Backtracking
An adjacency matrix requires more storage than a standard list
Dynamic
and the vertex-oriented list (due to m ≤ n2 − n).
programming
Greedy
432/ 550
Representing graphs: adjacency matrix II
DatAlg
Maritta Heisel
But an adjacency matrix is more transparent. Example:
Introduction
Alg. Design  
ADTs 1 2
0 1 1 1 0 0 0
Sorting

 0 0 0 1 1 0 0 

Alg. Theory
 0 0 0 0 0 1 0 
3 4 5  
Sorting (ct’d)

 0 0 1 0 0 1 1 

Binary Trees

 0 0 0 1 0 0 1 

Hashing 6 7  0 0 0 0 0 0 0 
Graphs 0 0 0 0 0 1 0
Topological sorting
Transitive closure
Graph traversal
Shortest paths
Backtracking
diagonal empty → no loops
Dynamic
matrix contains many zeros
programming
Greedy
433/ 550
Representing graphs: adjacency list
DatAlg
Maritta Heisel The graph is represented by n + 1 linked lists.

Introduction
L0 : list of all vertices; could also be represented as an
Alg. Design array.
ADTs L1 , . . . , Ln : list of successors for vertices 1, . . . , n.
Sorting
n + ni=1 outdeg (i) = n + m list elements are required.
P
Alg. Theory
Sorting (ct’d) Example:

1 2
Binary Trees
Hashing 2 .
.
.
Graphs
3
Topological sorting
.
Transitive closure
.. .. .
Graph traversal . . .
Shortest paths
7 / 6 /
Backtracking
Dynamic Each of the introduced graph representations can be

programming
transformed into all other representations.
Greedy
434/ 550
Transforming a standard list into an adjacency
matrix
DatAlg
(1) Set A[i, j] := 0 for 1 ≤ i ≤ n, 1 ≤ j ≤ n
Maritta Heisel
{n : first element of standard list}
Introduction
(2) If 2nd element of standard list 6= 0 :
Alg. Design
start with 3rd element
ADTs
while list not empty do
Sorting
Alg. Theory
read two successive values i and j and set A[i, j] := 1
Sorting (ct’d)
endwhile
Binary Trees Analysis:
Hashing
step (1) requires O(n2 ) steps
Graphs
Topological sorting step (2) requires O(m) steps
Transitive closure
Graph traversal
Shortest paths
total complexity: O(n2 )
Backtracking
Since all n2 elements of the adjacency matrix must be created,
Dynamic
programming O(n2 ) is a lower bound for this algorithm. Hence, this
Greedy
435/ 550
algorithm (concerning its asymptotic behavior) is optimal.
Transforming a standard list into an adjacency list
DatAlg
Maritta Heisel (1) create list L0 with elements 1, 2, . . . , n and

Introduction
assign to each element i a pointer on the empty list Li
Alg. Design (2) if m 6= 0:
ADTs start with the 3rd element
Sorting while list not empty do
Alg. Theory read two successive values i and j and add j to list Li
Sorting (ct’d)
endwhile
Binary Trees
Hashing Analysis:
Graphs step (1): O(n)
Topological sorting
Transitive closure
Graph traversal
step (2): O(m), if standard list is ordered
Shortest paths O(m · n), if standard list is not ordered
Backtracking
total complexity O(n · m) in general case
Dynamic
programming O(n + m) if using an array for L0
Greedy
436/ 550
Transforming an adjacency matrix into a
vertex-oriented list I
DatAlg
Maritta Heisel
Let the vertex-oriented list be represented as an array
Introduction a(1..2 + n + m)[Integer ]. Furthermore, we need an auxiliary
Alg. Design array for the output degrees.
ADTs
Sorting
var a : ARRAY (1..2 + n + m)[Integer ];
Alg. Theory outdeg : ARRAY (1..n)[Integer ]; i, r , s : Integer ;
Sorting (ct’d) (1) a[1] := n {dimension of matrix}
Binary Trees
(2) r := 3;
Hashing
{index of next array element that stores output degree of
Graphs
Topological sorting considered vertex}
Transitive closure
Graph traversal s := 4;
Shortest paths
{index of array element that stores next successor of
Backtracking
considered vertex}
Dynamic
programming
Greedy
437/ 550
vertex-oriented list II
DatAlg
Maritta Heisel
(3) repeat for all i from 1 to n
(a) outdeg [i] := 0
Introduction
(b) repeat for all j from 1 to n
Alg. Design
if A[i, j] = 1
ADTs
then a[s] := j;
Sorting outdeg [i] := outdeg [i] + 1;
Alg. Theory s := s + 1
Binary Trees endrepeat;
Hashing (c) a[r ] := outdeg [i];
Graphs r := r + outdeg [i] + 1;
Topological sorting
Transitive closure
s := r + 1;
Graph traversal
Shortest paths
endrepeat;
Backtracking (4) a[2] := r − n − 3
Dynamic
programming
{in the end r has the value 2 + n + m + 1}
Greedy
438/ 550
vertex-oriented list III
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Analysis:
Sorting
Alg. Theory
steps (1), (2), (4): O(1)
Sorting (ct’d) step (3): O(n2 )
Binary Trees
total complexity: O(n2 )
Hashing
Graphs
Topological sorting
Transitive closure
Graph traversal
Shortest paths
Backtracking
Dynamic
programming
Greedy
439/ 550
Complexity of graph operations I
DatAlg
Maritta Heisel
Introduction
Complexity for deletion and insertion of vertices and edges
Alg. Design
for graphs with n vertices and m edges.
ADTs New vertex has no edges.
Sorting
Deletion of vertices: all connected edges must be deleted.
Alg. Theory
Sorting (ct’d)
operation standard vertex− adj. list adj. matrix
Binary Trees
list oriented list
Hashing
insert edge O(1) O(n + m) O(n) O(1)
Graphs
Topological sorting
delete edge O(m) O(n + m) O(n + m) O(1)
Transitive closure insert vertex O(1) O(1) O(1) if last in O(1) O(n)
Graph traversal
Shortest paths
O(n + m) O(n) otherwise
delete vertex O(m) O(n + m) O(n · m) O(n2 )
Backtracking
Dynamic
programming
Greedy
440/ 550
Paths
DatAlg
Let G = (V , E ) be a graph, and let a, b ∈ V .
Maritta Heisel
Introduction
There is a path of size s from a to b in G iff there exist vertices
Alg. Design
v0 , v1 , . . . , vs with v0 = a and vs = b and (vi−1 , vi ) ∈ E for
ADTs 1 ≤ i ≤ s.
Sorting
Example: 1 2
Alg. Theory
Sorting (ct’d) 3 4 5
Binary Trees
Hashing 6 7
Graphs
Topological sorting
Vertices 1 and 4 are connected via
Transitive closure
Graph traversal a path of size 1: v0 = 1, v1 = 4
Shortest paths
a path of size 2: v0 = 1, v1 = 2, v3 = 4
Backtracking
a path of size 3: v0 = 1, v1 = 2, v3 = 5, v4 = 4
Dynamic
programming
There is no path from 5 to 1, or from 3 to 1, or . . . .
Greedy
441/ 550
Topological order I
DatAlg
A graph G = ({1, . . . , n}, E ) is in topological order iff for every
Maritta Heisel
two vertices k and l between which a path exists, k < l holds.
Introduction
Our example graph is not in topological order, because there is
Alg. Design
ADTs
a path from 5 to 3.
Sorting Renaming of vertices:
Alg. Theory
Sorting (ct’d) old 1 2 3 4 5 6 7

Binary Trees new 1 2 6 4 3 7 5
Hashing
Graphs
Topological sorting
1 2
Transitive closure
Graph traversal
Shortest paths
6 4 3
Backtracking
Dynamic
programming 7 5
Greedy
442/ 550
Topological order II
DatAlg
1
Maritta Heisel
2
Introduction
Alg. Design
ADTs Vertices on the same 3
Sorting level are not con-

4
Alg. Theory nected by a path.
Sorting (ct’d)
6 5
Binary Trees
Hashing
Graphs
practical value: for example 7
Topological sorting vertices =
b production steps
Transitive closure
Graph traversal edge from a to b = b step a must be performed before b
Shortest paths
or
Backtracking
Dynamic
vertices =
b courses
programming edge from a to b = b a must be attended before b
Greedy
443/ 550
Topological sorting
DatAlg
Maritta Heisel
Fact:
Introduction A graph G is free of cycles iff there is a topological order for G .
Alg. Design
ADTs G = ({1, . . . , n}, E ) can be topologically sorted if there exists a

Sorting renaming
Alg. Theory Φ : {1, . . . , n} → {1, . . . , n} with
Sorting (ct’d) G 0 = ({1, . . . , n}, {(Φ(a), Φ(b)) | (a, b) ∈ E ∧ Φ(a) < Φ(b)})
Binary Trees
Hashing
Graphs
Fact:
Topological sorting A graph G = (V , E ) can be sorted topologically iff it is empty
Transitive closure
Graph traversal or it includes a vertex v with indeg (v ) = 0 and the graph
G 0 = {V \ {v }, E \ {(v , u) | u ∈ V }} can be sorted
Shortest paths
Backtracking
Dynamic
topologically.
programming
Greedy
444/ 550
Recursive algorithm for topological sorting
DatAlg
Maritta Heisel
Introduction
(1) for each vertex v , determine indeg (v );
Alg. Design
ADTs
(2) choose some vertex v with indeg (v ) = 0;
Sorting {such a vertex exists, because otherwise the graph would
Alg. Theory contain a cycle}
Sorting (ct’d) (3) delete v and all edges leaving v ;
Binary Trees
(4) call the algorithm recursively with the resulting graph;
Hashing
result is a renaming of Φ0 : V \ {v } → {1, . . . , n − 1}
Graphs
Topological sorting
Transitive closure
(5) final renaming Φ:
Graph traversal Φ(v ) = 1, Φ(w ) = Φ0 (w ) + 1 for w ∈ V \ {v }
Shortest paths
Backtracking
Dynamic
programming
Greedy
445/ 550
Transitive closure
DatAlg
Maritta Heisel
Questions concerning the reachability of vertices in a graph,
Introduction starting from a given vertex:
Alg. Design
Which vertices can be reached?
ADTs
Sorting Is there a vertex from which all other vertices can be

Alg. Theory reached? (True for cycles)
Sorting (ct’d) To answer those questions, all reachabilities can be computed
Binary Trees
in advance.
Hashing
Graphs
Topological sorting
A directed graph G ∗ = (V , E ∗ ) is the reflexive, transitive
Transitive closure
Graph traversal
closure of a graph G = (V , E )
Shortest paths iff
Backtracking ((v , v 0 ) ∈ E ∗ iff there is a path from v to v 0 in G ).
Dynamic
programming
Greedy
446/ 550
Computing the transitive closure I
DatAlg
Maritta Heisel
Introduction
We denote the existence of a path from v to v 0 in G = (V , E )
Alg. Design
by v →E ∗ v 0 .
ADTs
Sorting If we know that there is a path v →E ∗ v 0 and a path

Alg. Theory v 0 →E ∗ v 00 , we can conclude v →E ∗ v 00 also.
Sorting (ct’d)
Binary Trees This leads to a first idea for an algorithm:

Hashing
Graphs
Let G be given as an adjacency matrix A. For each edge (i, j),
Topological sorting
Transitive closure
we look for edges (j, k) and store the resulting edges (i, k) in
Graph traversal the adjacency matrix.
Shortest paths
Backtracking
Dynamic
programming
Greedy
447/ 550
Computing the transitive closure II
DatAlg
step (1) repeat for each i from 1 to n
Maritta Heisel
A[i, i] := 1
Introduction endrepeat;
Alg. Design {reflexive edges in diagonal of adjacency matrix}
ADTs
Sorting
Alg. Theory
repeat for each j from 1 to n
Sorting (ct’d)
if A[i, j] = 1
Binary Trees
then repeat for each k from 1 to n
Hashing if A[j, k] = 1
Graphs then A[i, k] := 1
Topological sorting
Transitive closure
endif
Graph traversal
Shortest paths
endrepeat
Backtracking endif
Dynamic endrepeat
programming
endrepeat
Greedy
448/ 550
Computing the transitive closure III
DatAlg
Maritta Heisel
Example:
Introduction G:
1 4 3 2
Alg. Design
ADTs
A(G ) 1 2 3 4 A(G ∗ ) 1 2 3 4
Sorting
1 0 0 0 1 1 1 0 1 1
Alg. Theory
2 0 0 0 0 2 0 1 0 0
Sorting (ct’d)
Binary Trees
3 0 1 0 0 3 0 1 1 0
Hashing
4 0 0 1 0 4 0 1 1 1
Graphs
Topological sorting
Transitive closure
Graph traversal
Shortest paths
G ∗ = (V , E ∗ )
Backtracking 1 4 3 2
Dynamic
programming
Greedy
449/ 550
Computing the transitive closure IV
DatAlg
Maritta Heisel
Introduction
Alg. Design Problem: Not all paths are found by this algorithm! It only
ADTs finds paths of length ≤ 2.
Sorting
Alg. Theory
Straightforward solution: repeat application of algorithm
Sorting (ct’d) until no new edges are found.
Binary Trees
Hashing
However, this is not efficient: even one application of the
Graphs
algorithm is already of complexity O(| V |3 ) due to the three
Topological sorting nested for-loops in step (2).
Transitive closure
Graph traversal
Shortest paths
Backtracking
Dynamic
programming
Greedy
450/ 550
Computing the transitive closure V
DatAlg
Maritta Heisel
Introduction
Alg. Design More efficient solution: do not consider each possible

ADTs combination of subpaths, but only special combinations.
Sorting
Alg. Theory
A path from vertex i to vertex k is either an edge, or it can be
Sorting (ct’d)
decomposed into a path from i to j and a path from j to k,
Binary Trees such that j is the largest number of a vertex on the path
Hashing between vertex i to vertex k (without considering i and k
Graphs themselves). (The vertex numbers are the original ones, not
Topological sorting
Transitive closure
the ones obtained by topological sorting.)
Graph traversal
Shortest paths
Backtracking
Dynamic
programming
Greedy
451/ 550
Computing the transitive closure VI
DatAlg
Maritta Heisel
Introduction
Alg. Design
Now we compute paths in a way that assures that by
ADTs
combining the paths between i and j and between j and k only
Sorting paths are used that contain in-between vertices with a number
Alg. Theory smaller j (denoted i →Ej−1
∗ j and j →Ej−1
∗ k, respectively).
Sorting (ct’d)
Binary Trees
This can be achieved if the following invariant holds:
Hashing For the actual j, all paths (between all vertices i and k) are
Graphs known, which contain only in-between vertices with a number
Topological sorting
Transitive closure smaller than j.
Graph traversal
Shortest paths
Backtracking
Dynamic
programming
Greedy
452/ 550
Computing the transitive closure VII
DatAlg
Maritta Heisel
Introduction
Alg. Design It is obvious that this invariant is true in the beginning.

ADTs
To compute the paths for j + 1, one looks for paths
Sorting
i →Ej∗ j + 1 and j + 1 →Ej∗ k, which can then be combined to
Alg. Theory
Sorting (ct’d)
paths i →Ej+1
∗ k.
Binary Trees
This preserves the invariant. The resulting algorithm on
Hashing
slide 454 only exchanges the outer for-loops of the algorithm
Graphs
Topological sorting given on slide 448.
Transitive closure
Graph traversal
Shortest paths
Backtracking
Dynamic
programming
Greedy
453/ 550
Algorithm for transitive closure
DatAlg
Maritta Heisel
A[i, i] := 1
Introduction
endrepeat;
Alg. Design
{reflexive edges in diagonal of adjacency matrix}
ADTs
Sorting
step (2) repeat for each j from 1 to n
Alg. Theory
repeat for each i from 1 to n
Sorting (ct’d) if A[i, j] = 1
Binary Trees then repeat for each k from 1 to n
Hashing if A[j, k] = 1
Graphs then A[i, k] := 1
Topological sorting
Transitive closure endif
Graph traversal
Shortest paths
endrepeat
Backtracking endif
Dynamic endrepeat
programming
endrepeat
Greedy
454/ 550
Example
DatAlg
Maritta Heisel
Introduction
Alg. Design Executing the revised algorithm on the previous example:

ADTs
Sorting j =1 no new edge detected

Alg. Theory
j =2 no new edge detected
Sorting (ct’d)
Binary Trees j =3 new edge (4, 2) from (4, 3) and (3, 2)

Hashing
j =4 new edge (1, 2) from (1, 4) and (4, 2)
Graphs
Topological sorting new edge (1, 3) from (1, 4) and (4, 3)
Transitive closure
Graph traversal
Shortest paths
Backtracking
Dynamic
programming
Greedy
455/ 550
Traversing graphs: algorithm skeleton
DatAlg
Maritta Heisel
Similarly as for binary trees, one may want to visit all vertices
of a graph that are reachable from a given vertex:
Introduction
Alg. Design {visits all vertices in a given graph G = (V , E ), beginning from

ADTs vertex b}
Sorting var B: SET [Vertex]; {Set of already visited vertices}
Alg. Theory B := {b};
Sorting (ct’d)
repeat for all e ∈ E
Binary Trees
mark e as unused
Hashing
endrepreat;
Graphs
Topological sorting
while unused edges (v , v 0 ) ∈ E ∧ v ∈ B exist do
Transitive closure
Graph traversal
mark (v , v 0 ) as used;
Shortest paths
B := B ∪ {v 0 }
Backtracking
endwhile
Dynamic
programming {B contains all vertices reachable from b}
Greedy
456/ 550
Refining the algorithm skeleton
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs Introduce a set R ⊆ B (“Rand”) containing those vertices

Sorting of B from which there are still unused edges.
Alg. Theory
Replace the set B by a list, so that the order in which the
Sorting (ct’d)
vertices are visited becomes visible.
Binary Trees
Hashing
Assume that G is represented by an adjacency list. This
Graphs
makes it efficient to select the next edge to be taken.
Topological sorting
Transitive closure
Graph traversal
Shortest paths
Backtracking
Dynamic
programming
Greedy
457/ 550
Refined traversal algorithm
DatAlg
module TraverseG (G = (V , E ); b : Vertex)
Maritta Heisel B := hbi; R := {b};
Introduction mark all edges as unused;
Alg. Design while R 6= ∅ do
ADTs select vertex v ∈ R;
Sorting if no unused edge (v , v 0 ) ∈ E exists
Alg. Theory then R := R \ {v }
Sorting (ct’d)
else let (v , v 0 ) be the next unused edge ∈ E ;
Binary Trees
if v 0 ∈/B
Hashing
then B := B a hv 0 i;
Graphs
Topological sorting R := R ∪ {v 0 }
Transitive closure
Graph traversal
endif;
Shortest paths
mark (v , v 0 ) as used
Backtracking
endif
Dynamic
Greedy
458/ 550
endmodule
Remark
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting The traversal algorithm can be used to find connected

Alg. Theory components (“Zusammenhangskomponenten”) of a graph.
Sorting (ct’d)
Binary Trees Here, we have to assume that the graph is undirected.

Hashing
Graphs
Topological sorting
Transitive closure
Graph traversal
Shortest paths
Backtracking
Dynamic
programming
Greedy
459/ 550
Breadth-first vs. depth-first search
DatAlg
As for trees, there are different orders in which graphs can be
Maritta Heisel
traversed:
Introduction
Alg. Design
Breadth-first search: realize set R as a queue, i.e. replace
ADTs
R := {b} by R := enqueue(mt queue, b)
Sorting R 6= ∅ by R 6= mt queue
Alg. Theory select vertex v ∈ R by v := head(R)
Sorting (ct’d) R := R \ {v } by R := dequeue(R)
Binary Trees R := R ∪ {v 0 } by R := enqueue(R, v 0 )
Hashing
Depth-first search: realize set R as a stack , i.e. replace
Graphs
Topological sorting
R := {b} by R := push(b, mt stack)
Transitive closure
Graph traversal
R 6= ∅ by R 6= mt stack
Shortest paths select vertex v ∈ R by v := top(R)
Backtracking
R := R \ {v } by R := pop(R)
Dynamic
programming R := R ∪ {v 0 } by R := push(v 0 , R)
Greedy
460/ 550
Example
DatAlg
Maritta Heisel 1 2 3 4 5
Introduction
Alg. Design
ADTs 4 3 4 5 1
Sorting 4 . . . .
Alg. Theory
Sorting (ct’d)
5 3 3
Binary Trees
Hashing
Graphs 1 2 5
Topological sorting
Transitive closure
.
Graph traversal
Shortest paths
Backtracking Graph G with adjacency list

Dynamic
programming
Greedy
461/ 550
Example: breadth-first-search
DatAlg
4
Maritta Heisel
Introduction 5 3
Alg. Design
ADTs
1 2
Sorting
Alg. Theory
TraverseG; R as queue; starting vertex 1
Sorting (ct’d)
Binary Trees 5
Hashing 3 3 5
Graphs
Topological sorting 4 4 4 3 5
Transitive closure
Graph traversal 1 1 1 1 4 3 5
Shortest paths
Backtracking
Dynamic
programming
Resulting list: h1, 4, 3, 5i
Greedy
462/ 550
Example: depth-first-search
DatAlg
Maritta Heisel 4
Introduction
Alg. Design 5 3
ADTs
Sorting
1 2
Alg. Theory
Sorting (ct’d)
TraverseG; R as stack; starting vertex 1
Binary Trees
Hashing
5
Graphs
Topological sorting 4 4 4 3
Transitive closure
Graph traversal 1 1 1 1 1 1 1
Shortest paths
Backtracking
Dynamic Resulting list: h1, 4, 5, 3i

programming
Greedy
463/ 550
The shortest-path problem
DatAlg
Maritta Heisel
Introduction
Alg. Design Let G = (V , E ) be a graph.

ADTs Let c : E → R+ be a cost function for each edge.
Sorting
The problem is to find a path with minimal cost from a vertex
Alg. Theory
p to a vertex q in G . If the cost function is interpreted as the
Sorting (ct’d)
Binary Trees
distance between two vertices, then we find the shortest path.
Hashing Precondition: at least one path exists between p and q.
Graphs
Topological sorting
Transitive closure Question: how can we find out if this is the case?
Graph traversal
Shortest paths
Backtracking
Dynamic
programming
Greedy
464/ 550
Dijkstra’s algorithm for computing shortest paths I
DatAlg Dijkstra, E. W. A Note on Two Problems in Connexion with Graphs.

Maritta Heisel Numerische Mathematik, 1:269 – 271, 1959.
Introduction Uses the fact that for each shortest path (p, v1 , v2 , . . . vk , q)
Alg. Design each subpath (vi , . . . vj ) with 1 ≤ i < j ≤ k is a shortest path
ADTs from vi to vj .
Sorting
Alg. Theory
The vertices are subdivided into three sets:
Sorting (ct’d) A: the vertices for which the shortest path from p is
Binary Trees
known; vertices will be added to this set in order of
Hashing
increasing shortest path length from vertex p;
Graphs
Topological sorting B: the vertices from which the next vertex to be added to
Transitive closure
Graph traversal set A will be selected; this set comprises all those vertices
Shortest paths
that are connected to at least one vertex of set A but do
Backtracking
Dynamic
not yet belong to A themselves;
programming
C: the remaining vertices.
Greedy
465/ 550
Dijkstra’s algorithm for computing shortest paths II
DatAlg
Maritta Heisel
Introduction
Alg. Design The edges are also subdivided into three sets:
ADTs
I: the edges occurring in the minimal paths from vertex p
Sorting
to the vertices in set A
Alg. Theory
Sorting (ct’d) II: the edges from which the next edge to be placed in set
Binary Trees I will be selected; one and only one edge of this set will
Hashing lead to each vertex in set B;
Graphs
Topological sorting
III: the remaining edges (rejected or not yet considered).
Transitive closure
Graph traversal
Shortest paths
Backtracking
Dynamic
programming
Greedy
466/ 550
Dijkstra’s algorithm for computing shortest paths
III
DatAlg
The function pc : E → R+ stores the costs of the shortest
Maritta Heisel
known path from p to the vertice on which the edge points.
Introduction The set II is realized as priority queue with pc as priority
Alg. Design function.
ADTs
Sorting Algorithm:
Alg. Theory
{Initialization}
Sorting (ct’d)
A := {p};
Binary Trees
B := ∅;
Hashing
C := V \ {p};
Graphs I := ∅;
Topological sorting
Transitive closure II := new;
Graph traversal
Shortest paths
III := E;
Backtracking
pc = 0; {identically zero}
Dynamic
t := p; {node just transferred to A}
programming y := 0; {minimal pathcosts from p to t}
Greedy
467/ 550
IV
DatAlg
Maritta Heisel
Introduction repeat
Alg. Design {Step 1}
ADTs repeat for each e = (t,v) ∈ III
Sorting pc := pc[e 7→ y+c(e)];
Alg. Theory if v ∈ C then
Sorting (ct’d) B := B ∪ {v };
Binary Trees C := C \ {v };
Hashing II := insert(e, II);
Graphs III := III \ {e};
Topological sorting endif;
Transitive closure
Graph traversal
Shortest paths
Backtracking
Dynamic
programming
Greedy
468/ 550
Dijkstra’s algorithm for computing shortest paths V
DatAlg
Maritta Heisel
Introduction
Alg. Design if v ∈ B then

ADTs e’ := search(v, II); {search for the edge in II pointing to v}
Sorting if pc(e) < pc(e’) then
Alg. Theory II := insert(e, delete(e’, II));
Sorting (ct’d) III := III \ {e} ∪ {e 0 };
Binary Trees endif
Hashing endif
Graphs endrepeat;
Topological sorting
Transitive closure
Graph traversal
Shortest paths
Backtracking
Dynamic
programming
Greedy
469/ 550
VI
DatAlg
Maritta Heisel
Introduction
Alg. Design {Step 2}

ADTs e = (v,t) := min(II); {sets t}
Sorting A := A ∪ {t};
Alg. Theory B := B \ {t};
Sorting (ct’d) I := I ∪ {e};
Binary Trees II := delete min(II);
Hashing y := pc(e)
Graphs until q ∈ A endrepeat
Topological sorting
Transitive closure
Graph traversal
Shortest paths
Backtracking
Dynamic
programming
Greedy
470/ 550
Example: shortest path from P to Q
Initialization
DatAlg
Initialize A, B, C, I, II, III, t, and y.
Maritta Heisel
(5,0)
Introduction
R Q
Alg. Design
(2,0)
ADTs
Sorting (4,0)
Alg. Theory t=P
P (1,0)
Sorting (ct’d) y=0
Binary Trees
Hashing
(1,0) (7,0)
Graphs
Topological sorting
Transitive closure S T
Graph traversal
Shortest paths
(6,0)
Backtracking
Dynamic
programming
A B C (c(e),pc(e)) (c(e),pc(e)) (c(e),pc(e))
I: II: III:
Greedy
471/ 550
1st Iteration
DatAlg Step 1: Iterate all edges in III starting from t.

Maritta Heisel R,S ∈ C ⇒ move R,S to B and corresponding edges to II.
Introduction (5,0)
Alg. Design
R Q
ADTs
(2,2)
Sorting
Alg. Theory (4,0)

Sorting (ct’d) t=P
P (1,0)
Binary Trees y=0
Hashing
Graphs
Topological sorting (1,1) (7,0)
Transitive closure
Graph traversal
Shortest paths S T
Backtracking
(6,0)
Dynamic
programming
I: II: III:
Greedy
471/ 550
1st Iteration
DatAlg Step 2: Move minimal edge from II to I and its destination

Maritta Heisel from B to A.
Introduction
(5,0)
R Q
Alg. Design
ADTs (2,2)
Sorting
Alg. Theory (4,0)

Sorting (ct’d)
t=S
P (1,0)
Binary Trees y=1
Hashing
Graphs
(1,1) (7,0)
Topological sorting
Transitive closure
Graph traversal
Shortest paths
S T
(6,0)
Backtracking
Dynamic
programming A B C (c(e),pc(e)) (c(e),pc(e)) (c(e),pc(e))
I: II: III:
Greedy
471/ 550
2nd Iteration

Maritta Heisel T,Q ∈ C ⇒ move T,Q to B and corresponding edges to II.
Introduction (5,0)
Alg. Design
R Q
ADTs
(2,2)
Sorting
Alg. Theory (4,0)

Sorting (ct’d) t=S
P (1,0)
Binary Trees y=1
Hashing
Graphs
Transitive closure
Graph traversal
Shortest paths S T
Backtracking
(6,7)
Dynamic
programming
I: II: III:
Greedy
471/ 550
2nd Iteration

Introduction
(5,0)
R Q
Alg. Design
ADTs (2,2)
Sorting
Alg. Theory (4,0)

Sorting (ct’d)
t=R
P (1,0)
Binary Trees y=2
Hashing
Graphs
(1,1) (7,8)
Topological sorting
Transitive closure
Graph traversal
Shortest paths
S T
(6,7)
Backtracking
Dynamic
I: II: III:
Greedy
471/ 550
3rd Iteration

Maritta Heisel Q,T ∈ B ∧ new paths are both shorter ⇒ replace known paths
Introduction (5,7)
Alg. Design
R Q
ADTs
(2,2)
Sorting
Alg. Theory (4,6)

Sorting (ct’d) t=R
P (1,0)
Binary Trees y=2
Hashing
Graphs
Transitive closure
Graph traversal
Shortest paths S T
Backtracking
(6,7)
Dynamic
programming
I: II: III:
Greedy
471/ 550
3rd Iteration

Introduction
(5,7)
R Q
Alg. Design
ADTs (2,2)
Sorting
Alg. Theory (4,6)

Sorting (ct’d)
t=T
P (1,7)
Binary Trees y=6
Hashing
Graphs
(1,1) (7,8)
Topological sorting
Transitive closure
Graph traversal
Shortest paths
S T
(6,7)
Backtracking
Dynamic
I: II: III:
Greedy
471/ 550
4th Iteration

Maritta Heisel Q ∈ B ∧ new path is not shorter ⇒ keep known path
Introduction (5,7)
Alg. Design
R Q
ADTs
(2,2)
Sorting
Alg. Theory (4,6)

Sorting (ct’d) t=T
P (1,7)
Binary Trees y=6
Hashing
Graphs
Transitive closure
Graph traversal
Shortest paths S T
Backtracking
(6,7)
Dynamic
programming
I: II: III:
Greedy
471/ 550
4th Iteration

Maritta Heisel from B to A. Q ∈ A ⇒ algorithm terminates
Introduction (5,7)
Alg. Design
R Q
ADTs
(2,2)
Sorting
Alg. Theory (4,6)

Sorting (ct’d) t=Q
P (1,7)
Binary Trees y=7
Hashing
Graphs
Transitive closure
Graph traversal
Shortest paths S T
Backtracking
(6,7)
Dynamic
programming
I: II: III:
Greedy
471/ 550
DatAlg Result: I contains exactly one path from P to Q. This path is

Maritta Heisel the shortest path from P to Q.
Introduction (5,7)
Alg. Design
R Q
ADTs
(2,2)
Sorting
Alg. Theory (4,6)

Sorting (ct’d)
P (1,7)
Binary Trees
Hashing
Graphs
Transitive closure
Graph traversal
Shortest paths S T
Backtracking
(6,7)
Dynamic
programming
I: II: III:
Greedy
471/ 550
What have we learnt on graphs?
DatAlg
Maritta Heisel
Introduction Graphs are a data structure to represent net-like

Alg. Design structures.
ADTs
There are (at least) four different ways to represent
Sorting
graphs, which can be transformed into one another.
Alg. Theory
Sorting (ct’d)
Term representations of graphs are hardly useful.
Binary Trees Representing net-like structures makes it possible to
Hashing answer important questions about them.
Graphs
Topological sorting
Important algorithms on graphs include (among a great
Transitive closure
Graph traversal
many others) topological sorting, computing the transitive
Shortest paths
closure, visiting vertices, and shortest paths.
Backtracking
Dynamic
programming
Greedy
472/ 550
Backtracking
DatAlg
Maritta Heisel
General problem solving approach: determine the
Introduction solution(s) to a problem by trial and error.
Alg. Design
Break the problem down into substeps, which can be
ADTs
expressed recursively.
Sorting
Alg. Theory Build and traverse a searchtree. The nodes correspond to

Sorting (ct’d) the solutions of the substeps. Moving downwards in the
Binary Trees tree means moving towards a more complete solution.
Hashing
Build and traverse the searchtree in depth-first order.
Graphs
Moving upwards in the tree means backtracking, i.e.,
Backtracking
Dynamic
undoing the previous substep and returning to a node from
programming which it is promising to move downwards again.
Greedy
Complexity: often exponential.
Blockchain
Summary
473/ 550
General algorithm skeleton
DatAlg
Maritta Heisel
module try
Introduction initialize candidate selection
Alg. Design repeat select next candidate
ADTs if acceptable
Sorting then record candidate;
Alg. Theory if solution incomplete
Sorting (ct’d) then try next move
Binary Trees if ¬successful
Hashing then delete previous recording
Graphs endif
Backtracking endif
Dynamic
programming
endif
Greedy
until successful ∨ no further candidate
Blockchain
endrepeat
Summary
endmodule
474/ 550
Example: A knight’s tour
DatAlg
Consider an n × n board of chess. A knight can move
Maritta Heisel
according to the rules of chess.
Introduction The goal is now to find a tour covering the whole board.
Alg. Design Therefore, n2 − 1 moves must be calculated for one path,
ADTs so that every square has been visited exactly once.
Sorting
For covering the n × n squares on the board, one must
Alg. Theory
determine the next move or if a next move is possible,
Sorting (ct’d)
respectively.
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
475/ 550
A Knight’s tour: data representation, parameters
DatAlg
Maritta Heisel
Represent chess board by
Introduction
Alg. Design H : ARRAY (1..n)[ARRAY (1..n)[Integer ]],

ADTs
where n : Integer is the dimension of the board and
Sorting
H[x, y ] = i means that square (x, y ) was visited in the ith
Alg. Theory
move. (H[x, y ] = 0: square (x, y ) not yet visited)
Sorting (ct’d)
Binary Trees
Hashing
Parameters:
Graphs
x, y : Integer : coordinates from which the next move has
Backtracking
to be made.
Dynamic
i : Integer : number of the next move (to be recorded in H)
programming q : Bool: result parameter, indicates if trial was sucessful
Greedy or not.
Blockchain
Summary
476/ 550
A knight’s tour algorithm: algorithm
DatAlg
Maritta Heisel
module tryKnight(i, x, y : Integer ; q : Bool)
Introduction
var u, v : Integer ; q1 : Bool;
Alg. Design “initialize selection procedure”
ADTs repeat {u, v being possible moves according to chess rules}
Sorting if (1 ≤ u ≤ n) ∧ (1 ≤ v ≤ n) ∧ H[u, v ] = 0
Alg. Theory then H[u, v ] := i;
Sorting (ct’d) if i < n2
Binary Trees then tryKnight(i + 1, u, v , q1);
Hashing if ¬q1 then H[u, v ] := 0 endif
Graphs else q1 := true
Backtracking endif
Dynamic endif
programming
until q1 ∨ “no more candidates” endrepeat;
Greedy
Blockchain
q := q1
Summary
endmodule
477/ 550
Further backtracking problems
DatAlg
Maritta Heisel
Introduction
Alg. Design
Eight (n)-queens problem
ADTs
Sorting Ariadne’s thread

Alg. Theory Stable marriage problem
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Source:
Backtracking
N. Wirth: Algorithmen und Datenstrukturen, Teubner, 1975.
Dynamic
programming
Greedy
Blockchain
Summary
478/ 550
What have we learnt on backtracking?
DatAlg
Maritta Heisel
Introduction
Alg. Design
Backtracking is an algorithm principle that supports a
ADTs
trial-and-error-approach.
Sorting
Alg. Theory
Backtracking algorithms are recursive.
Sorting (ct’d) A search tree is built up and processed in a depth-first
Binary Trees manner.
Hashing
The complexity of backtracking algorithms is often
Graphs
exponential.
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
479/ 550
Dynamic Programming: Motivation I
DatAlg
Maritta Heisel
Introduction
Recall: recursive Fibonacci-algorithm:
Alg. Design
ADTs
module fibRecursive(n : Integer ) : Integer
Sorting
var i: Integer ;
Alg. Theory
i := n;
Sorting (ct’d)
if i = 0 ∨ i = 1
Binary Trees
Hashing
then return 1
Graphs
else return fibRecursive(i − 2) + fibRecursive(i − 1)
Backtracking
endif
Dynamic endmodule
programming
Greedy
Blockchain
Summary
480/ 550
Dynamic Programming: Motivation II
DatAlg Example i= 5 fib(5)

Maritta Heisel
Introduction fib(4) fib(3)

Alg. Design
ADTs
fib(3) fib(2) fib(2) fib(1)
Sorting
fib(2) fib(1) fib(1) fib(0) fib(1) fib(0)
Alg. Theory
Sorting (ct’d) fib(1) fib(0)

Binary Trees
Hashing
Repeated re-computation of values (e.g. computing three
Graphs
times from scratch fib(2) for i = 5).
Backtracking
Dynamic
The algorithm has exponential time complexity (O(2n )).
programming
Time complexity can be reduced to linear, if we save the
Greedy
solution we already computed : basic idea behind dynamic
Blockchain
Summary
programming.
481/ 550
Dynamic Programming: Basics I
DatAlg
Maritta Heisel
Introduction
Invented by Richard Bellman in the 1940’s - 50’s.
Alg. Design
ADTs Used to solve optimization problems.

Sorting
In this context, programming refers to optimizing and not
Alg. Theory
to the procedure of writing a program in a programming
Sorting (ct’d)
language.
Binary Trees
Hashing It is used to find the optimal solution for a problem,

Graphs not just a solution.
Backtracking Sometimes it is also used to compute the number of all
Dynamic
programming
possible solutions for a problem.
Greedy
Blockchain
Summary
482/ 550
Dynamic Programming: Basics II
DatAlg
Problem solving approach similar to divide-and-conquer:
Maritta Heisel
solve problem by reducing it to subproblems; however,
Introduction subproblems are not independent of each other, but
Alg. Design overlap.
ADTs
Naive solution: try out all possible combinations (see
Sorting
backtracking).
Alg. Theory
Sorting (ct’d) Problem with this approach: many partial combinations

Binary Trees are tried out over and over again.
Hashing Basic idea behind dynamic programming: save solutions of
Graphs already solved problems in a table.
Backtracking
Use the saved information if the same problem has to be
Dynamic
programming solved again later.
Greedy
Saving the solution for a problem and later
Blockchain
retrieving/reusing it is called memoization.
Summary
483/ 550
Knapsack problem I
DatAlg
Maritta Heisel
Knapsack problem:
Introduction
Alg. Design
You want to fill a knapsack with goods g from a set of
ADTs
goods M.
Sorting Each g ∈ M has a size s(g ) and a corresponding
Alg. Theory value v (g ) (with s(g ) and v (g ) being two positive natural
Sorting (ct’d) numbers).
Binary Trees
Each good can be packed repeatedly into the knapsack
Hashing
(k(g ) times). The capacity of the knapsack is C .
Graphs
Backtracking
Dynamic
Question:
programming How must we fill the knapsack in order to achieve the best
Greedy
total value without exceeding the capacity?
Blockchain
Summary
484/ 550
Knapsack problem II
DatAlg
Maritta Heisel
Introduction Formally: determine

Alg. Design
X
ADTs max{ (k(g ) ∗ v (g ))}
Sorting
g ∈M
Alg. Theory
Sorting (ct’d)
with k(g ) ∈ N and taking the condition
Binary Trees
Hashing
X
(k(g ) ∗ s(g )) ≤ C
Graphs
g ∈M
Backtracking
Dynamic
programming into account.
Greedy
Blockchain
Summary
485/ 550
Knapsack problem III
DatAlg
Maritta Heisel
Recursive definition of the optimal solution:
Let g1 , ..., gn be objects of M
Introduction ( n )
Alg. Design
X
Let P(g1 , ..., gn , C ) = max (k(gi ) ∗ v (gi )) be the
ADTs
i=1
Sorting
maximum.
Alg. Theory
Sorting (ct’d)
Cases:
Binary Trees 1. k(gn ) = 0 i.e. gn has not been packed.
Hashing
Then, P(g1 , ..., gn , C ) = P(g1 , ..., gn−1 , C )
Graphs
Backtracking 2. k(gn ) ≥ 1
Dynamic
programming Then, P(g1 , ..., gn , C ) = P(g1 , ..., gn , C − s(gn )) + v (gn ).
Greedy
If we choose gn once, we obtain a value of v (gn ), but the
Blockchain
remaining capacity is reduced by s(gn ).
Summary
486/ 550
Knapsack problem IV
DatAlg
Maritta Heisel
Introduction
Recursive equation:
Alg. Design
ADTs (
0 if n ≥ 1 ∧ C ≤ 0
Sorting
P(g1 , ..., gn , C ) = max{P(g1 , ..., gn−1 , C ),
Alg. Theory
P(g1 , ..., gn , C − s(gn )) + v (gn )} otherwise
Sorting (ct’d)
Binary Trees
Hashing Attention! An algorithm solving this equation directly would

Graphs have exponential complexity!
Backtracking
Dynamic Instead: proceed bottom-up and store intermediate values of P

programming
in a table (complexity is reduced to linear).
Greedy
Blockchain
Summary
487/ 550
Knapsack problem V
DatAlg
Maritta Heisel
Introduction Example:
Alg. Design
ADTs Object Size Value

Sorting
g1 3 3
Alg. Theory
g2 4 5
Sorting (ct’d)
g3 6 8
Binary Trees
g4 7 9
Hashing
Graphs
Backtracking As every object has a positive size, the maximum k(gi ) that
Dynamic
programming
can be chosen is C .
Greedy
Blockchain
Summary
488/ 550
Knapsack problem VI
DatAlg
Hence, we build up a table:
Maritta Heisel
Introduction 0 ... j ... C

Alg. Design
..
1 .
ADTs .. ..
Sorting . .
Alg. Theory ... ... P(g1 , ..., gi , j)
i gx
Sorting (ct’d)
..
Binary Trees .
Hashing n
Graphs
Backtracking gx denotes the last object packed.

Dynamic The table can be computed row-wise, because we can get
programming
P(g1 , ..., gi , j) by comparing P(g1 , ..., gi−1 , j) (same column,
Greedy
one row above) and P(g1 , ..., gi , j − s(gi )) + v (gi ), where
Blockchain
P(g1 , ..., gi , j − s(gi )) is in the same row, some columns ahead).
Summary
489/ 550
Knapsack problem VII
DatAlg
Maritta Heisel
Knapsack table:
Introduction
Alg. Design
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0 0 0 3 3 3 6 6 6 9 9 9 12 12 12 15
ADTs 1 g1 g1 g1 g1 g1 g1 g1 g1 g1 g1 g1 g1 g1
Sorting 2 0 0 0 3 5 5 6 8 10 10 11 13 15 15 16 18
Alg. Theory
g1 g2 g2 g1 g2 g2 g2 g2 g2 g2 g2 g2 g2
3 0 0 0 3 5 5 8 8 10 11 13 13 16 16 18 19
Sorting (ct’d) g1 g2 g2 g3 g3 g2 g3 g3 g3 g3 g3 g3 g3
Binary Trees 4 0 0 0 3 5 5 8 9 10 11 13 14 16 17 18 19
g1 g2 g2 g3 g4 g2 g3 g3 g3 g3 g4 g4 g4
Hashing
Graphs
Backtracking The table shows that the maximal value that can be obtained
Dynamic
programming
with capacity 15 is 19.
Greedy To find out what to pack, we move backwards in the table.
Blockchain
Summary
490/ 550
Knapsack problem VIII
DatAlg For our example: consider row 4.

Maritta Heisel
Take maximum: value is P(g1 , ..., g4 , 15), last packed g4 .
Introduction
Remove g4 and look at P(g1 , ..., g4 , 15 − 7), which is
Alg. Design
column 8.
ADTs
Sorting
There we find g2 ; we remove g2 and look at
Alg. Theory
P(g1 , ..., g4 , 8 − 4).
Sorting (ct’d) There we find g2 ; we remove g2 and look at
Binary Trees P(g1 , ..., g4 , 4 − 4) (column 0). This terminates the
Hashing process.
Graphs
Hence, the optimal packing is: once g4 and twice g2 .
Backtracking
Alternative optimal packing: twice g3 and once g1 . (Proceed as
Dynamic
programming shown above.)
Greedy In both cases the size of the goods is 15. However, it need not
Blockchain always be the case that the maximum with respect to the value
Summary
491/ 550
is also the maximum with respect to the size.
Complexity analysis
DatAlg
Maritta Heisel
Entry in table: O(1)
Introduction
Size of table: n ∗ (C + 1), hence table is created in
Alg. Design
O(n ∗ C )
ADTs Identifying maximum value: O(1) (or O(n), if linear
Sorting search is used)
Alg. Theory
Identifying the optimal packing: O(n + C ), as every
Sorting (ct’d)
backwards calculation either reduces the row-index or the
Binary Trees
column-index.
Hashing
Graphs The overall complexity of the knapsack problem: O(n ∗ C )

Backtracking
Dynamic
programming Note that the knapsack problem is NP complete if we only
Greedy consider n as input size. This means, that there is no known
Blockchain algorithm which solves the problem in O(p(n)), with p being
Summary polynomial.
492/ 550
General procedure
DatAlg
Maritta Heisel
Introduction
Alg. Design
Step 1 Define optimal solution.
ADTs
Sorting
Step 2 Reduce the optimal solution recursively to the
Alg. Theory solutions of smaller subproblems.
Sorting (ct’d) Step 3 Use a bottom-up approach to calculate the value
Binary Trees for the optimal solution.
Hashing
Step 4 Construct the solution from the calculated
Graphs
Backtracking
information.
Dynamic
programming
Greedy
Blockchain
Summary
493/ 550
Applicability conditions for dynamic programming
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs The optimal solution of the overall problem can be

Sorting constructed out of the optimal solutions of the
Alg. Theory subproblems.
Sorting (ct’d)
There exist overlapping subproblems (usually polynomial
Binary Trees
Hashing
many), with respect to the size of the input; only then it is
Graphs
of benefit to store the intermediate results.
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
494/ 550
What have we learnt on dynamic programming?
DatAlg
Maritta Heisel
Introduction
Alg. Design
Dynamic programming is used to solve optimization
ADTs
problems.
Sorting
Alg. Theory
The overall problem is reduced to overlapping
Sorting (ct’d) subproblems.
Binary Trees The solution is computed bottom-up; intermediate
Hashing results/values are stored in a table.
Graphs
From the table, the optimal solution can be obtained.
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
495/ 550
Greedy algorithms
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Similarly to dynamic programming, greedy algorithms are
Sorting
used to solve optimization problems.
Alg. Theory
Sorting (ct’d) In contrast to dynamic programming, no global optimal

Binary Trees solution is computed, but rather locally optimal decisions
Hashing are taken. That is why the approach is called greedy.
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
496/ 550
Example: Knapsack Problem I
DatAlg
Maritta Heisel
Introduction 2 possible greedy-strategies:

Alg. Design
ADTs
Sorting
select the most valuable element in each step that still fits
Alg. Theory
into the knapsack: Knapsack capacity: 15
Sorting (ct’d)
Binary Trees Object Size Value choose: g4 ; value: 9;

Hashing g1 3 3 remaining capacity: 8
Graphs g2 4 5 choose: g4 ; value: 9;
Backtracking g3 6 8 remaining capacity: 1
Dynamic g4 7 9 y not optimal
programming
Greedy
Blockchain
Summary
497/ 550
Example: Knapsack Problem II
DatAlg
Maritta Heisel select the relatively most valuable element (max( vs(g
(gi )
i)
))
Introduction
This will lead to the optimal solution in the case described
Alg. Design
above. However, this need not always be the case!
ADTs
Knapsack capacity: 50
Sorting v
Object Size Value s The strategy will choose
Alg. Theory
g1 30 90 3 g1 (value 90).
Sorting (ct’d)
g2 40 100 2,5 However, the optimal
Binary Trees
g3 50 110 2,2 would be g3 (value 110)
Hashing
Graphs
Backtracking Greedy algorithms do not always provide the optimal solution.

Dynamic In exchange, they are far less expensive than dynamic
programming
programming.
Greedy
Blockchain Complexity of the greedy-knapsack: O(C )

Summary
498/ 550
Optimal greedy algorithms
DatAlg
Maritta Heisel
There are problems where greedy algorithms produce optimal
solutions.
Introduction
Alg. Design
Example: resource planning problem.
ADTs
Sorting Given: set of activities S = {1, . . . , n} that want to use some

Alg. Theory resource, for example a lecture hall.
Sorting (ct’d)
The resource can only be used by one activity at a time.
Binary Trees
Hashing Each activity i has a starting time si and a completion time fi

Graphs with si < fi . The activity takes place in the time interval
Backtracking [si , fi ), i.e., the next activity can start at time fi .
Dynamic
programming Compatibility of activities: compatible(i, j) :⇔ si ≥ fj ∨ sj ≥ fi .
Greedy
Problem: compute the largest possible set of compatible
Blockchain
activities.
Summary
499/ 550
Greedy algorithm for resource planning I
DatAlg
Maritta Heisel
Introduction
The greedy algorithm for resource planning assumes that the
Alg. Design
ADTs
activities are ordered according to their completion times, i.e.
Sorting
f1 ≤ f2 ≤ · · · ≤ fn .
Alg. Theory
That condition can be established in time O(n log n) (see
Sorting (ct’d)
sorting algorithms).
Binary Trees
Let A be a set of activites. Then
Hashing
Graphs
comp(A) :⇔ (∀ i, j : A | i 6= j • compatible(i, j))
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
500/ 550
Greedy algorithm for resource planning II
DatAlg
Maritta Heisel Specification:

Introduction
Alg. Design var n : Integer ; s, f : ARRAY (1..n)[Integer ];

ADTs A : SET [Integer ];
Sorting
pre ordered(f ) ∧ (∀ i : 1..n • s[i] < f [i])
Alg. Theory
Sorting (ct’d) post A ⊆ {1, . . . , n} ∧ comp(A) ∧

Binary Trees ∀ B : SET [Integer ] • B ⊆ {1, . . . , n} ∧ comp(B)
Hashing ⇒ #B ≤ #A
Graphs
reads n, s, f
Backtracking
changes A
Dynamic
programming mem -
Greedy
where #M denotes the number of elements in the finite set M.
Blockchain
Summary
501/ 550
Greedy algorithm for resource planning III
DatAlg
Maritta Heisel
Algorithm:
Introduction
Alg. Design var n, i, j : Integer ; s, f : ARRAY (1..n)[Integer ];

ADTs A : SET [Integer ];
Sorting
A := {1};
Alg. Theory
Sorting (ct’d)
j := 1; {last activity added to A}
Binary Trees
i := 2;
Hashing
while i ≤ n do
Graphs if s[i] ≥ f [j]
Backtracking then A := A ∪ {i}; j := i
Dynamic endif;
programming
i := i + 1
Greedy
endwhile
Blockchain
Summary
502/ 550
Greedy algorithm for resource planning IV
DatAlg
Maritta Heisel
The “greediness” of this algorithm consists in choosing the
Introduction activity with the earliest possible completion time. Thus, as
Alg. Design
much time as possible is left for the remaining activities.
ADTs
Sorting
A correctness proof uses the loop invariant
Alg. Theory
Sorting (ct’d)
2 ≤ i ≤ n + 1 ∧ j < i ∧ pre ∧ postni−1
Binary Trees
Hashing
where postni−1 results from post by replacing n by i − 1.
Graphs
Backtracking
Overall complexity:
Dynamic
programming O(n log n) for sorting plus O(n) for the above algorithm, makes
Greedy O(n log n) in total.
Blockchain
Summary
503/ 550
Optimal storage on tapes I
DatAlg
Maritta Heisel
Given: n programs P1 , . . . , Pn with lengths l1 , . . . , ln .
Introduction
The programs are stored on a tape in order Pi1 , . . . , Pin .
Alg. Design
ADTs
Time to access Program Pij :
Sorting T (i1 , . . . , in , j) = li1 + · · · + lij .
Alg. Theory Average access time:
Sorting (ct’d)
n
X
1
Binary Trees T (i1 , . . . , in ) = n T (i1 , . . . , in , j)
Hashing j=1
j
n X
Graphs X
1
Backtracking
= n lij
Dynamic
j=1 i=1
programming Xn
1
Greedy = n (n − j + 1)lij
Blockchain j=1
Summary
504/ 550
Optimal storage on tapes II
DatAlg
Maritta Heisel
How to store the programs on the tape so that the
Introduction average access time becomes minimal?
Alg. Design 1
Let n be fixed; hence n can be ignored.
ADTs
Sorting
Alg. Theory Example: Let P1 , P2 , P3 have lengths 17, 5, 10.

Sorting (ct’d)
order acess time
Binary Trees
P1 P2 P3 17 + (17 + 5) + (17 + 5 + 10) = 71
Hashing
P1 P3 P2 17 + (17 + 10) + (17 + 10 + 5) = 76
Graphs
Backtracking
P2 P1 P3 59
Dynamic
P3 P1 P2 69
programming P2 P3 P1 52
Greedy
P3 P2 P1 57
Blockchain
Summary
505/ 550
Optimal storage on tapes III
DatAlg
Maritta Heisel
Introduction
Alg. Design
Fact:
ADTs
Sorting
If for the lenghts l1 , . . . , ln the relation l1 ≤ l2 ≤ · · · ≤ ln holds,
Alg. Theory
then the average access time is minimized by the order
Sorting (ct’d) P1 . . . Pn .
Binary Trees
Hashing Hence, a greedy strategy that always selects the shortest

Graphs program leads to the minimal average access time.
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
506/ 550
What have we learnt on greedy algorithms?
DatAlg
Maritta Heisel
Introduction
Alg. Design
Greedy algorithms treat optimization problems by taking
ADTs
locally optimal decisions.
Sorting
Alg. Theory
Hence, they may fail to determine the global optimum.
Sorting (ct’d) Greedy algorithms need less time than, e.g., dynamic
Binary Trees programming.
Hashing
There are problems where a greedy strategy leads to an
Graphs
optimal solution.
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
507/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Blockchain
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
508/ 550
Why blockchain?
DatAlg
Maritta Heisel
Initially built for supporting trading and tracking of virtual
Introduction values, for example currencies.
Alg. Design
Cash is only useful for local transactions.
ADTs
Sorting
High effort of maintaining transactions via the internet.
Alg. Theory A central authority is needed to validate and monitor the
Sorting (ct’d) transactions.
Binary Trees
Using a central system to store transactions leads to a
Hashing
single point of failure.
Graphs
Backtracking
Addtionally, cyber attacks may compromise that central
Dynamic system.
programming
Time for synchronizing the records of participants might
Greedy
Blockchain
be long.
Summary
509/ 550
Blockchain - Architecture
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
Graphs Traditional: Each party has its own records.

Backtracking Blockchain: Based on a peer-to-peer network.
Dynamic
programming Shared ledger containing all records.
Greedy Each party has the same set of records which is
Blockchain synchronized permanently.
Summary
510/ 550
Consensus models to validate transactions.
Blockchain - Data Structure I
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic Basically, a blockchain is a linked list.

programming
Greedy
Blockchain
Summary
511/ 550
Blockchain - Data Structure II
DatAlg
Maritta Heisel
Introduction Block Hash: Unique identifier for each block.

Alg. Design
Previous Block Hash: Link to the previous block
ADTs
(chaining).
Sorting
Alg. Theory Set of transactions: Each transaction is stored using a

Sorting (ct’d) hash function.
Binary Trees Transactions are static and cannot be altered or deleted.
Hashing
Graphs
To reverse or change a transaction, a new transaction is
Backtracking
necessary which is then visible to all participants
Dynamic
(traceability).
programming
Greedy
Blockchain
Summary
512/ 550
Blockchain - Permissioned vs. Permissionless
DatAlg
Maritta Heisel
Introduction
Alg. Design In a permissioned blockchain, the participants are

ADTs identifiable.
Sorting
For each transaction, permission rules can be definied
Alg. Theory
(read/write access).
Sorting (ct’d)
Binary Trees In a permissionless blockchain, participants are

Hashing anonymous.
Graphs Everyone can take part in the blockchain and has access to
Backtracking the transactions.
Dynamic
programming
Greedy
Blockchain
Summary
513/ 550
Blockchain - Use cases
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs Virtual currencies

Sorting
Trading
Alg. Theory
Sorting (ct’d) Electronic health records

Binary Trees Insurances
Hashing
eGovernment
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
514/ 550
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Bitcoin
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
515/ 550
Bitcoin
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs Invented by Satoshi Nakamoto in 2009.

Sorting
Virtual currency based on open source software.
Alg. Theory
Sorting (ct’d)
First usage of the blockchain technology.
Binary Trees Anonymous blockchain network.
Hashing
Coins are mined by solving mathematical puzzles.
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
516/ 550
Bitcoin Blockchain
DatAlg
Maritta Heisel
Each participant has a digital wallet and a key pair.
Introduction
Alg. Design
The wallet is stored in the blockchain.
ADTs To transfer bitcoins, a participant has to initiate a
Sorting transaction.
Alg. Theory
Such a transaction is broadcasted to the network.
Sorting (ct’d)
Binary Trees
The transaction has to be verified and commited to the
Hashing chain by other participants.
Graphs Using Proof-of-Work, a new block of transactions is
Backtracking appended to the blockchain.
Dynamic
programming Appending blocks to the chain is called Mining, because
Greedy Miners earn bitcoins.
Blockchain
Summary
517/ 550
Proof-of-Work Algorithm I
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Alg. Theory The networks maintains a list of new transactions.

Sorting (ct’d)
Miners select transactions from that list.
Binary Trees
Hashing Based on the transactions, the hash of the last block and
Graphs some other information, the new block hash is calculated.
Backtracking New block is broadcasted to the network and verified by
Dynamic
programming
other miners.
Greedy Network appends block to the chain.
Blockchain Currently, every 10 minutes a new block is published.
Summary
518/ 550 Problems?
Proof-of-Work Algorithm II
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting Computing a hash is very easy.

Alg. Theory
What happens if miners broadcast new blocks at the same
Sorting (ct’d)
time?
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
519/ 550
PoW Hashing / Mining I
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
520/ 550
PoW Hashing / Mining II
DatAlg
Maritta Heisel
Introduction
Alg. Design
Bitcoin uses SHA-256 for hashing.
ADTs
Sorting
Nonce is used to modify the hash.
Alg. Theory Target value is used to challenge the miners.
Sorting (ct’d)
Difficulty increases each 2016 blocks (around each 14
Binary Trees
days).
Hashing
Graphs Mining pools are used to increase the calculation power.

Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
521/ 550
Branching
DatAlg
Maritta Heisel
Introduction
Alg. Design
Miners compete for finding the next hash.
ADTs
Sorting
Only the first one will receive the bitcoins.
Alg. Theory When two blocks are broadcasted at the same time,
Sorting (ct’d) branching is possible.
Binary Trees
The longer the chain, the more work was necessary to
Hashing
build that chain.
Graphs
Backtracking
Assumption: Longest chain is the latest one.
Dynamic
programming
Greedy
Blockchain
Summary
522/ 550
51% Attack I
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
523/ 550
51% Attack II
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
524/ 550
51% Attack III
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
525/ 550
51% Attack IV
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
526/ 550
51% Attack V
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
Sorting
Alg. Theory
Sorting (ct’d)
Binary Trees
Hashing
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
527/ 550
Blockchain - Benefits
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
No central authority necessary.
Sorting All pariticipants hold the same copy of the ledger.
Alg. Theory Ensures transparency over all transactions.
Sorting (ct’d)
Can be used anonymous (privacy).
Binary Trees
Hashing Peer-to-peer ensures replication.

Graphs Hash functions provide security.
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
528/ 550
Blockchain - Disadvantages
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs
High power consumption to calculate hashes.
Sorting
Alg. Theory
Using majority voting is potentially unsecure (51% attack).
Sorting (ct’d) Anonymous participation allows usage for illegal stuff.
Binary Trees Scalability: Growing chain and increasing number of
Hashing
miners.
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
529/ 550
What have we learnt?
DatAlg
Maritta Heisel
Introduction
Alg. Design
A blockchain is a special kind of linked list.
ADTs Blocks are linked using unique identifiers.
Sorting
There are permissioned and permissionless networks.
Alg. Theory
Sorting (ct’d)
Records are stored in a shared ledger built on top of a
Binary Trees
peer-to-peer network.
Hashing To mine bitcoins, the miners are challenged with
Graphs mathematical puzzles.
Backtracking
A 51% attack allows double-spending of coins.
Dynamic
programming
Greedy
Blockchain
Summary
530/ 550
Summary: basic concepts
DatAlg
Maritta Heisel
The concept of an algorithm is central to computer
Introduction
Alg. Design
science.
ADTs Algorithms are timeless and do not depend on the
Sorting technology of the day.
Alg. Theory
An algorithm describes a process to be carried out by a
Sorting (ct’d)
processor.
Binary Trees
Hashing
To execute an algorithm on a computer, it must be
Graphs transformed into a program, expressed in some
Backtracking programming language.
Dynamic
programming
The description of an algorithm must be free of syntactic
Greedy
errors, semantic inconsistencies, and logical errors.
Blockchain
Summary
531/ 550
Summary: expressing algorithms
DatAlg
Maritta Heisel
Control structures for algorithms are sequence, selection,
Introduction
Alg. Design
and iteration. In combination with basic steps
ADTs
(assignments, evaluation of expressions), they suffice to
Sorting
express all conceivable algorithms.
Alg. Theory To re-use algorithms, we use modules.
Sorting (ct’d)
Modules have formal parameters, and they are called with
Binary Trees
actual parameters.
Hashing
Graphs
For each algorithm, it must be shown that it terminates.
Backtracking
Dynamic
programming For syntax definitions, we can use grammars or inductive
Greedy definitions.
Blockchain
Summary
532/ 550
Summary: developing algorithms by stepwise
refinement
DatAlg
Maritta Heisel
Algorithms can be developed by stepwise refinement.
Introduction
Alg. Design In stepwise refinement, we start out with a coarse-grained

ADTs algorithm and gradually replace non-basic steps by further
Sorting algorithms.
Alg. Theory
This approach corresponds to a top-down problem
Sorting (ct’d)
decomposition.
Binary Trees
Hashing
Stepwise refinement terminates when all steps of the
Graphs algorithm are basic, i.e., can be carried out by the
Backtracking envisaged processor.
Dynamic
programming
Therefore, it is necessary to know what steps are basic for
Greedy
a given processor.
Blockchain
Summary
533/ 550
Summary: specification notation, recursion
DatAlg
Maritta Heisel
Algorithms must be specified, using preconditions,
Introduction
postconditions and a classification of variables.
Alg. Design
ADTs
Specifications express what is to be done; an algorithm
Sorting
expresses how it is done. Hence, different algorithms may
Alg. Theory satisfy the same specification.
Sorting (ct’d) Specifications are necessary to apply the
Binary Trees design-by-contract principle. The caller of a module is
Hashing
responsible for guaranteeing the precondition, the module
Graphs
is responsible for establishing the postcondition.
Backtracking
Dynamic
Using recursion often leads to intuitive, comprehensible
programming algorithms. However, it must be demonstrated that the
Greedy
recursion terminates.
Blockchain
Summary
534/ 550
What have we learnt on abstract data types?
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs Explain what an abstract data type is

Sorting Give examples of ADTs and their characteristic properties
Alg. Theory
Define ADTs giving function signatures, preconditions,
Sorting (ct’d)
and axioms
Binary Trees
Hashing Explain the roles of constructors, selectors, and predicates

Graphs Implement ADTs in an object-oriented programming
Backtracking
language
Dynamic
programming
Greedy
Blockchain
Summary
535/ 550
What have we learnt about computability?
DatAlg
Maritta Heisel
Introduction
Not all problems are solvable by an algorithm.
Alg. Design
ADTs
A famous non-computable problem is the halting problem.
Sorting Proving that a problem is non-computable is done by
Alg. Theory contradiction.
Sorting (ct’d)
Other non-computable problems can be reduced to the
Binary Trees
halting problem or other problems known to be
Hashing
non-computable.
Graphs
Backtracking There are different degrees of non-computability: some

Dynamic problems are partially computable, others not even that.
programming
Greedy
Blockchain
Summary
536/ 550
What have we learnt about complexity? I
DatAlg
Maritta Heisel Many problems that are computable are nevertheless

Introduction
infeasible, because the algorithms solving the problems use
Alg. Design
too much resources.
ADTs Possible resources are time, memory and hardware.
Sorting
The complexity of an algorithm is the amount of resources
Alg. Theory
used.
Sorting (ct’d)
Binary Trees
The resources used are expressed in terms of a function on
Hashing the size of the input of an algorithm.
Graphs The asymptotic behavior of an algorithm results from the
Backtracking dominating term of the complexity function. It is often
Dynamic
programming
expressed using the big O notation.
Greedy Polynomial algorithms are often feasible, while exponential
Blockchain ones are clearly infeasible.
Summary
537/ 550
What have we learnt about complexity? II
DatAlg
Maritta Heisel
Introduction
Alg. Design We distinguish between worst-case and average-case

ADTs complexity of an algorithm.
Sorting
The complexity of a problem is the complexity of the best
Alg. Theory
algorithm solving the problem.
Sorting (ct’d)
Binary Trees That algorithm may be unknown.

Hashing Hence, we consider lower and upper bounds for the
Graphs
complexity of a problem.
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
538/ 550
What have we learnt about complexity? III
DatAlg
Maritta Heisel
Introduction
A complexity analysis can be performed by a recursive
Alg. Design procedure considering the syntactic constructs that make
ADTs up an algorithm.
Sorting The divide-and-conquer principle often leads to efficient
Alg. Theory
algorithms.
Sorting (ct’d)
Binary Trees
The complexity of recursive algorithms is determined by
Hashing
solving recurrence relations.
Graphs The sequential computation thesis supports the hypothesis
Backtracking that the class of feasible algorithms are the polynomial
Dynamic
programming
algorithms.
Greedy
Blockchain
Summary
539/ 550
What have we learnt about complexity? IV
DatAlg
Maritta Heisel Many practically relevant problems are judged to be

Introduction
infeasible.
Alg. Design Among them are the traveling salesperson problem, the
ADTs bin-packing problem, and the timetabling problem.
Sorting
In order to deal with infeasible problems, one can look for
Alg. Theory
approximate solutions, an algorithm which executes quickly
Sorting (ct’d)
Binary Trees
on the average input data, or a probabilistic algorithm.
Hashing
The problems in NP are characterized by the fact that a
Graphs
proposed solution can be verified in polynomial time.
Backtracking
Dynamic NP-complete problems are the hardest ones in NP. If for

programming
one of them, a polynomial algorithm is found, then P =
Greedy
NP.
Blockchain
Summary
540/ 550
What have we learnt about correctness?
DatAlg
Maritta Heisel A correctness proof provides more confidence in the

Introduction
correctness of an algorithm than testing. It embodies the
Alg. Design comprehension of how the algorithm solves its problem.
ADTs Total correctness consists of partial correctness and
Sorting termination.
Alg. Theory
Iteration and recursion are the difficult constructs in
Sorting (ct’d)
Binary Trees
proving correctness. They are treated by induction.
Hashing Induction needs an inductive hypothesis, a base case, and
Graphs an inductive step.
Backtracking
For loops, the inductive hypothesis is called loop invariant.
Dynamic
programming The termination of an algorithm is proved by showing that
Greedy some expression that is bounded from below strictly
Blockchain decreases with each iteration or recursive call.
Summary
541/ 550
What have we learnt on sorting algorithms?
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs You should now be able to

Sorting
Name and explain the sorting algorithms we have discussed
Alg. Theory
Apply the different sorting algorithms to an example
Sorting (ct’d)
Binary Trees Explain the complexity of the different algorithms

Hashing Explain the divide-and-conquer principle
Graphs
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
542/ 550
What have we learnt on trees?
DatAlg
Maritta Heisel
Introduction
Alg. Design
explain what a tree is, especially a binary tree
ADTs
explain the different functions defined on trees, e.g.,
Sorting
height, preorder traversal, etc.
Alg. Theory give the definition of search trees, and give and apply the
Sorting (ct’d) algorithms for searching, inserting, and deleting elements
Binary Trees in search trees
Hashing explain AVL trees as well as explain and apply the
Graphs algorithms for searching, inserting, and deleting elements
Backtracking in AVL trees
Dynamic
programming
give the motivation of using AVL trees instead of search
Greedy trees
Blockchain
Summary
543/ 550
What have we learnt on hashing?
DatAlg
Maritta Heisel
Introduction You should now be able to

Alg. Design
explain the ideas underlying hashing
ADTs
Sorting
give properties of good hash functions
Alg. Theory explain the differences between linking of collisions and
Sorting (ct’d) probing
Binary Trees
explain how searching, inserting and deleting work for the
Hashing
different hashing procedures
Graphs
Backtracking know the approximate number of search steps for the

Dynamic different hashing procedures
programming
Greedy
Blockchain
Summary
544/ 550
What have we learnt on graphs?
DatAlg
Maritta Heisel
Introduction Graphs are a data structure to represent net-like

Alg. Design structures.
ADTs
There are (at least) four different ways to represent
Sorting
graphs, which can be transformed into one another.
Alg. Theory
Sorting (ct’d)
Term representations of graphs are hardly useful.
Binary Trees Representing net-like structures makes it possible to
Hashing answer important questions about them.
Graphs
Important algorithms on graphs include (among a great
Backtracking
many others) topological sorting, computing the transitive
Dynamic
programming closure, visiting nodes , and computing shortest paths.
Greedy
Blockchain
Summary
545/ 550
What have we learnt on backtracking?
DatAlg
Maritta Heisel
Introduction
Alg. Design
Backtracking is an algorithm principle that supports a
ADTs
trial-and-error-approach.
Sorting
Alg. Theory
Backtracking algorithms are recursive.
Sorting (ct’d) A search tree is built up and processed in a depth-first
Binary Trees manner.
Hashing
The complexity of backtracking algorithms is often
Graphs
exponential.
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
546/ 550
What have we learnt on dynamic programming?
DatAlg
Maritta Heisel
Introduction
Alg. Design
Dynamic programming is used to solve optimization
ADTs
problems.
Sorting
Alg. Theory
The overall problem is reduced to overlapping
Sorting (ct’d) subproblems.
Binary Trees The solution is computed bottom-up; intermediate
Hashing results/values are stored in a table.
Graphs
From the table, the optimal solution can be obtained.
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
547/ 550
What have we learnt on greedy algorithms?
DatAlg
Maritta Heisel
Introduction
Alg. Design
Greedy algorithms treat optimization problems by taking
ADTs
locally optimal decisions.
Sorting
Alg. Theory
Hence, they may fail to determine the global optimum.
Sorting (ct’d) Greedy algorithms need less time than, e.g., dynamic
Binary Trees programming.
Hashing
There are problems where a greedy strategy leads to an
Graphs
optimal solution.
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
548/ 550
Literature I
DatAlg
Maritta Heisel Les Goldschlager and Andrew Lister.

Introduction Informatik – Eine moderne Einführung.
Alg. Design Hanser & Prentice-Hall, third edition, 1990.
ADTs
Manav Gupta.
Sorting
Blockchain for dummies.
Alg. Theory
for dummies, 2017.
Sorting (ct’d)
Binary Trees Bertrand Meyer.

Hashing Object-Oriented Software Construction.
Graphs Prentice-Hall, second edition, 1997.
Backtracking
Dynamic
Thomas Ottmann and Peter Widmayer.
programming Algorithmen und Datenstrukturen.
Greedy Spektrum Akademischer Verlag, fourth edition, 2002.
Blockchain
388
Summary
549/ 550
Literature II
DatAlg
Maritta Heisel
Introduction
Alg. Design
ADTs Robert Sedgewick.

Sorting Algorithmen in Java, Teil 1–4.
Alg. Theory Pearson Studium, third edition, 2003.
Sorting (ct’d)
Binary Trees
Niklaus Wirth.
Hashing
Algorithmen und Datenstrukturen.
Graphs
Teubner, second edition, 1979.
Backtracking
Dynamic
programming
Greedy
Blockchain
Summary
550/ 550

Algdat

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Algdat

Uploaded by

Copyright:

Available Formats

DatAlg

Graphs University Duisburg-Essen – Faculty of Engineering

Introduction Les Goldschlager, Andrew Lister

Alg. Design Implementation of ADTs

Maritta Heisel Industrial revolution

Sorting Performing that task manually would require a massive number

Dynamic =⇒ A program consists of a finite sequence of

Sorting (ct’d) tedious

ADTs Applications software

Sorting (ct’d) System software

Backtracking Computer hardware

Sorting Moreover algorithms are independent of the language in which

Introduction Recall: An algorithm is a description of how to carry out a

Introduction Computer programming is a comparatively recent activity;

Maritta Heisel The meaning of particular forms of expression in a language is

Sorting contains a potential inconsistency

Introduction 1. Make sense of the symbols in which the step is expressed

ADTs Logical error: omission of the factor of 2.

Binary Trees Therefore, algorithms have to be specified properly.

Maritta Heisel Design of nontrivial algorithms is very difficult.

Maritta Heisel Design of nontrivial algorithms is very difficult.

Maritta Heisel Design of nontrivial algorithms is very difficult.

Alg. Design The idea of stepwise refinement is to (repeatedly) break a

Maritta Heisel Initial version of an algorithm describing how to make a cup of

Introduction Refinement of the step put coffee in cup into a sequence of

Maritta Heisel The simplest form of an algorithm is a sequence of statements,

Alg. Design The statements are executed one at a time.

ADTs How could the sequence of statements be extended to

DatAlg We can design algorithms which contain nested

Hashing The choice between y and z can be refined similarly.

Maritta Heisel Final algorithm:

Introduction Lookup algorithm:

Maritta Heisel Power of iteration: allows a process of indeterminate

Sorting (ct’d) Failure to specify a terminating condition correctly is one of the

Sorting (ct’d) no side effect side effect

DatAlg For given inputs x, y , z of type integer , compute their

Maritta Heisel digit ::= 0 | 1 | ...| 9

Alg. Theory Why does this algorithm terminate?

Alg. Design Test if a number x has any nontrivial factors.

Binary Trees Remark: For x = 1 and x = 2 no possible factors exist. Hence,

Introduction var x, factor , y : Integer ;

Introduction Algorithm development task: determine largest of a list of

Introduction Preparation for detailed algorithm:

Introduction var l : LIST[Integer]; max, n : Integer;

Introduction Remember: The body of a post-tested loop is always executed

DatAlg Mathematical laws:

Introduction Definite Indefinite

Sorting Definite iteration is safer than indefinite iteration, because it is

Maritta Heisel repeat for each instance of an object

Maritta Heisel repeat for each instance of an object

ADTs var N, p, i : Integer ;

Introduction draw a square of side 10 cm;

Introduction For a square drawing module to be generally useful, it

DatAlg module drawsquare(size : Real)

Alg. Theory Add the return statement to the grammar.

DatAlg Modules can be nested, too.

Maritta Heisel module drawsquare(size : Real)

Alg. Theory For syntax definitions, we can use grammars or inductive

Alg. Design In stepwise refinement, we start out with a coarse-grained

Alg. Design caller must ensure precondition of module/algorithm

Introduction In order to develop correct algorithms (i.e., algorithms that

DatAlg We specify algorithms as follows:

ADTs x y x⇒y x y x⇔y