Professional Documents
Culture Documents
Chapter 01
Chapter 01
and Algorithms
Pham Quang Dung and Do Phan Thuan
1/1
First example
2/1
How many subsequences?
3/1
Direct algorithm — O(n3 )
n n2 +n
Scan all possible subsequences 2 +n = 2
Compute and keep the largest weight subsequence
4/1
Direct algorithm
Faster algorithm — O(n2 )
Pj Pj−1
Observation: k=i a[k] = a[j] + k=i a[k]
5/1
Divide And Conquer
Divide the sequence into 2 subsequences at the middle s = s1 :: s2
The largest subsequence might
I be in s1 or
I be in s2 or
I start at some position of s1 and end at some position of s2
Java code:
1 private long maxSeq ( int i , int j ){
2 if ( i == j ) return a [ i ];
3 int m = ( i + j )/2;
4 long ml = maxSeq (i , m );
5 long mr = maxSeq ( m +1 , j );
6 long maxL = maxLeft (i , m );
7 long maxR = maxRight ( m +1 , j );
8 long maxLR = maxL + maxR ;
9 long max = ml > mr ? ml : mr ;
0 max = max > maxLR ? max : maxLR ;
1 return max ;
2 }
3 public long algo3 ( int [] a ){
4 int n = a . length ;
5 return maxSeq (0 ,n -1);
6 }
6/1
Divide and Conquer — O(n log n)
7/1
Dynamic programming
General principle
Division: divide the initial problem into smaller similar problems
(subproblems)
Storing solutions to subproblems: store the solution to subproblems
into memory
Aggregation: establish the solution to the initial problem by
aggregating solutions to subproblems stored in the memory
8/1
Dynamic programming
Largest subsequence
Division:
I Let si be the weight of the largest subsequence of a1 , . . . , ai ending at
ai
Aggregation:
I s1 = a1
I si = max{si−1 + ai , ai }, ∀i = 2, . . . , n
I Solution to the original problem is max{s1 , . . . , sn }
Number of basic operations is n (best algorithm)
9/1
Dynamic programming — O(n)
10 / 1
Analyzing algorithms
11 / 1
Analyzing algorithms
n3 n2 n
algo1: T (n) = 6 + 2 + 3
n2 n
algo2: T (n) = 2 + 2
algo3:
I Count the number of addition (”+”) operation T (n)
(
0 if n = 1
T (n) = n n
T ( 2 ) + T ( 2 ) + cn if n > 1
12 / 1
Analyzing algorithms
Worst-case running time: the longest running time for any input of
size n
Best-case running time: the shortest running time for any input of
size n
Average-case running time: probabilistic analysis (make assumption of
a distribution of the input) yields expected running time
13 / 1
Order of growth
14 / 1
Asymptotic notations
15 / 1
Analysis of algorithms
Experiments studies
Write a program implementing the algorithm
Execute the program on a machine with different input sizes
Measure the actual execution times
Plot the results
16 / 1
Analysis of algorithms
17 / 1
Analysis of algorithms
18 / 1
Analysis of algorithms
19 / 1
Analysis of algorithms
20 / 1
Analysis of algorithms
21 / 1
Analysis of algorithms
22 / 1
Master theorem
Example
T (n) = 3T (n/4) + cn2 ⇒ T (n) = Θ(n2 )
T (n) = 2T (n/2) + n0.5 ⇒ T (n) = Θ(n)
T (n) = 16T (n/4) + n ⇒ T (n) = Θ(n2 )
T (n) = T (3n/7) + 1 ⇒ T (n) = Θ(logn)
23 / 1
Sorting
24 / 1
A sorting algorithm is called in-place if the size of additional memory
required by the algorithm is O(1) (which does not depend on the size
of the input array)
A sorting algorithm is called stable if it maintains the relative order
of elements with equal keys
A sorting algorithm uses only comparison for deciding the order
between two elements is called Comparison-based sorting
algorithm
25 / 1
Insertion Sort
At iteration k, put the k th element of the original list in the right
order of the sorted list of the first k elements (∀k = 1, . . . , n)
Result: after k th iteration, we have a sorted list of the first k th
elements of the original list
1 void insertion_sort ( int a [] , int n ){
2 int k ;
3 for ( k = 2; k <= n ; k ++){
4 int last = a [ k ];
5 int j = k ;
6 while ( j > 1 && a [j -1] > last ){
7 a [ j ] = a [j -1];
8 j - -;
9 }
0 a [ j ] = last ;
1 }
2 }
26 / 1
Selection Sort
Put the smallest element of the original list in the first position
Put the second smallest element of the original list in the second
position
Put the third smallest element of the original list in the third position
...
27 / 1
Bubble sort
Pass from the beginning of the list: compare and swap two adjacent
elements if they are not in the right order
Repeat the pass until no swaps are needed
28 / 1
Merge sort
Divide-and-conquer
Divide the original list of n elements into two lists of n/2 elements
Recursively merge sort these two lists
Merge the two sorted lists
29 / 1
Merge sort
30 / 1
Merge sort
31 / 1
Quick sort
32 / 1
Quick sort
33 / 1
Quick sort
1 int partition ( int a [] , int L , int R , int indexPivot )
2 int pivot = a [ indexPivot ];
3 swap ( a [ indexPivot ] , a [ R ]); // put the pivot in the e
4 int storeIndex = L ; // store the right position of
5
4 return storeIndex ;
5 }
34 / 1
Heap sort
35 / 1
Heap sort - Heap structure
10
9 7
4 5 6 1
3 2 1 2 3 4 5 6 7 8 9
10 9 7 4 5 6 1 3 2
36 / 1
Heap sort
37 / 1
Heap sort
38 / 1
Heap sort
1 void buildHeap ( int a [] , int n ){
2 // array is a [1.. n ]
3 for ( int i = n /2; i >= 1; i - -){
4 heapify (a ,i , n );
5 }
6 }
7
39 / 1
Comb sort
Comb sort is a relatively simple and quick sorting algorithm originally
designed by Dobosiewicz in 1980. Later it was rediscovered by Lacey and
Box in 1991. Comb sort improves on bubble sort.
Bubble sort: any two comparing elements has a gap of 1;
Comb sort: the gap can be much more than 1 and goes down (for
each iteration of outer loop) in steps of shrink factor: [ input size /
shrink factor, input size / shrink factorˆ2, input size / shrink
factorˆ3, . . . , 1 ];
The shrink factor has a great effect on the efficiency of comb sort.
The value 1.3 has been suggested as an ideal shrink factor by the
authors of the original article after empirical testing on over 200,000
random lists. A value too small slows the algorithm down by making
unnecessarily many comparisons, whereas a value too large fails to
effectively deal with turtles.
The final stage of the sort is thus equivalent to a bubble sort, but by
this time most turtles have been dealt with, so a bubble sort will be
efficient.
40 / 1
Comb sort
1 void combsort ( int a [] , int n ){
2 gap = n // Initialize gap size
3 shrink = 1.3; // Set the gap shrink factor
4 bool sorted = false ;
5
6 while (( gap > 1) || ( sorted == false )) {
7 // Update the gap value for a next comb
8 gap = floor ( gap / shrink );
9 if ( gap < 1) gap = 1;
10 int i = 0;
11 if ( gap == 1) sorted = true ;
12
13 // A single " comb " over the input list
14 while ( i + gap < n ) { // See Shell sort for a similar idea
15 if ( a [ i ] > a [ i + gap ]) {
16 swap ( a [ i ] , a [ i + gap ]);
17 if ( gap == 1) sorted = false ;
18 // If this assignment never happens within the loop ,
19 // then there have been no swaps and the list is sorted .
20 }
21 i := i + 1
22 }
23 }
24 }
41 / 1
Data structures
List
Stack
Queue
42 / 1
List
43 / 1
Stacks
An ordered list in which all insertions and deletions are made at one
end (called top)
Principle: the last element inserted into the stack must be the first
one to be removed (Last-In-First-Out)
Operations
I Push(x, S): push an element x into the stack S
I Pop(S): remove an element from the stack S, and return this element
I Top(S): return the element at the top of the stack S
I Empty(S): return true if the stack S is empty
44 / 1
Queues
An ordered list in which the insertions are made at one end (called
tail) and the deletions are made at the other end (called head)
Principle: the first element inserted into the queue must be the first
one to be removed (First-In-First-Out)
Applications: items do not have to be processed immediately but they
have to be processed in FIFO order
I Data packets are stored in a queue before being transmitted over the
internet
I Data is transferred asynchronously between two processes: IO buffered,
piples, etc.
I Printer queues, keystroke queues (as we type at the keyboard), etc.
45 / 1
Queues
Operations
I Enqueue(x, Q): push an element x into the queue Q
I Dequeue(Q): remove an element from the queue Q, and return this
element
I Head(Q): return the element at the head of the queue Q
I Tail(Q): return the element at the tail of the queue Q
I Empty(Q): return true if the queue Q is empty
46 / 1
Java Libraries
List
I ArrayList (dynamic array): get(int index), size(), remove(int index),
add(int index, Object o), indexOf(Object o)
I LinkedList (doubly linked list): remove, poll, element, peek, add, offer,
size
Stack
I push, pop, size
Queue
I LinkedList
I remove, poll, element, peek, add, offer, size
Set
I Collection of items
I Methods: add, size, contains
Map
I Map an object (key) to another object (value)
I Methods: put, get, keySet
47 / 1
Examples
1 package week2 ;
2
3 import java . util . ArrayList ;
4
5 public class E x a m p l e A r ra y L i s t {
6
7 public E x a m p l eA r r a y L i s t (){
8 ArrayList < Integer > L = new ArrayList ();
9 for ( int i = 1; i <= 10; i ++)
0 L . add ( i );
1 for ( int i = 0; i < L . size (); i ++){
2 int item = L . get ( i );
3 System . out . print ( item + " " );
4 }
5 System . out . println ( " size of L is " + L . size ());
6 }
7 public static void main ( String [] args ) {
8 E x a m p l e A r r a y L i s t EAL = new E x a m p l e A r r a y L i s t ();
9 }
0 }
48 / 1
Examples
1 package week2 ;
2 import java . util . HashSet ;
3 public class ExampleSet {
4
5 public void test (){
6 HashSet < Integer > S = new HashSet ();
7 for ( int i = 1; i <= 10; i ++)
8 S . add ( i );
9 for ( int i : S ){
0 System . out . print ( i + " " );
1 }
2 System . out . println ( " S . size () = " + S . size ());
3 System . out . println ( S . contains (20));
4 }
5 public static void main ( String [] args ) {
6 ExampleSet ES = new ExampleSet ();
7 ES . test ();
8 }
9 }
49 / 1
Examples
1 package week2 ;
2 import java . util . LinkedList ;
3 import java . util . Queue ;
4 public class ExampleQueue {
5 public static void main ( String [] args ) {
6 Queue Q = new LinkedList ();
7 /*
8 * Q . element (): return the head of the queue without removing it .
9 * If Q is empty , then raise exception
0 * Q . peek (): return the head of the queue without removing it .
1 * If Q is empty , then return null
2 * Q . remove (): remove and return the head of the queue .
3 * If Q is empty , then raise exception
4 * Q . poll (): remove and return the head of the queue .
5 * If Q is empty , then return null
6 * Q . add ( e ): add an element to the tail of Q .
7 * If no space available , then raise exception
8 * Q . offer ( e ): add an element to the tail of Q . If no space availa
9 * then return false . Otherwise , return true
0 */
1 for ( int i = 1; i <= 10; i ++) Q . offer ( i );
2 while ( Q . size () > 0){
3 int x = ( int ) Q . remove ();
4 System . out . println ( " Remove " + x + " , head of Q is " + Q . peek ())
5 } 50 / 1
Examples
1 package week2 ;
2
3 import java . util .*;
4 public class ExampleStack {
5 public ExampleStack (){
6 /*
7 * S . push ( e ): push an element to the stack
8 * S . pop : remove the element at the top of the stack and return it
9 */
0 Stack S = new Stack ();
1 for ( int i = 1; i <= 10; i ++)
2 S . push ( i + " 000 " );
3 while ( S . size () > 0){
4 String x = ( String ) S . pop ();
5 System . out . println ( x );
6 }
7 }
8 public static void main ( String [] args ) {
9 ExampleStack S = new ExampleStack ();
0 }
1 }
51 / 1
Examples
1 package week2 ;
2
3 import java . util . HashMap ;
4 public class E xampleH ashMap {
5
6 public Examp leHashMa p (){
7 HashMap < String , Integer > m = new HashMap < String , Integer >();
8 m . put ( " abc " ,1);
9 m . put ( " def " , 1000);
0 m . put ( " xyz " , 100000);
1 for ( String k : m . keySet ()){
2 System . out . println ( " key = " + k + " map to " + m . get ( k ));
3 }
4 }
5 public static void main ( String [] args ) {
6 Examp leHashM ap EHM = new ExampleH ashMap ();
7 }
8 }
52 / 1
Examples
1 package week2 ;
2 import java . util . Scanner ;
3 import java . util . HashMap ;
4 import java . io . File ;
5 public class CountWords {
6 public CountWords ( String filename ){
7 HashMap < String , Integer > count = new HashMap < String , Integer >();
8 try {
9 Scanner in = new Scanner ( new File ( filename ));
0 while ( in . hasNext ()){
1 String s = in . next ();
2 if ( count . get ( s ) == null )
3 count . put (s , 0);
4 count . put (s , count . get ( s ) + 1);
5 }
6 for ( String w : count . keySet ())
7 System . out . println ( " Word " + w + " appears " + count . get ( w ) +
8 in . close ();
9 } catch ( Exception ex ){
0 ex . p r in tS ta c kT ra ce ();
1 }
2 }
3 public static void main ( String [] args ) {
4 CountWords CW = new CountWords ( " data \\ week2 \\ CountWords . txt " );
5 } 53 / 1
Checking parentheses expression
54 / 1
Water Jug Problem
There are two jugs, a a-gallon one and a b-gallon one (a, b are positive
integer). There is a pump with unlimited water. Neither jug has any
measuring marking on it. How can you get exactly c-gallon jug (c is a
positive integer)?
55 / 1
Water Jug Problem
Search problem
State (x, y ): quantity of water in two jugs
Neighboring states
I (x, 0)
I (0, y )
I (a, y )
I (x, b)
I (a, x + y − a) if x + y >= a
I (x + y , 0) if x + y < a
I (x + y − b, b) if x + y >= b
I (0, x + y ) if x + y < b
Final states: (c, y ) or (x, c)
56 / 1
Water Jug Problem
Algorithm 1: WaterJug(a, b, c)
Q ← ∅;
Enqueue((a, b), Q);
Visited(a, b) ← TRUE;
while Q is not empty do
(x, y ) ← Dequeue(Q);
foreach (x 0 , y 0 ) ∈ neighboring states of (x, y ) do
if (x 0 = c ∨ y 0 = c) then
Solution();
BREAK;
else
if NOT Visited(x 0 , y 0 ) then
Enqueue((x 0 , y 0 ), Q);
Visited(x 0 , y 0 ) ← TRUE;
57 / 1