Data Structures and Algorithms Lecture 1: Introduction

Dwight Sabio Week 1

1

Christian Achievers for God and Country

What is an Algorithm?
A well-defined computational procedure that takes some value (or set of values) as input and produces some value (or set of values) as output Provides a step by step method for solving a computational problem. Is not dependent on any particular programming language, machine or 2 compiler.

Christian Achievers for God and Country

What is an Algorithm?
An organized sequence or list of clear steps or operations needed to solve a given programming problem 3 components: Inputs Steps or instructions Output Any well-defined computational procedure that takes some value(s) as input and produces some value(s) as output Sequence of computational steps that transform the input to output 3

Christian Achievers for God and Country

Algorithms and Cooking Recipes

A cooking recipe is a step-by-step procedure on how to cook/bake a dish An algorithm is similar to a cooking recipe because it provides a step-bystep procedure on how to solve a computational problem

Christian Achievers for God and Country

Algorithms and Computer Engineering

Can a particular task be accomplished by a computing device? A problem can be solved in many different ways Many algorithms can be used to solve the same problem

Christian Achievers for God and Country

Programmers
Different tools Different ways of specifying the procedure(s)

Chefs
Different Output Different Ingredients to create their final product

Christian Achievers for God and Country

Algorithms Not Recipes

An Algorithm
Takes an input Produces an output Well-defined computational procedure Solves a computational problem Independent of language and machine

A Computational Problem is a well-defined specification of the relationship between the input and the output
7

Christian Achievers for God and Country

Criteria
Input there are zero or more quantities which are externally supplied. Output at least one quantity is produced Definiteness each instruction must be clear and unambiguous Finiteness it terminates after a finite number of steps Effectiveness every instruction must be sufficiently basic that it can in principle be carried out by a person using only pencil 8 and paper.

Christian Achievers for God and Country

Computational Problem (Example)

Input: A set of n real numbers (a1, a2, , an) Output: A value S where S=g+l g = max (a1, a2, , an) l = min (a1, a2, , an) The sum of the greatest number and lowest number.
9

Christian Achievers for God and Country

Algorithm (Example)
Let A[i] be the ith number on the list (a1,a2,an)

1 2 3 4 5 6

Max, min = A[1] For i = 2 to n If A[i] > max then max = A[i] ElseIf A[i] < min then min = A[i] Next i Return max + min

10

Christian Achievers for God and Country

Design Issues in Algorithms

Correctness does the algorithm solve the computational problem? Efficiency how fast can the algorithm run?

11

Christian Achievers for God and Country

Efficient Algorithms
We choose between different algorithms based on their efficiency Usual measure of efficiency: Speed how long an algorithm takes to produce its result

12

Christian Achievers for God and Country

Assess efficiency of algorithms, independent of:

type of computer used, programming language, programming

Technology improves things by a constant factor only Even a supercomputer cannot rescue a bad algorithm
a faster algorithm on a slower computer will always win for sufficiently large inputs
13

Christian Achievers for God and Country

Algorithm Design Technique

Divide and Conquer
Divide: divide the problem into a number of subproblems Conquer: solve the subproblems recursively or in a straightforward manner Combine: merge the solutions to come up with the global solution

14

Christian Achievers for God and Country

Merge-Sorting in Arrays

15

Christian Achievers for God and Country

Merging: An Example

16

Christian Achievers for God and Country

Merge-Sort

17

Christian Achievers for God and Country

Analyzing Merge-Sort

18

Christian Achievers for God and Country

//Function to merge two pre-sorted arrays void MergeSort(apvector <int> &arrayA, apvector <int> &arrayB, apvector <int> &arrayC) { int indexA = 0; // initialize variables for the subscripts int indexB = 0; int indexC = 0; while((indexA < arrayA.length( )) && (indexB < arrayB.length( )) { if (arrayA[indexA] < arrayB[indexB]) { arrayC[indexC] = arrayA[indexA]; indexA++; //increase the subscript } else { arrayC[indexC] = arrayB[indexB]; indexB++; //increase the subscript } indexC++; //move to the next position in the new array } // Move remaining elements to end of new array when one merging array is empty while (indexA < arrayA.length( )) { arrayC[indexC] = arrayA[indexA]; indexA++; indexC++; } Example: Ascending while (indexB < arrayB.length( )) { Order arrayC[indexC] = arrayB[indexB]; Array A: {7, 12} indexB++; indexC++; Array B: {5, 7, 8} } return; }

19

Christian Achievers for God and Country

//Function to merge two pre-sorted arrays void MergeSort(apvector <int> &arrayA, apvector <int> &arrayB, apvector <int> &arrayC) { int indexA = 0; // initialize variables for the subscripts int indexB = 0; int indexC = 0; (( 0 < 2) && (0 < 3)) while((indexA < arrayA.length( )) && (indexB < arrayB.length( )) { //arrayA[0] < arrayB[0] if (arrayA[indexA] < arrayB[indexB]) // 7 < 5 FALSE { arrayC[indexC] = arrayA[indexA]; indexA++; } else { arrayC[indexC] = arrayB[indexB]; //arrayC[0] =5 indexB++; // indexB = 1 } indexC++; // indexC = 1 } // Move remaining elements to end of new array when one merging array is empty while (indexA < arrayA.length( )) { arrayC[indexC] = arrayA[indexA]; indexA++; indexC++; } while (indexB < arrayB.length( )) { arrayC[indexC] = arrayB[indexB]; indexB++; indexC++; } return; }

20

Christian Achievers for God and Country

//Function to merge two pre-sorted arrays void MergeSort(apvector <int> &arrayA, apvector <int> &arrayB, apvector <int> &arrayC) { int indexA = 0; // initialize variables for the subscripts int indexB = 0; int indexC = 0; (( 0 < 2) && (1 < 3)) while((indexA < arrayA.length( )) && (indexB < arrayB.length( )) { //arrayA[0] < arrayB[1] if (arrayA[indexA] < arrayB[indexB]) // 7 < 7 FALSE { arrayC[indexC] = arrayA[indexA]; indexA++; } else { arrayC[indexC] = arrayB[indexB]; //arrayC[1] =7 indexB++; // indexB = 2 } indexC++; // indexC = 2 } // Move remaining elements to end of new array when one merging array is empty while (indexA < arrayA.length( )) { arrayC[indexC] = arrayA[indexA]; indexA++; indexC++; } while (indexB < arrayB.length( )) { arrayC[indexC] = arrayB[indexB]; indexB++; indexC++; } return; }

21

Christian Achievers for God and Country

//Function to merge two pre-sorted arrays void MergeSort(apvector <int> &arrayA, apvector <int> &arrayB, apvector <int> &arrayC) { int indexA = 0; // initialize variables for the subscripts int indexB = 0; int indexC = 0; (( 0 < 2) && (2 < 3)) while((indexA < arrayA.length( )) && (indexB < arrayB.length( )) { //arrayA[0] < arrayB[2] if (arrayA[indexA] < arrayB[indexB]) // 7 < 8 TRUE { arrayC[indexC] = arrayA[indexA]; //arrayC[2] = 7 indexA++; //indexA = 1 } else { arrayC[indexC] = arrayB[indexB]; indexB++; } indexC++; // indexC = 3 } // Move remaining elements to end of new array when one merging array is empty while (indexA < arrayA.length( )) { arrayC[indexC] = arrayA[indexA]; indexA++; indexC++; } while (indexB < arrayB.length( )) { arrayC[indexC] = arrayB[indexB]; indexB++; indexC++; } return; }

22

Christian Achievers for God and Country

//Function to merge two pre-sorted arrays void MergeSort(apvector <int> &arrayA, apvector <int> &arrayB, apvector <int> &arrayC) { int indexA = 0; // initialize variables for the subscripts int indexB = 0; int indexC = 0; (( 1 < 2) && (2 < 3)) while((indexA < arrayA.length( )) && (indexB < arrayB.length( )) { //arrayA[1] < arrayB[2] if (arrayA[indexA] < arrayB[indexB]) // 12 < 8 FALSE { arrayC[indexC] = arrayA[indexA]; indexA++; } else { arrayC[indexC] = arrayB[indexB]; // arrayC[3] = 8 indexB++; //indexB = 3 } indexC++; // indexC = 4 } // Move remaining elements to end of new array when one merging array is empty while (indexA < arrayA.length( )) { arrayC[indexC] = arrayA[indexA]; indexA++; indexC++; } while (indexB < arrayB.length( )) { arrayC[indexC] = arrayB[indexB]; indexB++; indexC++; } return; }

23

Christian Achievers for God and Country

//Function to merge two pre-sorted arrays void MergeSort(apvector <int> &arrayA, apvector <int> &arrayB, apvector <int> &arrayC) { int indexA = 0; // initialize variables for the subscripts int indexB = 0; int indexC = 0; (( 1 < 2) && (3 < 3)) while((indexA < arrayA.length( )) && (indexB < arrayB.length( )) // FALSE { //arrayA[1] < arrayB[2] if (arrayA[indexA] < arrayB[indexB]) { arrayC[indexC] = arrayA[indexA]; indexA++; } else { arrayC[indexC] = arrayB[indexB]; indexB++; } indexC++; } // Move remaining elements to end of new array when one merging array is empty while (indexA < arrayA.length( )) { arrayC[indexC] = arrayA[indexA]; indexA++; indexC++; } while (indexB < arrayB.length( )) { arrayC[indexC] = arrayB[indexB]; indexB++; indexC++; } return; }

24

Christian Achievers for God and Country

//Function to merge two pre-sorted arrays void MergeSort(apvector <int> &arrayA, apvector <int> &arrayB, apvector <int> &arrayC) { int indexA = 0; // initialize variables for the subscripts int indexB = 0; int indexC = 0; (( 1 < 2) && (3 < 3)) while((indexA < arrayA.length( )) && (indexB < arrayB.length( )) // FALSE { //arrayA[1] < arrayB[2] if (arrayA[indexA] < arrayB[indexB]) { arrayC[indexC] = arrayA[indexA]; indexA++; } else { arrayC[indexC] = arrayB[indexB]; indexB++; } indexC++; } // Move remaining elements to end of new array when one merging array is empty while (indexA < arrayA.length( )) //1 < 2 TRUE { arrayC[indexC] = arrayA[indexA]; // arrayC[4] = 12 indexA++; // indexA = 2 indexC++; // indexC = 5 } while (indexB < arrayB.length( )) //(3 < 3) FALSE { arrayC[indexC] = arrayB[indexB]; indexB++; indexC++; } return; }

25

Christian Achievers for God and Country

//Function to merge two pre-sorted arrays void MergeSort(apvector <int> &arrayA, apvector <int> &arrayB, apvector <int> &arrayC) { int indexA = 0; // initialize variables for the subscripts int indexB = 0; int indexC = 0; (( 1 < 2) && (3 < 3)) while((indexA < arrayA.length( )) && (indexB < arrayB.length( )) // FALSE { //arrayA[1] < arrayB[2] if (arrayA[indexA] < arrayB[indexB]) { arrayC[indexC] = arrayA[indexA]; indexA++; } else { arrayC[indexC] = arrayB[indexB]; indexB++; } indexC++; } // Move remaining elements to end of new array when one merging array is empty while (indexA < arrayA.length( )) //2 < 2 FALSE { arrayC[indexC] = arrayA[indexA]; indexA++; indexC++; } while (indexB < arrayB.length( )) //(3 < 3) FALSE { arrayC[indexC] = arrayB[indexB]; indexB++; indexC++; } return; }

26

Christian Achievers for God and Country

Analyzing Algorithms
predicting the resources that the algorithm requires
memory communication bandwidth computer hardware computational time

27

Christian Achievers for God and Country

Why is there a need for algorithm analysis?

To study its behavior

What happens if the input size is increased?

To predict its performance

Time (processing speed) Space (memory)

Given two algorithms, A1 and A2 solving the same problem: which is better?
28

Christian Achievers for God and Country

Assumption
We are using a generic processor, Random Access Memory (RAM) model of computation
Instructions are executed ONE AT A TIME, no concurrent operations

Algorithms implemented as computer programs

29

Christian Achievers for God and Country

Algorithm Efficiency Analysis Methodologies

A priori analysis determines the efficiency of an algorithm based on or derived from mathematical or logical facts A posteriori analysis determines the efficiency of an algorithm based on actual experiments

30

Christian Achievers for God and Country

A Posteriori Analysis (Example)

10
Algo 1 1 sec

100
10 sec

Algo 2

3 sec

15 sec

Algo 1 seems more efficient than Algo2

31

Christian Achievers for God and Country

A Posteriori Analysis (Example)

10

100

1000

Algo 1 1 sec 10 sec 100 sec

Is Algo1 still faster than Algo2?

Algo 2 3 sec 15 sec

30 sec

32

Christian Achievers for God and Country

A Priori Analysis (Example)

Let A[i] be the ith number on the list (a1,a2,an)

1 2 3 4 5 6

Max, min = A[1] For i = 2 to n If A[i] > max then max = A[i] ElseIf A[i] < min then min = A[i] Next i Return max + min

33

Christian Achievers for God and Country

A Priori Analysis (Example)

Assumptions:
Instructions will be executed sequentially Each instruction takes c time units

From the example algorithm presented previously:

Line 1 = c Line 2-5 = [n-1] [2c (worst-case comparison) + c (assign)]

Line 6 = c Total = (3n 1) c

34

Christian Achievers for God and Country

GENERAL RULES FOR OPERATION COUNT

35

Christian Achievers for God and Country

Rule1: Declarations with no initialization Declarations with no initialization have no operation count (i.e. 0).

36

Christian Achievers for God and Country

Rule 2: Delimiters (such as { and }) Delimiters have no operation count (i.e. 0).

37

Christian Achievers for God and Country

Rule 3: Function Heading Function heading such as main() has no operation count (i.e. 0).

38

Christian Achievers for God and Country

Rule 4: Operators Each operator (whether arithmetic, logical, relational) has a running time of 1 (for simplicity only, though it may not hold in reality).

39

Christian Achievers for God and Country

Rule 5: Expressions The operation count of an expression depends on the number of operators (arithmetic, logical, relational) are there.

40

Christian Achievers for God and Country

Rule 6: Assignment Statement The operation count of an assignment statement is 1 plus the number of operators in the expression. Arithmetic operators (such as +=, -=, *=) have an operation count of 2.

41

Christian Achievers for God and Country

Rule 7: Function call 1(function call) + operation count for the operators + operation count of the function

42

Christian Achievers for God and Country

Rule 8: if/if-else Statement Given an if-else statement of the form: if (<condition>) <S1>; else <S2>; The overall operation count of the if-else statement is the operation count of the <condition> plus the maximum of the operation counts of <S1> and <S2>. The keyword else does not consume any operation 43 count.

Christian Achievers for God and Country

Rule 9: for Statement Given a for statement of the form: for (<initialization>; <condition/test>; <increment>) { <S>; } The total operation count of the for statement is operation count of the <initialization> + operation count of the <condition/test> * (number of iterations + 1) + operation count of <increment> * number of iterations + operation count of <S> * number of 44 iterations

Christian Achievers for God and Country

Rule 10: while Statement Given the while statement of the form while (<condition>) { <S>; } The operation count is the <condition> * (number of iterations) + operation count of <S> * number of iterations.
45

Christian Achievers for God and Country

Rule 11: do-while Statement Given the do-while statement of the form do { <S>; } while (<condition>); The operation count is the <S> * number of iterations + operation count of the <condition> * number of iterations.
46

Christian Achievers for God and Country

Rule 12: Nested Loops Analyze these inside out. The total operation count of a statement inside a group of nested loops is the operation count of the statement multiplied by the product of all the sizes of all the loops.

47

Christian Achievers for God and Country

Analyze the following code

for (j = 1; j < 10; j++) printf (I love Ice Cream);
1 10 9 (2)

T(n) = 38

Christian Achievers for God and Country

Analyze the following code

for (j = 1; j <= 10; j++) printf (I love Ice Cream);
1 11 10(2)

10

T(n) = 42

Analyze the following code

for (j = 5; j < 10; j++) printf (I love Ice Cream);
1 6 5(2)

Christian Achievers for God and Country

T(n) = 22

Christian Achievers for God and Country

Analyze the following code

for (j = 5; j <= 10; j++) printf (I love Ice Cream);
1 7 6(2)

T(n) = 26

Christian Achievers for God and Country

Lets Generalize
for (j = 1; j < 10; j++) printf (I love Ice Cream); 1 + 10 + 9(2) 9

for (j = 5; j < 10; j++) printf (I love Ice Cream);

1 + 6 + 5(2) 5

Christian Achievers for God and Country

Lets Generalize
for (j = a; j < b; j++) printf (I love Ice Cream);
1 ba+1 ba ba

Christian Achievers for God and Country

Lets Generalize
for (j = 1; j <= 10; j++) printf (I love Ice Cream); 1 + 11 + 10(2) 10

for (j = 5; j <= 10; j++) printf (I love Ice Cream);

1 + 7 + 6(2) 6

Christian Achievers for God and Country

Lets Generalize
for (j = a; j <= b; j++) printf (I love Ice Cream);
1 ba+2 ba+1 ba+1

Christian Achievers for God and Country

Sample Problems
(a) for (j = 0; j < n; j++) printf (Sample Problem 1\n);

(b) for (j = 0; j < n; j++) { printf (Operation Count - ); printf (Sample Problem 2\n); }

Christian Achievers for God and Country

(c) int factorial (int nVal) { int j; int nFactorial = 1;

for (j = 1; j <= nVal; j++) nFactorial *= j; return nFactorial;

Christian Achievers for God and Country

(d) int factorial (int nVal) { int j; int nFactorial = 1;

for (j = nVal; j > 0; j--) nFactorial *= j; return nFactorial;

Christian Achievers for God and Country

(e)

59