# Data Structures and Algorithms (Introduction

)

Jose Rizal University, Mandaluyong City, Philippines

References:
• Data Structures and Algorithms in JAVA Goodrich and Tamassia (2004) John Wiley & Sons, Inc. • Data Structures & Algorithm Analysis in C Mark Allan Weiss. (1993) Benjamin Cummings Publishing Company, Inc./Jemma, Inc., Philippines

2

Questions you need to consider in this topic
• What are data structures and algorithms? • What good will it do me to know about them? • When does it make sense to apply what I learn about data structures and algorithm?
3

Data Type: the beginning
• A data type is a well-defined collection or category of data with a well-defined set of operations on it. (i.e., integers, float, characters) A data type consists of
1. a domain (= a set of values) 2. a set of operations. Question: Why are data types important in programming? Can we consider data types as basic representation of data inside the computer?
4

What are data structures?
• Data structures are ways in which data is arranged in your computer's memory (or stored on disk). • It is a general method of storing and accessing data that optimizes or organizes one or more aspects of data access. • Data structures are used to implement Abstract Data Types, which are mathematical models with a collection of operations defined on the model. They specify the type of data stored, what its operations do but not how it is done. Question: If we already have data types to represent data, why do we need data structures? What is the difference between ADTs and data types?
5

Database: An Example
• Let’s define a database as a collection of interrelated data. • A database as defined above will be composed of records. Records are the basic unit that provide a format for storing information. • A record in turn, is usually divided into several fields. A field holds a particular kind of data. Question
How do records and fields in a database depict the relationship between data types and data structures? What possible advantage does the structure of a database provide to the users of a program.
6

Algorithm
• Any method of solving a certain kind of problem. • A precise method usable by a computer for the solution of a problem. • A finite set of instructions which, if followed accomplishes a particular task. Question: How can we determine if a particular algorithm is efficient or not?

7

A note on algorithm
Often the algorithms being used to manipulate the data structures will stem from the design of the data structures themselves. For example, if we create a data structure designed to represent a list, we will need algorithms to manage a collection of such objects (i.e. the list). Question: Which is more important, the design of the data structure or the algorithm?
8

Foundation of Algorithms
Can a particular task be accomplished by a computing device? What is the minimum number of operations for any algorithm to perform a certain function?

9

Analysis of algorithms
Know the behavior of the algorithm. This includes the pattern and performance profile of an algorithm that can be measured in terms of its execution time and the amount of space consumed. Question: Why is there a need to analyze an algorithm if it is already capable of solving the problem at hand?
10

Properties of Algorithm
• Input - the domain of the algorithm which could be zero or more quantities • Output - the set of one or more resulting quantities which is also called the range of the algorithm • Finiteness - means an algorithm must terminate after a finite number of steps; traceability • Definiteness - ensured if every step of an algorithm is precisely defined and unambiguous. • Effectiveness - ensured if all the operations in the algorithm are sufficiently basic that they can, in principle, be done exactly and in finite time by a person using paper and pen Question: Does it mean that when all properties are present in an algorithm, it is already the best algorithm? 11

Merging Data Structures and Algorithms
There are many types of data structures and algorithms used to create abstract data types. Here are a few of the more common examples used in computer programming. • Stack • Queue • Linked List
12

• Stack => A stack is a container for items which can be pushed onto it, or removed from the top of it. Navigation through the stack is not normally allowed, and access is limited to one unit (or node) at a time. • Queue => A queue is similar to a stack in that items can be added or removed from it. A LIFO queue (last-in-first-out) models the same behaviour as a stack, while a FIFO queue (fist-in-first-out) allows items to be added at one end, and removed from the other. • Linked List =>A linked list represents a queue of items in which individual nodes can be removed, added at arbitrary points, and the whole list freely navigated. Each node is linked to the next and/or previous node. Question: Can algorithms designed for a certain DS work for another DS? Why?
13

Program development life cycle (revisited)
1. Requirements Information given (input) and the results to be produced (output) must be explicitly specified, i.e., understand the problem 2. Design Write an algorithm that will solve the problem according to the requirements 3. Analysis Trying to come up with another algorithm and compare it with the previous one. 4. Refinement and Coding A representation is chosen and a complete version of the program is made. 5. Verification Consists of 3 aspects: program proving, testing and debugging Question: Is it possible to develop a program without passing through some of the stages of the life cycle?
14

Example 1
Problem: Accept N inputs and get their sum. Requirement: Input: Set of N integers Output: Sum of N integers Design: Set a temporary variable to 0. For each value entered, add the value to the temporary variable.
15

Example 1 (cont.)
Refinement and Coding: tempsum = 0; for (i=1; i<=n; i++) { x=System.in.read(); tempsum = tempsum + x; }
16

Example 2
Problem: Sort a set of n>=1 integers in nondescending order Requirement: Input: Set of N integers (N>=1) Output: Sorted set of N integers Design: Find the smallest and place it on the first position of the list. For the integers which remain to be unsorted, find the smallest and place it next in the sorted list.
17

Example 2 (cont.)
Refinement and coding: Representation: 1-dimensional array void sort(int A[ ]) { int temp, i, k, smallest; for (i=0; i<=n-1; i++) { smallest = i; for (k=i+1; k<=n; k++) if A[smallest]>A[k] smallest = k; temp = A[i]; A[i] = A[smallest]; A[smallest] = temp; } }

18

How to analyze programs
2 Phases: 1. Priori estimates Obtain a function which bounds the algorithm’s complexity time. The amount of time a single execution will take or the number of times a statement is executed
However, It is impossible to know the exact amount of time to execute any command unless the following are known: a. the machine we are executing b. machine instruction set c. time required by each machine instruction d. translation of the compiler from source to machine language
19

How to analyze programs (cont.)
2. Posteriori estimates Something to do with memory consumed. Question: Which is better? Arrays or Linked list? spaces

20

Time Complexity (Big-Oh)
Complexity of Algorithms • Several algorithms could be created to solve a single problem. These algorithms may vary in the way they get, process and output data. Hence, they could have significant difference in terms of performance and space utilization • Algorithm efficiency is measured in two criteria: space utilization and time efficiency.
– Space utilization is the amount of memory required to store the data – Time efficiency is the amount of time required to process the data Question: What makes an algorithm if you sacrifice either of the criterion?
21

Time Complexity (Big-Oh)
• Execution time is the amount of time spent in executing instructions of a given algorithm. It is dependent on the particular computer (hardware) being used. • To express the execution time we use the notation: T(n), where T is the function and n is the size of the input • Factors that affect the execution time:
1. 2. 3. 4. 5. input size instruction type machine speed quality of source code of the algorithm implementation quality of the machine code generated from the source code by the compiler
22

Time Complexity (Big-Oh)
The Big-Oh Notation • Although T(n) gives the actual amount of time in the execution of an algorithm, it is easier to classify complexities of algorithm using a more general notation, the Big-Oh (or simply O) notation. • This notation is used to describe the time or space complexity of an algorithm. It gives an approximate measure of the computing time of an algorithm for large number of input. • Asymptotic notations mean n → ∞ If A(n) = Amnm + …+ A1n1 + A0 is a polynomial of degree m, then A(n) = O(nm).
23

Time Complexity (Big-Oh)
• The following are examples of computing times in algorithm analysis:

24

Time Complexity (Big-Oh)
O(1) constant; most instructions are executed once or at most only a few times. O(log n) program slightly slower as N grows; normally in programs that solve a big problem by transforming it into a small problem, cutting the size by some constant factor. O(n) linear; proportional to the size of N
25

Time Complexity (Big-Oh)
O(n log n) occurs in algorithms that solve a problem by breaking it up into smaller subproblems, solve them independently and then combining the solution. O(n2) quadratic; can be seen in algorithms that process all pairs of data items O(n3) cubic; algorithms that process triples of data items O(2n) exponential; brute-force solution
26

Time Complexity (Big-Oh)
• To make the difference clearer, let's compare based on the execution time where n=100000 and time unit = 1 msec:

27

Example
Given 2 algorithms performing the same task on N inputs: P1 10n O(n) P2 n2/2 O(n2)

which is faster and efficient?
28

Operation Count
GENERAL RULES
RULE 1: Declaration with no initialization Declarations with no initialization have no operation count (i.e. 0). RULE 2: Delimiters (such as { and }) Delimiters have no operation count (i.e. 0). RULE 3: Function heading Function heading such as main() has no operation count (i.e. 0). RULE 4: Operators Each operator (whether arithmetic, logical, relational) has a running time of 1 (for simplicity only, though it may not hold in reality). RULE 5: Expressions The operation count of an expression depends on the number of operators (arithmetic, logical, relational) there are.
29

Operation Count(cont.)
RULE 6: Assignment Statement The operation count of an assignment statement is 1 plus the number of operators in the expression. Arithmetic operators (such as ++, --, +=, -=, *=, /=, %=) have an operation count of 2. RULE 7: Function Call Each function call has an operation count of 1. For System.in.read(), the operation count is 1 regardless of how many variables are involved. For System.out.println(), the operation count is 1 plus the operation counts of the parameters involved. For arbitrary function, the operation count is 1 plus the operation counts of the parameters. RULE 8: if or if-else Statement Given an if-else statement of the form: if(<condition>) <S1>; else <S2>; The overall operation count of the if-else statement is the operation count of the condition plus the maximum of the operation counts of <S1> and <S2>. The keyword else does not consume any operation count.
30

Operation Count(cont.)
RULE 9: for Statement Given a for statement of the form: for (<initialization>; <condition>; <change of state>) { <S>; } The total operation count of the for statement is the operation count of the <initialization> + operation count of the <condition> * (number of iterations + 1) + operation count of <change of state> * (number of iterations) + operation count of <S> * (number of iterations). RULE 10: while Statement Given the while statement of the form while(<condition>) { <S>; } The total operation count is the operation count of the <condition> * (number of iterations + 1) + operation count of <S> * (number of iterations).
31

Operation Count(cont.)
RULE 11: do-while Statement Given the do-while statement of the form do { <S>; } while (<condition>); The total operation count is the operation count of the <S> * (number of iterations) + operation count of the <condition> * (number of iteration). RULE 12: Nested Loops Analyze these inside out. The total operation count of a statement inside a group of nested loops is the operation count of the statement multiplied by the product of all the loops.

32

Examples
1. k= 100; for (i=1; i<=k; i++) x= x+1; n= 200; 2. for (k=1; k<=n+1; k++) { System.out.println(k); System.out.println(n-k); } 3. while (i<=1) { k=k+1; I= I+1; } 4. while (i<n) { k = k +1; i = i + 1; } 5. for (i=1; i<=c; i++) for (j=c; j>=1; j--) System.out.println(i +” “+j); 6. ctr = 1; while (ctr <= m-2) { for (k=1; k<=n; k++) System.out.println(ctr +” “+ k); ctr = ctr+1; }
33