Introduction To Data Structures and Algorithms: COMP2015 HKBU/CS/JF/2023 1

Chapter 1
Introduction to Data Structures

and Algorithms
COMP2015 HKBU/CS/JF/2023 1
Why study Data Structures and
Algorithms?
To run computer programs efficiently!
n Computer program:
q Accepts input (data)
q Performs a sequence of actions with the input
q Generates output (data)
n Using a computer
q Solve computational problems?
q Want it to go faster?
q Ability to process more data?
Course Motivation
n To Run Computer Programs Efficiently:

q Efficient Management of Data ð Data Structures
q Efficient Sequence of Actions ð Algorithms
n How does Google find the documents matching your query

so fast?
q Uses sophisticated algorithms to create index structures,

which are just data structures
q Algorithms and data structures are ubiquitous.
Data Structures
n Data structures
q Objects created to organize data used in computation
n Means of storing and organizing data to facilitate access

and modifications
n Several data structures exist

q Array
q Linked list
q Stack
q Queue
Algorithms
n Algorithm
q A problem-solving method suitable for implementation as
a computer program
n Given a well-spicified problem, an algorithm forms
computational steps to transform inputs to outputs, i.e.
Solution.
n Example: Sorting problem
Input Output
A sequence A permutation
of numbers Algorithm (reordering) <a1’,
(a1, a2, ..an) a2’, ..., an’> of the
input sequence,
a1’ ≤ a2’, ... ≤ an’
Data Structures and Algorithms
n Data structures exist as the by-product or end product of

algorithms
q Understanding data structures is essential to

understanding algorithms and hence to problem-
solving.
q Simple algorithms can give rise to complicated data
structures.
q Complicated algorithms can use simple data
structures.
Example: Selection Problem
n Problem: Given a group of N numbers, determine the kth

largest, where N > k .
n Solution 1:
(1) read N numbers into an array,

(2) sort the array in decreasing order,
(3) return the element in position k.
n Solution 2:
(1) read the first k elements into an array and sort them
in decreasing order,
(2) each remaining element is read one by one,

(2.1) it is ignored if it is smaller than the kth element
in the array
(2.2) otherwise, it is placed in its correct spot in the
array, bumping one element out of the array.
(3) the element in the kth position is returned as the

answer
n Two natural questions:
q Which solution is better ?
n By simulation
n By theoretical analysis
q Is either algorithm good enough (particularly when N is

very large)?
n A simulation using a random file of 1 million elements
and k = 500,000 will show that NEITHER algorithm
finishes in a reasonable amount of time (each requires
several days of computer processing to terminate).
n An alternative algorithm gives a solution in about a
second.
n What is the conclusion?
Conclusion from the Previous
Example
n An important concept
q In many problems, writing a working program is not
good enough (The program may be inefficient!).
q If the program is to be run on a large data set, then the
running time becomes an issue.
n We will discuss
q How to estimate the running time of a program for large
inputs
q How to compare the running time of two programs
q Techniques to improve the speed of a program, and to
determine the program bottlenecks
Attributes of Algorithms
n Desirable characteristics in an algorithm
q Correctness
q Ease of understanding (clarity)
q Elegance
q Efficiency
n Correctness
q Does the algorithm solve the problem it is designed for?
q Does the algorithm solve the problem correctly?
n Ease of understanding (clarity)
q How easy is it to understand or alter the algorithm?
q Important for program maintenance
n Elegance
q How clever or sophisticated is the algorithm?
q Sometimes elegance and ease of understanding work
at cross-purposes.
n Efficiency
q How much time and/or space does the algorithm
require when executed?
q Perhaps the most important desirable attribute
Study of Algorithms
n Study of Algorithms
q Design of algorithms
n Efficient vs inefficient
n Design an efficient algorithm ð use good data structures
q Proving correctness of algorithms
q Formal study of efficiency of algorithms: algorithm analysis

n To estimate the resources required by the algorithm
q Running time
q Storage required
Analysis of Algorithm
n Objective of algorithm analysis:
q To determine the resources required by an algorithm

n Time - running time main concern
n Space - memory space
n Types of algorithm analysis

q Empirical analysis
q Mathematical analysis
Empirical Analysis
n Empirical analysis
q Implement the algorithm

q Input and other factors
n Actual data
n Random data (average-case behavior)
n Perverse data (worst-case behavior)
q Run empirical tests
Mathematical Analysis
n Mathematical analysis
q To predict the performance of an algorithm
(speed/running time)
n Characterizes running time as a function of the input
size, N
n Takes into account all possible inputs, often
analyzing the worst case
q Allows us to evaluate the speed of an algorithm
independent of the hardware/software environment

q To compare different algorithms
n What to analyze?
q The most important resource to analyze is the running
time!
n Factors affect the running time of a program:
q Platforms: the compiler and the computer (not to be

considered in the analysis)
q The algorithm used

Main Concerns
q The input to the algorithm
n In order to predict resource requirements, we need a

computational model.
n Random-Access Machine (RAM) model assumed
q Instructions are executed one after another. No

concurrent operations. Þ Not parallel computers
q Each of instructions takes a constant amount of time.
n Random-Access Machine (RAM) model

q It is too tedious to define each of the instructions and their
associated time costs.
q Instead, use instructions commonly found in real

computers:
n Arithmetic: add, subtract, multiply, divide, remainder,
floor, ceiling. Also, shift left/shift right (good for
multiplying/dividing by 2k ).
n Data movement: load, store, copy.
n Control: conditional/unconditional branch, subroutine

call and return.
n How do we analyze an algorithm’s running time?
q The time taken by an algorithm depends on the input.

n Example: Sorting 1000 numbers takes longer time than sorting
3 numbers.
q The size of the input is typically the main consideration.

n The input size depends on the problem being studied.
q Characterizes Running Time as a function of the Input Size N.

n Running time vs Input size (N)
q Running time: On a particular input, it is the number of primitive

operations (steps) executed. We do not distinguish between the
basic operations.
n We usually concentrate on finding the worst-case running

time: the longest running time for any input of size N.
q Reason:
n The worst-case running time gives a guaranteed
upper bound on the running time for any input.
q Why not analyze the average case?

n An average-case analysis does not provide such a
bound, and it is usually much more difficult to
compute. In some instances, the definition of
“average” can affect the result.
n We define:
q Tavg(N): the average-case running time;
q Tworst(N): the worst-case running time;
q Clearly, Tavg(N) £ Tworst(N)
A Simple Example
n A program segment to calculate
The declarations count for no time
int Sum(int N) Lines 1 and 4 count for one unit each

{
int i, PartialSum; Line 2 has the hidden costs on
initializing i, testing i £ N, and
/* 1 */ PartialSum = 0; incrementing i. The total cost is 1 to
/* 2 */ for (i = 1; i <= N; i++) initial, N+1 for all the tests, and N
/* 3 */ PartialSum += i*i*i; for all the increment, which is 2N+2
/* 4 */ return PartialSum;
Line 3 counts for four units per time
} executed (2 multiplications, 1
addition and 1 assignment) and is
The total operations for this function is executed N times, for a total of 4N
1+1+2N+2+4N = 6N + 4
Review: Mathematical Foundation
• Series and summation:
( + 1) (arithmetic series)
= 1 +2 + 3 + ⋯+ =
2
1−
= 1+ + + ⋯+ = (geometric series)
1−
• Sum of squares:
( + 1)(2 + 1)
= 1 + 2 + ⋯+ =
6
Review: Recursion
n In some problems, it may be natural to define the

problem in terms of the problem itself.
n Recursion is useful for problems that can be represented

by a simpler version of the same problem.
n Recursion is one way to decompose a task into smaller

subtasks. At least one of the subtasks is a smaller
example of the same task.
n The smallest example of the same task has a non-

recursive solution.
Example 1: Factorial Function
n Example: the factorial function

6! = 6 * 5 * 4 * 3 * 2 * 1
q We could write as
6! = 6 * 5!
v In general, we can express the factorial function as follows:

n! = n * (n-1)!
v Is this correct?
v We should be a bit more precise:

n! = 1 (if n = 0)
n! = n * (n-1)! (if n > 0)
Example 2: Fibonacci Numbers
Ø Recall definition of Fibonacci numbers:
0, 1, 1, 2, 3, 5, 8, 13, 21, …
F(n) = F(n-1) + F(n-2) for n > 1

F(0) = 0
F(1) = 1
Ø Computing the nth Fibonacci number recursively:
F(n)
F(n-1) + F(n-2)
F(n-2) + F(n-3) F(n-3) + F(n-4)

......
Recursion
n We must always make sure that the recursion bottoms

out.
n A recursive function must contain at least one non-

recursive branch.
n The recursive calls must eventually lead to a non-

recursive branch.
Tutorial
Evaluate the following sums:

(1)
4
(2)
4

Introduction To Data Structures and Algorithms: COMP2015 HKBU/CS/JF/2023 1

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Introduction To Data Structures and Algorithms: COMP2015 HKBU/CS/JF/2023 1

Uploaded by

Copyright:

Available Formats

Chapter 1

Introduction to Data Structures

q Performs a sequence of actions with the input

q Generates output (data)

q Ability to process more data?

n To Run Computer Programs Efficiently:

q Efficient Sequence of Actions ð Algorithms

n How does Google find the documents matching your query

q Uses sophisticated algorithms to create index structures,

n Means of storing and organizing data to facilitate access

n Several data structures exist

n Data structures exist as the by-product or end product of

q Understanding data structures is essential to

n Problem: Given a group of N numbers, determine the kth

(1) read N numbers into an array,

(2) each remaining element is read one by one,

(3) the element in the kth position is returned as the

q Is either algorithm good enough (particularly when N is

n Desirable characteristics in an algorithm

q Ease of understanding (clarity)

q Does the algorithm solve the problem it is designed for?

q Does the algorithm solve the problem correctly?

n Ease of understanding (clarity)

q How easy is it to understand or alter the algorithm?

q Important for program maintenance

n Design an efficient algorithm ð use good data structures

q Proving correctness of algorithms

q Formal study of efficiency of algorithms: algorithm analysis

n Objective of algorithm analysis:

q To determine the resources required by an algorithm

n Types of algorithm analysis

q Implement the algorithm

n Random data (average-case behavior)

n Perverse data (worst-case behavior)

q Run empirical tests

independent of the hardware/software environment

n Factors affect the running time of a program:

q Platforms: the compiler and the computer (not to be

q The algorithm used

n In order to predict resource requirements, we need a

n Random-Access Machine (RAM) model assumed

q Instructions are executed one after another. No

n Random-Access Machine (RAM) model

q Instead, use instructions commonly found in real

n Control: conditional/unconditional branch, subroutine

q The time taken by an algorithm depends on the input.

q The size of the input is typically the main consideration.

q Characterizes Running Time as a function of the Input Size N.

q Running time: On a particular input, it is the number of primitive

n We usually concentrate on finding the worst-case running

q Why not analyze the average case?

q Tavg(N): the average-case running time;

q Tworst(N): the worst-case running time;

q Clearly, Tavg(N) £ Tworst(N)

int Sum(int N) Lines 1 and 4 count for one unit each

• Series and summation:

n In some problems, it may be natural to define the

n Recursion is useful for problems that can be represented

n Recursion is one way to decompose a task into smaller

n The smallest example of the same task has a non-

n Example: the factorial function

v In general, we can express the factorial function as follows:

v We should be a bit more precise: