Lecture 4.

RAM Model, Space and Time Complexity

1
 Recap

 Time and space tradeoff factors should be focus during designing of algorithms,
 Performance of algorithm should be measure rather than performance of its implemented program.
 Switching between CPU and IO circuit should be in mind before writing the instructions of an algorithm.

2
 The RAM Model

 Random Access Machine (not R.A. Memory)

 An idealized notion of how the computer works
– Each "simple" operation (+, -, =, if) takes exactly 1 step.
– Each memory access takes exactly 1 step
– Loops and method calls are not simple operations, but depend
upon the size of the data and the contents of the method.
 Measure the run time of an algorithm by counting the
number of steps.

3
 Random Access Machine
 A Random Access Machine (RAM) consists of: . . .
– a fixed program input tape
– an unbounded memory
– a read-only input tape Program

...
– a write-only output tape
Memory
 Each memory register can hold an arbitrary integer (*). output tape
 Each tape cell can hold a single symbol from a finite . . .
alphabet s.
Instruction set:
x  y, x  y {+, -, *, div, mod} z
goto label
if y {<, , =,  ,> , } z goto label
x  input, output  y
4 halt
 Space Complexity
 The amount of memory required by an algorithm to run to
completion
– The term memory leaks refer to the amount of memory
required is larger than the memory available on a given
system

5
 Space Complexity (Cont !!!)

 Fixed part: The size required to store certain data/variables, that is

independent of the size of the problem:

– Such as int a (2 bytes, float b (4 bytes) etc

 Variable part: Space needed by variables, whose size is dependent on

the size of the problem:
– Dynamic array a[]

6
 Space Complexity (cont’d)
 S(P) = c + S(instance characteristics)
– c = constant
 Example:
float sum (float* a, int n)
{
float s = 0;
for(int i = 0; i<n; i++) {
s+ = a[i];
}
return s;
}
Constant Space:
one for n, one for a [passed by reference!], one for s, one for I , constant space=c=4

7
 Running Time of Algorithms

 Running time
– depends on input size n
 size of an array
 polynomial degree
 # of elements in a matrix
 # of bits in the binary representation of the input
 vertices and edges in a graph

– number of primitive operations performed

 Primitive operation
– unit of operation that can be identified in the pseudo-code

8
 Steps To determine Time Complexity

Step-1. Determine how you will measure input size. Ex:

– N items in a list
– N x M table (with N rows and M columns)
– Two numbers of length N

Step-2. Choose the type of operation (or perhaps two operations)

– Comparisons
– Swaps
– Copies
– Additions

Note: Normally we don't count operations in input/output.

9
 Steps To determine Time Complexity (Cont !!!)

Step-3. Decide whether you wish to count operations in the

– Best case? - the fewest possible operations
– Worst case? - the most possible operations
– Average case? This is harder as it is not always clear what is meant by an
"average case". Normally calculating this case requires some higher
mathematics such as probability theory.
Step-4. For the algorithm and the chosen case (best, worst, average), express the
count as a function of the input size of the problem.

10
 Steps To determine Time Complexity (Cont !!!)

5. Given the formula that you have determined, decide the complexity class of
the algorithm.

Q. What is the complexity class of an algorithm?

A. a complexity class is a set of problems of related resource-based
complexity, such as P, NP classes
Q. Is there really much difference between
3n
5n + 20
and 6n -3
11 A. Yes but when n is large?
 Primitive Operations in an algorithm

 Assign a value to a variable (i.e. a=5)

 Call a method (i.e. method())
 Arithmetic operation (i.e. a*b, a-b*c)
 Comparing two numbers ( i.e. a<=b, a>b &&a>c)
 Indexing into an array (i.e. a[0]=5)
 Following an object reference (i.e. Test obj)
 Returning from a method (i.e. return I )

12
 Assignment Statement

 The running time of a an assignment is considered as constant

 Examples:
– A=5;
– A[5]=7
– C=a+b;

Note:- Running time of assignments can be

considered as 1 or C (Refer to constant
values)
13
 for Loops

 The running time of a for loop is at most the running time of

the statements inside the for loop (including tests) times the
number of iterations.
Example Let
For(i=0;i<=n-1;i++) 1.running time of basic operations is
constant C
A[i]=0; 2.and loop is iterated n times then
Total Running time will be
Note: n*c
(number of basic steps in loop body)
* (number of iterations)
14
 Nested for Loops

 The total running time of a statement inside a group of nested

for loops is the running time of the statement multiplied by the
product of the sizes of all the for loops.
 Example Let
for(i=0;i<=n-1;i++) 1.running time of basic operations is
constant C
for(j=0;j<=m-1;j++)
2.Running time of outer loop (i.e. i)
K++; is n
3.And running time of Inner loop
(i.e. j) is m
4.Total Running time will be
15 m*n*c
 Consecutives for Loops

 The total running time of consecutive loops is the sum of

running of all consecutives loops
 Example
for(i=0;i<=n-1;i++) Let
K++; 1.running time of basic operations is
constant C
for(j=0;j<=m-1;j++) 2.Running time of loop (i.e. i) is n
K++; 3.And running time of loop (i.e. j) is
m
4.Total Running time will be
n*c + m * c
16
 Conditional Statements

 The running time of an if-else statement is never more than the

running time of the test plus the larger of the running times of S1
and S2. Let
 Example 1.Condition is true and path have one
basic operation (i.e. S1) then Ruining
if(Condition) time will be constant C1
S1; 2.Similarly, if condition is false and
else path have one basic operation (i.e.
S2) then Ruining time will be
S2; constant C2
Note:
Number of basic steps on branch
17 that is executed
 Primitive Operations: Example -1

 Count the number of primitive operations in this program :

Int method() Primitive Operations Yes/No Total
{ Assign a value to a
variable
i = 0;
Call a method
a = 0;
Arithmetic operation
for (i=1;i<=n;i++)
Comparing two numbers
{ printf(“%d”,i);
Indexing into an array
a=a+i; }
Following an object
return I reference
} Returning from a
method
18
 Primitive Operations: Example -1 (Cont !!!)

Primitive Operations Yes/No Total

Assign a value to a variable Yes 5
Call a method No 0
Arithmetic operation Yes 2
Comparing two numbers Yes 1
Indexing into an array No 0
Following an object No 0
reference
19 Returning from a method Yes 1
 Primitive Operations - Example 2

 Count the number of primitive operations in this program :

Int method()
Primitive Operations Yes/No Total
{ i = 0;
Assign a value to a
a = 0; variable
for (i=1;i<=n;i++) Call a method
{ printf(“%d”, i); Arithmetic operation
for (a=1;a<=n;a++) Comparing two numbers
printf(“%d”,a);
Indexing into an array
}
Following an object
return I
reference
} Returning from a
method
20
 Primitive Operations - Example 2 (Cont !!!)

Primitive Operations Yes/No Total

Assign a value to a variable yes 6
Call a method No 0
Arithmetic operation yes 2
Comparing two numbers yes 2
Indexing into an array No 0
Following an object reference No 0
Returning from a method yes 1
21
 Summary

 RAM model is used to measure the run time of an algorithm by counting the number of steps.
 Space complexity of an algorithm refer to the amount of memory required to run.

 Time complexity of an algorithm refer to its running time, which depends on input size.
 Primitive operations refer to the unit of operation that can be identified in the pseudo-code

22
 In Next Lecturer

 In next lecture, we will talk about the types of algorithm complexity and how to find running time of different pseudo
codes.

23