Chapter 4: Computational Complexity

Basics of algorithm analysis: Big O-notation

The field of complexity analysis is concerned with the study of the efficiency of algorithms. To compare the
efficiency of algorithms, a measure of the degree of difficulty of an algorithm called computational
complexity was developed. Computational complexity indicates how much effort is needed to execute an
algorithm, or what its cost is. This cost can be expressed in terms of execution time (time efficiency, the
most common factor) or memory (space efficiency).
• Time Complexity: Determine the approximate number of operations required to solve a problem of
size n.
• Space Complexity: Determine the approximate memory required to solve a problem of size n.
The computational complexity of an algorithm is a measure of the cost (usually in execution time) incurred
by applying the algorithm.
The asymptotic complexity of an algorithm is an approximation of the computational complexity that holds
for large amounts of input data.
Big-O Notation
The most commonly used notation for specifying asymptotic complexity, that is, for estimating the rate
of growth of complexity functions.
Given two positive-valued functions f and g, consider the following definition:
Definition:
The function f(n) is O(g(n)) if there exist positive numbers c and N such that f(n) ≤ c.g(n) for all n ≥ N.
This definition states that g(n) is an upper bound on the value of f(n). In other words, in the long run (for
large n) f grows at most as fast as g.
Example:
f(n) = n2 + 5n, f(n)=O(n2 )
 Complexity Class Number of operations performed based on size of input data n
Name Big-O n=10 n=100 n=1000 n=10,000 n=100,000 n=1,000,000
Constant O(1) 1 1 1 1 1 1
Logarithmic O(log n) 3.32 6.64 9.97 13.3 16.6 19.93
Linear O(n) 10 100 1000 10,000 100,000 1,000,000
n log n O(n log n) 33.2 664 9970 133,000 1.66 * 106 1.99 * 107
Quadratic O(n2) 100 10,000 106 108 1010 1012
Cubic O(n3) 1000 106 109 1012 1015 1018
Exponential O(2n) 1024 1030 10301 103010 1030103 10301030

Computational complexity
The Turing machines we have studied have all been deterministic, i.e., when the transition rules are
represented as five-tuples (s, x, s’, x’, d), were s, x are the current state and tape symbol, s’, x’ are the next
state and tape symbol, and d is the move direction, then no two transition rules begin with the same pair (s,
x).
In a nondeterministic Turing machine, there may be more than one transition rule beginning with the same
pair (s, x).
P class of decision problems
A decision problem is in class P of polynomial-time problems if it can be solved by a deterministic Turing
machine in a polynomial time as function of the size of the input string,
i.e., there is a deterministic Turing machine T that solves the problem and halts in a final state after no more
than p(n) transitions when the input to T is a string of length n.
Problems in P are called tractable while problems not in P are called intractable.
Examples of tractable problems:
(1) determining whether an item is in a list of n elements.
(2) determining whether a given positive integer n is prime.
NP class of decision problems
A decision problem is in class NP of nondeterministic polynomial-time problems if it can be solved by a
nondeterministic Turing machine in a polynomial time as function of the size of the input string,
i.e., there is a nondeterministic Turing machine T that solves the problem and halts in a final state after no
more than p(n) transitions when the input to T is a string of length n.
Note: Non-Deterministic TM’s
A non-Deterministic Turing Machine N allows more than one possible action per given state-tape symbol
pair. In a Non-Deterministic Turing machine, in each branch - we do both possibilities - and only when we
are done, we "choose" which one is the one we need for the solution (if one exists).
P and NP
One of the most challenging questions in computer science is whether P = NP, i.e., whether every problem
in P is also in NP.
A problem S is NP-complete if it belongs to NP and if S can be shown to belong to P then every problem in
NP must also belong to P, i.e., the existence of polynomial-time algorithm to solve S, implies the existence
of a polynomial-time algorithm for every problem in NP.
Example of NP-complete problem:
Travelling Salesman Problem