Computational Complexity 1

Colin Campbell
Engineering Mathematics Department

C.Campbell@bris.ac.uk

Colin Campbell Engineering Mathematics Department Computational Complexity 1

Computational Complexity

Turing Machines are therefore a mathematical model of

computation and we have considered several variants:
the standard TM which has a single tape and is deterministic
an extended TM e.g. which has multiple tapes or is
non-deterministic.
the universal Turing Machine which can simulate any TM on
any input
a Turing-complete device or programming language which
can, in principle, act as a universal Turing Machine

Colin Campbell Engineering Mathematics Department Computational Complexity 1

Computational Complexity

Complexity analysis examines the efficiency of algorithms (right)

and we consider two aspects to complexity:
time: how many steps are executed.
space: how many tape cells are used.

Colin Campbell Engineering Mathematics Department Computational Complexity 1

Some Definitions and Reminders

The language of a Turing Machine, M, denoted L(M ), is the

set of strings that are accepted by M. Formally we can write
this as L(M ) = {w |M accepts w }.
A language is Turing-recognizable if it is accepted by some
Turing Machine.
A decider is a Turing Machine that halts on all inputs.
A Turing Machine decides a language L it it accepts every
string in L, and rejects every string not in L.
A language is Turing-decidable if it is decided by some
Turing Machine.
Every Turing-decidable language is Turing-recognizable, but
not every Turing-recognizable language is Turing-decidable.

Colin Campbell Engineering Mathematics Department Computational Complexity 1

Example 1

Let’s consider some examples:

Example 1: let’s revisit the Turing Machine example for inverting
a binary string (one’s complement):

Colin Campbell Engineering Mathematics Department Computational Complexity 1

Example 1

in this case Σ = {0.1} and Γ = Σ ∪ {S, B }

The transition table is:
Initial state Input Output Move New state
q0 0 1 R q0
q0 1 0 R q0
q0 B B L q1

Colin Campbell Engineering Mathematics Department Computational Complexity 1

Example 1

Let’s consider the input 1101, the TM will halt with the tape
containing 0010.
Let’s use the notation in the following table with [. . .] indicating
the position of the R/W head. Then the tape content and state is
as follows:

Colin Campbell Engineering Mathematics Department Computational Complexity 1

Example 1

[1] 1 0 1 B q0
0 [1] 0 1 B q0
0 0 [0] 1 B q0
0 0 1 [1] B q0
0 0 1 0 [B ] q0
0 0 1 [0] B q1

From which we deduce that for an input with n binary digits, the
Turing Machine terminates after (n + 1) transitions (one row to
next row counts as +1 so 6 − 1 = 5).

Colin Campbell Engineering Mathematics Department Computational Complexity 1

Times and Space Complexity

At this point we can formally introduce some definitions:

The time complexity of an algorithm (Turing Machine) is a
function f : N → N (natural numbers) where the function
f (n ) represents the maximum number of transitions that
occur in processing an input of size n (tape cells).
The space complexity of an algorithm (Turing Machine) is a
function f : N → N (natural numbers) where the function
f (n ) represents the maximum number of tape cells that are
used in processing an input of size n.
In example 1 our time complexity was (n + 1). In this section we
focus on the time complexity only.

Colin Campbell Engineering Mathematics Department Computational Complexity 1

Order notation

The big − O notation represents the complexity of an algorithm by

a function f (n ) which represents the number of steps for the
algorithm to succeed and halt, given an input of size n. Formally
we say that f is O (g (n )) if and only if there exist positive integers
c, n0 such that f (n ) ≤ cg (n ) for all n ≥ n0 .
Example 1
Suppose the algorithm has f (n ) = n + 1 steps. Now n + 1 ≤ 2n
so with c = 2 this gives g (n ) = n so we say the algorithm is order
O (n ).
Thus O (g (n )) is an upper bound on the complexity of f (n ).

Colin Campbell Engineering Mathematics Department Computational Complexity 1

Example 2

Example 2: let’s consider a more tricky example which we will

implement in 3 ways (we refer to these as implementations: two
described here, later with an extended Turing Machine we give a
third implementation). The TM is to recognise the language
A = {0k 1k |k ≥ 0} that is 01, 0011, 000111, . . ..

Colin Campbell Engineering Mathematics Department Computational Complexity 1

Implementation 1

Implementation 1
By scan we mean read, by crossing off we mean write a blank,
more terminology for how these things are written!
1. Scan to end of tape, reject if 0 is found to the right
of a 1.
2. Repeat stage 3 while both 0’s and 1’s remain on the
tape:
3. Move to start of tape; move right crossing off single
0 and 1.
4. Move to start of tape; scan for 0 or 1 (reject) or
reach end (accept).

Colin Campbell Engineering Mathematics Department Computational Complexity 1

Implementation 1

An analysis for an input of n bits:

1. First stage uses n steps O (n ).
2. Stage 3 is repeated up to n/2 times (2 symbols are
crossed off each time)
3. Stage 3 requires up to 2n steps (move to start then
to end) hence now O (n ) × O (n ) = O (n2 )
(neglecting constant factors)
4. Final stage uses n steps + n steps: O (n ).
We only consider the dominant terms since n2  n as n → ∞.
Hence, overall, the method has time complexity O (n2 ).

Colin Campbell Engineering Mathematics Department Computational Complexity 1

Implementation 2

Implementation 2
However, let’s try this alternative (we omit “move to start of tape”
since this is O (n )):
1. Scan to end of tape, reject if 0 is found to the right of a 1.
2. Repeat stages 3 and 4 while both 0s and 1s remain on the
tape.
3. Scan tape, check whether total number of 0s and 1s is odd.
Reject if odd.
4. Cross off the first 0 and every second 0 after that; repeat for
1s
5. If no 0s or 1s remain on the tape, accept, otherwise reject.

Colin Campbell Engineering Mathematics Department Computational Complexity 1

Implementation 2

An analysis for an input of n bits:

1. Stage 1 is O (n )
2. Stage 3, 4 are repeated up to 1 + log2 (n ) times
3. O (n ) and
4. O (n ) so together O (n ) × O (log2 n ) = O (n log n )
5. Stage 5 is O (n )
Overall, this implementation has a lower time complexity of
O (n log n ).

Colin Campbell Engineering Mathematics Department Computational Complexity 1

Example 3

Example 3: the n-queens problem

Let’s now consider what appears to be a straightforward problem,
and look at its computational complexity. The n-queens problem
is: how can we place n queens on an n × n chessboard without any
queen attacking another.

Colin Campbell Engineering Mathematics Department Computational Complexity 1

Example 3

For a 4 × 4 game:

A queen attacks another if it in the same row, column or diagonal.

Colin Campbell Engineering Mathematics Department Computational Complexity 1

Example 3

Let’s suppose we can generate and check 1 million configurations

per second. The expected end-time was computed, based on a
number of steps, and the results are as follows for a start time of
11 : 00 on Nov. 10, 2017. The time taken is as follows:
n End of calculation
1 Fri Nov 2017 11:00:00
... ... ... ... ...
10 Fri Nov 2017 13::46:40
11 Mon Nov 2017 18:15:11
12 Wed Feb 2018 15:41:40
13 Wed Jun 2027 22:58:26
14 Fri Dec 2369 15:33:45
15 Thu Mar 15894 18:46:20
Clearly the time complexity is rising exponentially with n.

Colin Campbell Engineering Mathematics Department Computational Complexity 1