Complexity Theory

CHAPTER 6 Advanced topic on complexity theory

Topic 1: The PCP theorem and hardness of approximation
• The PCP Theorem is concerned with the notion of “proofs” for mathematical statements.
• probabilistically checkable proof (PCP) is a type of proof that can be checked by a
randomized algorithm using a bounded amount of randomness and reading a bounded
number of bits of the proof. The algorithm is then required to accept correct proofs and
reject incorrect proofs with very high probability.
• Let us begin with a somewhat informal definition:

Definition 1.1 A traditional proof system works as follows:

• A statement is given. (e.g., “this graph G = · · · is 3-colorable” or “this CNF formula
F = (x1 ∨ x2 ∨ x10) ∧ · · · is satisfiable”).
• A prover writes down a proof, in some agreed-upon format.
• A verifier checks the statement and the proof, and accepts or rejects.
From the perspective of theory of computer science, we usually fix a constant size alphabet and
assume the statement and proof are strings over this alphabet; we also usually write n for the length
of the statement and measure lengths of strings and running times in terms of n. The familiar
complexity class NP can be cast in this setup:
Remark 1.1 Language L is in NP iff there is a polynomial time deterministic verifier V (a Turing
Machine, say) and an arbitrarily powerful prover P, with the following properties:
• “Completeness”: For every x ∈ L, P can write a proof of length poly(|x|) that V accepts.
• “Soundness”: For every 𝑥 ∉ L, no matter what poly(|x|) - length proof P writes, V rejects.
To equate the notion of the verifier being efficient with it being a deterministic polynomial time
algorithm is nowadays a bit quaint; ever since the late ’70s we have been quite happy to consider
randomized polynomial time algorithms to be efficient.

One pinnacle of research in this area is The PCP Theorem:

Theorem 1.1 (due to Arora-Safra (AS) and Arora-Lund-Motwani-Sudan-Szegedy (ALMSS))

“The PCP (Probabilistically Checkable Proof) Theorem”:

All languages L ⊆ NP have a P.C.P. system wherein on input x ∈ {0, 1} n:

• Prover P writes down a poly(n)-bit-length proof.
• Verifier V looks at x and does polynomial-time deterministic computation. Then V uses
O(log n) bits of randomness to choose C random locations in the proof. Here C is an
absolute universal constant; say, 100. V also uses these random bits to produce a
deterministic test (predicate) φ on C bits.

Compiled by: Samuel G. 1

 Complexity Theory

• V reads the bits in the C randomly chosen locations from the proof and does the test φ on
them, accepting or rejecting.
• Completeness: If x ∈ L then P can write a proof that V accepts with probability 1.
• Soundness: For every x ∉ L, no matter what proof P writes, V accepts with probability at
most 1/2.
Remark 1.2 This P.C.P. system has “one-sided error”: true statements are always accepted, but
there is a chance a verifier might accept a bogus proof. Note that this chance can be made an
arbitrarily small constant by naive repetition; for example, V can repeat its same spot-check 100
times independently, thus reading 100C bits and accepting false proofs with probability at most
2−100.
hardness of approximation studies the algorithmic complexity of finding near-optimal
solutions to optimization problems. Instead of making a formal definition we will just give some
examples. Briefly, these are “find the best solution” versions of classic NP-complete problems.
Definition 1.2 MAX-E3SAT: Given an E3CNF formula — i.e., a conjunction of “clauses” over
boolean variables x1, . . . , xn, where a clause is an OR of exactly 3 literals, xi or xi find an
assignment to the variables satisfying as many clauses as possible.
It is well-known that this problem is NP-hard. MAXE3SAT, there was a polynomial time
algorithm with the following guarantee: Whenever the input instance has optimum OPT — i.e.,
there is an assignment satisfying OPT many clauses — the algorithm returns a solution satisfying
99.9% × OPT many clauses. Such an algorithm would be highly useful, and would tend to refute
the classical notion that the NP-hardness of MAX-E3SAT means there is no good algorithm for it.
Let us make some definitions to capture these notions:

Definition 1.3 Given a combinatorial optimization maximization problem, we say algorithm A

is an approximation algorithm (for 0 < α ≤1) if whenever the optimal solution to an instance has
value OPT, A is guaranteed to return a solution with value at least α · OPT. We make the analogous
definition for minimization problems, with α ≥ 1 and A returning a solution with value at most
α · OPT. Unless otherwise specified, we will also insist that A runs in polynomial time.

Definition 1.4 A maximization (resp., minimization) combinatorial optimization problem is said

to have a PTAS (Polynomial Time Approximation Scheme) if it has a (1 − ϵ )-approximation
algorithm (resp., (1 + ϵ )-approximation algorithm) for every constant ϵ > 0.
Theorem 1.2 (credited to an unpublished 1992 result of Arora-Motwani-Safra-Sudan-Szegedy):
Speaking roughly, the PCP Theorem is equivalent to the statement, “MAX-E3SAT has no PTAS
assuming P ≠ NP”.

Compiled by: Samuel G. 2

 Complexity Theory

Topic 2: Randomized computation and complexity

In this topic we formally study probabilistic computation and complexity classes associated with
it. We now define probabilistic Turing machines (PTMs). Syntactically, a PTM is no different
from a nondeterministic TM: it is a TM with two transition functions δ0, δ1. The difference lies in
how we interpret the graph of all possible computations: instead of asking whether there exists a
sequence of choices that makes the TM accept, we ask how large is the fraction of choices for
which this happens. More precisely, if M is a PTM, then we envision that in every step in the
computation, M chooses randomly which one of its transition functions to apply (with probability
half applying δ0 and with probability half applying δ1). We say that M decides a language if it
outputs the right answer with probability at least 2/3.
Failure Probability
PTM Can Make Mistakes
A major aspect of randomized algorithms or probabilistic machines is that they may fail to produce
the desired output with some specified failure probability.
One-sided error: The machine may err only in one direction i.e. either on ‘yes’ instances or on
‘no’ instances.
I Either false positive or false negative, not both.
Two-sided error: The machine may err in both directions i.e. it may result a ‘yes’ instance as a
‘no’ instance and vice versa.
I Both false positive and false negative can occur.
Zero-sided error: The algorithm never provides a wrong answer. But sometimes it returns ‘don’t
know’
Now let’s see the definition of some of probabilistic complexity classes,
Definition 2.1: RP (Randomized Polynomial time)
The complexity class RP is the class of all languages L for which there exists a polynomial
PTM M such that: -
1
x ∈ L⇒ Prob[M(x) = 1] ≥ 2

x ∉ L⇒ Prob[M(x) = 0] = 1

Definition 2.2: coRP

The complexity class coRP is the class of all languages L for which there exists a
polynomial PTM M such that: -

Compiled by: Samuel G. 3

 Complexity Theory

x ∈ L⇒ Prob[M(x) = 1] = 1
1
x ∉ L⇒ Prob[M(x) = 0] ≥ 2

Definition 2.3: BPP (Bounded-error Probabilistic Polynomial time)

The complexity class BPP is the class of all languages L for which there exists a polynomial
PTM M such that: -
2
x ∈ L⇒ Prob[M(x) = 1] ≥ 3
1
x ∉ L⇒ Prob[M(x) = 0] ≤ 3

M answers correctly with probability 2/3 on any input x regardless if x ∈ L or x ∉ L

BPP is closed under complement. ⇒ BPP = coBPP

Some examples to demonstrate how randomness can be a useful tool in computation includes
Probabilistic Primality Testing, Polynomial identity testing and Testing for perfect matching in a
bipartite graph. For farther understanding on these terms refer your text book.
we have the following relations between the probabilistic complexity classes:

ZPP = RP ∩ coRP

RP ⊆ BPP

coRP ⊆ BPP

RP, coRP and ZPP are subclasses of BPP corresponding to probabilistic algorithms with one-sided
and “zero-sided” error.

Topic 3: Communication Complexity

To explain what communication complexity means, say there are players Alice and Bob and there
is a wall between them. Alice receives an n-bit string x ∈ X and Bob receives an n-bit string y ∈ Y.
Alice and Bob are both trying to compute f: X ×Y → {0,1} for their inputs by sending 1-bit
messages. By the end, both Alice and Bob should know f(x, y). We would like to minimize the
communication bits given that the two players have infinite computation power. This has been
studied for two players, but it gets a lot harder for a large number of players.
Formal definition for Communication Complexity
A t-round communication protocol for f is a sequence of function pairs (S1,C1), (S2,C2) , . . . ,
(St,Ct), (f1, f2). The input of Si is the communication pattern of the first i − 1 rounds and the output
is from {1, 2}, indicating which player will communicate in the ith round. The input of Ci is the
input string of this selected player as well as the communication pattern of the first i − 1 rounds.

Compiled by: Samuel G. 4

 Complexity Theory

The output of Ci is the bit that this player will communicate in the ith round. Finally, f1, f2 are 0/1
valued functions that the players apply at the end of the protocol to their inputs as well as the
communication pattern in the t rounds in order to compute the output. These two outputs must be
f(x, y). The communication complexity of f is
C(f) = min max {Number of bits exchanged by P on x, y.}
Protocols P x, y
Notice, C(f) ≤ n + 1 since the trivial protocol is for one player to communicate his entire input,
whereupon the second player computes f(x, y) and communicates that single bit to the first. Can
they manage with less communication?
Communication Complexity Classes
We can define the following communication complexity classes:
PCC, RPCC, BPPCC, NPCC, PPCC
as the set of functions f(x, y) that can be solved with poly-logarithmic communication complexity.
RPCC are those communication complexity functions that can be solved in time polylog(n) by a
randomized, 1-sided error protocol. That is, on "yes" instances, the protocol is correct with
probability 1, and on "no" instances the protocol errs with probability at most 1/4.
BPPCC are those functions that can be solved in time polylog(n) by a randomized, 1-sided error
protocol. On "yes" instances, the protocol is correct with probability 3/4, and on "no" instances the
protocol is also correct with probability 3/4. Note that the error is 2-sided and bounded away from
1/2.
PPCC are those functions that can be solved in time polylog(n) by a randomized protocol with
unbounded error. That is, on all instances, the protocol is correct with probability greater than 1/2.
Finally, ZPPCC are those functions that can be solved in average case time polylog(n) by a
randomized protocol with zero error.
Remark: Unlike the computational complexity classes where NO non-trivial relationship is known,
here we know almost everything, i.e.,

PCC⊊RPCC⊊BPPCC⊊NPCC:

Topic 4: Quantum computation

Quantum computation investigates the computational power and other properties of computers
based on quantum-mechanical principles. An important objective is to find quantum algorithms
that are significantly faster than any classical algorithm solving the same problem.
In classical computation, the unit of information is a bit, which can be 0 or 1. In quantum
computation, this unit is a quantum bit (qubit), which is a superposition of 0 and 1. Consider a

Compiled by: Samuel G. 5

 Complexity Theory

system with 2 basis states, call them |0› and |1›. We identify these basis states with the two

orthogonal vectors (10) and (01), respectively.

quantum complexity classes
EQP. The class of problems that can be solved exactly by quantum computers using polynomial
time. This class depends on the set of elementary gates one allows, and is not so interesting.
BQP. The problems that can be solved by quantum computers using polynomial time (and with
error probability ≤1/3 on every input). This class is the accepted formalization of “efficiently
solvable by quantum computers”.
QPSPACE. The problems that can be solved by quantum computers using polynomial space. This
turns out to be the same as classical PSPACE.
The relation between these classes could be represented in the form of

P ⊆BPP ⊆BQP ⊆ PSPACE

It is believed that BQP contains problems that aren't in BPP, for example factoring large integers:
this problem (or rather the decision-version thereof) is in BQP because of Shor's algorithm, and is
generally believed not to be in BPP.

Topic 5: Application of complexity theory on Cryptography (Cryptography and

complexity)
Understanding the foundations of Cryptography is understanding the foundations of Hardness.
Complexity Theory studies the nature of computational hardness {i.e. lower bounds on the time
required in solving computational problems} not only to understand what is difficult, but also to
understand the utility hardness offers.
Whereas cryptography studies on how to devise a secure and efficient method to transfer secret
messages. To complicate the task, we require that the method should also work in case the parties
that want to exchange messages have never met before, so that they have not had the chance of
agreeing in private on some key to be used. Such a method indeed exists, based on a complexity-
theoretic assumption.
Public key cryptography
In public-key cryptography, every participant has a public key known to everybody and a private
key only known to himself. If participant Alice wants to send participant Bob a secret message
over a, presumably unsafe, channel, she encrypts the message using Bob’s public key. The protocol
is devised in such a way that the only way to recover the message is to use Bob’s private key. This
ensures that only Bob can decrypt the message, and that a possible eavesdropper cannot decipher

Compiled by: Samuel G. 6

 Complexity Theory

it. RSA algorithm (stands for the names of its inventors Rivest, Shamir, and Adleman) is an
example for public key cryptography protocol.
Note you can study how the RSA works from text book named complexity theory, chapter 8
RSA works with the assumption that there is no polynomial time algorithm for factoring. (In
contrast, note that we can decide in polynomial time whether a number is prime or not).
One-way functions
Informally, a one-way (or trapdoor) function is a function that is easy to compute, but hard to
invert. Such functions may serve as the basis for a cryptographic scheme.
Note first that some easily computable functions may be hard to invert for trivial reasons. For
example, if the function is not injective then there is no inverse at all. We may counter this by
requiring that an inverse should compute some inverse value, that need not be unique. A second
trivial reason that a function may not be easily invertible is that it might map large strings to small
ones.

The Formal definition: - A function f: {0, 1} * → {0, 1} * is a one-way function if

1. f is injective. (As indicated above, this requirement is not always imposed.)

2. f is honest: There is a constant k such that for all x, |x| ≤ |f(x)|k.
3. f is polynomial time computable.
4. f−1 is not polynomial time computable.

Note that the existence of one-way functions implies that P ≠ NP. (Because of the honesty,
computing the – unique – inverse of f(x) is a typical NP task: we can guess an inverse and use f to
check its correctness.) Even if we assume that P ≠ NP, however, the existence of one-way
functions is not known. Their existence is tied to a special complexity class called the class UP.
Definition 8.4.1. Call a nondeterministic Turing machine unambiguous if for every input there is
at most one accepting computation. UP is the class of sets that are accepted by some unambiguous
machine in polynomial time.

Obviously, we have the inclusions P ⊆ UP ⊆ NP. Nothing more than this is known. The following
result ties the class UP to the existence of one-way functions.

Theorem (Valiant) One-way functions exist precisely when P ≠ UP.

Proof. Suppose that f is a one-way function. Define the set

L ={(x, y) : ∃z ≤ x(f(z) = y)}

Since f is injective, L ∈ UP. Also L ∉ P: If L ∈ P then we could find an inverse of f in polynomial

time by binary search.

Compiled by: Samuel G. 7

 Complexity Theory

Conversely, if L is a set in UP−P then we can define a one-way function f as follows. Suppose M
is an unambiguous machine accepting L. Given a computation path y of M on input x, we let f
map y to 1^x if y is accepting, and to 0^y if y is not accepting. Note that x can be effectively
retrieved from y. Since accepting paths of M are unique, f is injective. Also, f is honest since the
computations of M are of polynomial length. So to prove that f is one-way it suffices to show that
its inverse cannot be computed in polynomial time. (Note that f is not surjective, so the inverse is
only defined on the range of f.) Suppose we could do this. Then we could decide L in polynomial
time as follows: Given x, compute f−1(1^x). If this yields a y we know that x ∈ L, and if not we
know that x ∉ L. This contradicts the assumption that L∉ P, hence f is one-way.

Compiled by: Samuel G. 8