Computational Theory

CS 323

Chapter 0
I NTRODUCTION
Introduction
• The theory of computation comprises the
fundamental mathematical properties of
computer hardware, software, and certain
applications thereof.
• In studying this subject, we seek to determine:
– what can and cannot be computed,
– How quickly,
– with how much memory,
– and on which type of computational model.

3/1/2023 2
0.1 AUTOMATA, COMPUTABILITY, AND
COMPLEXITY

• Three traditionally central areas of the theory

of computation:
– automata,
– computability,
– and complexity
• They are linked by the question:What are the
fundamental capabilities and limitations of
computers?

3/1/2023 3
COMPLEXITY THEORY
• Central question: What makes some problems
computationally hard and others easy?
• Some computer problems are easy, and some are hard
– sorting problem vs. scheduling problem
• In most fields, an easy computational problem is preferable
to a hard one
– because easy ones are cheaper to solve.
• Cryptography is unusual because it specifically requires
computational problems that are hard, rather than easy.
– Complexity theory has pointed cryptographers in the direction
of computationally hard problems
– around which they have designed revolutionary new codes.

3/1/2023 4
3/1/2023 5
3/1/2023 6
COMPUTABILITY THEORY

• In complexity theory, the objective is to

classify problems as easy ones and hard ones;
• whereas in computability theory, the
classification of problems is by those that are
solvable and those that are not.

3/1/2023 7
AUTOMATA THEORY
• Automata theory deals with the definitions
and properties of mathematical models of
computation.
• Models:
– finite automaton,
• used in text processing, compilers, and hardware
design.
– context-free grammar,
• used in programming languages and artificial
intelligence.

3/1/2023 8
0.2 MATHEMATICAL NOTIONS AND
TERMINOLOGY

• SETS
• A set is a group of objects represented as a
unit.
• Sets may contain any type of object, including:
– numbers, symbols, and even other sets.
• The objects in a set are called its elements or
members.

3/1/2023 9
SETS

3/1/2023 10
SETS
• An infinite set contains infinitely many
elements.

3/1/2023 11
SETS
• Empty set: The set with zero members and is
written ∅.
• Singleton set: A set with one member
• Unordered pair: a set with two members
• Describing a set containing elements
according to some rule

3/1/2023 12
Venn diagram

3/1/2023 13
Venn diagram

3/1/2023 14
Venn diagram

3/1/2023 15
Venn diagram

3/1/2023 16
SEQUENCES AND TUPLES

• A sequence of objects is a list of these objects

in some order.
– e.g. (7, 21, 57)
• The order doesn’t matter in a set, but in a
sequence it does.
– (7, 21, 57) is not the same as (57, 7, 21)
• repetition does matter in a sequence, but it
doesn’t matter in a set.

3/1/2023 17
SEQUENCES AND TUPLES

• Finite sequences often are called tuples. A

sequence with k elements is a k-tuple. Thus (7,
21, 57) is a 3-tuple.
– A 2-tuple is also called an ordered pair.
• The power set of A is the set of all subsets of A.
– If A is the set {0, 1},
• the power set of A is the set { ∅, {0}, {1}, {0, 1} }.
• The set of all ordered pairs whose elements are
0s and 1s is { (0, 0), (0, 1), (1, 0), (1, 1) }.

3/1/2023 18
Cartesian product
• If A and B are two sets, the Cartesian product
or cross product of A and B, written A × B, is
the set of all ordered pairs wherein the first
element is a member of A and the second
element is a member of B.

3/1/2023 19
Cartesian product
• We can also take the Cartesian product of k
sets, A1, A2, . . . , Ak,
– Written A1 × A2 ×· · ·×Ak.
– It is the set consisting of all k-tuples
• (a1, a2, . . . , ak) where ai ∈ Ai.

3/1/2023 20
Cartesian product

3/1/2023 21
FUNCTIONS AND RELATIONS

• A function is an object that sets up an input–

output relationship.
– A function also is called a mapping

• f is a function with domain D and range R

A function that uses all the elements of
the range is said to be onto the range.

3/1/2023 22
FUNCTIONS AND RELATIONS

3/1/2023 23
FUNCTIONS AND RELATIONS

• When the domain of a function f is A1×· · ·×Ak

for some sets A1, . . . , Ak, the input to f is a k-
tuple (a1, a2, . . . , ak) and we call the ai the
arguments to f.
– A function with k arguments is called a k-ary
function, and k is called the arity of the function.
• unary function
• binary function

3/1/2023 24
FUNCTIONS AND RELATIONS

• A predicate or property is a function whose

range is {TRUE, FALSE}.
– even(4) = TRUE and even(5) = FALSE.
• A property whose domain is a set of k-tuples
A×· · ·×A is called a relation, a k-ary relation,
or a k-ary relation on A.
– if R is a k-ary relation, the statement R(a1, . . . , ak)
means that R(a1, . . . , ak) = TRUE.

3/1/2023 25
binary relation
• If R is a binary relation, the statement aRb
means that aRb = TRUE.

3/1/2023 26
Equivalence relation
• A binary relation R is an equivalence relation if
R satisfies three conditions:
1. R is reflexive if for every x, xRx;
2. R is symmetric if for every x and y, xRy implies
yRx; and
3. R is transitive if for every x, y, and z, xRy and yRz
implies xRz.

3/1/2023 27
Equivalence relation

Theoretical Informatics (CS 422)

3/1/2023 28
Prof. Hussein Al-Zoubi
GRAPHS
• An undirected graph, or simply a graph, is a
set of points with lines connecting some of
the points.
• The points are called nodes or vertices, and
the lines are called edges,

3/1/2023 29
GRAPHS

G = (V,E)

3/1/2023 30
GRAPHS
• The number of edges at a particular node is
the degree of that node.
• No more than one edge is allowed between
any two nodes.
– We may allow an edge from a node to itself, called
a self-loop

3/1/2023 31
Labeled graph

3/1/2023 32
Subgraph

3/1/2023 33
GRAPHS
• A path in a graph is a sequence of nodes
connected by edges.
• A simple path is a path that doesn’t repeat any
nodes.
• A graph is connected if every two nodes have a
path between them.
• A path is a cycle if it starts and ends in the same
node.
• A simple cycle is one that contains at least three
nodes and repeats only the first and last nodes.

3/1/2023 34
Tree
• A graph is a tree if it is connected and has no
simple cycles
– root
– leaves

3/1/2023 35
Directed graph

3/1/2023 36
Directed graph
• Outdegree
• Indegree

3/1/2023 37
Directed graphs and binary relations

• A path in which all the arrows point in the

same direction as its steps is called a directed
path.
• A directed graph is strongly connected if a
directed path connects every two nodes.

3/1/2023 38
Directed graphs and binary relations

3/1/2023 39
STRINGS AND LANGUAGES

• An alphabet is any nonempty finite set.

• The members of the alphabet are the symbols
of the alphabet.

3/1/2023 40
Strings
• A string over an alphabet is a finite sequence
of symbols from that alphabet,
– usually written next to one another and not
separated by commas.
• If Σ1 = {0,1}, then 01001 is a string over Σ1.
• If Σ2 = {a, b, c, . . . , z}, then abracadabra is a
string over Σ2.

3/1/2023 41
Strings
• If w is a string over Σ, the length of w, written
|w|, is the number of symbols that it contains.
– The string of length zero is called the empty string
and is written ε.
• If w has length n, we can write w = w1w2 · · ·wn
where each wi ∈ Σ.

3/1/2023 42
Strings
• The reverse of w, written , is the string
obtained by writing w in the opposite order

• String z is a substring of w if z appears

consecutively within w.
– cad is a substring of abracadabra.
• concatenation

3/1/2023 43
Lexicographic order
• shortlex order or simply string order,
– Shorter strings precede longer strings.
• string ordering of all strings over the alphabet
{0,1} is (ε, 0, 1, 00, 01, 10, 11, 000, . . .).
• string x is a prefix of string y if a string z exists
where xz = y,
– x is a proper prefix of y if in addition x ≠ y.
• A language is a set of strings.
3/1/2023 44
BOOLEAN LOGIC
• Boolean logic is a mathematical system built
around the two values TRUE and FALSE.
• Boolean operations:
– negation or NOT
– conjunction or AND (∧)
– disjunction or OR (∨)

3/1/2023 45
BOOLEAN LOGIC
• Suppose that P is the Boolean value
representing the truth of the statement “the
sun is shining” and Q represents the truth of
the statement “today is Monday”
– In P ∧ Q or P ∨ Q
• P and Q are called the operands of the operation.

3/1/2023 46
Boolean operations
• exclusive or, XOR ⊕
• equality operation ↔
• Implication, →

3/1/2023 47
Boolean operations
• We can express all Boolean operations in
terms of the AND and NOT operations

• The distributive law for AND and OR

3/1/2023 48
3/1/2023 49
0.3 DEFINITIONS, THEOREMS, AND PROOFS

• Definitions describe the objects and notions

that we use.
• Mathematical statements: a statement
expresses that some object has a certain
property.
• A proof is a convincing logical argument that a
statement is true.
• A theorem is a mathematical statement
proved true.
3/1/2023 50
Lemmas and corollaries
• Occasionally we prove statements that are
interesting only because they assist in the
proof of another, more significant statement.
Such statements are called lemmas.
• Occasionally a theorem or its proof may allow
us to conclude easily that other, related
statements are true. These statements are
called corollaries of the theorem.

3/1/2023 51
3/1/2023 52
FINDING PROOFS
• “P if and only if Q”, often written “P iff Q”
– forward direction, P ⇒ Q
– reverse direction, P ⇐ Q
– “P if and only if Q” as P ⇐⇒ Q
– To prove a statement of this form, you must prove
each of the two directions.
• Counterexample: an object that fails to have the
property
– If the statement actually is true, you will not be able
to find a counterexample.

3/1/2023 53
FINDING PROOFS

3/1/2023 54
FINDING PROOFS

3/1/2023 55
Tips
• Be patient
• Come back to it
• Be neat
• Be concise

3/1/2023 56
FINDING PROOFS

3/1/2023 57
0.4 TYPES OF PROOF
• PROOF BY CONSTRUCTION
• We define a graph to be k-regular if every
node in the graph has degree k.

3/1/2023 58
PROOF BY CONSTRUCTION

3/1/2023 59
PROOF BY CONTRADICTION

3/1/2023 60
PROOF BY CONTRADICTION

3/1/2023 61
PROOF BY INDUCTION

• Basis: proves that P(1) is true

• Induction step: proves that for each i ≥ 1, if
P(i) is true, then so is P(i+ 1)
• the assumption that P(i) is true is called the
induction hypothesis

Theoretical Informatics (CS 422)

3/1/2023 62
Prof. Hussein Al-Zoubi
PROOF BY INDUCTION

3/1/2023 63
PROOF BY INDUCTION

3/1/2023 64