Computation Theory: Three Basic Concepts: Languages, Grammars, and Automata

Computation Theory
Chapter 1
Three Basic Concepts:

Languages, Grammars, and Automata
Languages, Grammars, and Automata 1

Why study the theory of computing?
Core mathematics of CS (has not changed in

over 30 years)
Many applications, especially in design of
compilers and programming languages
Important to be able to recognize
incomputable and intractable problems
Need to know this in order to be a computer
scientist, not simply a computer programmer

A pioneer of automata theory
Alan Turing (1912-1954)
▪ Father of Modern Computer Science

▪ English mathematician
▪ Studied abstract machines called
▪Turing machines even before
computers existed

Theory of Computation: A
Historical Perspective

Languages & Grammars

The Chomsky Hierarchy

Languages

A language is a set of strings
String: A sequence of letters
Examples: “cat”, “dog”, “house”, …
Defined over an alphabet:

 = a, b, c,, z

Alphabets and Strings
We will use small alphabets:  = a, b
Strings
a
ab u = ab
abba v = bbbaaa
baba w = abba
aaabbbaabab
String Operations
w = a1a2 an abba

v = b1b2 bm bbbaaa
Concatenation
wv = a1a2 anb1b2 bm abbabbbaaa

w = a1a2 an ababaaabbb
Reverse
w = an  a2 a1
R
bbbaaababa

String Length
w = a1a2 an
Length: w =n
Examples: abba = 4
aa = 2
a =1
Length of Concatenation
uv = u + v
Example: u = aab, u = 3
v = abaab, v = 5
uv = aababaab = 8
uv = u + v = 3 + 5 = 8
Empty String
A string with no letters: 
Observations:  =0
w = w = w
abba = abba = abba

Substring
Substring of string:
a subsequence of consecutive characters
String Substring
abbab ab
abbab abba
abbab b
abbab bbab
Prefix and Suffix
abbab
Prefixes Suffixes
 abbab w = uv
a bbab
prefix
ab bab
suffix
abb ab
abba b
abbab 
Another Operation
w = ww


n

w
n
Example: (abba ) = abbaabba

2
Definition: w =
0
(abba ) =  0

The * Operation
 * : the set of all possible strings from
alphabet 
 = a, b
* =  , a, b, aa, ab, ba, bb, aaa, aab,

The + Operation
+ : the set of all possible strings from

alphabet  except 
 = a, b
* =  , a, b, aa, ab, ba, bb, aaa, aab,
+
 =  * −
+
 = a, b, aa, ab, ba, bb, aaa, aab,
Languages
A language is any subset of *
Example:  = a, b
* =  , a, b, aa, ab, ba, bb, aaa,
Languages: 
a, aa, aab
{ , abba, baba, aa, ab, aaaaaa}
Note that:
Sets  = { }  {}
Set size {} =  = 0
Set size {} = 1

String length  =0
Another Example
An infinite language L = {a b : n  0} n n

ab
L abb  L
aabb
aaaaabbbbb
Operations on Languages
The usual set operations
a, ab, aaaa  bb, ab = {a, ab, bb, aaaa}

a, ab, aaaa  bb, ab = {ab}
a, ab, aaaa − bb, ab = a, aaaa
Complement: L =  * −L
a, ba =  , b, aa, ab, bb, aaa,

Reverse
Definition: L = {w : w  L}
R R
Examples: ab, aab, baba = ba, baa, abab R
L = {a b : n  0}
n n
L = {b a : n  0}
R n n

Concatenation
Definition: L1L2 = xy : x  L1, y  L2 
Example: a, ab, bab, aa
= ab, aaa, abb, abaa, bab, baaa

Another Operation
Definition: L =
n

LL L
n
a, b = a, ba, ba, b =

3
aaa, aab, aba, abb, baa, bab, bba, bbb

0
Special case: L = 
a , bba , aaa 0 = 

More Examples
L = {a b : n  0}
n n
L = {a b a b : n, m  0}
2 n n m m
aabbaaabbb  L 2

Star-Closure (Kleene *)
Definition: L* = L  L  L 
0 1 2
Example:
 , 
a, bb, 
 
a, bb* =  
 aa , abb , bba , bbbb, 
aaa, aabb, abba, abbbb,
Positive Closure
Definition: +
L = L  L 1 2
= L * −
a, bb, 
+  
a, bb = aa, abb, bba, bbbb, 
aaa, aabb, abba, abbbb,
 
Grammars

Grammars
A grammar G is defined as a 4-tuples:
G = (V, T, S, P)
Where:
V : is a finite set of objects called variables (non terminals)
T : is a finite set of objects called terminal symbols
S : S  V is a special symbol called the Start symbol
P : is a finite set of productions or "production rules"
Sets V and T are nonempty and disjoint

Grammars
Production rules have the form:
x→y
where x is an element of (V  T)+ and y is in (V  T)*
Given a string of the form
w = uxv
and a production rule
x→y
we can apply the rule, replacing x with y, giving
z = uyv
We can then say that
wz
Read as "w derives z", or "z is derived from w"
Grammars
If u  v, v  w, w  x, x  y, and y  z, then we say:

uz
*
This says that u derives z in an unspecified number of

steps.
Along the way, we may generate strings which

contain variables as well as terminals. These are called
sentential forms.

Grammars
What is the relationship between a language and a

grammar?
Let G = (V, T, S, P)
The set
L(G) = {w  T* : S  w}
*
is the language generated by G.

Grammars
Consider the grammar G = (V, T, S, P), where:

V = {S}
T = {a, b}
S = S,
P = S → aSb
S→

Grammars
What are some of the strings in this language?

S  aSb  ab
S  aSb  aaSbb  aabb
S  aSb  aaSbb  aaaSbbb  aaabbb
It is easy to see that the language generated by this grammar
is:
L(G) = {anbn : n  0}

Grammars
Let's go the other way, from a description of a language to a
grammar that generates it. Find a grammar that generates:
L = {anbn+1 : n  0}
So the strings of this language will be:
b (0 a's and 1 b)
abb (1 a and 2 b's)
aabbb (2 a's and 3 b's) . . .
In order to generate a string with no a's and 1 b, you might

want to write rules for the grammar that say:
S → ab
a→
But you can't do this; a is a terminal, and you can't change a
terminal, only variables
Grammars
So, instead of:
S → ab
a→
we create another variable, A (we often use capital letters to
stand for variables), to use in place of the terminal, a:
S → Ab
A→
Now you might think that we can use another S rule here to
generate the other part of the string, the anbn part
S → aSb
But you can't, because that will generate ab, aabb, etc.
Note, however, that if we use A in place of S, that will solve
our problem:
A → aAb
Grammars
So, here are our rules:
S → Ab
A → aAb
A→
The S → Ab rule creates a single b terminal on the right,

preceded by other strings (including possibly the empty
string) on the left.
The A →  rule allows the single b string to be generated.
The A → aAb rule and the A →  rule allows ab, aabb,

aaabbb, etc. to be generated on the left side of the string.

Finite Automata

Finite Automaton
Input
String
Output
Finite String
Automaton

Finite Accepter
Input
String
Output
“Accept”
Finite
or
Automaton
“Reject”

Transition Graph
Abba -Finite Accepter a, b
q5
a, b
b a a b
q0 a q1 b q2 b q3 a q4
initial final
state state
transition
state “accept”

Initial Configuration
Input String
a b b a
a, b
q5
a, b
b a a b

Input finished
a b b a
a, b
q5
a, b
b a a b
Output: “accept”
Ex2: Rejection
a b a
a, b
q5
a, b
b a a b

Input finished
a b a
a, b
Output:
q5 “reject”
a, b
b a a b

Ex 3 : Another Rejection
a, b
q5
a, b
b a a b


a, b
q5
a, b
b a a b
Output:
“reject”
Another Example
a a b
a a, b
b a, b
q0 q1 q2

Input finished
a a b
a a, b
Output: “accept”
b a, b
q0 q1 q2

Rejection
b a b
a a, b
b a, b
q0 q1 q2

Input finished
b a b
a a, b
b a, b
q0 q1 q2
Output: “reject”

Formalities
Deterministic Finite Accepter (DFA)
M = (Q, , , q0 , F )
Q : set of states
 : input alphabet
 : transition function
q0 : initial state
F : set of final states

Input Alphabet:  = a, b
Set of States: Q = q0 , q1, q2 , q3 , q4 , q5
Initial State: q0
Set of Final States : F = q4 
a, b
q5
a a, b
b a b

Transition Function 
 a b
q0 q1 q5
q1 q5 q2
q2 q5 q3
q3 q4 q5 a, b
q4 q5 q5
q5 q5 q5 q5
a a, b
b a b

Extended Transition Function  *
 * : Q  * → Q
a, b
q5
a a, b
b a b

 * (q0 , ab) = q2
a, b
q5
a a, b
b a b

 * (q0 , abbbaa) = q5
a, b
q5
a a, b
b a b

Observation: There is a walk from q to q
with label w
 * (q, w) = q
q w q
w =  1 2  k
1 2 k
q q

Example: There is a walk from q0 to q5
with label abbbaa
 * (q0 , abbbaa) = q5
a, b
q5
a a, b
b a b

Recursive Definition
 * (q ,  ) = q
 * (q, w ) =  ( * (q, w), )
q w q1  q
 * ( q , w ) = q
 * (q, w ) =  (q1, )
 (q1, ) = q
 * (q, w ) =  ( * (q, w), )
 * (q, w) = q1

 * (q0 , ab ) =
 ( * (q0 , a ), b ) =
 ( ( * (q0 ,  ), a ), b ) =
 ( (q0 , a ), b ) =
 (q1 , b ) =
q2 a, b
q5
a a, b
b a b
Languages Accepted by DFAs
Take DFA M
Definition:
The language L(M ) contains
all input strings accepted by M
L(M ) = { strings that drive M to a final state}

Example
L(M ) = abba M
a, b
q5
a a, b
b a b
accept

Another Example
L(M ) =  , ab, abba M
a, b
q5
b a a a, b
b
accept accept accept

Formally
For a DFA M = (Q, , , q0 , F )
Language accepted by M :
L(M ) = w  * :  * (q0 , w)  F 
q0 w q q  F

Observation
Language rejected by M :
L(M ) = w  * :  * (q0 , w)  F 
q0 w q q  F

Example 1
L(M ) = {a b : n  0}n
a a, b
b a, b
q0 q1 q2
accept trap state

Example 2
L(M ) = { all strings with prefix ab }
a, b
q0 a q1 b q2
b a accept
q3 a, b

Example 3
L (M ) = { all strings without
substring 001 }
1 0 0,1
1
 0 1
0 00 001
0
Regular Languages
A language L is regular if there is

a DFA M such that L = L(M )
All regular languages form a language family

Examples of regular languages:
abba  , ab, abba {a b : n  0}

n
{ all strings with prefix ab }

{ all strings without substring 001 }
There exist automata that accept these

Languages (see previous slides).
Another Example
The language L = awa : w  a, b*
is regular:
a
b
L = L( M ) b
q0 a q2 q3
b a
q4
a, b
There exist languages which are not Regular:
Example: L= {a b : n  0}
n n
There is no DFA that accepts such a language
(we will prove this later in the class)

Any Questions

Computation Theory: Three Basic Concepts: Languages, Grammars, and Automata

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Computation Theory: Three Basic Concepts: Languages, Grammars, and Automata

Uploaded by

Copyright:

Available Formats

Computation Theory

Three Basic Concepts:

Languages, Grammars, and Automata 1

Core mathematics of CS (has not changed in

Languages, Grammars, and Automata 2

▪ Father of Modern Computer Science

Languages, Grammars, and Automata 3

Languages, Grammars, and Automata 4

Languages, Grammars, and Automata 5

Languages, Grammars, and Automata 6

Languages, Grammars, and Automata 7

String: A sequence of letters

Examples: “cat”, “dog”, “house”, …

Defined over an alphabet:

Languages, Grammars, and Automata 8

w = a1a2 an abba

wv = a1a2 anb1b2 bm abbabbbaaa

Languages, Grammars, and Automata 10

Languages, Grammars, and Automata 11

abba = abba = abba

Example: (abba ) = abbaabba

Languages, Grammars, and Automata 17

Languages, Grammars, and Automata 18

Set size {} = 1

a, ab, aaaa  bb, ab = {a, ab, bb, aaaa}

a, ba =  , b, aa, ab, bb, aaa,

Examples: ab, aab, baba = ba, baa, abab R

Languages, Grammars, and Automata 24

Definition: L1L2 = xy : x  L1, y  L2 

Example: a, ab, bab, aa

= ab, aaa, abb, abaa, bab, baaa

Languages, Grammars, and Automata 25

a, b = a, ba, ba, b =

aaa, aab, aba, abb, baa, bab, bba, bbb

a , bba , aaa 0 = 

Languages, Grammars, and Automata 27

Languages, Grammars, and Automata 30

Languages, Grammars, and Automata 31

If u  v, v  w, w  x, x  y, and y  z, then we say:

This says that u derives z in an unspecified number of

Along the way, we may generate strings which

Languages, Grammars, and Automata 33

What is the relationship between a language and a

is the language generated by G.

Languages, Grammars, and Automata 34

Consider the grammar G = (V, T, S, P), where:

Languages, Grammars, and Automata 35

What are some of the strings in this language?

Languages, Grammars, and Automata 36

In order to generate a string with no a's and 1 b, you might

The S → Ab rule creates a single b terminal on the right,

The A →  rule allows the single b string to be generated.

The A → aAb rule and the A →  rule allows ab, aabb,

Languages, Grammars, and Automata 39

Languages, Grammars, and Automata 40

Languages, Grammars, and Automata 41

Languages, Grammars, and Automata 42

Languages, Grammars, and Automata 43

Languages, Grammars, and Automata 44

Languages, Grammars, and Automata 46

Languages, Grammars, and Automata 47

Languages, Grammars, and Automata 48

Languages, Grammars, and Automata 50