==============

Undecidability
==============
Notes from: Automata Theory, Languages, and Computation
We use a TM to capture the essential idea of what any computing
device is -- at best -- capable of doing.
- ignore memory limitations, address-space-size limitations...
Decidable problems for TMs:
---------------------------
1) does a TM have S states?
2) is there any input that would cause a TM to make at least five moves?
- if a TM makes 5 moves then it does so via looking at only the 9
cells of its tape surrounding the initial head position
-- so we can brute force through all of the input symbols and
possible tape configs and determine whether the TM is still
alive after any such run
- if so, the answer is yes; if not... no.
+++++++++++++++++++++++++++++++
Undecidable problems: two types
+++++++++++++++++++++++++++++++
========================
(1) RE but not recursive
========================
- Those for which a TM exists but not an algorithm
- algorithm is short hand for: a TM that always (eventually)
halts (on any input), whether it accepts this input or not
- so the essential problem here is that we have a TM that
may run forever on inputs that the TM doesn't accept
-- so if the TM is still running (for a given input), we
don't know which world we're in:
- the world where this input IS in the language, we
just haven't gotten to the accepting state yet
- the world where this input is NOT in the language
- e.g. the universal language, L_u, which is:
L_u = { (M,w) | M,w bitstrings and w is in L(M) }
- i.e. the TM encoded by M accepts input bitstring w
==========
(2) Non-RE
==========
- no TM exists; e.g. the diagonalization language, L_d, which is:
L_d = { all bitstrings w such that the TM encoded by w
does not accept itself as input (i.e. input w) }
-------------------
Miscellaneous facts
-------------------
1) If a language is recursive then so is its complement.
2) If a language and its complement are both RE then those
two languages are also both recursive.
3) A RE language's complement is not necessarily RE (otherwise
all RE languages would also be recursive languages).
3) If there is a *reduction* from P_1 to P_2
(a) if P_1 is undecidable then so is P_2
(b) if P_1 is non-RE then so is P_2
4) Every non-trivial property of the RE languages is undecidable.
property of RE languages: subset of RE languages
trivial property of RE langs: the set is all RE langs or NO RE langs
--------------------------------------------
Proof that L_d is not recursively enumerable
--------------------------------------------
L_d = { the set of strings w_i such that w_i is not in L(M_i) }
// where w_i is a bitstring
// and M_i is the Turing Machine encoded by that bitstring
// thus it consists of all TMs that do not accept themselves as input
L_d has no TM which accepts it (is not RE).
Intuitively, L_d cannot be accepted by a TM because if it were
then the TM that accepts L_d would have to disagree with
itself when given itself as input.
Encoding TMs
------------
1) enumerate all bitstrings; assign an integer to all binary strings
bitstring i
--------- -
epsilon 0
0 1
1 2
00 3
01 4
... ...
2) how to encode a TM as a bitstring
Q : q_1, ..., q_r (for some r)
q_1 : always the start state
q_2 : always the only accept state
 TAU : the tape alphabet
X_1, X_2, ..., X_s (for some s)
where X_1 == 0, X_2 == 1, X_3 == blank
directions: L = D_1 and R = D_2
Then we can encode a TM's transition function as follows:
DELTA( q_i, X_j ) = ( q_k, X_m, D_n )
0^i 1 0^j 1 0^k 1 0^m 1 0^n i,j,k,m,n >= 1
The code for an entire TM M consists of all the above codes
for the transitions in some order, separated by 11
C_1 11 C_2 11 ... 11 C_n
where each C_i is the code for one transition of M
Note that there may be several different encodings for the same TM
since the order in which transition function codes are listed is
irrelevant. Note also that there will be some bitstrings that
do not encode valid TMs. For example, any bitstring that doesn't
begin with a 0 represents an invalid TM. Bitstrings which do not
represent valid TMs are treated as TMs that have a single non-
accepting state and no transitions: so they immediately halt
without accepting regardless of the input. Thus, the language
accepted by all such TMs is the empty language.
E.g. for transition DELTA( q_3, 0 ) = ( q_1, 1, R ), we get
0^3 1 0^1 1 0^1 1 0^2 1 0^2
--- --- --- --- ---
q_3 0 q_1 1 R
So now we can build a matrix where each row represents a value for i
and thus represents some bitstring -- and thus represents some TM.
The columns are increasing j values, which also represent bitstrings.
The cell (i,j) in the matrix is defined as:
-------------------------------------------
1 if M_i accepts w_j as input
0 otherwise
Then the i'th row of this matrix can be thought of as the
"characteristic vector" for the language L(M_i); all 1's in
the row indicate strings that are members of this language.
The diagonal of this matrix then is the language of TMs that
accept themselves as input, since cell (i,i) is 1 when M_i
accepts w_i as input and 0 otherwise. We can think of this
diagonal as itself a characteristic vector.
Then, to construct L_d, we complement this diagonal. We claim that
the characteristic vector formed by complementing the diagonal is
NOT the language of any row. Why? Because the diagonal disagrees
with every row of this matrix in some column. But we've already
said that every possible TM is encoded by some row in this matrix
and thus this complemented diagonal cannot be the characteristic
vector of any TM and thus there is no TM that accepts L_d.
What have we just proven? That L_d is not recursively enumerable.
=========================================================
There are two classes of recursively enumerable languages
=========================================================
(1) Those that have an algo; these are recursive. DECIDABLE
(2) Those that do not have an algo. UNDECIDABLE
+++++++++++++++++++
Recursive languages
+++++++++++++++++++
A language L is recursive if L = L(M) for some TM M such that:
1) if w is in L, M accepts and halts
2) if w is not in L, M halts without accepting
Thm: Recursive languages are closed under complementation.
Pf: Let L = L(M) for some TM M that always halts. Construct the
TM for !M such that: !L = L(!M).
- the accepting states of M are made non-accepting states of !M
- !M has a new accepting state r
- for each non-accepting state p in M and a tape symbol T such that
DELTA( p, T ) = {} // M has no transitions out on p given T
add a transition in !M to the accepting state r
-------------------------------------------------------------------
Thm: If both a language L and its complement are RE then both L and
its complement are recursive.
-------------------------------------------------------------------
Proof:
L = L(M_1)
!L = L(M_2)
Construct the TM M that takes input w and simulates both M_1 and M_2
on w. If M_1 accepts w then accept. If M_2 accepts w then reject.
If input w is in L then M_1 will eventually accept then halt. If so,
the constructed TM M will accept and halt. Otherwise if input w is
not in L then M_2 will eventually accept w then halt - in which case
M halts without accepting. Thus on all inputs M halts and thus
L(M) is recursive.
=====================================================================
So there are three possibilities for relationships between a language
and its complement:
=====================================================================
1) a language and its complement are both RE and thus recursive
2) a language and its complement are both not RE
3) the language is RE but its complement is not RE
E.g. consider the diagonalization language, L_d
L_d = { w_i | M_i does not accept w_i }
where M_i = the TM encoded by the i'th bitstring
w_i = the i'th bitstring
so L_d is the TMs that do not accept themselves as input.
L_d is NOT recursively enumerable (proof above). The complement of L_d:
L_nd = { w_i | M_i accepts w_i }
- so we take the TM encoded by the i'th bitstring and run it on itself
-- if it accepts then w_i is in L_nd
- it's clear how we could reduce from an instance of L_nd (i.e. we
have w_i) TO an instance of L_u
-- feed: w111w to L_u (where L_u expects input of the form: M111w)
if ( L_u accepts) then the TM for L_nd accepts
- but it's less clear how we would go from an instance of L_u (arbitrary
(M,w)) TO an instance of L_nd -- except for if we just waited for L_u
to choose (M,w) such that the bitstring for M == the bitstring for w and
then piggybacked on that.
--> but maybe I'm thinking too hard
--> we can simulate U as a subroutine of the TM for L_nd so perhaps
that alone lets us get the information that we need
==================================================
The universal language, L_u : RE but not recursive
==================================================
L_u = { (M,w) | TM M accepts input string w }
- the set of strings representing a TM and an input accepted by that TM
There is a universal Turing machine U such that: L(U) = L_u
- we can build this TM and hence L_u is RE (details: pages 387 - 389)
L_u is NOT recursive
--------------------
Suppose that L_u were recursive but then the complement of L_u (which
we'll call L_nu) would also be recursive.
L_nu = { (M,w) | TM M does not accept input string w }
But if we had an algorithm that accepted L_nu then we could build a TM
to accept L_d, where:
 L_d = { M | TM M does not accept input string M }
we would just feed every input M to L_d into L_nu as: (M,M). L_nu will
accept (M,M) when the bitstring M is *not accepted* by the TM encoded
by M, i.e. when the TM M does not accept itself as input. So we'll just
accept whenever L_nu accepts and reject whenever L_nu rejects.
But we proved above that L_d is not recursively enumerable and hence
our assumption must have been false and L_u is NOT recursive.
=========================
Theorems about reductions
=========================
If there is a reduction from P_1 to P_2 then
1) if P_1 is undecidable then so is P_2
2) If P_1 is non-RE then so P_2 is also non-RE
Proof by contradiction
----------------------
Suppose P_1 is undecidable and P_2 is decidable.
Suppose we're given an instance w of P_1. We'll apply the reduction algorithm
to w to get an instance x of P_2.
Then since P_2 is decidable, we'll use its algorithm to determine whether
P_2 accepts x. If so, then P_1 should accept w. If not, then P_1 should
reject w. Thus we have constructed an algorithm to decide P_1, contradicting
our assumption that P_1 was undecidable.
======================================
Turing machines and the empty language
======================================
L_e = { all encoded TMs whose language is empty }
-- not recursively enumerable
L_ne = { all encoded TMs that accept at least one string }
-- recursively enumerable (but obviously not recursive)
-----------------------------------
Thm: L_ne is recursively enumerable
-----------------------------------
Proof: construct a non-deterministic TM M that accepts L_ne
M takes as input a TM code M_i
- guesses an input w that M_i might accept
- tests whether M_i accepts w (via simulating the universal TM U)
- if M_i accepts w then M accepts M_i
--------------------------
Thm: L_ne is NOT recursive
--------------------------
Proof: We'll reduce L_u to L_ne.
 - L_u is undecidable (RE but not recursive) and hence so is L_ne.
- Recall that L_u takes (M,w) and accepts if w is in L(M)
- And L_ne takes M and accepts if L(M) != {}
Once we perform this reduction, we can use it to claim that SINCE
L_u reduces to L_ne AND since L_u is not recursive, neither is L_ne.
So what do we need to do this?
------------------------------
We need an algorithm that converts a specific instance (M,w) of L_u
into a Turing machine M' such that:
M' accepts <==> L_u accepts (M,w)
The algo:
---------
1) M' receives some input x; i.e. the contents of the tape of M' are x
2) M' overwrites these tape contents to be (M,w)
3) M' simulates the universal TM U on (M,w)
4) If U accepts (M,w) then M' accepts
If U never accepts (M,w) then M' never accepts
Note that our conversion algo creates M' for a *specific* L_u instance;
i.e. the # of states of M' will be a factor of the length of M and w.
M' is carefully designed for a particular P_1 instance.
So we can really think of this conversion algorithm as a wrapper fxn;
it basically overwrites the tape for M' with the given instance (M,w)
then runs the U TM on that (M,w). If the single instance (M,w) is
accepted by U, then L(M') accepts **every input x** that it receives.
If, on the other hand, U never accepts this specific (M,w) input
then L(M') will never accept *any* of its inputs.
- if M accepts w then the code for TM M' is in L_ne
- if M never accepts w then the code for TM M' is not in L_ne
Because such a reduction exists and because L_u is not recursive, we
can use this to claim that L_ne is also not recursive.
- if L_ne were recursive, we could use it to build an algo for L_u
1) reduce instance of L_u to TM M' as above
2) since we assumed L_ne is recursive, use its algorithm to
determine whether or not L(M') = {}. If L(M') == {}, then
have L_u reject (M,w); else L_u accepts (M,w).
but we already know that L_u is not recursive and hence no such
algorithm for L_u exists. So we have contradicted the assumption
and consequently L_ne is not recursive.
----------------------------------------------------------------
Since we know that L_ne is RE but not recursive, we can conclude
that L_e is not RE.
----------------------------------------------------------------
+++++++++++++++++++++++++++++++++++++++++++++++++
Rice's theorem and the properties of the RE langs
+++++++++++++++++++++++++++++++++++++++++++++++++
So, as we have seen above, we don't have an algo for determining
whether a particular encoded TM M accepts at least one string or
if, instead, M never accepts any input (i.e. L(M) = {}).
Property: a set of RE languages
- the property of being context-free is the set of all CFLs
- the property of being regular is the set of all reg langs
- the property of being empty is the set consisting of only
the empty language
Trivial properties: any property which applies to ALL REs or NO REs.
- note that the property of being empty is not trivial since
the cardinality of that set is 1 -- since it contains the
empty language itself
Sample non-trivial property: the language is a CFL.
L_e : all RE langs that do not accept any input
L_ne : all RE langs that accept at least one string
Both of these are undecidable. In fact, all non-trivial properties of
RE langs are undecidable.
- impossible to build a TM M that takes as input TMs encoded as
bitstrings and determines whether each encoded TM has a
particular property
- for example, we can't build a TM that takes M_i's and
if ( L(M_i) is a CFL ) then accept
- b/c figuring out whether a given TM M_i accepts a CFL is undecidable
So whenever a problem is undecidable (e.g. determining whether the
language accepted by a TM is a CFL) then the property for that problem
is undecidable as well. Since all a property is is a set of TMs and
since we can't figure out which TMs belong in that set then the property
is undecidable.
You may be thinking: well, if we can't build an algorithm that takes a
TM as input and determines something about the language of that TM,
how about taking the language itself as input (i.e. taking all strings
of a language as input) and then seeing whether the property holds
over all of those strings.
- problem is that languages are generally infinite
- and the input to a TM must be finite
Thus, if P is a property of the RE langs, then:
L(P) = { the set of encoded TMs M_i such that L(M_i) is a language in P }
=========================================================================
Rice's theorem: Every non-trivial property of the RE langs is UNDECIDABLE
=========================================================================
Proof: We'll reduce L_u to L_p
Let P be some non-trivial property of the RE langs.
Assume that the empty language is not in P.
Since P is non-trivial, there must be some non-empty language L that
is in P. Let M_L be the TM which accepts L.
Reduction:
----------
Input: (M,w) // an instance of L_u
Output: TM M' // an instance of L_p
(1) simulate M on w using the universal TM U
- if ( M accepts w ) then M' simulates M_L on input x
--> that is, if M accepts w then L(M') = L
--> we add the code for M' to L_p
- if ( M does not accept w ) then M' does not accept any of its inputs
--> so L(M') = {}
--> Since we assumed above that the empty language is not in P,
we're done here
So we only simulate M_L on an input if ( M accepts w )
This algo reduces an instance of L_u to an instance of L_p; we know
that L_u is undecidable, ergo L_p must be as well.
(2) the case where the empty language IS in L_p
... see page 399
---
Since we have reduced L_u to L_p and since L_u is undecidable so is L_p.
--------------------------------------------
Undecidable vs. Decidable problems about TMs
--------------------------------------------
- Any problem that involves only the language that a TM accepts is undecidable
-- is the language accepted by a TM empty?
-- is the language accepted by a TM finite?
-- is the language accepted by a TM regular?
-- is the language accepted by a TM CF?
- Problems about the states of the TM may be decidable.
-- does a TM have five states?
-- is there some input for which the TM makes at least five moves?
- since the TM can only move a single cell per move, we know that
 any permutation of five moves will only look at the tape in at
most 9 slots: the four on either side of the initial position
moreover, the input symbol and tape symbol alphabets are finite
and a TM can only consume at most one input symbol per move
so we can try all possible tape configurations (values in cells)
and simulate the TM on each. if any such is still alive after 5
moves...
===================================
Post's correspondence problem (PCP)
===================================
Prove that PCP is undecidable via reducing L_u to PCP.
An instance of PCP consists of two equal-length strings over SIGMA:
A = w_1, w_2, ..., w_k for some integer k
B = x_1, x_2, ..., x_k
For each i, (w_i, x_i) is a corresponding pair.
An instance of PCP has a solution if there is a sequence of one or more
integers i_1, i_2, ..., i_m such that:
w_(i_1) w_(i_2) ... w_(i_m) == x_(i_1) x_(i_2) ... x_(i_m)
That is, if we concatenate the strings in A at indices i_1, i_2, ..., i_m
(in order) to get an output string; that output string equals the string
we get when we concatenate the strings in B at those same indices (and
also in order).
E.g. for k = 3, an instance might be:
A = 1, 10111, 10
B = 111, 10, 0
Then a solution would be: 2, 1, 1, 3
w_2 w_1 w_1 w_3 = 101111110
x_2 x_1 x_1 x_3 = 101111110
------------
Modified PCP
------------
The first pair of the solution must be the first corresponding pair.
A, B as above
But MPCP has a solution if there is a sequence of ZERO or more ints i_1,
i_2, ..., i_m such that:
w_1 w_(i_1) w_(i_2) ... w_(i_m) == x_1 x_(i_1) x_(i_2) ... x_(i_m)
The first corresponding pair is forced to be at the beginning of the strings.
=============================
Proof of PCP's undecidability
=============================
Reduce L_u to L_MPCP then reduce L_MPCP to L_PCP.
First we note that intuitively MPCP is easier to solve than PCP since
instead of considering k possible corresponding pairs for the first
spot, we only look at the first corresponding pair of the two lists.
Consequently, reducing MPCP to PCP should be possible since we're
reducing an "easier" problem to a "harder" problem and we can always
go in this direction with a reduction: P_2 is at least as hard as P_1
(but may be harder).
Reducing MPCP to PCP
--------------------
Given an instance of MPCP over alphabet SIGMA, we construct an instance
of PCP as follows:
So the game is that we have to transform our MPCP instance so that
PCP will be forced to choose the first corresponding pair for the
start of the solution.
The way we do this:
for each w_i = a_1 a_2 ... a_n
construct y_i = a_1 * a_2 * ... * a_n *
So we alter each string w_i in A's list by inserting an asterisk AFTER
each symbol in w_i.
for each x_i = b_1 b_2 ... b_m
construct z_i = * b_1 * b_2 * ... * b_m
So we alter each string x_i in B's list by inserting an asterisk BEFORE
each symbol in x_i.
then all y_i start with some symbol in SIGMA and all z_i start with *
to complete the transformation, we introduce a new pair, the 0'th:
y_0 = * y_1 // y_1 does NOT begin with an asterisk but y_0 does now
z_0 = z_1 // z_1 begins with an asterisk
So, since all y_i values besides zero do NOT BEGIN with an asterisk AND
all z_i values DO begin with an asterisk, we have to start with the y_i
value that does begin with an asterisk, which is y_0.
Thus the first corresponding pair for any solution must necessarily be (y_0,z_
0).
Then, y_0 ends with an asterisk (since y_1 ends with an asterisk) and all
z_i values begin with an asterisk so the problem continues equivalent to how
it would have continued for MPCP.
We also add a pair ($, *$) as the (k+1)'th pair which allows us to end the str
ings.
E.g. MPCP instance
A = 1, 10111, 10
B = 111, 10, 0
corresponding PCP instance:
C = *1*, 1*, 1*0*1*1*1*, 1*0*, $
D = *1*1*1, *1*1*1, *1*0, *0, *$
since the only string in C that begins with an asterisk is the first one (*1*)
and since all strings in D begin with asterisks, we must use the first
corresponding pair as the first pair of our solution.
-------------------------
Thm: MPCP reduces to PCP.
-------------------------
Proof: in the construction above. If the MPCP instance has a solution then
w_1 w_(i_1) w_(i_2) ... w_(i_m) == x_1 x_(i_1) x_(i_2) ... x_(i_m)
So if we replace all w_i and x_i by y_i and z_i, we get two strings that
are almost the same:
y_1 y_(i_1) ... y_(i_m) != z_1 z_(i_1) ... z_(i_m)
They are not exactly the same since:
z_1 begins with an * and y_1 does not
y_(i_m) ends with an * and z_(i_m) does not
But we note that y_0 == * y_1 and z_0 == z_1 so then we replace the first
pair of each string with the 0'th pair to get:
now both strings begin with an asterisk but are otherwise unaltered
If we append y_(i_(k+1)) to the end of the first list and z_(i_(k+1)) to
the end of the second list, then that amounts to adding a $ to the end
of the first string and adding *$ to the end of the second string, which
takes care of the incompatible endings.
Ergo, { 0, i_1, i_2, ..., i_m, k+1 } is a solution to the PCP instance.
Now, going in the other direction: proving that a solution to the
constructed PCP instance will yield a solution to MPCP.
- Our constructed PCP instance has y_0, y_1, ..., y_(k+1) and
z_0, z_1, ..., z_(k+1).
- The given PCP solution must begin with 0'th pair (since that's the
only corresponding pair between the two lists that agrees in the
first character) and it must end with the (k+1)'th pair (since that's
the only corresponding pair between the two lists that agrees in
the last character.
--------------------
Reducing L_u to MPCP
--------------------
Given a pair (M,w) construct an instance (A,B) of MPCP such that M accepts
w if and only if (A,B) has a solution.
- pages 407 - 412
+++++++++++++++++++++++++++
Other undecidable problems: Reduce PCP to the prob we wish to prove undecidable
+++++++++++++++++++++++++++
(1) Reducing PCP to the question of whether a given CFG is ambiguous
(2) Reducing PCP to the problem of complementing a list language