# Dabu Claudia-Maria IS anul I

Introduction to Complexity Theory

Space Complexity

Time and space are two of the most important considerations when we seek practical solutions to computational problems. Space complexity shares many of the same features of time complexity. Therefore, space complexity serves as a further way of classifying problems according to their computational difﬁculty. We will continue to use the TM as our model, but now we look at the tape space it consumes during its computation. Space Complexity of an algorithm is total space taken by the algorithm with respect to the input size. Space complexity includes both Auxiliary space and space used by input. Auxiliary Space is the extra space or temporary space used by an algorithm. Complexity classes correspond to bounds on resources. One such resource is space: the number of tape cells a TM uses when solving a problem. Defining complexity classes The measure DSPACE is used to define complexity classes, sets of all of the decision problems which can be solved using a certain amount of memory space. For each functionf(n), there is a complexity class SPACE(f(n)), the set of decision problems which can be solved by a deterministic Turing machine using space O(f(n)). There is no restriction on the amount of computation time which can be used, though there may be restrictions on some other complexity measures (like alternation). Several important complexity classes are defined in terms of DSPACE. These include:

REG = DSPACE(O(1)), where REG is the class of regular languages. In fact, REG = DSPACE(o(log log n)) (that is, Ω(log log n) space is required to recognize any nonregular language). L = DSPACE(O(log n))

1

that is. but many complexity classes are based on nondeterministic Turing machines. quantum Turing machines. for all c ≥ 1 and f such that f(n) ≥ 1. such as L (logarithmic space). or for the size of the output. etc. optimization problems. A complexity class is a set of problems of related complexity. This justifies usage of big O notation in the definition. Simpler complexity classes are defined by the following factors:  The type of computational problem: The most commonly used problems are decision problems.counting problems. The resource (or resources) that are being bounded and the bounds: These two properties are usually stated together. to be defined in terms of the amount of space used by all of the work tapes (excluding the special input and output tapes). smaller than the size of the input. "logarithmic space". a typical complexity class has a definition like the following: The set of decision problems solvable by a deterministic Turing machine within time f(n). monotone circuits. complexity classes can be defined based on function problems. However. etc. except that the input tape may never be written-to. would not truly capture the memory space used. etc. "constant depth". promise problems. Thus.Dabu Claudia-Maria IS anul I  PSPACE =  EXPSPACE = DSPACE is traditionally measured on a deterministic Turing machine. such as "polynomial time". Thus. and the output tape may never be read from. This is solved by defining the multi-string Turing machine with input and output. which is a standard multi-tape Turing machine.   Of course.) 2 . This allows smaller space classes. The model of computation: The most common model of computation is the deterministic Turing machine. the class of languages recognizable in c f(n) space is the same as the class of languages recognizable in f(n) space. Since many symbols might be packed into one by taking a suitable power of the alphabet. some complexity classes have complex definitions that do not fit into this framework. (This complexity class is known as DTIME(f(n)). "charging" the algorithm for the size of the input. Boolean circuits. Several important space complexity classes are sublinear.

This forms the basis for the complexity class P. Cobham-Edmonds thesis states that "the time complexities in any two reasonable and general models of computation are polynomially related" (Goldreich 2008. which is the set of decision problems solvable by a deterministic Turing machine within polynomial time. Some important complexity classes of decision problems defined in this manner are the following: Complexity class Model of computation Resource constraint DTIME(f(n)) Deterministic Turing machine Time f(n) P Deterministic Turing machine Time poly(n) EXPTIME Deterministic Turing machine Time 2poly(n) NTIME(f(n)) Non-deterministic Turing machine Time f(n) NP Non-deterministic Turing machine Time poly(n) NEXPTIME Non-deterministic Turing machine Time 2poly(n) DSPACE(f(n)) Deterministic Turing machine Space f(n) L Deterministic Turing machine Space O(log n) 3 . but necessarily requires quadratic time in the model of single-tape Turing machines. Chapter 1. For instance.2). The corresponding set of function problems is FP. the language {xx| x is any binary string} can be solved in linear time on a multi-tape Turing machine. Many important complexity classes can be defined by bounding the time or space used by the algorithm. If we allow polynomial variations in running time.Dabu Claudia-Maria IS anul I But bounding the computation time above by some concrete function f(n) often yields complexity classes that depend on the chosen machine model.

This allows smaller space classes. that is. and the output tape may never be read from. which are defined using quantum Turing machines. for all c ≥ 1 and f such that f(n) ≥ 1. would not truly capture the memory space used. smaller than the size of the input. except that the input tape may never be written-to. Several important space complexity classes are sublinear. Classes like IP and AM are defined using Interactive proof systems. which are defined using Boolean circuits and BQP and QMA. ZPP and RP. #P is an important complexity class of counting problems (not decision problems). Since many symbols might be packed into one by taking a suitable power of the alphabet.Dabu Claudia-Maria IS anul I PSPACE Deterministic Turing machine Space poly(n) EXPSPACE Deterministic Turing machine Space 2poly(n) NSPACE(f(n)) Non-deterministic Turing machine Space f(n) NL Non-deterministic Turing machine Space O(log n) NPSPACE Non-deterministic Turing machine Space poly(n) NEXPSPACE Non-deterministic Turing machine Space 2poly(n) It turns out that PSPACE = NPSPACE and EXPSPACE = NEXPSPACE by Savitch's theorem. such as L (logarithmic space). Thus. to be defined in terms of the amount of space used by all of the work tapes (excluding the special input and output tapes). which is a standard multi-tape Turing machine. DSPACE is traditionally measured on a deterministic Turing machine. or for the size of the output. AC and NC. This is solved by defining the multi-string Turing machine with input and output. Other important complexity classes include BPP. "charging" the algorithm for the size of the input. This justifies usage of big O notation in the definition. ALL is the class of all decision problems. which are defined using probabilistic Turing machines. 4 . the class of languages recognizable in c f(n) space is the same as the class of languages recognizable in f(n) space.

If M is a nondeterministic TM where in all branches halt on all inputs we deﬁne its space complexity f(n) to be the maximum number of tape cells that M scans on any branch of its computation on any input of length n. where f(n) is the maximum number of tape cells that M scans on any input of length n. then we also say that M runs in space f(n). SPACE(f(n)) ={L|Lis decided by anO(f(n))space TM} NSPACE(f(n)) ={L|L is decided by an O(f(n))space NTM} Low Space Classes Definitions (logarithmic space classes): • • L = SPACE(logn) NL = NSPACE(logn) Problem! How can a TM use only logn space if the input itself takes n cells?! 3Tape Machines 5 .Dabu Claudia-Maria IS anul I Deﬁnition: Let M be a deterministic TM that halts on all inputs. Deﬁnition: Let f : N→R+ be a function. The space complexity classes. We deﬁne the space complexity of M to be the function f : N→N. are deﬁned as follows.SP ACE(f(n)) and NSP ACE(f(n)). If the space complexity of M is f(n).

For the reminder of this lecture notes. 3. “Turing Machine" will refer to a 3-tape Turing Machine. output tape. input tape. work tape. For any language L define XL (x) so that if x XL (x) = 1 otherwise XL (x) = 0. We use this model because it is easier to deal with it. Read and write. We remind that any multi-tape TM can be simulated by an ordinary TM with a loss of eficiency that is only polynomial. Define W M (x) to be the index of the rightmost cell on the worktape scanned by M on input x. Space complexity is measured by the bounds on the machine's position on this tape. Write-only. Usually considered unidirectional: this assumption is not essen-tial but useful. Thus we could only measure space complexities which are at least linear. Writing is not allowed on the input tape: this way space is measured only on the worktape. as considered below. The 3 tapes are: 1. Read-only 2. For decision problems. In order to consider also sublinear space bounds we restrict the input tape to be read-only. 6 L then . one can omit the output-tape altogether and have the decision in the machine's state.Dabu Claudia-Maria IS anul I The model of computation we will use is a 3-tape Turing Machine. If we allowed writing on the input tape then the length of the input itself should be taken into account when measuring space. Define S M (n) = max|x|=n W M(x).

Therefore if |x|= n a configuration gives information about the following: state of M (O(1) bits) contents of the work tape (s(n) bits) head position in the input tape (log (n) bits) head position in the work tape (log (s(n)) bits) Sub-Logarithmic Space Complexity Working with sublogarithmic space is not so useful. One may b e tempted to think that whatever can be done in o(log (n)) space can also be done in constant space. and since obviously that in fact we may also (incorrectly) argue This intuition comes from the following imprecise observation: if space is not constant. is forced to use constant space. Therefore any M that uses less than O(log (n)) cells. machine M must determine how much space to use.Dabu Claudia-Maria IS anul I Definition 4. Definition: A configuration of M is an instantaneous representation of the computation carried on by M on a given input x. where x is the input.and therefore will not be done when discussing such space bounds. Otherwise. 7 .1 (Dspace): We may multiply s( ) by log2 |TM | where TM is the alphabet used by M. . Determining how much space to use seems to require the machine counting up to at least |x|= n which needs O(log (n)) space. (However.) This is done in order to have a correspondence to the number of configurations. we could always linearly compress the number of space cells using a bigger alphabet. this convention disallow treatment of sublogarithmic space. It turns out that this intuition is wrong and the reason is that the language itself can help in deciding how much space to use. We may also add log2(|x|) to s( ). Formally this would mean Dspace(o(log (n))) .

…. entering an infinite loop and never stop. associated with any position on the input tape. If the sequence of such configurations is C i = C1 i. We can write s(n) 2o(s(n)) = 2o(s(n)) and since s is at least logarithmic. Theorem Dspace (o(log2log2(n)) = Dspace (O(1)) Proof: Consider a s( ) -bounded machine M on the alphabet { 0. Let be C the number of possible configurations of M on input x. The kind of argument used to prove their equivalence extends the one used to prove the following simpler fact. then M can be on every cell on the input tape at most k = 2 s(n) *s(n)*|Q M |= O( 2s(n) ) times. Otherwise. n is the number of possible locations of the head on the input tape. For any s(n) : s(n) > log(n) Proof: Fix an input x: | x |= n and a deterministic machine M that accepts x in space s(n). is bounded by . 8 . we consider all possible semi-configurations of M when passing location i. For every location i on the input tape. The number of possible diferent sequences of semi-configurations of M. M will go through the same configuration at least twice. . s(n) is the number of possible locations of the head on the worktape and 2o(s) is the number of possible contents in the worktape { the exponent is o(s) because the alphabet is not necessarily binary. Then necessarily M has to run in time t(n)*2o(s) .1 }. Therefore C < 2o(s(n)) M cannot run on input x for a time t(n) > 2 s(n) . Claim: given input x : |x| = n such that M accepts x. n<2o(s(n)) . Then an upper bound for C is: where Q M is the set of states of M.Dabu Claudia-Maria IS anul I Going further on. We will show that these two complexity classes are equivalent. we can consider Dspace (o(log log (n)) and Dspace (O(1)). The reason being that if M were to be on the cell more than k times then it would be in the same configuration twice. and thus never terminate. We dene a semi-configuration as a configuration with the position on the input tape replaced by the symbol at the current input tape position. Theorem . Cr i then by the above claim its length is bounded: r<O (2 s(n) ).

More precisely. Thus there are pairs of complexity classes such that one is properly included in the other. for every space-constructible but . the answer to such questions is given by the time and space hierarchy theorems respectively. So labelling n 1 positions on the input tape by at most n1 /3 sequences means that there must be at least three positions with the same sequence of semi-configurations. it is desirable to prove that relaxing the requirements on (say) computation time indeed defines a bigger set of problems. In particular. we can proceed to make quantitative statements about how much more additional time or space is needed in order to increase the number of problems that can be solved. there exists some language L which is decidable in space . although DTIME(n) is contained in DTIME(n2). For instance. the time hierarchy theorem tells us that P is strictly contained in EXPTIME. it would be interesting to know if the inclusion is strict. Having deduced such proper set inclusions. Hierarchy theorems The space function not in space hierarchy theorem shows that. Say Where each of the positions with symbol a has the same sequence of semi-configurations attached to it. For the complexity classes defined in this way. We then show and therefore there exists that . For time and space requirements. The space hierarchy theorem states that . such Thus Dspace (O(1)) proving the theorem. the time hierarchy theorem states that . 9 .Dabu Claudia-Maria IS anul I Since s(n) = o(log2log2n) then that . The number of sequences of semi-configurations at any position in the input tape is < n1/3. and the space hierarchy theorem tells us that L is strictly contained in PSPACE. They are called hierarchy theorems because they induce a proper hierarchy on the classes defined by constraining the respective resources. The time and space hierarchy theorems form the basis for most separation results of complexity classes.