The document provides an introduction to complexity theory, which classifies computational problems based on the resources required to solve them, such as running time. It discusses different models of computation including finite automata, pushdown automata, and Turing machines. Complexity theory aims to determine which problems are feasible to solve with limited resources and whether some problems are inherently harder than others. It introduces complexity classes like P (problems solvable in polynomial time) and NP (problems verifiable in polynomial time) and discusses the open question of whether P equals NP.
The document provides an introduction to complexity theory, which classifies computational problems based on the resources required to solve them, such as running time. It discusses different models of computation including finite automata, pushdown automata, and Turing machines. Complexity theory aims to determine which problems are feasible to solve with limited resources and whether some problems are inherently harder than others. It introduces complexity classes like P (problems solvable in polynomial time) and NP (problems verifiable in polynomial time) and discusses the open question of whether P equals NP.
The document provides an introduction to complexity theory, which classifies computational problems based on the resources required to solve them, such as running time. It discusses different models of computation including finite automata, pushdown automata, and Turing machines. Complexity theory aims to determine which problems are feasible to solve with limited resources and whether some problems are inherently harder than others. It introduces complexity classes like P (problems solvable in polynomial time) and NP (problems verifiable in polynomial time) and discusses the open question of whether P equals NP.
Introduction Theory of Computation tries to answer the following questions: What are the mathematical properties of computer hardware and software? What is a computation, and what is an algorithm? What are the limitations of computers? Can “everything” be computed?(As we proceed, the answer to this question is “no”.) Central Question in the Theory of Computation: What are the fundamental capabilities and limitations of computers Theory of computation can divided in to Complexity Theory, Computability Theory, and Automata Theory.
11/26/2022 Complexity Theory 2
Automata, Computability and Complexity Theory They are linked on question of “What are the fundamental capabilities and limitation of a computer”. Complexity Theory What makes some problem computationally hard and others easy? Computer problems comes in different varieties, some are easy and some are hard. For example: the sorting problem is an easy one. Scheduling problem is much hard than sorting.
11/26/2022 Complexity Theory 3
Cont.. Complexity theory classify problem according to computational resource required to Running time Storage space Parallelism Randomness Round of interaction, communication others. Attempts to answer what is computationally feasible with limited resource? Central Question in Complexity Theory: Classify problems according to their degree of “difficulty”. Give a rigorous proof that problems that seem to be “hard” are really “hard”. It works on the objective to classify problems as easy ones and hard ones;
11/26/2022 Complexity Theory 4
Complexity Class Complexity have two class P(Polynomial)and NP(Non polynomial). P:the set of problems decidable in polynomial time. NP: The set of problem undecidable in given polynomial time. Need to model of computation to define classes that capture important aspects of computation. Our model of computation :Turing machine
11/26/2022 Complexity Theory 5
Complexity Class NP Solving a problem is harder than verifying a given solution to the problem. Class NP captures the set of decision problem whose solutions are efficiently verifiable.
11/26/2022 Complexity Theory 6
Computability Theory
11/26/2022 Complexity Theory 7
Automata Theory Automata theory deals with definition and properties of mathematical models of computations. Study of abstract computing devices, or “machines” • Automaton = an abstract computing device These models play a role in several area of computer science. One model is called ,Finite automata ,is used in text processing, compiler design and hardware design. An other called context free grammar is used in programming language and artificial intelligence. Central Question in Automata Theory: Do these models have the same power, or can one model solve more problems than the other?
11/26/2022 Complexity Theory 8
Model of Computation
11/26/2022 Complexity Theory 9
Example of Computation 3 f ( x) x
11/26/2022 Complexity Theory 10
Cont..
11/26/2022 Complexity Theory 11
Cont.. Temporary memory
11/26/2022 Complexity Theory 12
Model of Automaton
11/26/2022 Complexity Theory 13
Automaton
11/26/2022 Complexity Theory 14
Kinds of Automata Different Kinds of Automata Automata are distinguished by the temporary memory Finite Automata: no temporary memory Pushdown Automata: stack Turing Machines: random access memory
power) 11/26/2022 Complexity Theory 16 Pushdown Automaton
Example: Compilers for Programming Languages (medium
computing power) 11/26/2022 Complexity Theory 17 Turing Machine
Examples: Any Algorithm (highest computing power)
11/26/2022 Complexity Theory 18
Power of Automata
11/26/2022 Complexity Theory 19
Goal of Complexity Theory obtaining an understanding of the relations between various computational phenomena (e.g., relating one fact regarding computational complexity to another). That is, it provides tools to measure the difficulty of computational problems both absolutely and in comparison with other problems. Generally: Absolute Questions: How much time is needed to perform the task? How much resources will be needed? Relative Questions: More difficult than other tasks? Are there “most difficult" tasks?
11/26/2022 Complexity Theory 20
Cont.. It is concerned with the study of the intrinsic complexity of computational tasks. That is, a typical complexity theoretic study refers to the computational resources required to solve a computational task (or a class of such tasks), rather than referring to a specific algorithm or an algorithmic schema. Its “final” goals include the determination of the complexity of well defined computational task.
11/26/2022 Complexity Theory 21
Hard Computational Problems Some computational problems are hard Despite a numerous attempts we do not know any efficient algorithms for these problems We are also far away from the proof that these problems are indeed hard to solve, in other words NP=P or NPP, this is a question
11/26/2022 Complexity Theory 22
Decision/Optimisation Problems A decision problem (DP) is a computational problem for which the intended output is either yes or no In an optimisation problem (OP) we rather try to maximise or minimise some value An OP can be turned into a DP if we add a parameter k, and then ask whether the optimal value in OP is at most or at least k Note that if a DP is hard, then its related its optimisation version must be hard too.
11/26/2022 Complexity Theory 23
Example Optimisation problem - Given graph G with integer weights on its edges. What is the weight of a minimum spanning tree (MST) in G? Decision problem – Given graph G with integer weights on its edges, and an integer k. Does G have a minimum spanning tree of weight at most k?
11/26/2022 Complexity Theory 24
Problems and Languages We say that the algorithm A accepts an input string x if A outputs yes on input x A decision problem can be viewed as a set L of (binary) strings – the strings that should be accepted by an algorithm that correctly solves the problem We often refer to L as a language We say that an algorithm A accepts a language L if A outputs yes for each x in L and outputs no otherwise
11/26/2022 Complexity Theory 25
Cont… One possibility: function from strings to strings F : ∑* ∑* Function problem: given x, compute f(x) Decision problem: f:∑* {yes, no} given x, accept or reject
11/26/2022 Complexity Theory 26
Cont.. Decision problems: f:∑* {yes, no} equivalent notion: language L ∑* L = set of “yes” instances Examples: set of strings encoding satisfiable formulas set of strings that encode pairs (n,k) for which n has factor <k decision problem associated with L: Given x, is x in L?
Jean Gallier, Jocelyn Quaintance - Linear Algebra and Optimization With Applications To Machine Learning - Volume II - Fundamentals of Optimization Theory With Applications To Machine Learning. 2-Wor