2 views

Uploaded by Mian Saeed Rafique

iyeriyqty8

- Lecture 24
- PPT Jurnal ROP
- 0804.3401v1
- Mcs-21
- Discont.ps
- Balls
- 1110.3425
- 1,2-Analysis and Design of Algorithm
- Policies
- Bin Sort
- Intriduction to Algorithms (Lec1)
- 1007.4063
- Practice Quiz
- h p Refinement
- Online Scheduling & Project Scheduling
- Hardness Results for Intersection Non-Emptiness
- Coherent-States Anna 2012 05
- Complete Homework Will Contain… …a Complete Heading.
- Cal Sample Final Test
- f 0422052058

You are on page 1of 33

Theory of Computation

1 / 33

Incompressible strings

Minimal Length Descriptions

Descriptive Complexity

Theory of Computation

2 / 33

Incompressible strings

K (x) |x| b

1

as long as the string itself.

Theory of Computation

3 / 33

Incompressible strings

Many properties of random strings are also true for incompressible strings.

For example

Theory of Computation

4 / 33

Incompressible strings

Theorem

For every n there is a incompressible string of length n.

Theorem

The number of b-incompressible strings of length b is at least

2n 2nb+1 + 1

The proof is simple counting!

Theory of Computation

5 / 33

Incompressible strings

Theory of Computation

6 / 33

Incompressible strings

Theorem

A computable property that holds for almost all strings also holds for

incompressible strings.

Let us formalize this statement. A property p is a mapping from set of all

strings to {TRUE , FALSE }. Thus

p : {0, 1} {true, false}

Theory of Computation

7 / 33

Incompressible strings

|{x {0, 1}n : p(x) = false }|

2n

approaches 0 as n approaches infinity.

Theory of Computation

8 / 33

Incompressible strings

Theorem

Let p be a computable property. If p holds for almost all strings the for

any b > 0, the property p is false on only finitely many strings that are

incompressible by b.

Let M be the following algorithm:

1

Find the ith string s such that p(s) = False. Considering the strings

lexicographically.

Output s.

Theory of Computation

9 / 33

Incompressible strings

property p. Let x be a string that does not have property p. Let ix be the

index of x in the list of all strings that do not have property p ordered

lexicographically. Then hM, ix i is a description of x. The length of this

description is

|ix | + c

Theory of Computation

10 / 33

Incompressible strings

Now we count. For any give b > 0 select n such that at most

1

2b+c

fraction of the strings of length n do not have property p. Note that

since p holds for almost all strings, this is true for all sufficiently large n.

Theory of Computation

11 / 33

Incompressible strings

Then we have

ix

2n

2b+c

Therefore, |ix | n b c. Hence the length of hMiix is at most

n b c + c = n b. Thus

K (x) n b

which means x is b-compressible. Hence, only finitely many strings that

fail p can be incompressible by b.

Theory of Computation

12 / 33

Incompressible strings

Is it possible to compress a string that is already compressed?

The answer to this question seems to be no! Let us prove something that

is quite close to this.

Theory of Computation

13 / 33

Incompressible strings

Theorem

There is a constant b such that for every string x the minimal description

d(x) is incompressible by b.

Roughly the idea of the proof is that

If d(x) were compressible we could use the compressed version of d(x) to

obtain a smaller description of x than d(x).

Theory of Computation

14 / 33

Incompressible strings

Consider the following TM M:

1

On input hR, y i

Let b = 2|hMi| + 3. We now show that b satisfies the statement of the

theorem. Suppose on the contrary that d(x) is b compressible for some

string. The

|d(d(x))| |d(x)| b.

But what happens if we run M on d(d(x)). Note that it outputs

x

Theory of Computation

15 / 33

Incompressible strings

2|hMi| + 2 + |d(d(x))| = (b 1) + (|d(x) b) = |d(x)| 1

This contradicts the fact that d(x) is the minimal description of x.

Theory of Computation

16 / 33

Complexity Theory

Two of the most important resources are going to be

1

Time

Space

Theory of Computation

17 / 33

Complexity Theory

Let

A = {0k 1k : k 0}

Consider the following TM that accepts A

1

Scan across the tape to check if all 0s are before 1s if not reject.

Goto Step 2

We want to analyze this algorithm and talk about how many steps (time)

does it take.

Theory of Computation

18 / 33

Complexity Theory

A step of the Turing machine is assumed to take unit time. Lets start with

a definition. Let M be a TM that halts on all inputs.

The running time or the time complexity of M is a function

f :NN

such that f (n) is the maximum number of steps that M uses on any input

of length n.

1

M is f (n)-time TM.

Theory of Computation

19 / 33

Big O notation

The exact running time t(n) is going to be very complicated expression. In

some cases we will not be able to compute it.

But what we are interested in is getting a rough idea of how t(n) grows for

large inputs. So we will use asymptotic notation.

Theory of Computation

20 / 33

Big O notation

f (n) = 5n3 + 13n2 6n + 48

This function has four terms. He most important term is

5n3

which will be the bulk if n is large.

Theory of Computation

21 / 33

Big O notation

In fact

5n3

the constant 5 does not contribute to the growth of this function. Thus

we will say that f (n) is order n3 . We will denote this by

f (n) = O(n3 ).

Let us state this precisely:

Theory of Computation

22 / 33

Big O notation

f (n) = O(g (n))

if there are positive integers c, n0 such that for all n n0

f (n) cg (n)

When f (n) = O(g (n)) we say that g is an asymptotic upper-bound on f .

Sometime we will just say that g is an upper bound on f .

Theory of Computation

23 / 33

Big O notation

Intuitively if f (n) = O(g (n)) you can think that f is less than or equal to

some multiple of g . For most functions the highest order term is is an

obvious candidate to be g .

Theory of Computation

24 / 33

Big O notation

Let

f (n) = 13n4 + 7n2 + 13

Then

f (n) = O(n4 )

Furthermore,

f (n) = O(n5 )

But it is not true that

f (n) = O(n3 )

Theory of Computation

25 / 33

Big O notation

When we say that f (n) = O(g (n)) we are only claiming that g (n) is an

asymptotic upper bound on f . It may not be the best one. Lets look at

other examples.

Theory of Computation

26 / 33

Big O notation

Suppose that

f (n) = logb n

where b is a constant. Note that

logb n =

log2 n

.

log2 b

f (n) = O(log n)

Let

f2 (n) = 5n log n + 200n log log n + 17

then

f2 (n) = O(n log n)

Theory of Computation

27 / 33

Big O notation

We can use O notation in exponents also. Thus

2O(n)

stands for a function that is upper bounded by

2cn

for some constant c. We have to be careful though

For example

2O(log n)

Well remember that O(log n) has a constant attached to it. Thus this is

upper bounded by

2c log n

by some constant c. Now, we get

2c log n = (2log n )c = nc

Dr. Sarmad Abbasi ()

Theory of Computation

28 / 33

Big O notation

c

the form 2n are called exponential bounds.

Theory of Computation

29 / 33

small o notation

want to say

f grows strictly slower that g .

Let f , g : N R + . We say that f (n) = o(g (n)) if

f (n)

=0

n g (n)

lim

Theory of Computation

30 / 33

small o notation

n = o(n)

n log n = o(n2 )

n2 = o(n3 ).

Theory of Computation

31 / 33

small o notation

A = {0k 1k : k 0}

1

Scan the tape and make sure all the 0s are before 1s.

Scan the tape and if the total number of 0s and 1s is odd reject.

Scan and cross of every other 0 and do the same with every other 1.

Theory of Computation

32 / 33

small o notation

O(n log n)

Thus the time taken by this TM is o(n2 ). Hence, this reflects a real

algorithmic improvement.

Theory of Computation

33 / 33

- Lecture 24Uploaded byEric Parker
- PPT Jurnal ROPUploaded byNurul Azizah
- 0804.3401v1Uploaded byAnonymous Jx2rEZ
- Mcs-21Uploaded byfirstname234
- Discont.psUploaded byKevin Mondragon
- BallsUploaded bykishoje
- 1110.3425Uploaded byHung Pham
- 1,2-Analysis and Design of AlgorithmUploaded bySwati Sharma
- PoliciesUploaded byEdwin Okoampa Boadu
- Bin SortUploaded byMohammed Jeelan
- Intriduction to Algorithms (Lec1)Uploaded byCarlos Bcar
- 1007.4063Uploaded byVigneshwar Natesan
- Practice QuizUploaded byRachit Shah
- h p RefinementUploaded bylakshmigsr6610
- Online Scheduling & Project SchedulingUploaded byzamanian
- Hardness Results for Intersection Non-EmptinessUploaded bypepe gonzales
- Coherent-States Anna 2012 05Uploaded byAydınDeğer
- Complete Homework Will Contain… …a Complete Heading.Uploaded byapi-26199179
- Cal Sample Final TestUploaded bylarita98
- f 0422052058Uploaded byInternational Journal of computational Engineering research (IJCER)
- Complete Homework Will Contain… …a Complete Heading.Uploaded byapi-26199179
- DocumentUploaded byVijayalaxmi Biradar
- Sequences of pyramidal numbersUploaded byAnonymous 0U9j6BLllB
- article 4 - earth scienceUploaded byapi-285720668
- 1-s2.0-S1366554509000313-main.pdfUploaded byMuhammad Imran Khan
- CSN 104 COMPUTER PROGRAMMING Practical Assignment 5 & 6.docxUploaded byAbhinav Bansal
- Activity#2(Baflor)Uploaded byJoevelleBaflor
- M53 E4 ReviewerUploaded byAllen Lois Lanuza
- ROCK clustering exampleUploaded byshehzad791
- Computer Networking a Top-Down ApproachUploaded byAnonymous ZntoXci

- Lecture 44Uploaded byMian Saeed Rafique
- Lecture 32Uploaded byMian Saeed Rafique
- Lecture 28Uploaded byMian Saeed Rafique
- Lecture 10Uploaded byMian Saeed Rafique
- Fall 2012_CS614_1_MC110400801Uploaded byMian Saeed Rafique
- Mrk_Spring 2013_CS619_516_1736_S13025FFE9Uploaded byMian Saeed Rafique
- مطالعہ پاکستان پوزیشن وننگ سوالات جماعت نہمUploaded byMian Saeed Rafique
- Spring 2012_CS614_1Uploaded byMian Saeed Rafique
- Lecture 16Uploaded byMian Saeed Rafique
- Lecture 11Multi Version Timestamp Ordering ProtocolUploaded bySun Java
- Lecture 12Uploaded byMian Saeed Rafique
- Lecture 15Uploaded byMian Saeed Rafique
- Lecture 43Uploaded byMian Saeed Rafique
- Lecture 19Uploaded byMian Saeed Rafique
- Lecture 42Uploaded byMian Saeed Rafique
- LecMulti Version Timestamp Ordering Protocolture 05Uploaded bySun Java
- Lecture 02Uploaded byMian Saeed Rafique
- Fall 2012_CS501_1_MC110400801Uploaded byMian Saeed Rafique
- Lecture 45Uploaded byMian Saeed Rafique
- Turing Slides 1Uploaded byMian Saeed Rafique
- Fall 2012_CS501_1Uploaded byMian Saeed Rafique
- CS 701Uploaded byMian Saeed Rafique
- CS 701 Research Paper (MS150200090)Uploaded byMian Saeed Rafique
- Assignment No. 03 MS150200090Uploaded byMian Saeed Rafique
- Fall 2012_CS614_1Uploaded byMian Saeed Rafique
- Lecture 01Uploaded byMian Saeed Rafique
- ion Timestamp Ordering Protocol03Uploaded bySun Java
- Avast keyUploaded byMian Saeed Rafique
- satyanarayanan01pervasiveUploaded byDennis Ebenezer Dhanaraj

- Essay StructureUploaded byBasia Puszkar
- Nursing TheoriesUploaded byJude Bello-Alvear
- aasl learning standards 2007Uploaded byapi-290574902
- Parenting over GeneticsUploaded bySkillnja
- DissertationUploaded byawwo342010
- Convergent Thinking vs Divergent by Chuck ClaytonUploaded bygmelinda
- Explorations SyllabusUploaded byESGaringo
- Document 1Uploaded byJoseph Atienza
- definition_of_motivation.pdfUploaded byDavid Olorato Ngwako
- Interview InstructionsUploaded bymbramall
- 3. Management-Barriers of on the Job Learning-Lilla HortovanyiUploaded byBESTJournals
- 5-Attitude & BehaviourUploaded byAnand Dubey
- SpeakingUploaded byEbhy Cibie 'gates'
- SAT Essay Prompt ArchetypesUploaded byMadeleine Beliveau
- Ian HackingUploaded bySashaNicholas
- ERIC IdentifierUploaded byChristopher Oca Dumlao
- Facilitating LearningUploaded byjenniferespanol
- Aristotle de Interpretatione, Fate or Free WillUploaded by257185
- COESAT HandbookUploaded bysee1tear
- Article for workshop 1_Qualitative Bullying Article.pdfUploaded byCT Barokah
- self-reflectionUploaded byapi-235251890
- Ai Challenge ProblemsUploaded byharezmi
- woodcock-johnson iii categoriesUploaded byapi-253754657
- Pre-int Unit 1bUploaded bygallipatero
- dschool bootlegUploaded byapi-253592912
- L5 Deconstruction of Meaning in Spring and Fall.doc 3Uploaded bybhuvaneswar Chilukuri
- A Study of PBL Learning Model for Develop Student’s Communication and Working skill in ChonburiUploaded byOon Koo
- syllabus_mapeh_1_2.docUploaded byRodjan Moscoso
- DishaUploaded bypmcmbharat264
- animal farm final projects honorsUploaded byapi-222057179