This article was downloaded by: [Stanford University Libraries]

On: 27 September 2012, At: 11:09

Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK

International Journal of General Systems

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/ggen20

COMPLEXITY OF DESCRIPTIONS OF SYSTEMS: A

FOUNDATIONAL STUDY
a
LARS LÖFGREN
a
University of Lund, Lund, Sweden

Version of record first published: 06 Apr 2007.

To cite this article: LARS LÖFGREN (1977): COMPLEXITY OF DESCRIPTIONS OF SYSTEMS: A FOUNDATIONAL STUDY, International
Journal of General Systems, 3:4, 197-214

To link to this article: http://dx.doi.org/10.1080/03081077708934766

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to
anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation that the contents
will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should
be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims,
proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in
connection with or arising out of the use of this material.
 l nt. J. General Systems © Gordon and Breach Science Publishers LId.
1977, YoU, pp. 197-214 Printed in Great Britain

COMPLEXITY OF DESCRIPTIONS OF SYSTEMS:

A FOUNDATIONAL STUDY
LARS LllFGREN
University oj Lund, Lund, Sweden
(Received January 25, /976)

By definition (Webster) a system is deemed complex and complicated when the interconnection or arrangement of
its parts is difficult to trace or understand. Tracing and understanding are associated with cerebral learning
processes. By the aid of a previously developed model for such processes we can study the difficulties that define the
complexity impressions and thus provide ,a foundational, rather than methodological, study of complexity. The
nature of complexity as difficulty of perception suggests that a complexity classification should. in general, be more
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012

difficult than a simplicity classification. This thesis is discussed and applied with a mathematical notion of degree of
difficulty.
INDEX TERMS Learning processes, description processes, foundational study, perception, complexity
classification, simplicity classification.

INTRODUCTION Various well known complexity problems will be

Complexity is one of those concepts which we know
well by intuition but find difficult to explain. As we
are about to see the concept goes well beyond that of
simple numerosity. Eventually complexity may be
considered the opposite of simplicity. However,
instead of studying complexity in terms of simplicity
we want to provide an independent, foundational
study of complexity.
Let us look for a moment on the use of
complexities. In some contexts we find it desirable to
increase the complexity of a system, thereby hoping
to call forth emergent properties. In other contexts
we may want to reduce the complexity of a system,
which otherwise would be· too difficult to under-
stand, describe, or manage,
Indeed, various complexity measures have been
suggested for areas like thermodynamics, automata
theory, information theory, computer science, etc.
With a few exceptions they may appear unrelated.
This apparent diversity of complexities may be
reduced by a foundational study that looks into the
nature of complexity impressions as cognition
phenomena. This is how we are going to approach
complexity, and the view immediately suggests that
complexity be associated with descriptions, rather
than being thought of as an intrinsic properly of
objects, A classification of describing and interpret-
ing processes will generate a classification of
complexities, in which the familiar complexity
measures will have their places as well as other such
measures.
8
197
discussed in the light of the developed complexity
view. Concerning the philosophical problem
whether the real world is complex or simple, we will
explain objects as existential perceptions, resulting
by an extreme form of complexity reduction,
2 ON THE ORIGIN OF
COMPLEXITY SENSATIONS
Intuitively, a system is deemed complex and
complicated when the interconnection or arrange-
ment of its parts is difficult to trace or understand
(cf, Webster i"). Consider for example the following
string-shaped objects: 000000000011111111111, de-
noted A, and 10101100101011010001, denoted B.
When asked which string is the more complex,
people usually point at B, although A in fact is
slightly longer than B. Numerosity of parts may not
be a decisive measure of system complexity, Instead,
it may be the easiness with which we trace the
arrangement of the parts in A that is responsible for
our impression that A is less complex than B. We
tend to see A as "(ten) O's to the left of (eleven) I's." It
is more difficult to find a similar property, or rule,
which permits a correspondingly short description
of B.
Again, when a system is conceived as a de-
scription (and not as an object as in the above
example), it is deemed complex or complicated
when its meaning, i.e., that which is described, is
difficult to understand, Consider for example the
 L. LOFGREN
following two symbol-strings as descriptions (and
not as objects in the above sense): "the sum of the
numbers ten and onc-hundrcd-and-twentytwo,"
denoted C, and "the smallest set which contains
itself as a member," denoted D. When asked about
the complexity of the two descriptions, most people
(with a sufficient back-ground) would point at D as
the more complex although C is slightly longer than
D. To be sure it may be the relative easiness by which
the meaning of the description C is comprehended
that accounts for the impression that C is less
complex than D.
The examples suggest t hat the concept of com-
plexity basically refers to cognition processes. We
want to analyse it in terms of those processes which
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012
generate our inner (cerebral) descriptions and those
which arc responsible for our awareness of mean-
ings of descriptions (of systems intended as de-
scriptions, like the printed pages of this paper,
cerebral description, etc.),
As we have seen, the complexity of a system
depends upon whether the system is considered an
object or a description. Yet it may be argued that
objects arc in fact descriptions (that objects are
existential perceptions occurring in the cerebral
description of the data-now that stimulates our
receptors; sec Section 7), or that descriptions are in
fact objects (typed sentences, DNA-molecules, etc).
How could we then make a distinction between
object and description when dealing with their
complexities?
We want to avoid the problem by talking about
description process versus interpretation process
instead of object versus description. After all, the
difficulties of tracing or understanding a string
depend on whether we describe it (as an object) or
interpret it (as a description). Although there is an
interdependence between description- and
interpretation-extractions, we shall be able to
isolate the corresponding difficulties, cf. Sections 4
and 5.
Figure I illustrates the inverse character of
description processes (learning, inheritance) and
interpretation processes.
lnheritance~
~learning ~~
systemY- ~ description
'interpretation/
FIGURE 1 Description and interpretation processes.
We first recognize two ways, the two ways, in
which descriptions arc generated. Either a de-
scription of a system is extracted from the system by
a learning process (compare an acquired character
in a biological context), or it is generated by an
hereditary-like mechanism (compare an inherited
character and the biological dichotomy of charac-
ters into acquired and inherited characters).
Next, a description would not be a description if
not associated with an interpretation. By interpret-
ing the description we get that which is described.
Notice that the "system" indicated in Figure I
may be considered an object as well as a description.
In the latter case the description may for example be
described in a higher-level language (eventually in
the same language as that used for the primary
description, provided that it is rich enough to permit
self-reference). Again, as we have argued, the
generated description may be looked at as an object.
However, its character of description is unmistak-
able as soon as it is associated with an in-
terpretation.
The learning or description mechanism indicated
in Figure I is supposed to be of a general kind. It
covers verbal descriptions in natural or formal
languages as well as descriptions in cerebral
languages. It will be explained further in Section 3
(where we shall see that learning actually involves
interpreta tion).
Let us now return to the classification of
complexities with reference to Figure 1. For the
moment we consider only description by learning
but not by inheritance, because the latter is not
directly associated with cognition phenomena and
thus not with complexity sensations. This does not
prevent us, later on, from considering complexities
of genetic descriptions (as objects and as de-
scriptions ).
What remains are two mutually inverse pro-
cesses, a learning or description process and an
interpretation process. Each of them is associated
with cognition phenomena, and we are thus led to
two corresponding kinds of complexity.
Associated with the learning process is the
complexity of describing, the d-complexity. It is
measured by the difficulty associated with extract-
ing the description of a system S, the descri ption
responsible for how we "see" S and feel its
complexity. The complexities discussed above for
the strings A and B are of this type. After having
discussed the learning process in Section 3 we will
return to a further classification of d-complexities.
Associated with the interpretation process is the
 COMPLEXITY OF DESCRIPTIONS OF SYSTEMS
complexity of interpreting, the i-complexity. It is
measured by the difficulty associated with extract-
ing the interpretation (meaning) of a description.
The complexities discussed above for the strings C
and D are of this type. In Section 5, i-complexities
will be further classified (depending on the language
and the nature of the II priori knowledge with
respect to which the description is considered a
description).
3 THE LEARNING OR
DESCRIPTION PROCESS
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012
We think of the basic function of our brains as a
learning or description process, which produces
short and communicable descriptions of the enor-
mous, and in itself unmanageably complex, data-
flow that is our direct experience of an outer world
(surrounding). We will thus think of the real world,
that we "see" in terms of existing objects, as a
cerebral description of the data-flow that stimulates
our receptors, the inputs to our brains.
We have in earlier worksP: 13 developed ideas
about learning as a description process and for-
mulated the following learning-hypothesis:
An object A can lellrn from a surrounding S to the extent that it
can extract order (regularities) from S so as to produce a
description of S relative to A (A shall be able to make inferences
from the description). The more regularities that have been
found, the more genuine is the learning, and the shorter the
description (which utilizes the regularities) compared with a
representative part of S itself. The learning ability (capacity) of A
reflects both the kinds of surroundings that A can learn and the
genuineness of the learnings. If S is self-describing relative to A,
such that A can obtain the properties (description) of S without
any proper order-extraction work, A is said not to have learned S,
but to have obtained the description of S genetically (by
inheritance: or, in a computer context, to have been programmed
with the description), In this case, A's description of S is not short
when compared with a representative part of S, but equal to that
part.
Involved is the idea that the more regularities
have been found, the more can they be utilized (be
referred to) in the description to make it shorter
(than a lengthy listing of uncorrelated facts).
Furthermore, the predictive power of the de-
scription will increase with the number of re-
gularities found. Hence, the shorter the description
can be made, the more communicable will it be, and
the more genuine will the learning be. Then more
reliable predictions can be made on the basis of the
learned description, and more safe inferences can be
made of how to behave in the surrounding.
199
ProJr... for 1l. .. cr1ptiOtl of llle dau_
tile generation <_-------, nw {tile aurroundin& or
FIG URE 2 Structure oftearning (description) process.
Let us by the diagram in Figure 2 indicate a
structure of a learning process. It is sufficiently
abstract to be thought of as the structure of a
description process, or of a theory-formation pro-
cess, or of a cerebral process.
The indicated program for the generation of
hypotheses of the surrounding does not in general
consist of a collection of ready-made hypotheses.
Rather, it is a set of instructions of how to form new
hypotheses from the knowledge of how the old was
refuted. Again, the process of simplifying the actual
theory may involve extensions, i.e., the formation of
new hypotheses.
A universally quantified hypothesis cannot in
general be verified, but it can be falsified (refuted) or,
again, supported by relevant experiments. The
question of relevance (cf. Definition I below) has
been sufficiently developed!": t6 to suggest a pro-
gram for the generation of relevant questions. The
entries to this program are the actual hypothesis and
the actually confirmed theory.
Decision as to rejection or confirmation of an
hypothesis H is made according to a confirmation
logic. If H has obtained a sufficient amount of
support, it will be confirmed and added as a ncw
axiom to the previously confirmed theory (de-
scription).
The actual theory is steadily being simplified by
reformulations, involving naming processes and
simplifying generalizations. These latter are actually
part of (directs) the hypothesis generation process.
Involved here is the metahypothesis that if two
theories both describe the same set of experimental
facts but predict differently the outcome of further
experiments, then the simpler is the most likely to be
correct (simplicity is here measured by the shortness
 200
of the proper axioms, provided that the theories
have thc same logical basis).
A strict definition of the confirmation relations
can be given!": 16 in terms of the deduction relation
A ~.,.11 ("thc formula B can be derived from the
formula A in the theory T" or equivalently "B is a
theorem in the theory T(A), i.e., Taugmented with
A as a new axiom"). Equivalently, the definition can
be given in terms of complexity values I W which I,
arc defined in Section 5.1. Such equivalent for-
mulations are given within parenthesis in the
following definition.
DEFINITION I Let B be a description of an
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012
observation and H a description of an hypothesis
with respect to an a priori knowledge T, such that B,
as well as H, is consistent with respect to T (i.e., I
I)
> 111 I and I > IH and such that H is genuinely
hypothetical, i.c., ,jt-.,.H (I HI> 0). Then:
i) 11 verifies H in T:
B~.,.H (I BI ~I HI),
ii) B'supports H in T:
H~·rBand,y-.,.B (I HI~I BI>O),
iii) Bfalsifies H in T:
T(H,B)inconsistent i1HI~I,BI),
iv) B is neutral for H in T:
~1'B (I B I=0),
v) B is compatible with H in T:
H ,jt-'rB and B,jt-.,.H and T(H, B)
consistent
(IHI~IBI and I >IHI v IBI),
1) B COilfirms H in T if (i) or (ii),
2) B is relevant for H in T if (I) or (iii),
3) B is irreleoant for H in T if (iv) or (v).
III Section 4.2 we will need the following theorem
which is proved in Ref. 16.
THEOREM The following statements are equioa-
lent:
Bfalsifies H ill T,
B verifies I H in T,
I B supports H in T
L. LOFGREN
Finally we want to make the comment that some
of the subprocesses of Figure 2 may themselves be
regarded learning processes, again with a structure
according to Figure 2. Consider for example the
generation of relevant questions. It depends on a
fixed basis for the actual theory. How is this basis, a
logical basis for example, determined? By a learning
process which in turn requires a still more basic
basis, etc. Finally we may come down to a basis that
is genetically determined. In general it is thus
reasonable to consider hierarchies of learning
structures of the type of Figure 2.
4 COMPLEXITIES OF DESCRIBING
(d-COMPLEXITIES)
In the previous section we looked at a theory as a
description and talked about simplicity in terms of
the shortness of the proper axioms of the theory.
Surely a measure of simplicity of a system is the
shortness of its shortest description. Hence, looking
at complexity as the opposite of simplicity we are at
once led to identify complexity of a system with the
length of its shortest description.
In this section we do want to develop d-
complexities of a system in terms of the length of its
shortest description. Then we do not want, however,
to build upon an already established concept of
simplicity. Instead, we want to supply independent
- motivations of d-complexities as difficulties of
tracing or understanding a system by learning (cf.
the characterization of d-complexity in Section 2).
Such independent motivations are required beca-
use it is not from the outset clear that, for example, a
. string lacking expressible rules, i.e., a string with a
maximal shortest description, is the one which is the
most difficult to learn. In providing evidence, we will
make a distinction between two cases. I n the first we
will consider learning processes where most of the
-learning work takes place on a subconscious level.
Here we will experience difficulties as difficulties of
being aware of the mental description or "picture"
that we "see." In the second case we will consider
processes where greater parts of the learning process
(cf. Figure 2) take place on a conscious level and are
responsible for impressions of difficulty.
4.1 Conscious Difficulties in
Subconscious Learning
Let us reconsider the two strings A and B of Section
2:0ur instinctive judgement of A as easy (simple)
 COMPLEXITY OF DESCRIPTIONS OF SYSTEMS
and B as difficult (complex) takes place in our
subconscious minds. The strings are of such lengths
that they are not recognized in a single act. The
visual center of attention sweeps over the strings
and covers two or a few more figures at a time. When
seeing two O's close to each other in string A, the
subconscious hypothesis is formed that also the
neighboring figures are O's, The hypothesis is
supported, strengthened, etc., and we tend to "see"
half of the string as only O's and, in a similar way, the
other half as only I's. Since this part of the learning
process is carried out subconsciously, we may even
get the impression that we are recognizing A in a
single act. That this is not the case, however, may be
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012
argued from the way we are aware of the other
string, B. Here simple hypotheses of the above kind
will be rejected. Most likely the hypotheses that can
at all be generated in this type of subconscious
"seeing" are very limited. We may not be able to
express any rules in the internal language, in terms
of which we can describe B sufficiently short to be
aware of it the way we are aware of A with its very
short internal description. When letting the eye
sweep over B we can only become aware of it as "a
string of'O's and I's." The shortening of the detailed
description of B, which is necessary to have it
contained in the limited awareness space, makes us
miss a lot of facts of B (which are sequentially
generated-and forgotten-when letting the eye
sweep over B). Compare the fact that it is much
easier to remember A than B in such a detail that we
can reconstruct the strings from their short-term
memorizations.
It is my impression that it is not more difficult to
classify B as "difficult" than it is to classify A as
"easy," i.e., when looking at the strings for just a
short while. This probably depends upon the fact
that the learning processes at play are entirely
subconscious and, furthermore, so fast that we are
not even aware of the learning times. Should there
be any difference in the difficulties of learning, we
will simply not be aware of them. Yet we do classify
B as "difficult" and A as "easy." These different
impressions may be due to the difference in the way
we are aware of the strings. It is easy to be aware of A
in such a detail that we can reconstruct A from the
short-term memory we immediately develop. Our
awareness of B is different. When we find this, which
we do easily, we may consciously try to find rules for
B in the hope that they will permit a sufficiently
short description. That search process may generate
a direct impression of difficulty. But if we do not
have any real need for remembering B, we let the
201
type of awareness we have for B predict for us that it
will be difficult for us to find rules. We may hence
refrain from trying and simply classify B as difficult,
or complex, the easy way, i.e., by an easy prediction
of a forthcoming difficulty.
Our discussions so far suggest a dichotomy of
strings, z, into complex (difficult) and simple (easy)
strings, depending upon whether their learned
descriptions are longer or shorter than a given
length, IV. The learned description depends both on
the descriptive language, L, and the actual learning
process. For languages with limited expressibility, it
is reasonable to assume that the learning process is
adapted to L such that the learned description
coincides with the shortest description of z that is
expressible in L, denoted s(z, L). With the notation
[s(z, L)] for the length of the shortest description
s(z, L), we are thus led to the following complexity
and simplicity predicates.
DEFINITION 2 er(z, IV), the d-complexit)' predicate
"the string z is complex with respect to the number
IV," is defined by:
er(z, IV): [s(z, L)] > IV.
6(z, IV), the d-simplicity predicate "the string z is
simple with respect to the number IV," is defined by:
6(z, w): ,er(z, IV) (: [s(z, L)] ~ IV).
The extensions of these predicates define com-
plexity and simplicity classes as follows.
DEFINITION 3 C w , the d-complexit y class IVitIJ
respect to IV, is defined by:
C w = {z :[(z, IV)} (= {z: [s(z, L)] > IV)).
Sw' the d-simplicit y class IVitIJ respect to IV, IS
defined by:
Sw = {z: 6(z, IV)} (= {z: [s(z, L)] ~ IV}).
Our impression that it is as easy to classify a string
as easy (simple) as it is to classify a string as difficult
(complex) is valid only within this case of subcon-
scious learning. It may be compared with an alleged
saying of Salvador Dali. Asked ifhe found it difficult
to paint, Dali admitted: "No, it is either easy or it is
impossible l"
Later on, when dealing with more expressible
languages and more conscious learning processes,
we will find that it is in fact more difficult to classify a
 2(J2 L. LOFGREN
difficult string as difficult than it is to classify an easy
string as casy (cf. Sections 4.2 and 4.5).
4.2 CllllScio/ls DijJic/llt ies ;11
Partly CO/lSC;O/lS Learning
Let us recall from Section 3 that learning essentially
is a search process. It searches for rules hidden in a
string (an observation report) by suggesting hy-
potheses for test and eventual confirmation into
such rules. A rule is naturally considered the more
powerful the more observable facts it concerns, and
a measure of the power of the rules that have been
found is the shortening of the description (of the
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012
observation report) that can be obtained by
utilizing the rules,
The domain of search may be lixed and small (cf.
the fixed strings A and B of the previous discussion).
Again, it may be unlimited, as in the case where Z
= Z(II) is considered an observation report for the
lirst II observations of a behavior that we want to
describe, Hypotheses then naturally concern Z for
all II, and the problem arises how to decide on such
hypotheses with linitc tests.
Consider an hypothesis concerning an inlinite
domain. For example, a universally quantilied
hypothesis that can be particularized to an infinity
of distinct test-propositions. Should the hypothesis
be true, it would have a very large (inlinite)
shortening cffcct on the description of the (steadily
increasing] observation report. Of course, such an
hypothesis cannot be verified by a finite test
sequence, It can be supported, however, and in cases
of strong support it may be considered confirmed.
However, if such a universally quantified hypothesis
is falsilicd, its negation, i.e., an existentially quan-
tified hypothesis, will be verified (cf. Theorem 1). An
existentially quantified hypothesis, however, will
have almost no shortening effect on the description.
Rather, it has the character of a single observation,
or step, in the test sequence z.
If an hypothesis H: vx Rx, where x indicates
(varies over) thc steps of z, is obtaining increasing
support until a step X o is reached where H is falsified,
i.e., I Rx o, then we may look at R as a rule that
holds with certain exceptions. This means that we
arc in fact eonsidcring a new hypothesis
H, :l/x(xi'xo)=>Rx, which is slightly longer and
slightly Icss powcrfulthan H. Should there be many
exceptions also from this new hypothesis, certainly a
much longer and weaker hypothesis can be formed.
However, thc shortening effect of this hypothesis
may be so small that, instead, we may start to look
for an entirely different hypothesis or, eventually, for
an hypothesis concerning a rule for the exceptions.
Let us reconsider, for a moment, the string A of
Section 2. Here the rule that a 0 has a 0 as neighbor
and a I a 1 as neighbor becomes increasingly
supported both in the left part of the string and in
the right part. In the middle, however, there is an
exception. Nevertheless, the rule with the exception
permits a shortening of the description of the string
such that we can easily memorize it. We may even
tend to "see" the string as two disconnected strings,
each satisfying the rule without exception. However,
when looking at the whole string as a test sequence
to be continued to the right, the exception from the
rule makes us feel uncertain of how to predict the
next figure. Should it be a I or should we expect
another exception') If the exceptions occur re-
gularly, it is about time to expect another exception!
Consider instead the following sequence, E:
101010101010101010101. Here the rule "the suc-
cessor of a 1(0) is 0(1)" is valid throughout without
exception (i.e., wherever there is a successor). When
going through the sequence as a test sequence, the
corresponding hypothesis is getting an increasing
number of supports and we feel safer in predicting E
than in predicting A.
Universally quantified hypotheses are of special
interest for learning processes because of the strong
shortening effects they have on the description,
provided that they are confirmed. If such an
hypothesis H is obtaining a steadily increasing
support when the test sequence Z is increased (when
increasing n in Z(ll)), then the redundancy of Z with
respect to H is steadily increasing. Compare
Definition I, saying that a certain step B in the test
sequence z supports H with respect to the a priori
knowledge T if and only if B can be deduced from H
in T, i.e., when B is redundant with respect to H.
Hence, we may measure the support that z gives to
H by the shortening (removal of redundancy) that
can be obtained by describing z in terms of H.
It is when an hypothesis is getting a continuously
increasing support, i.e., support with no in-
termediary rejection, that it may be considered for
confirmation. In this case of steadily increasing
redundancy of the sequence z (with respect to the
hypothesis) the ratio [d(z, L]/[z], where d(z, L) is
the actual description of z in L which utilizes the
hypothesis, is steadily decreasing with increasing
length of z. In fact, the behavior of this ratio may
indicate how close we are to a reasonable termi-
nation of the learning process.
Let us in the actual case of partly conscious
 COMPLEXITY OF DESCRIPTIONS OF SYSTEMS
learning associate feelings of increasing difficulty of
learning with an increasing number of test steps that
have to be performed before the learning process is
terminated. At each step we have to decide if the
observation (the new element of the test sequence) is
irrelevant for the hypothesis H, falsifies H, or
supports H (cf. Definition I).
Let z , (n) and z 2 (u) be two test sequences, each of
a reasonably large test length, n, such that the ratio
[s(z, L)]/[z] is large (close to 1) for z, (Il) and small
(close to 0) for z2(n). Which is the most difficult to
learn?
We can conclude that z, contains fewer rules to be
found than Z2 because of its longer shortest
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012
description. In fact, since the ratio for z\ is close to I,
we know that the test sequence z, (Il) already
produced can contain but little support for eventual
hypotheses. Hence, the learning process for z ,
cannot be terminated at this point (n), but will have
to be continued with increasing difficulty oflearning
as a result.
The learning process for Z2' on the other hand,
may be terminated already at this point Il,provided
that so many of the hidden rules (we know that they
are there) have been found, that also the ratio
[d(z, L)]/[z] is close to O. Recall at this point that we
assume that Il is reasonably large, such that we may
assume that {s(z,L)] and [d(z,L)] are about the
same.
Hence, we conclude that z\ with its larger
[s(z, L)]/[z]-ratio is more difficult to learn than Z2.
Indeed, it is the behavior of the ratio [s(z,L)]/[z]
for increasing [z] that determines whether a string z
shall be classified as difficult or easy. Let <p(z) be a
delimiting ratio-behavior such that sequences with
ratios above <p(z) are classified as difficult and
sequences with ratios below <p(z) are classified as
easy. Then, without having to go into a specification
of <p(z) at this point, we can define relative
complexity and simplicity classes with respect to a
<p-function as follows.
DEFINITION 4 r~, the relative d-complexity class
with respect to the ratio-function '11, is defined by:
r ~ = {z: [s(z, L )]/[z] > '11 (z)} = {;:(I:(z, [z]<p (z))}.
I~, the relative d-simplicity class with respect to the
ratio-function '11, is defined by:
I~ = {z: [s(z, L )]/[z] ~ <p(z)} = {z: 5(z, [z]<p(z))}.
4.3 N umerosit y
Consider a primitive learning mechanism that can
203
but distinguish parts of a system, for example the
balls on a billiards table. Let us assume that the
involved descriptive language, L, has a low power of
expressibility such that it cannot express any
relations between the parts except being distinct.
For example, the color and form of the balls, the way
they bounce against each other, are properties that
cannot be expressed in L. Furthermore, let us
assume that L has an alphabet consisting of just a
single letter, a.
A most natural description of a system in this
language L will be a string of a's, like aaaa, with a
one-to-one correspondence between the letter-
occurences and the parts of the system as "seen" by
the learning mechanism. Any such description will
be a shortest description, and its length will indicate
the number of parts, the numerosity, of the system.
The d-complexity predicate [(z, w) (cf Definition
2) will become a numerosity predicatc ".zconsists of
more than w parts." The d-complexity class with
respect to w, C", will become the class of systems
with more than w parts.
ln many contexts we deal with numerosity as a-
simple form of complexity also when we have at our
disposal a more powerful language than that
described above. This means, however, that we arc
then deliberately abstracting from properties that
go beyond the mere distinctness of parts that
underlies numerosity as a complexity-measure.
4.4 Intricacy and Over-All
d-Complexity
Should I try to describe the whole world, I would
certainly find it more complicated than if I tried to
describe just a part of it, provided that I used one
and the same depth of description.
However, even if [ restrict myself to systems with
the same numerosity, I may get very different
impressions of difficulty, as shown by considering
the two strings A and B of Section 2.
The part of the complexity impression that is
independent of numerosity, has to do with the
nature of the interrelations of the parts, the intricacy
of the system. A reasonable measure of the intricacy
of a string z is [s(z, L)]/[z], because [z] measures
the numerosity of z (considered as a system of
positioned letters). Hence, using the notation of
Definition 4, we may define the intricacy class with
respect to the ratio-constant Y. as r X' where Y. denotes
the function '11 which has the value :,{ for all z.
Notice that intricacy, as well as numerosity, arc
complexity measures primarily associated with
 204 L. LOFGREN
strings or fixed length (i.e., the domain or search for
the learning process is fixed). On the other hand, r "',
wit h <p depending on z, was developed with the view
that Z(II) was a steadily increasing observation
report (i.e., the domain of search for the learning
process is considered unlimited).
Should I want to compare two fixed strings z, and
Z2 or different lengths, I may lind that the
numerosity or z, is larger than the numerosity or Z2,
and yet that the intricacy of Zl is smaller than the
intricacy of Z2' Which, then, has the greatest over-all
complexity? Well, since the strings are fixed, a
natural answer is to apply Definition 3, i.e., to
compare z, and Z2 with respect to the length or their
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012
shortest descriptions in a common descriptive
language L.
On the other hand, should the strings be
considered initial observation reports, i.e., should
they not be considered as fixed strings, they may
instead be compared with respect to Definition 4.
Should they already from the beginning be large
enough, their overall complexities may then be
compared with the ratios [s(zt, L)]/[zt] and
{S(Z2' L)]/[Z2] respectively, where zt is the initial
segment of Zl which has the same length as Z2' The
reason is that if Z2 is large enough, then, if one of
these ratios is larger than the other, it can be
expected to remain so also for increased sequences.
4.5 Comput ationa! Difficulty oj
Difficulty Classlfication-«
A Support
A most reasonable requirement on descriptions of
strings is that there shall be an effective way of
constructing or computing a string from its de-
scription. Otherwise, if descriptions were allowed to
be effectively unintelligible, there would not be
much sense in using them for communications.
From a computational point or view it is thus
natural to consider descriptions with respect to a
universal Turing machine, i.e., a machine that can
simulate any effective computation process.P
DEFINITtON 5 Let z be a string of symbols from
the alphabet ofa universal Turing machine, U. Then
a string x on the same alphabet is a description oj z
lI'itll respect to U if U computes z from the argument
x, i.e., t/J,:(x)=z, where t/Ju is the function computed
by U. Moreover, s(z, U) is the shortest description oj
z with respect to U, i.e., [s(z, U)] ~[x] holds for
every description x of z with respect to U.
With s(z, U) substituted for s(z,L) in Definitions
2, 3, and 4, we obtain the following results
concerning the computability or complexity classes.
LEMMA 1 For /10 universal Turing machille U is
s(z, U) a computable (recursive)Jullction of z.
This is Theorem 11.2 or Relative Explanations of
Systems. 1 2 II was first proved in Ref. 9.
LEM MA 2 For no universal Turing machine U is
there all algorithm (effective process) for the de-
termination ofa z such that [s(z, U)] > x, where x is an
arbitrarily given number.
This is Theorem 11.3 or Relative Explanations or
Systems.'?
THEOREM 2 The d-complexit y predicate \£(z, 11')
is neither computable liar semicomput able. The d-
simplicity predicate 6(z,lI') is not computable but
semicomputahle.
Proof Suppose that the simplicity predicate
6(z, 11') is computable. Then f(z) = 4,rIl1l'6(z, II' )(the
smallest number II' such that 6(z,w» would be
partially computable and total, i.e., computable. But
f(z)=[s(z, U)], and ir f(z) is computable, so is
s(z, U). Hence, by Lemma I, 6(z, 11') cannot be
computable. The semieomputability of 6(z,w) is
seen bythefollowingseriesofequivalent forms, where
we ha ve utilized the notations of Ref. 12:
6(z, 11')= [s(z, U)] ~w
=3xt/Ju(x)=zand [x]~w
=3x3yT(u,x,y)and U(y)=zand [x]~II'.
The last form is an existential quantification over a
computable predicate. Hence, 6(z, 11') is semicom-
putable. Notice that a corresponding proof cannot
be given for the complexity predicate \£(z, 11')
=([s(z,U)]>w)=>(3xt/Ju(x)=z and [x]>w) be-
cause here the last implication does not go the other
way around. Instead, \£(z,w) is not semicornput-
able. This directly follows from the equivalence
\£(z, 11')= -,6(z, 11'). Since 6(z,w) is semicomput-
able but not computable, its negation, i.e., l£(z,w)
cannot be semicornputable (and hence not comput-
able).
THEOREM 3 Let <p(z) be a computable ratio-
[unction such that L", is infinite, and such that 0
~<p(z)< 1 for all z longer than a given number k.
Then the relative simplicity class L", is simple ill the
mathematical sense,' i.e.,
i) L", is recursively enumerable,
ii) !", = r '" is illfinite,
 COMPLEXITY OF DESCRIPTIONS OF SYSTEMS
iii) r, = r" containes 110 illfillite, recursively ell-
umerable subset.
Proof (i):E" is recursively enumerable, because
S(z, w), and hence also S(z, [z]cp(z)), is sernicom-
putable (Theorem 2). (ii) Suppose that r" is finite.
Then [s(z,U)]>[z]cp(z) holds for just a finite
number of strings z, Let m be the length of the
longest of these, and let \I' = d"max (m, k) + i. If a is
the number of symbols in the Turing machine
alphabet, there are precisely a W + I distinct z-strings
of length \I' + I and each of these must have a
shortest description such that [s(z, U)] ~ [z]cp(z)
< [z], i.e., a description of at most length w.
However, there can be at most «" + a W - I +~...+ a
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012
=(a w + 1 -a)/(a-i) such distinct descriptions, and
this number is less than a W + 1. Hence, the assum-
ption must be false, i.e., r" must be infinite. (iii)
Suppose that r" contains an infinite, recursively
enumerable subset. This means that there is a
recursive enumerating function g(Il), such that for
every number 11: g (11) = z; and [sf z". U )] > [z,,]cp (z,,).
Hence. the Iunction ntv). defined by:
iI(x] = d"g(jlll([g(1l )]cp (g(Il)) > x j),
is partially computable. Furthermore, h(x) is total
because Z, is infinite, i.e., [z]cp(z) cannot be finitely
bounded. This means that there is a computable
function iI(x) such that [s(h(x), U)] >x, which
contradicts Lemma 2. Hence, r can contain no
infinite, recursively enumerable s;bset.
Theorem 2 shows that it is in general more
difficult, in a mathematical sense, to decide if a string
IS complex than it is to decide if a string is simple (cf.
Definition 3).
Theorem 3 shows the same result also for relative
complexity (cf. Definition 4). Let us first argue that
any realistic :E,,-class is infinite, the premise for
Theorem 3. If:E" were not infinite, there would be a
largest relatively simple string z. However, the
extension of z according to all the rules possessed by
z must obviously also be considered relatively
simple, a contradiction showing the infinity of' E .
The simplicity of :E", thus showed by Theorem
implies that :E" but not its complement, the relative
complexity class r ", is recursively enumerable.
Hence, r" is more difficult to decide than f .
All this is in support of the suggested co;;'plexity
classifications which indeed intend to capture the
difficulties of learning complex (difficult) strings.
(Some further results are given in Ref. iO.)
Thc nature of the mathematical difficultv-
classification implied by Theorems 2 and 3 is as
c
205
follows. If a set is recursive (computable), then and
only then is there an effective method of deciding for
arbitrarily given x:s whether x belongs to the set or
not. If a set is recursively enumerable (the extension
of a semicomputable predicate), then there is an
efTeetive method for deciding of arbitrarily given
x:s, that in fact do belong to the set, that they belong
to the set, i.e., the set is positively decidable in an
efTective way. However, if a set is not recursively
enumerable, then it is efTectively impossible to
decide for arbitrarily given x:s whether x belongs to
the set even for all those x:s that in fact do belong to
the set.
Hence, the simplicity sets S.. (cf. Definition 3 and
Theorem 2) and :E" (cf. Theorem 3) are positively
decidable, whereas the complexity sets C.. (cf.
Definition 3 and Theorem 2) and r (cf. Theorem 3)
are impossible to decide, even positively. Thus when
we say that the sets C w and r" are more difficult than
the sets S w and :E", we refer to difficulty in terms of
effective impossibility. And should we feel equipped
with superefTective powers, a higher such power will
be required for deciding C w and r" than for deciding
s, and :E".
4.6 all VOIl Neumann's Distillctioll
between Passive and
Active Selj-Description
In his studies of self-reproducing automata von
Neumanrr'! dealt with the following complexity
problem. A constructing automaton may be expec-
ted to be more complex than the automaton it
constructs, because the primary must contain a
complete plan or description .of the secondary, and
III this sense the primary is more complex than the
secondary.
A solution to this apparent difficulty against self-
reproduction may be found in the fact thai a
description car. be much shorter than that de-
scribed. For that reason a description may be
properly contained in the primary automaton in
spite of the fact that it describes that very
3 automaton. Indeed, there are self-describing Turing
machines.t '
In discussing the difficulty, von Neumann sug-
gested two distinct methods of accomplishing self-
reproduction, called the "passive" and "active"
method by Burks 2s who says about a self-
reproducing automaton that it:
must be able to obtain a description or itself.
: .. VO~l Neul~lilnn considers two methods for accomplishing
this, which I Will call the "passive" and "active" methods. In (he
 206 L. LOFGREN

passive method the self-reproducingautomaton contains within measured by the difficulty associated with extract-
itself a passive description of itself and reacts this description in ing the interpretation (meaning) of a description.
such a way that the description cannot interfere with the In the following subsections we will discuss
automaton's operations. In the active method the self-
reproducingautomaton examines itself and thereby constructs a various types of interpretations, and see how
description of itself Von Neumann suggests that this second corresponding types of i-complexity result.
method would probably lead to paradoxes of the Richard type,
and for this reason he adopts the first method.
5.1 Complexity and Syntactical
The two methods of generating self-descriptions can Information
immediately be identified with the two ways of
Let Wand V be well-formed formulas (wff's),
producing a description as are illustrated in Figure
considered as descriptions with reference to a theory
I, namely inheritance and learning.
T. In a definite sense the meaning of W includes the
Inheritance, where a genetic description of an
meaning of V, if V is a consequence of W in T. For
organism is contained within the organism and
example, the meaning of "a snowball" ineludes the
released in thc act of reproduction, corresponds to
meaning of sentences like "it feelscold in my hands,"
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012

the passive method. As is well known 12 there are

"it will melt in my hands," etc.
Turing machines which, in their act of computation,
release their own description numbers (in spite of Hence, whatever the meaning of the wff's Wand
the fact that this very release-computation is what is V are, we can say that the difficulty of extracting the
dcscri bed), meaning of V is not greater than the difficulty of
Learning, where the learning organism (auto- extracting the meaning of W, if V is a consequence of
maton) investigates itself and thereby tries to Win T, denoted W~T V. Furthermore, wff's that are
produce a description of itself, corresponds to the equivalent with respect to T have the same
active method. meaning-extraction difficulty.
As we know, learning is in general less powerful
than inheritance: there are effectively describable DEFINITION 6 The IW I-complexity of a wff Win
rules (behaviors) that cannot be effectively lear- a theory T is a measure which satisfies:
ned. t 2. 13 Hence, we know that we can always
IWI=IVI ifandonlyif W~TV and V~TW,
complicate even the most powerful learning auto-
maton with a sufficiently complex normal behavior IWI~IVI ifandonlyif W~TV.
(behavior beside learning) such that it will be unable
to learn itself. This is thus in support of the von DEFINITION 7 Let T be a first order theory in a
Neumann hypothesis about paradoxes connected full first order predicate language L. The complexity
with self-description according to the active algebra A for descriptions Win L with respect to Tis
method. A=({I WI:WEL}, ~>.
This does not mean, however, that there are no
learning automata that can learn their own de- The following theorem is given in.!"
scriptions. Indeed it is possible to exhibit very
simple such learning automata which work with a THEOREM 4 The complexity algebra A is a
learning structure of Figure 2. However, such nonatomic Boolean algebra where:
automata can be said to learn themselves because of
their simplicity (resulting from a poor and ready-
made hypothesis-generating program) rather than
by the power of a complicated learning mechanism. (=inf{1 WI 1,1 W2 1} ),
In general, many of the functions indicated in the
learning structure of Figure 2 are noneffective
processes, i.e., processes that cannot be algorithmi- I WI*=li Wi (=complement of! Wi)·
cally executed (for example the minimalization step, i-complexity naturally relates to (syntactical)
the confirmation step, the relevant-question step). information.P The larger the information of a
description W, i.e., the more W describes, the more i-
5 COMPLEXITIES OF INTERPRETING complex do we expect W to be.
(i-COM PLEXITI ES). On a more formal level, let W be a description in a
Let us recall from Section 2 that the i-complexity is language L with respect to the a priori knowledge T.
 COMPLEXITY OF DESCRIPTIONS OF SYSTEMS
Then the syntactical information 12 of W with
respect to T, 1(W,T)=d,r{X:(XEL) and (W~TX)
and (1'- T X )} satisfies Definition 6 for I W 1-
complexity in the sense that:
I I,
I(H-;T)= I(V, T) if and only if I Wi = V
1(W,T)~I(V,T)ifandonlyifl WI~I VI.
Information, and thus complexity, is frequently
associated with independence from the a priori
knowledge T (or with degree of freedom). If Wand
V are two descriptions such that I WI> I V I, then W
is more deductively independent than V because
W~TV but not V~TW Independence from T may
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012
also be looked at as uncertainty with respect to the
knowledge T. Hence, the more uncertainty, with
respect to the a priori knowledge T, that is removed
by the observation of an event described by W, the
more complex is the event (the higher is the I W 1-
value).
5.2 Complexity and Statistical
Information
If the a priori knowledge T concerning a class of
events is not given as a formal theory but as a
statistical knowledge, then statistical rather than
syntactical information will measure the un-
certainty that is removed upon the observation of an
event. Just as syntactical information measures
I W I-complexity, statistical information measures i-
complexity in the case of a statistical a priori
knowledge.
By way of example, suppose that a string of O's
and l's is produced by a source about which we
know, a priori, that it emits symbols from the
alphabet iX={O, I} with the given probabilities p(O)
and p( I) at each symbol emission. The nature of this
a priori knowledge T is such that it permits no
deductions from the observation of a certain emitted
symbol. except that a certain uncertainty, with
respect to T, is removed at the observation. Indeed,
the "meaning" or "interpretation" of the observed
symbol will have to be identified with the cor-
responding removal of uncertainty. Whatever the
work of interpreting the emitted symbol is, it is
naturally associated with the amount of removed
uncertainty.
As is well known.Pvan appropriate measure for
this amount of removed uncertainty is the infor-
mation I (x) = log l/p(x), where x ranges over the
symbol alphabet iX. I (x) thus measures the
interpreting-complexity of the symbol x, when
207
emitted from the source. The i-complexity of a
whole string A emitted from the source will be I (A)
=~ I(Xi)' where Xi is the symbol at the i-th position
of A.
The probability that the i-th symbol of A is x,
p(x), is independent of i and hence the average i-
complexity, C(n, H), ofa string of length n produced
by the source will be C(n, H) = nH, where
H= l: p(x) log 'l/p(x)
xea
is the entropy of the source. Here we may compare n
with the numerosity and H with the intricacy for d-
complexities (where, instead, the problem is to
determine the properties of the source; cf. Sections
4.3 and 4).
For a fixed n, the C(n, H)-complexity is maximal
for a source where the a priori probabilities p(x) are
alike for all symbols x E iX, i.e., when the source
produces symbols from iX at random.P
Concerning a nonstatistical definition of random-
ness, see Section 6.2 where randomized sequences
are developed as having maximal d-complexity.
5.3 Computational Complexity
In Section 5.1 we considered interpretations where
the meaning of a wIT W was identified with the
meaning of all deductions from W in the a priori
knowledge T. We have seen that merely the
syntactical concept of deduction suffices to define a
partial order on the set 01"1 H1-complexities. This type
of complexity may thus be looked upon as a
complexity of nonreferential interpreting, since no
reference beyond the describing language itself was
actually utilized for the definition of the complexity.
By way of contrast, we will now consider a very
referential type of interpretation, associated with
computational complexity. Here we will actually
make use of an extralinguistic interpretation,
namely an interpretation of a description as a
program that orders a machine how to compute.
The difficulties of interpreting will thus be as-
sociated with the amount of computation that is
directed by the program (description).
Computational complexity has been studied
comparatively extensively,": I and attempts to axio-
matize the concept have been made and;': 22
furthermore, discussed both from the aspect of
being too abstract (leaving out some genuine
complexity property), and of being too concrete'
(containing properties that are not genuine com-
 L.
plcxity properties). In the following we will sum-
marizc thc axiomatic description and provide
motivational aspects partly with reference to the
recursive Turing machine predicate 12 T(z, X, y) and
partly with reference to the ideas of expected
difficulties of difficulty classification, advanced in
Sect ion 4.5. . undefined.
Let us recall that T(z, x, y) is true if and only if t he
Turing muchinc Z, with code number z, when
starting from the argument x performs the com-
putation sequence Y, with code number y. The
computation sequence is a description of what the
machine does, step for step such that each segment
of the sequence describes the state of the machine,
the scanning position, and the contents of the tape at
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012
the corresponding step of the computation. Thus
the computation sequence Y itself, or its code
number .1', may be taken as a representative of the
actual computation and its complexity. However, Y
is a complete description of the computation and
contains further properties beside those which
directly bears on the complexity of computation. In
trying to abstract away from such irrelevant
properties it has been suggested to measure the
complexity of the computation "',(x) with a
measure-function (!>,(x), with natural numbers as
values, according to the following definition." 4
DEFINITION 8 <.)),(x) is a complexity measure for
the computation "',(x) if:
i) (I>, (x) is defined if and only if'" ,(x) is defined,
ii) the simplicity predicate <!J,(x) ~ II' is comput-
able in z, x, 11'.
Let us first see how these two properties of a
complexity measure (I>,(x) are to be found among
the properties of the computation sequence itself.
i) By definition "',(x) is defined if and only if
3yT(z. :x, .1'), i.e., if and only if the code
number .1'( =(I>,(x)) is defined.
ii) The predicate T(z, x, .1') is recursive, and with
it the predicate 3y(y~1I' and T(z,x,y))
(=(I>.(.x) ~ w], because the existential quanti-
fication is bounded.
Condition (i) is natural because "'_(x) is defined if
and only if the corresponding computation sequ-
ence Y is defined, and a complexity of computation,
(I>,(x), is not naturally defined if the computation is
not defined. Conversely, if the computation Y is
defined, its complexity should also be defined.
Otherwise the measure would not be considered
sufficiently well developed.
L6FGREN
However, if"',(x) is undefined for a given z and x,
it is true that a corresponding computation sequ-
ence y is not defined as a number, but nevertheless
an infinite behavior of the machine Z is defined.
Accordingly it has been suggested to define <!J.(.x) as
a certain nonnumber (infinity) when "'.(.x) is
Condition (ii) implies that there is an effective way
of deciding, for arbitrarily given z, x, IV, if <!J,(x) ~ II'
is true or false. Here ~ is the "less than" relation
among numbers, such that if<!J,(x) is undefined as a
number, then it is false that <!J,(x) ~ II' and, of course,
also false that <!J, (x ) > 11'.
Why should, in this case of i-complexity (com-
plexity of interpreting), the predicate <!J,(x) ~ II' be
computable (recursive)? After all, in the case of d-
complexity (complexity of describing), the cor-
responding predicate 6(z, IV), i.e., [s(z, L)] ~ II' is in
general noncomputable (cf, Definition 2 and Theo-
rem 2).
A machine oriented motivation of condition (ii)
may be developed as follows. The type of in-
terpretation we are considering, namely an in-
terpretation of a description as a program that
orders the machine how to compute, is obviously
effective (computable). Describing processes, on the
other hand, are in general noneffective (cf. Section
4.5). Now, the difficulty of interpreting is, in the
actual case of computational complexity, realized
by the computer (for example as a length of
computation, number of tape positions used, etc).
Hence, if<!J,(x) is defined as a number, that number
is computable and with it the predicates <!Jz(x) ~ IV
and <!J,(x) > 11'. On the other hand, if <!J,(x) is not
<
defined as a number, the defined program z, x) still
effectively determines a machine behavior, an
infinite behavior, and we can from the triple z, x, II'
effectively determine that <!J,(x)~1I' is false (but not
that <!J,(x) > II' is false; cf. the following Theorem 5),
at least if<!J, (x ] is measured by y.
A II10re abstract motivation of condition Iii) may
be developed with reference to the idea that a
complexity classification should be more difficult
than a simplicity classification (cf. Section 4.5). To
begin with, let us observe that condition (i) implies
that the complexity predicate <!J,(X) > II' (cf. [(z, 11')
of Definition 2) is noncomputable.
THEOREM 5 Let <I>,(x) be any functions that
satisfy condition (i) of Definition 8, such that <!J,(x),
when defined, takes values from the natural numhers
1,2,3, .... Then the complexity predicate <!J,(x ) > II' is
noncomputable.
 COMPLEXITY OF DESCRIPTIONS OF SYSTEMS
Proof Let us assume that the complexity pre-
dicate <I>,(X) > II' is computable. Then the simplicity
predicate <I>,(X);£ II' will also be computable. This
can be seen as follows. The simplicity predicate is
equivalent with a bounded existential quantifi-
cation over complexity predicates:
<I>,(x);£ 11';; 3y (0 <)';£ 11') and (<I>,(x)=.I')
;;3)' (0 <.1';£ 11') and (<I>,(x)j y)
and (<I>,(X)> y-I).
Hence, since the complexity predicate is assumed
computable, the simplicity predicate will also be
computable. Finally, by condition (i) of Definition 8,
ifi,(x) is defined if and only if <I>,(x) is defined, and
<Il,(x) is undefined if and only if both <I>,(x» II' and
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012
<1>, (x);£ II' are false. Hence we can effectively decide if
ifi,(x) is defined or not. This contradiction against
the non-computability of the halting problem for
Turing machines proves the theorem.
Hence, if we accept condition (i], the complexity
predicate will be noncomputable, and the com-
putability of the simplicity predicate, i.e., condition
(ii), is a natural assurance for the complexity
classifications to be more difficult than the simp-
licity classifications. Furthermore, condition (ii)
implies that the complexity predicate is semicom-
putable and that <I>,(x) is partially computable in z
and x.
THEOREM 6 Let <I>,(x) be a complexity measure
according 10 Definition 8. Theil <I>,(x) is partial
recursive ill z and x, and the complexity predicate
<1>, (x) > \I' semicomputable (hut 1I0t computable; cf
Theorem 5).
Proof Since <I>,(x);£II', according to condition
(ii), is computable the function f(z,x)=dofJlII'[<I>,(x)
;£ 11'] is partially computable and equal to <I>,(x), i.e.,
<I>,(x) is partially computable in Z,X. Hence the
complexity predicate <I>,(x) > II' is semicomputable.
Definition 8 has interesting consequences con-
cerning the computational complexity ofJimctiulIs.
Notice that <I>,o(x o ), if defined, specifies the com-
plexity of computing the specific function value
ifi,o(xo), when computed with the computation
sequence Yo such that T(zo, XO,)I O)' On the other
hand, the complexity of computing a whole func-
tion, say ifi,o' will have to be determined by Zo, i.e.,
by <1>'0 rather than by an isolated computation
».
sequence Yo (determined by (zo, x o In the case of
computing functions rather than function values we
will thus speak of computations specified by z-
values rather than by y-values.
209
Given a function cp(x), it may be computed by all
infinity of computations
Z"Z2"" (cp(x)=ifi'l(x)=ifi,,(x)= ... ).
It is natural to ask for a computation z; with a
smallest complexity. On that point Definition 8
implies the interesting result that there are comput-
able functions that do not have a computation of
minimal complexity. The following theorem is
proved by Blum.'
THEOREM 7 Let <I>,(x) be a complexit y measure
according to Definition 8, and let f(x) he allY
monotone increasing recursive function. I1Jel1 there
exists a recursive function cp(x) having /10 simples I
computation in the [ollowing sense. Given (lily
computation z, fur cp(ifi, (x)=cp(x)), there ex ists
another computation Z2/or cp(ifi,,(x)=Cp(x)) such
that f (<1>" (x)) ;£<1>'1 (:dfor all sufficienti» large x.
Thus, however fastly growing the function f is
chosen to be, there is a function cp such that any
computation z 1 for cp will turn out to be at least f
times as complex as needed by another computation
Z2' This reasoning can again be applied on the
computation Z2, and so on.
It is true, however, that function complexities here
are being compared for all but a finite number of
arguments ("for almost all x "), It may thus happen
that those arguments, that for some reason are
found to be the most interesting, all belong to the
finite number which are excluded by the above
theorem (a so called speed-up theorem).
Computational complexity is a well studied field
although rather broad and, perhaps, not very clearly
defined. Some authors' include within com-
putational complexity not only complexity of
computations of functions (a so called dynamic
measure of computational complexity). but also
complexity of describing functions (with their
computational programs; a so ca.led static measure
of computational complexity). In our terminology,
a static measure of computational complexity is a d-
complexity, whereas a dvnarnic measure is an i-
complexity. This is why we have studied questions
concerning static measures already in Section 4.5.
Furthermore, computational aspects on proofs in
formal theories (cf. the computability requirements
on the axioms and rules of inference in a formal
theory), may suggest the area of proof-complexlt y as
a branch of computational complexity.
Parts of proof-complexity are akin to d-
complexity. For example when the complexity is
considered as the difficulty of finding (learning) a
 210 L. LOFGREN
proof to a given theorem, or when considered as the
difficulty of memorizing a given proof. On the other
hand, the work needed to check if an alleged proof
really is a proof is akin to computational complexity
in the form of i-complexity (also akin to the work of
checking the wellformedness of a given formula that
implies part of the meaning of the formula).
Concerning proof-complexity measured by the
length of the proofs we want to mention a result of
Godcl-Mostowski '8 which has a certain re-
semblance with Theorem 7. Let 5 be a first-order
arithmetic and 5 I an extended arithmetic, essen-
tially containing also second-order quantifications
(for details, see!"). Then there are wffs W which are
theorems in both 5 and 5, such that the shortest 5,-
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012
proof of W is arbitrarily (in the sense of the
following theorem) shorter than the shortest 5-
proof of W
THEOREM 8 For every recursivefullctioll f there
is a 111J W of 5 such that both ~sW and ~s Walld
such that the lengths of the millimal proofs cifw
ill 5
and 5" p(W) and pdW) respectively, satisfy
Is», (W))<p(W).
For a proof, the reader is referred to Mostowski.!"
6 INCREASE OF COMPLEXITY
The higher the i-complexity of a description is, the
richer is the description in contents. Hence high i-
complexity may be associated with high potentiality
of behavior, richness in expressibility, etc., and it
may be natural and desirable to ask for an increase
in the i-complexity of a system.
d-complexities, on the other hand, are mostly
wanted small. There are exceptions, however, like
desires for arbitrarily long randomized sequences,
i.e., sequences which really should be difficult to
learn, i.e., sequences of high d-complexity.
6./ OIlVOIl Neumann's Cut-Point
for Complexity
In his work on self-reproducing automata, von
Neumann" considered the high potentiality to do
things that may be found in very complicated
automata. Indeed, he more or less identified
complexity with such a potentiality (and thereby
used complexity in a restricted sense that we have
tried to avoid in this more general study of the
concept of complexity). On p. 78 of his book" he
writes:
There is a concept which will be quite useful here, of which we
have a certainintuitiveidea, but which is vague, unscientific, and
imperfect. This concept clearly belongs to the subject of
information, and quasi-thermodynamical considerations are
relevant to it. I know no adequate name for it, but it is best
described by calling it "complication." It is effectivity in
complication, or the potentiality to do things. I am not thinking
about how involved the object is, but how involved its purposive
operations are. In this sense, an object is of the highest degree of
complexity ifit can do very difficult and involved things.
Evolving biological organisms have the ability to
produce something more complicated than them-
selves. Certain artilicial automata on the other
hand, like machine tools, are more complicated than
the elements which can be made with them. Von
Neumann suggests the hypothesis that corn-
plication is degenerative below a certain level but
not above that level or complexity cut-point."
We do not now know what complication is, or how to measure it,
but I think that something like this conclusion is true even if one
measures complication by the crudest possible standard, the
number of elementary parts. There is a minimum number of parts
below which complication is degenerative, in the sense that if one
automaton makes another the second is less complex than the
first, but above which it is possible for an automaton to construct
other automata of equal or higher complexity. Where this
number lies depends upon how you define the parts.
There is thus this completely decisive property of complexity,
that there exists a critical size below which the process of
synthesis is degenerative, but above which the phenomenon of
synthesis, if properly arranged, can become explosive, in other
words, where syntheses of automata can proceed in such a
manner that each automaton will produce other automata which
are more complex and of higher potentialities than itself.
Now, none of this can get out of the realm of vague statem~nt
until one has defined the concept of complication correctly. And
one cannot define the concept of complication correctly until one
has seen in greater detail some critical examples, that is, some of :
the constructs which exhibit the critical and paradoxical
properties of complication. There is nothing new about this. It
was exactly the same with conservation and non-conservation
properties in physics, with the concepts of energy and entropy,
and with other critical concepts. The simplest mechanical and
thermodynamic systems had to be discussed for a long time
before the correct concepts of energy and entropy could be
abstracted from them.
It would seem that von Neumann here in fact is
using distinct meanings of complexity, partly the
potentiality aspect of i-complexity and partly a d-
complexity akin to numerosity. Without a more
precise definition of complexity, von Neumann's
cut-point hypothesis may lack in contents, and it is
often remarked that trivially simple automata may
be considered self-reproductive.
Since the basic question of concern is the
potentiality aspect of complexity, we want to
consider r-complexities alone, first in connection
with anonreferential type of interpretation and later
in connection with a referential.
 COMPLEXITY OF DESCRIPTIONS OF SYSTEMS
As a starting point for the development of a
non referential complexity aspect, let us consider a
suggestion of von Neumann that when an auto-
maton is not very complicated, the description of the
functions of that automaton is simpler than a
description of the automaton itself but that the
situation is reversed with respect to complicated
automata. The suggestion has been commented
upon by Burks and Godel ; see pp. 54-56 of Ref. 25.
Let T be a theory of automata such that T(w,,),
i.e., T augmented with the proper axioms w",
describes the automaton A. In the same way, let
T(~) describe the behavior of the automaton A,
provided that ~ are the full behavior axioms for A.
For automata like Turing machines it can happen
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012
that T(w,,)c T(~) in the sense that the complete
behavior description T(~) may require new terms,
defined by ~, beyond those defined by T(w,,) (cf. a
remark of Godel'"). Furthermore, T(W.) may be
rich enough to include T(Wa). From T(w,,)c T(~)
we can conclude that I ~ I> I w" I (cf. Section 5.1),
i.e., the complexity of the description of the behavior
is larger than the complexity of the description of the
automaton.
For simpler types of automata, on the other hand,
the behavior may not require new terms beyond
those already defined in T(w,,) but may be
completely defined by the description T(w,,) of the
automaton. Furthermore, the behavior may be poor
enough not to permit a deduction of the automaton
structure A. Then we have T(~)cT(w,,), i.e., I~I
< I w" I, i.e., the complexity of the description of the
automaton is larger than the complexity of the
description of the hehavior.
In between the above two cases we have I w" I
=1 ~I,i.e., T(w,,) and T(~)areequivalentandthe
behavior description is thus as well as structure
description. This case may be regarded a kind of
self-description. It should be noticed, however, how
critically this self-reference point for description
depends on the theory T.
Concerning this kind of self-description, the
involved deductions are defined with respect to an
externally given theory T. Hence it is beyond the
point whether the deductions are performed by the
automaton, its surrounding, or somebody else.
However, when proceeding from self-description
to self-reproduction we have to consider the
behavior of the automaton as a description (of
another automaton) with respect to the interpreting
properties of a surrounding, i.e., we switch to "
referential type of interpretation. If that surround,
211
ing is thought of as neutral with respect to the
deseribing process, which is a natural assumption in
the case of constructing automata, then we have to
consider whether the deseription is to be produced
by the automaton according to the passive or active
method (cf. Section 4.6).
Let us assume that the passive method is at play
(cf. our intention to consider i-complexities alone).
Let a, be a productive automaton with a behavior
which is interpreted by the surrounding as a
description of another automaton a2 which again is
productive, etc. Then, since a i produces a i + l e we can
say that a, is more complex than ai + 1 if we consider
a complexity which results from I Wi-complexity
upon replacing deductions with productions.
In terms of this referential production complexity,
a production process (ai produces ai+ I) can be
degenerative or at most, in the case of self-
reproduction (a i produces a i ), complexity preserv-
ing. But the complexity can never be increased as
long as the generated automaton is completely
determined by the generating automaton. Hence
self-reproduction cannot be a cut-point for referen-
tial production complexity but is rather a limit
point.
However, in the case of a mixed referential and
nonreferential aspect of i-complexity we can have
self-reproduction a true cut-point as follows. Let
part of the behavior of an automaton a, be
referentially interpreted as the description. of
another automaton a i + 1 and another part of the
behavior nonreferentially interpreted. That latter
part is thus not interpreted by the surrounding as a
description of another automaton. Instead it may,
for example, be interpreted by an outsider as an
enumeration of the theorems of some theory 7i.
As shown in Refs. 19, 12, there exists a productive
automaton ai which, beside producing the auto-
maton ai+ l ' enumerates a theory T; such that 7i
c 7i+ i- In other words, the I W I-complexity of the
enumerating behavior is steadily increasing from
generation to generation (ai + 1 can produce a new
theorem of 7i+ 1 which a, cannot), in spite of the fact
that the a,.,.! automaton is completely determined
by the a, automaton.
Thus it is possible to increase effectively beyond
limit the I W I-complexity, and thus the potentiality,
of productive automata (and, as previously ar-
gued,'? at the same time their d-complexity in the
form of numerosity of parts). Such an effective
increase is however lIot possible for d-complexity in
general as we are about to see in the next section.
 212 L. LOFGREN
6.2 Liniit ations jor WI Effective
Increase ofthe Length of
Randomized Sequences
In discussing von M iscs' axiom of randomness
Popper:" writes:
The axiom or randomness or, as it is sometimes called, "the
principle or the excluded gambling system," is designed to give
mathematical expression to the chance-like character or the
sequence. Clearly, a gambler would be able to improve his
chances by thc usc or a gambling system if sequences or penny
losses showed regularities such as. say, a fairly regular
appearance of tuils after every run or three heads. Now the axiom
or randomness pustulates or all collectives that there does 110t
exist a gambling systcin that can be successfully applied to them.
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012
Instead of founding the concept of randomness
upon the powers of gamblers, I suggested" that we
replace the gambler with an elTcctive machine, a
Turing machine, and consider a sequence as
random if no Turing machine can find any rules in
it with which to shorten the description of it (if there
are rules they can bc used to shorten the description;
ef. Section 4.2 and 4.5). Hence I suggested that a
sequence is randomized if its shortest description
with respect to a universal Turing machine is
maximal, and that a sequence is the more rando-
mized thc longer its shortest description.": "
DEFINITtON 9 Let z, and Z2 be two strings of
equal length, written on the alphabet of a universal
Turing machine U. Then, in relation to U, z, is //lore
randomized than Z2 if and only if s(z" U) > S(Z2, U).
With U dcfining thc semantics of L, Definitions 9
and 2 immediately suggest that randomization be
identified with d-complexity as follows,
COROLLARY ofDefinit ion Zand 9. Let 2, and Z2 be
two st rings 0/ equal lengiu written on the alphabet of
a universal Turing //lachine U. Then z, is more
randomized than 22 if and only if z, is more d-
complex than Z2 according to Definition 2 (with
s(z, U) suhstit uted for s(z, L)),
Argument z, is more d-complex than Z2 accord-
ing to Definition 2 if and only if there is a number II'
such that [(z" 11') and 6(Z2' 11'), i.e, such that z , IS
complex and Z2 simple with respect to 11'. This is
equivalent with the condition thai s(z" U)
>S(Z2' U), i.e., with the condition that z, is more
randomized than Z2'
I previously defined '2 a degree of randomization
as follows and showed that there is no effective way
of generating arbitrarily long seq uences randomized
to a nonzero degree.
DEFINITION lOA string z of length v, written on
an alphabet of n symbols, is randomized 10 a degree I'
in relation to U if s(z, U)~s(w, U) holds for I' 'n'- of
the n" strings II' of length I', where r is a rational
number such that 0<1';£ I.
THEOREM 9 For /10 universal Turing machine U
and for no degree of randomization 1'>0 is there Wl
algorithm forthe generation of a string z(v), where v
is an arbit rariiy given length of the string, such that
z(v) is randomized to degree I' relative to U.
In view of the above Corollary we may as well
interpret Theorem 9 as saying that there is no
effective way of generating sequences with an
arbitrarily large over-all d-cDmplexity (cf. Section
4.4). (Cf. also a theorem!" which says that rando-
mized sequences can only be effectively generated
provided that they are shorter than a fixed length,
which depends on U.)
Other definitions of randomized sequences have
been suggested which also involve the notion of
effective computability in one way or another.": 17lt
would seem that they all imply that it is impossible
to effectively generate arbitrarily long randomized
sequences.
7 REDUCTION OF COMPLEXITY
d-complexities are usually wanted small just as we
want the difficulties of learning or describing our
surroundings 10 be made small.
Let us recall that a d-complexity is not an intrinsic
property of objects but rather a concept that
depends on our learning capacity. Hence we may
well consider to reduce the complexity of an object,
not by changing the object, but by changing our
views of it, for example by describing it in a more
powerful language.
We want to illustrate such a complexity reduction
with a type of reduction that occurs within our
cerebral description processes and which is likely to
generate our existential perceptions of concrete
objects. Thus we will outline a (foundational) study
of objects by looking into their nature as per-
ceptions. The interested reader is referred to
LOfgren 15 for a fuller study. Distinct methodologi-
cal studies of things and objects have been given by
Bunge 5 and Goguen."
Let us first notice that it is not the objects of the
real world that are the entries to our cerebral
description mechanisms. Rather, it is an enormous
 COMPLEXITY Of DESCRIPTIONS Of SYSTEMS
data-f1ow into our receptors that is the direct entry.
In trying to describe that f1ow-in itself inde-
scribably large-the description process must re-
duce its complexity. We want to suggest that it is this
reduction that calls forth perceptions of existing
objects. Thus the objects, in terms of which we "see"
the real world, are the results of the cerebral
description process rather than being the entries to
that process.
Let us, for the development of the arguments,
recall a principle according to which the extensions
of concepts vary inversely with their intensions.
Nagel?" considers it a logical principle.
Let Ext A={A"A 2, ... } be the extension of a
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012
concept A, i.e., the set of objects A I' A z,'" denoted
by the concept. Let IntA={P 1X,P25,', ... } be the
intension of the concept A, i.e., the set of all
properties (predicates expressible in a given lan-
guage) that hold for each member, x, of Ext A. By
way of example, the extension of the concept
"triangle" is the set of all specific triangles. The
corresponding intension is the set of all properties
which apply to every specific triangle ("having three
sides," "having an angular sum of 180 degrees," etc.).
Suppose that some properties are deleted from
the intension of a concept A. In other words, some
properties are abstracted away. Then, further new
objects, Ai' will satisfy the remaining intension
(which has become less restrictive). That is, the
smaller the intension, the larger the extension, and
the more abstract (less concrete) the concept. Again,
the more concrete a concept is, the larger will its
intension be, and the smaller its extension. A
concept like a concrete object ("the table before
me") which has a minimal extension (containing just
one element), will thus have maximal intension.
Abstract concepts are easily demonstrated in
writing. The reason is that a description expresses
properties (the elements of the intension), and
abstract concepts have small, and therefore easily
expressible, intensions. The concept "triangle" for
example, is rather easily described in terms of its few
basic properties (which everyone knows which is
familiar with axiomatic Euclidean geometry).
Concrete concepts, on the other hand, are very
difficult to demonstrate in writing unless denoting
phrases are used and allowed to be extralinguisti-
cally interpreted (cf. "the table before me"). The
reason is their maximal intension.
Let us now consider writings involved in a
cerebral description (learning) process with a
structure according to Figure 2. We will assume that
the language, which is used in the description
D
213
process, is sufficiently advanced to allow a renaming
process, essentially of a type known from de-
finability theory.i"
For example, if Wx is a very large wIT that occurs
frequently in a description, it may be economical to
give it a short name and to use this name, instead of
Wx itself, wherever possible in the description. For
example, if Q is a not yet used predicate symbol,
such a naming can be accomplished by adding the
wlTltx(Qx= Wx), the definition, as an axiom to the
actual theory (description). Such a naming process
will be harmless in the sense that it fulfills the
requirements of eliminability and noncreativity.i"
Yet it can be quite elTective in the sense that it may
shorten the description of the data-f1ow con-
siderably.
Furthermore, if the large wIT Wx uniquely
determines (within the theory) an x such that Wx,
that very x may be named c, provided that c is a not
yet used constant and that the wIT3 ! x Wx and x = c
is added as a new axiom. Although an individual
constant. c will by the naming completely represent
the predicate Wx.
Let us assume that a complex configuration of
properties have occured frequently in the past. Then
an hypothesis H will be formed, namely the
hypothesis that: "the configuration is unique and
will occur as such also in the future." Upon such a
strong support for H, that H will be confirmed, the
renaming process will be excited and a name, c, will
be introduced for the unique bearer of the complex
property configuration.
There is reason to believe that the cerebral
processes which generate perceptions are directed
from the confirmation processes (that cognition is a
form of very strong belief). It is therefore reasonable
that primary perceptions may occur upon a
confirmation that is of such strength that it excites
the naming process. What kind of perception will
then result? It will be the perception of something
concrete; which has a maximally unmanagable
intension and therefore is the best choice for
simplification by naming. Thus, it will be the
perception, as a single concept internally repre-
sented by the name c, of something concrete, i.e., an
existential perception (of a concrete object).
Notice that this essay on the emergence of
existential perceptions is not intended as a causal
explanation of perceptions. It may well be, as in
lower forms of animals, that the very term-naming
mechanism we have outlined may be at play,
however without resulting in perceptions. The
question if such perceptions occur first at a certain
 214 L. LOFGREN
level of confirmation, or first at a certain level of
cxprcssibility of the language used in the description
process, is beyond the reach of this essay. Tnstead
we have intended to support the likelihood of just
existential perceptions, taken for granted that
perceptions in general do occur.
As we have seen, the possibility of reducing
complexity depends essentially on the nature of the
language. Our ideas about languages arc mostly
based on our acquaintance with the external
communication languages. It is important, how-
ever, to conceive of languages very broadly when
associating general learning processes with lan-
guages (in ·which the produced descriptions are
Downloaded by [Stanford University Libraries] at 11:09 27 September 2012
formulated ).
The inner cerebral language we have just out-
lined, which produces objects in describing data, has
a non referential interpretation theory. On that
point it is unlike an outer communication language
which is referential and produces properties or data
in describing objects.
Very likely the growing knowledge of semantics
of programming languages will be helpful for a
further understanding of the hierarchy of inner
cerebral languages. For an outline we refer to Ref.
15.
1701' a pllOl()1{raph and biography of Professor Lofgrell see
Voll/me 3, No. I (1976). p. J (iflMs Journal.
REFERENCES
I. M. Arbib, Theories of Abstract Automata. Prentice-Hall,
N.J., t969.
2. G. Ausicllo, "Abstract Computational Complexity and
Cycling Computations," Journal of Computer and System
Sciellees. 5, 1971, PI'. 118-128.
3. M. Blum, "A Machine Independent Theory of the Com-
plexity of Recursive Functions." J. ACM, 14, 1967, pp. 322-
336.
4. A. Horodin, "Computational Complexity: Theory and
Practice." Currents is the Theory of Compunng. edited by A.
Aho, Prentice-Hull, N.J .• 1973.
5. M. Bunge, "Things." lilt. J. General Systems, I, 1974, pp,
229-236.
6. A. Church, "On the Concept of a Random Sequence." Bull.
Amer. Math. Soc .. 46, 1940, PI'. 130-135.
7. M. Davis. Computability and Unsolvabilnv. McGraw-Hili,
New York, 1958.
8. J. Goguen, "Objects." lilt. J. General Systems, I, 1975, PI'.
237-243.
9. L. Lofgren, "Recognition of Order and Evolutionary
Systems." Computer and Information Sciences, Vol.lI,edited
by J. Tou, Academic Press, New York, 1967.
10. L. Lofgren, "Relative Recursiveness of Randomization and
Law-Recognition." Notices Amer. Math. Soc., 16, 1969, p.
685.
11. L. LOfgren, "Complexities of Descriptions and Random
Numbers." J. Symbolic Logic, 36, 1971, p. 360. [Abstract.)
12. L. Lofgren, "Relative Explanations of Systems." Trends ill
General Systems Theory, edited by G. Klir, John Wiley, New
York,1972.
13. L. LOfgren, "On the Formalizability of Learning and
Evolution." Logic, Methodology alld Philosophy of Science
IV, edited by P. Suppes, er. al., North-Holland, Amsterdam,
\973.
14. L. Lofgren, "On Hempel's Paradox of Confirmation." J.
Symbolic LORic, 41, 1976, pp. 273-274. (Abstract.)
15. L. LOfgren, "On Existence and Existential Perception."
Synthese. (Forthcoming.)
16. L. LOfgren, "A Theory of Confirmation with a Resolution of
the Hempel Paradox." (Forthcorning.)
17. D. Loveland, "A New Interpretation of the VOII Mises'
Concept of Random Sequence." Zeitschr f math Logik und
Grundiugen d Math, 12, 1966, PI'. 279-294.
18. A. Mostowski, Sentences Undecidable in Formalized
Arithmetic, all exposition ofthe theory ofKurl Giidel, North-
Holland, Amsterdam, 1952.
19. J. Myhill, "The Abstract Theory of Self-Reproduction."
Views on General Systems Theory, edited by M. Mesarovic,
John Wiley, New York, 1964.
20. E. Nagel, The Structure of Scieece, Routledge and Kegan
Paul, London, 1961.
21. K. Popper, The Logic of Scientific Discovery. Basic Books.
New York, 1961.
22. M. Rabin, "Degree of Difficulty of Computing a Function
and a partial ordering of recursive sets." Tech. Rept. 2,
Hebrew University, Jerusalem, 1960.
23. C. Shannon and W. Weaver, The Mathematical Theory of
Communication, Univ. of Illinois Press, Urbana, 1962.
24. P. Suppes, Introduction to Logic, Van Nostrand, Princeton,
1957.
25. J. von Neumann, Theory of Self-Reproducing Automata.
edited and completed by A. Burks, University of Illinois
Press, Urbana, 1966.
26. Webster's New Internarional Dictionary of the English
Language. Second edition, unabridged. Springfield: Mer-
riam Co., 1960.