Professional Documents
Culture Documents
This essay is structured around two papers with the same title, The Architecture of
Complexity, written, one in 1962 by Herbert A. Simon and the other one in 2007 by
Albert-Lszl Barabsi1. Simons basic assertion is that a large portion of the
complex systems observed in nature has hierarchical structure, which has a
property, near-decomposability, that greatly simplifies understanding their behavior
and its description and, furthermore, facilitates understanding how to store the
information required to describe or reproduce the system in a relatively short
space.
From another point of view, Simons central argument is that these
hierarchical systems have common properties (in particular, near-decomposability)
that are independent of their specific content, which means that a wide diversity of
complex systems shows hierarchical structure. Thus, hierarchy is one of the crucial
structural patterns in the architecture of complexity.
On the other hand, in his paper, Barabsi proposes that, in complex
systems, interactions form networks. After exposing several ideas related to this
topic, he states that network theory is not an approximation for a theory of
complexity, but addresses the emergence and structural evolution of the frame of a
complex system, that is, of its architecture. He also notes that it is not enough to
understand the global behavior of a complex system but that, additionally, it is
imperative to understand the nature of the dynamic processes taking place in the
network and asserts that many phenomena that involve humans behave in a way
described by power law functions.
Since both authors develop their approach to the architecture of complexity
by referring to complex systems, the first section of this paper deals with this
important concept, around which a whole area of research has been building up
since the last third of the last century. In the second section, the main ideas and
conclusions raised by Simon with respect to the hierarchical structure of complex
systems are examined so that, in the third section, the same is done with
1 Quotations with no specific reference were drawn from these two articles.
1
Barabsis ideas and the network structure of this type of systems. The essay
concludes with a section of conclusions and outstanding issues.
Complex systems
Many authors agree that there is no universally accepted definition of complexity
or complex systems and so, their work is based on approximations. Simon, without
proposing a formal definition, assumes that a complex system is a set of a large
number of elements that interact in a non-simple way, in the sense that inferring
the properties of the whole is not a trivial matter.
Melanie Mitchell, in a description of complex systems that includes the
network concept used by Barabsi, considers them as "large networks of
components with no central control and simple rules of operation [that] give rise to
complex collective behavior, sophisticated information processing and adaptation
via learning or evolution "(Mitchell, 2009, 13).
Meanwhile, Alejandro Gallardo Cano (2012) describes complex reality as
"phenomena, processes or complex systems, [considering] a lot of elements linked
by dense networks of relationships, showing novel emergency phenomena, arising
from a rare combination of causal and teleological interactions and causing, in turn,
constant instability in the system they belong to. 'Clouds' [networks] of elements or
agents permanently lingering at a precarious balance between order and chaos.
Sets - many of them of a different nature - interacting with each other in multiple
ways and with divergent purposes, persisting due to internal processes of self-
information and self-organization, and that, as a consequence of all these aspects,
behave in novel and unpredictable ways and... all this cocktail is precisely the
intimate composition of complex reality".
2 For further details on the different views of complexity nowadays, see Daz Mata, Alfredo (2012)
2
On the other hand, by the time Simons article was published in 1962, General
System Theory had already begun its strong presence, condensed a few years
later in 1968, in a classic text with that title, written by Ludwig von Bertalanffy,
whom Simon quotes in his articles first footnote as one of the leading exponents of
this general system theory. In this book, Bertalanffy, although not specifically
speaking of complex systems, writes about some of the features currently
associated with them - like some of those mentioned in the preceding paragraph -
or related to them: self-regulation processes, nonlinear interactions, self-guided
systems and organisms, emergency, open system theory, hierarchies of organized
wholes, law isomorphism, interdisciplinarity and network theory.
Furthermore, Bertalanffy cites themes and authors, himself included, that are now
considered, among others, important contributors to the process of constructing a
theory of complex systems, which is itself still immersed in the process, such as
those mentioned by Simon and Barabsi: Norbert Wiener , with his book
Cybernetics, published in 1948, Claude Shannon and Warren Weaver, with the
mathematical theory of communication, formalized in a book published in 1949,
and John von Neumann and Oskar Morgenstern with a text on game theory which
saw the light in 1947. Additionally, the presently most common computer
architecture is named after von Neumann, an honor that comes from his
participation in the ENIAC (Electronic Numeric Integrator And Computer) project,
the first fully electronic computer, with no moving parts and the direct ancestor of
today's computers.
An interesting part of the title match of the two articles on the architecture of
complexity is that they help linking, as is done in the preceding paragraphs, various
topics included currently in complexity and complex systems theories and that
were already present in academia and science since the mid 1900s, as evidenced
by Simons article and Bertalanffy's book which, incidentally, is an important
reference when the analysis of systems is involved.
3
Herbert A. Simon and hierarchical complex systems
Simon won the 1978 Nobel Prize in Economics "for his pioneering research into
the decision-making process within economic organizations" 3. From the approach
he makes to complex systems, which is cited above, this author addresses four
issues in his paper. The first analyzes that very frequently complexity takes the
form of hierarchy. The second idea deals theoretically with the relationship between
the hierarchical structure of a complex system and the time it takes it to emerge
through evolutionary processes and, specifically, he argues that hierarchical
systems evolve much more quickly than those who are not. In the third section,
Simon explores the dynamic properties of hierarchically organized systems and the
way they can be decomposed to analyze them. Specifically, in this third part, he
explains the feature of near-decomposability of complex systems that is central to
his argument. Finally, in Part Four, he analyzes the relationship between complex
systems and their descriptions.
Hierarchies
Many complex systems are integrated by hierarchically interrelated subsystems,
until a lowest level of elementary basic subsystem is reached but this, on the other
hand, does not necessarily imply a subordinate relationship. Simon refers to social,
biological, physical and symbolic hierarchical systems, emphasizing that hierarchy
should be defined in terms of the intensity of interaction among elements, rather
than in terms of spatial proximity. In particular, he notes that social hierarchies are
described in terms of the interrelationships of individuals.
Evolving systems
When discussing the evolution of complex systems, and based on an example,
Simon states that "the time required for the evolution of a complex form from
simple elements depends critically on the numbers and distribution of potential
intermediate stable forms". This means that hierarchical systems include
4
subsystems that are stable intermediate forms that facilitate (and shorten) the
process of moving from the simple to the complex, somewhat like stages in which
a reorganization takes place, in a stop in its way to another reorganization at a
higher stage of complexity. These stages allow for the condensation of forces and
inertia, a recomposition of achievements, that act as stepping stones for those that
follow.
5
Figure 1. Figurative illustration of a hierarchy of components with subcomponents
Herbert Simon summarizes the main theoretical findings obtained from his
approach to hierarchical nearly decomposable systems in two propositions:
"(a) in a nearly decomposable system, the short-run behavior of each of the
component subsystems is approximately independent of the short-run behavior of
6
the other components; (b) in the long run, the behavior of any one of the
components depends in only an aggregate way on the behavior of the other
components. "
From the above, it is clear that systems that can be represented by a nearly
decomposable matrix are necessarily hierarchical systems including Leontieffs
input-output matrix of the economy and various physicochemical systems.
Regarding social systems, Simon states that, in their dynamics, "where members
of a system communicate with, and influence, other members, the near
decomposability is generally very prominent." Furthermore, it appears that, as is
the case in physical systems, in social systems, the higher frequency dynamics are
associated with subsystems and low frequency dynamics with larger systems.
7
First, it is possible to simplify the description of complex systems using the
redundancy of the original structure, which can be displayed using its hierarchical
structure. A second way to simplify the description, for the common case where the
systems are nearly decomposable, is to include only aggregate properties in the
description of the interaction between parts. A third way to simplify this description
is "recoding" the existing redundancy, through a suitable "recoding" of that
redundancy.
Thus, the complexity or simplicity of a structure depends on the way you can
describe it and, since many real complex structures are highly redundant, you can
use this redundancy to simplify the description, as long as the adequate
representation has been found. In the natural sciences, the dynamic laws,
expressed in the form of systems of differential or difference equations have
8
offered, in a very large number of cases, the key to simple dynamic description of
the complex.
At the end of this brief history, he suggests that, in the midst of these scientific
advances (and many others), until relatively recently there were no large maps of
networks available so all that could be done was to speculate about the structure of
real networks, until this situation changed dramatically in the late 1990s, due to the
Internet. With this large group of interconnected computers, it became possible to
obtain maps of interconnected documents on the World Wide Web (WWW),
Hollywood actors linked by movies, or metabolites linked by reactions. "These
maps not only catalyzed the emergence of network science, but forced on it a
methodology involving simultaneous data collection, model building and analytical
work." (Barabsi, 2005, 69)
9
Barabsis fundamental statement is based on the existence of many real
networks that he identifies as scale-free networks, which have the following
properties that distinguishes them from the "classical" model of random networks,
the Erds-Rnyi model:
P ( k ) k
For example, using a robot (Web crawler) to surf the World Wide Web and collect
link data, it was found that P(k) follows a power law distribution, described by the
above expression with out 2.45 and 2.1 . Figure 2, illustrates the behavior
of this power law distribution when =2.45 . Random network theory would
predict that the probability P(k) that a WWW page selected at random has exactly
k output or input links follows a Poisson distribution and not the power law that can
be seen in Figure 2.
Citing as examples the WWW case cited above, a network of Hollywood actors
and the power grid of the West of the United States, Barabsi concludes that many
large random networks share the common feature that the distribution of their local
connectivity is not a Poisson distribution, but follows a power law, for large k, with
exponent between 2.1 and 4 (Barabsi, 1999, 510).
10
2. Freedom of scale. The scale of a random network is established by the average
of links for all nodes, because it is assumed that most of the nodes have
approximately the same number of links; that average characterizes the network.
For networks that follow a power law distribution, with P ( k ) k , and <3, the
second moment of the distribution (from which the variance is obtained) diverges
(goes to infinity), which means that the average is not characteristic. These
networks lack a characteristic scale, which is why they are called scale-free.
The fact that many real networks of various types are scale-free indicates
that drastically different networks share similar organizational principles, such as
those proposed by Barabsi.
3. Presence of hubs. Scale-free networks have many nodes with few links and
relatively few nodes with an extraordinarily large number of connections (few
nodes with high degree). These important nodes, called "hubs" greatly affect the
overall performance of the network, and they do make the network robust against
accidental (random) failures but vulnerable to coordinated attacks.
The topology of scale-free networks like the Internet implies that they cannot be
split into isolated pieces by removing random nodes. This great strength regarding
11
random failures is integrated into the inhomogeneous topology of scale-free
networks. However, this same inhomogeneity makes them especially vulnerable to
targeted attacks. "In fact, while the Internet is not expected to break under the
random failure of routers and links, well-informed hackers can easily handicap the
network by targeting hubs for attacks".
k
( k )=
i k i
The discovery and classification of these growth mechanisms not only led to
detailed models of specific networks but also to the development of techniques to
predict large-scale topology of a wide range of real networks, to the reproduction of
observations and to allow accurate predictions for the scale exponents (the ones
that measure the number of connections of the nodes) and its dependence on
microscopic parameters (growth rate of the network, attachment rule). (Barabsi,
2005, 69)
12
5. Self-organization. The results of numerous studies indicate that large networks
self-organize into a scale-free state, a feature that none of the existing models
predict. (Barabsi, 1999, 510). "Stationary scale-free distributions... indicate that
the development of large networks is governed by robust self-organizing
phenomena that go beyond the particulars of the individual systems." (Barabsi,
1999, 509)
Ricard Sol (2009, 48), makes an interesting observation regarding this matter:
"Thus, unwanted phenomena such as the spread of a pandemic or the
amplification of economic instabilities will easily make its way through social or
trading networks... On the other hand, this ability to propagate modifications also
13
tells us that our actions, even if only at a small scale, can reach the whole system.
Our networked society makes it increasingly possible for small changes to trigger
large effects. Each of us, properly using what networks offer, can participate in
history and its course." This may explain, at least partially, the sudden success of
movements like Occupy Wall Street.
Barabsi uses networks involving human beings, such as social, technological and
economic networks, to describe recent advances in the quantitative understanding
of human dynamics, which is driven by the collective dynamics (interactions) of
individual human actions. This author presents empirical evidence to show that the
timing of many human activities, which are patterns ranging from communication to
entertainment and work, show statistics that follow a power law distribution and not
the Poisson distribution, as is traditionally assumed for random networks. Those
statistics are characterized by rapidly occurring bursts of events separated by long
periods of inactivity. According to Barabsi, this bursty nature of human behavior
is due to a decision based, waiting line type of process. He mentions two human
14
activity patterns that provide evidence of power law distribution: the email
communication from a data set that captures the sender, receiver, time and the
size of each post, and the pattern of Einstein communication based on his letters,
which follow approximately a power law4.
The wide range of human activity patterns following power law statistics suggest
that the bursty nature observed in these patterns reflects a fundamental and
potentially generic characteristic of human dynamics.
x
e
P (x )=
x!
And, when x = 0
e 0
P ( 0 )= =e
0!
P ( 0 )=e
This last function is plotted in Figure 3, and it has the same form of the power law
expression considered earlier,
P ( k ) k
So it is no surprise that the graphs of both expressions are practically identical .
4 Barabsi provides more examples of this behavior in his 2009 paper, on page 413.
15
Figure 3. Plot of P ( 0 )=e
1.0000
0.9000
0.8000
0.7000
0.6000
0.5000
0.4000
0.3000
0.2000
0.1000
0.0000
1 6 1 1 16 2 1 26 3 1 36 4 1 46 5 1 5 6 61 6 6 71 7 6 81 8 6 91 96
16
and that both approaches (and both authors) are important contributors to the
construction of a theory of complex systems - still in process - that "Over the last
decade have sparked a stampede... of complex systems researchers to create
what has been called the "new science of networks'" (Mitchell, 2009, 229).
And, from Simons and Barabsis approaches to the architecture of
complexity, hierarchical networks emerge from the fact that many real and
drastically different networks of various types share similar organizational
principles: freedom of scale-free, hierarchical structure and near decomposability,
among others.
Also, the self-organizing process of many hierarchical networks that makes
them scale free appears to be related to their sel
It is also easy to visualize that scale-free networks are very hierarchical in the
sense that hubs belong to the highest hierarchical level of nodes, if hierarchy in
terms of the amount of links is considered. One has yet to explore ways in which it
may be possible to use Simons properties of hierarchy, particularly near-
decomposability, to simplify the analysis of hierarchical scale-free networks.
17
the morphology of organisms (and no new species emerging) punctuated by
(relatively) short periods of large change in morphology, resulting in the emergence
of new species "(Mitchell, 2009, 84-85). And Bertalanffy (1968/1976, 71) reports
intermediate hierarchies that facilitate the evolution of complex systems: "The
reason for the predominance of segregation in living nature appears to be due to
the fact that segregation into subordinate partial subsystems implies an increase in
the systems complexity".
Meanwhile, referring to the article by Herbert Simon 5, Rolando Garcia (2006, 80-
81) points out two principles that are characteristic of complex systems:
"arrangement of its elements by levels of organization [hierarchy] with their own
dynamics but interacting with each other; and an evolution that does no progress
via continuous development but by means of successive reorganizations". In this
last quotation, Garcia, to begin with, refers to the two propositions in which Simon
summarizes the theoretical findings that emerge from his analysis of nearly
decomposable hierarchical systems6, related to the different dynamics at different
hierarchical levels. Moreover, Garcia also favors the theory of evolution by
successive reorganizations, which Gould and Eldredge called "punctuated
equilibrium".
5 It should be noted that Rolando Garca writes about decomposable and non decomposable
systems and makes no reference to Simons nearly decomposable systems.
6 Page 7 of this paper.
7 Page 14 of this paper.
8 Page 8 of this paper.
18
Finally, it is worth mentioning another outstanding issue raised by Tesson (2006,
chap. 6), who says that traditional network theory, including the scale-free
networks analyzed by Barabsi, addresses networks as "nodal" entities,
emphasizing the relationships among nodes and excluding the flow that occurs in
these relationships ("flow management in tubular networks" as opposed to "nodal
networks as constructs"), as well as the environment in which these networks
operate: "there is no way to represent context inherent in a nodal network."
References
Barabsi, Albert-Lszl (1999), Emergence of Scaling in Random Networks,
Science, vol. 286, October 15, p. 509-512.
Barabsi, Albert-Lszl (2005), Taming complexity, Nature Physics, Vol. 1, p. 68-
70.
Barabsi, Albert-Lszl (2007), The Architecture of Complexity, IEEE Control
Systems Magazine, August, p. 33-42.
Barabsi, Albert-Lszl (2009), Scale-Free Networks: A Decade and Beyond,
Science, vol. 325, July 24, p. 412-413.
Bertalanffy, Ludwig von (1968/1976), Teora general de sistemas, Fondo de Cultura
Econmica, Mexico, DF
DAZ MATA, Alfredo (2012), Tres aproximaciones a la complejidad, Contadura y
Administracin, FCA, UNAM, vol. 57, num. 1, january-march, pp. 241-264.
GALLARDO CANO, Alejandro (2012) Para comprender la complejidad, in Daz
Mata, Alfredo (Coord.) El enfoque de la complejidad, diversas perspectivas,
UNAM, Mxico D.F.
GARCA, Rolando (2006), Sistemas complejos, Gedisa, Barcelona.
MITCHELL, Melanie (2009), Complexity, A Guided Tour, Oxford University Press,
New York.
RICHMOND, Samuel B. (1968), Operations Research for Management Decisions,
The Ronald Press Company, New York.
SIMON, Herbert A. (1962), The Architecture of Complexity, Proceedings of the
American Philosophical Society, vol. 106, num. 6, December, p. 467-482.
19
SIMON, Herbert A. (2002), "Near decomposability and the Speed of Evolution",
Industrial and Corporate Change, vol. 11, num. 3.
SOL, Ricard (2009), Redes complejas, Tusquets, Barcelona.
TESSON, Karen Jane (2006), Dynamic Networks. An interdisciplinary study of
network organization in biological and human social systems, thesis presented for
the degree of Doctor of Philosophy, University of Bath, Department of Psychology,
June 2006, www.jackwhitehead.com/teesonphd/006c5.pdf , April 12 , 2013.
He also notes that it is not enough to understand the global behavior of a complex
system but that, additionally, it is imperative to understand the nature of the
dynamic processes taking place in the network and asserts that many phenomena
that involve humans behave in a way described by power law functions
hierarchical systems evolve much more quickly than those who are not
20
This means that hierarchical systems include subsystems that are stable
intermediate forms that facilitate (and shorten) the process of moving from the
simple to the complex, somewhat like stages in which a reorganization takes place,
in a stop in its way to another reorganization at a higher stage of complexity. These
stages allow for the condensation of forces and inertia, a recomposition of
achievements, that act as stepping stones for those that follow.
This trial and error process, guided by selectivity, explains the way huge and
complicated problems can be solved, also through ever higher hierarchical levels
that represent intermediate solutions.
"The explanation for the ubiquity of the ND [nearly decomposable] property is that,
under the usual conditions of mutation and/or crossover and natural selection, ND
systems increase in fitness, and therefore reproduce, at a much faster rate than
systems that do not possess the ND property "
21
individuals. And it can be said that, in principle, this mechanism has applications
outside the field of biology, in many entities that evolve, such as organizations or
human societies.
22
it is still necessary to analyze the advances made in the study of the dynamic
processes taking place in these networks in order to understand the interactions
between topology and dynamics of networks.
This author presents empirical evidence to show that the timing of many human
activities, which are patterns ranging from communication to entertainment and
work, show statistics that follow a power law distribution and not the Poisson
distribution, as is traditionally assumed for random networks. Those statistics are
characterized by rapidly occurring bursts of events separated by long periods of
inactivity. According to Barabsi, this bursty nature of human behavior is due to a
decision based, waiting line type of process
The wide range of human activity patterns following power law statistics suggest
that the bursty nature observed in these patterns reflects a fundamental and
potentially generic characteristic of human dynamics.
23