What Is Random?: Chance and Order in Mathematics and Life

Book cover of What Is Random?

Edward Beltrami

What Is Random?

Chance and Order in Mathematics and Life

2nd ed. 2020

Logo of the publisher

Logo of the publisher

Edward Beltrami

Department Applied Mathematics and Statistics, Stony Brook University, Stony Brook, NY, USA

ISBN 978-1-0716-0798-5e-ISBN 978-1-0716-0799-2

https://doi.org/10.1007/978-1-0716-0799-2

Mathematics Subject Classification (2010): 60-01

© Springer Science+Business Media, LLC, part of Springer Nature 1999, 2020corrected publication2020

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Copernicus is part of Springer, an imprint published by Springer NatureThe registered company is Springer Nature Switzerland AG

The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

A Word About Notation

It is convenient to use a shorthand notation for certain mathematical expressions that appear often throughout the book. For any two numbers designated as a and b the product "a times b" is written as ab or equivalently a × b, while "a divided by b" is a/b. The product of "a multiplied by itself b times" is a b so that, for example, 2¹⁰ means 1024. The expression 2−n is synonymous with 1/2 n .

In a few places I use the standard notation $$ \sqrt{a} $$ to mean "the square root of a," as in $$ \sqrt{25}=5 $$ .

All "numbers greater than a and less than b" are expressed as (a, b). If greater than is replaced by greater than or equal to then the notation is [a, b).

A sequence of numbers, such as 53371…, is generally indicated by a 1 a 2 a 3… in which the subscripts 1, 2, 3, … indicate the first, second, third, and so on terms of the sequence, which can be finite in length or even infinite (such as the unending array of all even integers).

Preface

We all have memories of peering at a TV screen as myriad little balls churn about in an urn until a single candidate, a number inscribed on it, is ejected from the container. The hostess hesitantly picks it up and, pausing for effect, reads the lucky number. The winner thanks Lady Luck, modern descendant of the Roman goddess Fortuna, blind arbiter of good fortune. We, as spectators, recognize it as simply a random event and know that in other situations Fortuna’s caprice could be equally malicious, as Shirley Jackson’s dark tale The Lottery chillingly reminds us.

The Oxford Dictionary has it that a random outcome is one without perceivable cause or design, inherently unpredictable. But, you might protest, isn’t the world around us governed by rules, by the laws of physics? If that is so, it should be possible to determine the positions and velocities of each ball in the urn at any future time, and the uncertainty of which one is chosen would simply be a failing of our mind to keep track of how the balls are jostled about. The sheer enormity of possible configurations assumed by the balls overwhelms our computational abilities. But this would be only a temporary limitation: a sufficiently powerful computer could conceivably do that brute task for us, and randomness would thus be simply an illusion that can be dispelled. After thinking about this for a while you may begin to harbor a doubt. Although nature may have its rules, the future remains inherently unknowable because the positions and velocities of each ball can never really be ascertained with complete accuracy.

The first view of randomness is of clutter bred by complicated entanglements. Even though we know there are rules, the outcome is uncertain. Lotteries and card games are generally perceived to belong to this category. More troublesome is that nature’s design itself is known imperfectly, and worse, the rules may be hidden from us, and therefore we cannot specify a cause or discern any pattern of order. When, for instance, an outcome takes place as the confluence of totally unrelated events, it may appear to be so surprising and bizarre that we say that it is due to blind chance. Jacques Monod, in his book Chance and Necessity, illustrates this by the case of a man hurrying down a street in response to a sudden phone call at the same time that a roof worker accidentally drops a hammer that hits the unfortunate pedestrian’s head. Here we have chance due to contingency, and it doesn’t matter whether you regard this as an act of divine intervention operating according to a predestined plan or as an unintentional accident. In either case the cause, if there is one, remains indecipherable.

Randomness is the very stuff of life, looming large in our everyday experience. Why else do people talk so much about the weather, traffic, and the financial markets? Although uncertainty may contribute to a sense of anxiety about the future, it is also our only shield against boring repetitiveness. As I argue in a later chapter, chance provides the fortuitous accidents and capricious wit that gives life its pungency. It is important, therefore, to make sense of randomness beyond its anecdotal meanings. To do this we employ a modest amount of mathematics in this book to remove much of the vagueness that encumbers the concept of random, permitting us to quantify what would otherwise remain elusive. The mathematics also provides a framework for unifying how chance is interpreted from the diverse perspectives of psychologists, physicists, statisticians, computer scientists, and communication theorists.

In the first chapter, I tell the story of how, beginning a few centuries ago, the idea of uncertainty was formalized into a theory of chance events, known today as probability theory. Mathematicians adopt the convention that selections are made from a set of possible outcomes in which each event is equally likely, though unpredictable. Chance is then asked to obey certain rules that epitomize the behavior of a perfect coin or an ideal die, and from these rules one can calculate the odds. The Taming of Chance, as this chapter is called, supplies the minimal amount of probability theory needed to understand the law of large numbers and the normal law, which describe in different ways the uncanny regularity of large ensembles of chance events. With these concepts in hand, we arrive at our first tentative answer to the question what is random?

To crystalize our thinking, most of the book utilizes the simplest model of a succession of random events, namely a binary sequence (a string of zeros and ones). Although this may seem almost like a caricature of randomness, it has been the setting for some of the most illuminating examples of the workings of chance from the very beginnings of the subject three centuries ago to the present.

If some random mechanism generates a trail of ten zeros and ones, let us say, then there are 2 to the power 10, namely 1024, different binary strings that are possible. One of the first conundrums to emerge is that under the assumption that each string is equally likely, there is a (very) small possibility that the string generated consists of ten zeros in succession, something that manifestly is not random by any intuitive notion of what random means. So it is necessary to distinguish between a device that operates in a haphazard manner to spew forth digits without rhyme or reason, a random process, and any particular realization of the output that such a process provides. One may argue that the vagaries of chance pertain to the ensemble of possibilities and not to an individual outcome. Nevertheless, a given binary string, such as 0101010101010101, may appear so orderly that we feel compelled to deny its randomness, or it may appear so incoherent, as in the case of 0011000010011011, that we yearn to call it random regardless of its provenance. The last situation conforms to what I mentioned earlier, namely that random is random even though the cause, if any, is unknown. In keeping with this idea the emphasis will shift away from the generating process in succeeding chapters and focus instead on the individual outcomes. This change in outlook parallels the actual shift that has taken place in the last several decades among a broad spectrum of thinkers on the subject, in contrast to the more traditional views that have been central over the last few centuries (and that are discussed in the first chapter). The chapter closes with a new topic for this edition, conditional probability, with Bayes’ Theorem developed in the Technical Notes.

In Uncertainty and Information, the second chapter, I introduce the notion of information formulated by Claude Shannon nearly three quarters of a century ago, since this gives us an additional tool for discussing randomness. Here one talks of information bits and entropy, redundancy, and coding, and this leads me to pose a second test to resolve the question what is random? The idea is that if some shorter binary string can serve as a code to generate a longer binary message, the longer message cannot be random since it has been compressed by removing some of the redundancies within the string. One application concerns the perception of randomness by people in general, a topic much studied by psychologists.

The third chapter, Janus-Faced Randomness, introduces a curious example of a binary sequence that is random in one direction but deterministic when viewed in reverse. Knowledge of the past and uncertainty about the future seem to be two faces of the same Janus-faced coin, but more careful scrutiny establishes that what appears as predictable is actually randomness in disguise. I establish that the two faces represent a tradeoff between ignorance now and disorder later. Though there are randomly generated strings that do not appear random, it now emerges that strings contrived by deterministic rules may behave randomly. To some extent, randomness is in the eye of the beholder. Just because you do not perceive a pattern doesn’t mean there isn’t one. The so-called random-number generators that crop up in many software packages are of this ilk, and they give support to the interpretation of chance as a complex process, like balls in an urn, which only appear random because of the clutter.

The chapter continues with a brief discussion of the early days of the study of thermodynamics in the nineteenth century, when troublesome questions were raised about the inexorable tendency of physical systems to move from order to disorder over time, in apparent contradiction to the laws of physics, which are completely reversible. I show that this dilemma is closely related to the question of binary strings that appear random in spite of being spawned by precise rules. We close with a new topic for this edition, a brief discussion of how uncertainty emerges as ineluctable randomness in quantum theory.

In the fourth chapter Algorithms, Information, and Chance, I follow the mathematicians Andrei Kolmogorov and Gregory Chaitin who say that a string is random if its shortest description is obtained by writing it out in its entirety. There is, in effect, no pattern within the string that would allow it to be compressed. More formally, one defines the complexity of a given string of digits to be the length, in binary digits, of the shortest string (i.e., the shortest computer program written in binary form) that generates the successive digits. Strings of maximum complexity are called random when they require programs of about the same length as the string itself. Although these ideas echo those of Chapter 2, they are now formulated in terms of algorithms implemented on computers and the question what is random? takes on a distinctly different cast.

Godel’s celebrated incompleteness theorem in the version rendered by Alan Turing states that it may not be possible to determine whether a universal computer, namely one that can be programmed to carry out any computation whatever, will ever halt when it is fed a given input. Chaitin has reinterpreted this far-reaching result by showing that any attempt to decide the randomness of a sufficiently long binary string is inherently doomed to failure; his argument is reproduced in this chapter.

The penultimate chapter is more speculative. In The Edge of Randomness, I review recent work by a number of thinkers that suggests that naturally occurring processes seem to be balanced between tight organization, where redundancy is paramount, and volatility, in which little order is possible. One obtains a view of nature and the arts and the world of everyday affairs as evolving to the edge of these extremes, allowing for a fruitful interplay of chance and necessity, poised between surprise and inevitability. Fortuitous mutations and irregular natural disturbances, for example, appear to intrude on the more orderly processes of species replication, providing evolution with an opportunity for innovation and diversity.

To illustrate this concept in a particular setting, I include a brief and self-contained account of naturally occurring events that display similar patterns at different scales; they satisfy what is known as power laws and describe processes in which there is a high frequency of small events interspersed with a few large magnitude occurrences. This is reminiscent of the order exhibited by chance events in the large as discussed in the first chapter. However, now the prediction of any individual occurrence remains inscrutable because it is contingent on the past history of the process. The emphasis on power laws is new to this edition.

The last chapter, Fooled by Chance, looks at some entertaining puzzles and surprising inferences that spring from chance in unexpected ways, spotlighting the often counter-intuitive consequences of randomness in everyday life. Up until now this has not been the focus of the present work but it seemed appropriate for me to include this diversion. A powerful mathematical tool, the Poisson distribution, is also introduced in this chapter to provide a fresh insight into the workings of chance. The entire chapter is a further add-on for the second edition.

The book is intended to provoke, entertain, and inform by challenging the reader’s ideas about randomness, providing first one and then another interpretation of what this elusive concept means. As the book progresses, I tease out the various threads and show how mathematics, communication engineering, computer science, philosophy, physics, and psychology all contribute to the discourse by illuminating different facets of the same idea.

The material in the book should be readily accessible to anyone with a smattering of college mathematics, no calculus needed. I provide simple numerical examples throughout, coded in MATLAB, to illustrate the various iterative processes and binary sequences that crop up. Three appendices provide some of the background information regarding binary representations and logarithms that are needed here and there, but I keep this as elementary as possible. Although an effort is made to justify most statements of a mathematical nature, a few are presented without corroboration, since they entail close-knit arguments that would detract from the main ideas. You can safely bypass the details without any loss and, in any case, the fine points are available in the Technical Notes assembled at the end.

The current second edition is a revision of