P. 1
Random password generators: High-impact Strategies - What You Need to Know: Definitions, Adoptions, Impact, Benefits, Maturity, Vendors

Random password generators: High-impact Strategies - What You Need to Know: Definitions, Adoptions, Impact, Benefits, Maturity, Vendors

|Views: 1,382|Likes:
Published by Emereo Publishing
A random password generator is software program or hardware device that takes input from a random or pseudo-random number generator and automatically generates a password. Random passwords can be generated manually, using simple sources of randomness such as dice or coins, or they can be generated using a computer.

While there are many examples of "random" password generator programs available on the Internet, generating randomness can be tricky and many programs do not generate random characters in a way that ensures strong security. A common recommendation is to use open source security tools where possible, since they allow independent checks on the quality of the methods used. Note that simply generating a password at random does not ensure the password is a strong password, because it is possible, although highly unlikely, to generate an easily guessed or cracked password.

A password generator can be part of a password manager. When a password policy enforces complex rules, it can be easier to use a password generator based on that set of rules than to manually create passwords.

This book is your ultimate resource for Random password generators. Here you will find the most up-to-date information, analysis, background and everything you need to know.

In easy to read chapters, with extensive references and links to get you to know all there is to know about Random password generators right away, covering: Random password generator, Password, 1dl, 2D Key, ATM SafetyPIN software, Canonical account, Challenge-Handshake Authentication Protocol, Challenge-response authentication, Cognitive password, Default password, Diceware, Draw a Secret, Duress code, LM hash, Munged password, One-time password, OpenID, OTPW, Partial Password, Passmap, PassPattern system, Passphrase, Password authentication protocol, Password cracking, Password fatigue, Password length parameter, Password management, Password manager, Password notification e-mail, Password policy, Password strength, Password synchronization, Password-authenticated key agreement, PBKDF2, Personal identification number, Pre-shared key, Privileged password management, Risk-based authentication, S/KEY, Secure Password Authentication, Secure Remote Password protocol, SecurID, Self-service password reset, Shadow password, Single sign-on, Swordfish (password), Windows credentials, Zero-knowledge password proof, Bach's algorithm, Barrett reduction, BB84, Beaufort cipher, Block cipher modes of operation, CDMF, Ciphertext stealing, Common Scrambling Algorithm, CryptGenRandom, Crypto++, Cryptographically secure pseudorandom number generator, Cycles per byte, Feedback with Carry Shift Registers, Feige-Fiat-Shamir Identification Scheme, Generating primes, GGH encryption scheme, Hash chain, HOTP, Industrial-grade prime, ISMACryp, JOSEKI (cipher), Key schedule, Key Wrap, Kochanski multiplication, KR advantage, Linear feedback shift register, Mental poker, Modular exponentiation, Montgomery reduction, MOSQUITO, Pairing-based cryptography, Randomness extractor, RC algorithm, Residual block termination, Rip van Winkle cipher, Schoof's algorithm, Secret sharing using the Chinese remainder theorem, SecureLog, Shamir's Secret Sharing, Snuffle, Substitution-permutation network, Summation generator, Symmetric-key algorithm, Time-based One-time Password Algorithm, Type 1 product, Type 2 product, Type 3 product, Type 4 product, Verifiable random function.

This book explains in-depth the real drivers and workings of Random password generators. It reduces the risk of your technology, time and resources investment decisions by enabling you to compare your understanding of Random password generators with the objectivity of experienced professionals.
A random password generator is software program or hardware device that takes input from a random or pseudo-random number generator and automatically generates a password. Random passwords can be generated manually, using simple sources of randomness such as dice or coins, or they can be generated using a computer.

While there are many examples of "random" password generator programs available on the Internet, generating randomness can be tricky and many programs do not generate random characters in a way that ensures strong security. A common recommendation is to use open source security tools where possible, since they allow independent checks on the quality of the methods used. Note that simply generating a password at random does not ensure the password is a strong password, because it is possible, although highly unlikely, to generate an easily guessed or cracked password.

A password generator can be part of a password manager. When a password policy enforces complex rules, it can be easier to use a password generator based on that set of rules than to manually create passwords.

This book is your ultimate resource for Random password generators. Here you will find the most up-to-date information, analysis, background and everything you need to know.

In easy to read chapters, with extensive references and links to get you to know all there is to know about Random password generators right away, covering: Random password generator, Password, 1dl, 2D Key, ATM SafetyPIN software, Canonical account, Challenge-Handshake Authentication Protocol, Challenge-response authentication, Cognitive password, Default password, Diceware, Draw a Secret, Duress code, LM hash, Munged password, One-time password, OpenID, OTPW, Partial Password, Passmap, PassPattern system, Passphrase, Password authentication protocol, Password cracking, Password fatigue, Password length parameter, Password management, Password manager, Password notification e-mail, Password policy, Password strength, Password synchronization, Password-authenticated key agreement, PBKDF2, Personal identification number, Pre-shared key, Privileged password management, Risk-based authentication, S/KEY, Secure Password Authentication, Secure Remote Password protocol, SecurID, Self-service password reset, Shadow password, Single sign-on, Swordfish (password), Windows credentials, Zero-knowledge password proof, Bach's algorithm, Barrett reduction, BB84, Beaufort cipher, Block cipher modes of operation, CDMF, Ciphertext stealing, Common Scrambling Algorithm, CryptGenRandom, Crypto++, Cryptographically secure pseudorandom number generator, Cycles per byte, Feedback with Carry Shift Registers, Feige-Fiat-Shamir Identification Scheme, Generating primes, GGH encryption scheme, Hash chain, HOTP, Industrial-grade prime, ISMACryp, JOSEKI (cipher), Key schedule, Key Wrap, Kochanski multiplication, KR advantage, Linear feedback shift register, Mental poker, Modular exponentiation, Montgomery reduction, MOSQUITO, Pairing-based cryptography, Randomness extractor, RC algorithm, Residual block termination, Rip van Winkle cipher, Schoof's algorithm, Secret sharing using the Chinese remainder theorem, SecureLog, Shamir's Secret Sharing, Snuffle, Substitution-permutation network, Summation generator, Symmetric-key algorithm, Time-based One-time Password Algorithm, Type 1 product, Type 2 product, Type 3 product, Type 4 product, Verifiable random function.

This book explains in-depth the real drivers and workings of Random password generators. It reduces the risk of your technology, time and resources investment decisions by enabling you to compare your understanding of Random password generators with the objectivity of experienced professionals.

More info:

Published by: Emereo Publishing on Aug 02, 2011
Copyright:Traditional Copyright: All rights reserved
List Price: $39.95


Read on Scribd mobile: iPhone, iPad and Android.
This book can be read on up to 6 mobile devices.
Full version available to members
See more
See less



  • Random password generator
  • Password
  • 2D Key
  • ATM SafetyPIN software
  • Canonical account
  • Challenge-Handshake Authentication Protocol
  • Challenge-response authentication
  • Cognitive password
  • Default password
  • Diceware
  • Draw a Secret
  • Duress code
  • LM hash
  • Munged password
  • One-time password
  • OpenID
  • OTPW
  • Partial Password
  • Passmap
  • PassPattern system
  • Passphrase
  • Password authentication protocol
  • Password cracking
  • Password fatigue
  • Password length parameter
  • Password management
  • Password manager
  • Password notification e-mail
  • Password policy
  • Password strength
  • Password synchronization
  • Password-authenticated key agreement
  • PBKDF2
  • Personal identification number
  • Pre-shared key
  • Privileged password management
  • Risk-based authentication
  • S/KEY
  • Secure Password Authentication
  • Secure Remote Password protocol
  • SecurID
  • Self-service password reset
  • Shadow password
  • Single sign-on
  • Swordfish (password)
  • Windows credentials
  • Zero-knowledge password proof
  • Randomness
  • randomness
  • Algorithmic information theory
  • Algorithmically random sequence
  • Applications of randomness
  • Bernoulli stochastics
  • Biology Monte Carlo method
  • Clock drift
  • Control variates
  • Determinism
  • /dev/random
  • Dice
  • Diehard tests
  • Differential entropy
  • Entropy (information theory)
  • Entropy estimation
  • Fisher–Yates shuffle
  • Global Consciousness Project
  • Hardware random number generator
  • History of randomness
  • Ignorance space
  • Indeterminacy (philosophy)
  • Indeterminacy in philosophy
  • Infinite monkey theorem
  • Edward Kofler
  • Lavarand
  • LavaRnd
  • Linear partial information
  • Mendelian randomization
  • A Million Random Digits with 100,000 Normal Deviates
  • A Million Random Digits with 100,000 Normal Deviates is a 1955
  • Monte Carlo method
  • Nothing up my sleeve number
  • Philosophical interpretation of classical physics
  • Physical Unclonable Function
  • Random binary tree
  • Random compact set
  • Random number generation
  • Random number table
  • Random permutation
  • Random sample
  • Random sequence
  • Random stimulus
  • Random variable
  • Random variate
  • Randomization
  • Randomizer
  • Randomness extractor
  • Randomness tests
  • Seven states of randomness
  • Shuffle play
  • Shuffling
  • Shuffling machine
  • Spinner (game)
  • Subrandom numbers
  • From random numbers
  • Two-stage model of free will
  • Yao's principle
  • Bach's algorithm
  • Bach's algorithm[1]
  • Barrett reduction
  • BB84
  • Beaufort cipher
  • Block cipher modes of operation
  • CDMF
  • Ciphertext stealing
  • Common Scrambling Algorithm
  • CryptGenRandom
  • Crypto++
  • Cryptographically secure pseudorandom number generator
  • Cycles per byte
  • Feedback with Carry Shift Registers
  • Feige-Fiat-Shamir Identification Scheme
  • Generating primes
  • GGH encryption scheme
  • Hash chain
  • HOTP
  • Industrial-grade prime
  • ISMACryp
  • JOSEKI (cipher)
  • Key schedule
  • Key Wrap
  • Kochanski multiplication
  • Kochanski multiplication[1]
  • KR advantage
  • Linear feedback shift register
  • A linear feedback shift register
  • Mental poker
  • Modular exponentiation
  • Montgomery reduction
  • Pairing-based cryptography
  • RC algorithm
  • Residual block termination
  • Rip van Winkle cipher
  • Schoof's algorithm
  • Secret sharing using the Chinese remainder theorem
  • SecureLog
  • Shamir's Secret Sharing
  • Snuffle
  • Substitution-permutation network
  • Summation generator
  • Symmetric-key algorithm
  • Time-based One-time Password Algorithm
  • Type 1 product
  • Type 2 product
  • Type 3 product
  • Type 4 product

Random password

High-impact Strategies - What You Need to Know:
Definitions, Adoptions, Impact, Benefits, Maturity, Vendors
Kevin Roebuck
A random password generator is software program or hardware device that takes input from a random or
pseudo-random number generator and automatically generates a password. Random passwords can be
generated manually, using simple sources of randomness such as dice or coins, or they can be generated
using a computer.
While there are many examples of “random” password generator programs available on the Internet, gen-
erating randomness can be tricky and many programs do not generate random characters in a way that en-
sures strong security. A common recommendation is to use open source security tools where possible, since
they allow independent checks on the quality of the methods used. Note that simply generating a password
at random does not ensure the password is a strong password, because it is possible, although highly un-
likely, to generate an easily guessed or cracked password.
A password generator can be part of a password manager. When a password policy enforces complex rules,
it can be easier to use a password generator based on that set of rules than to manually create passwords.
This book is your ultimate resource for Random password generators. Here you will find the most up-to-
date information, analysis, background and everything you need to know.
In easy to read chapters, with extensive references and links to get you to know all there is to know about
Random password generators right away, covering: Random password generator, Password, 1dl, 2D Key,
ATM SafetyPIN software, Canonical account, Challenge-Handshake Authentication Protocol, Challenge-
response authentication, Cognitive password, Default password, Diceware, Draw a Secret, Duress code, LM
hash, Munged password, One-time password, OpenID, OTPW, Partial Password, Passmap, PassPattern sys-
tem, Passphrase, Password authentication protocol, Password cracking, Password fatigue, Password length
parameter, Password management, Password manager, Password notification e-mail, Password policy,
Password strength, Password synchronization, Password-authenticated key agreement, PBKDF2, Personal
identification number, Pre-shared key, Privileged password management, Risk-based authentication, S/KEY,
Secure Password Authentication, Secure Remote Password protocol, SecurID, Self-service password reset,
Shadow password, Single sign-on, Swordfish (password), Windows credentials, Zero-knowledge password
proof, Bach’s algorithm, Barrett reduction, BB84, Beaufort cipher, Block cipher modes of operation, CDMF,
Ciphertext stealing, Common Scrambling Algorithm, CryptGenRandom, Crypto++, Cryptographically secure
pseudorandom number generator, Cycles per byte, Feedback with Carry Shift Registers, Feige-Fiat-Shamir
Identification Scheme, Generating primes, GGH encryption scheme, Hash chain, HOTP, Industrial-grade
prime, ISMACryp, JOSEKI (cipher), Key schedule, Key Wrap, Kochanski multiplication, KR advantage, Linear
feedback shift register, Mental poker, Modular exponentiation, Montgomery reduction, MOSQUITO, Pairing-
based cryptography, Randomness extractor, RC algorithm, Residual block termination, Rip van Winkle
cipher, Schoof’s algorithm, Secret sharing using the Chinese remainder theorem, SecureLog, Shamir’s Se-
cret Sharing, Snuffle, Substitution-permutation network, Summation generator, Symmetric-key algorithm,
Time-based One-time Password Algorithm, Type 1 product, Type 2 product, Type 3 product, Type 4 product,
Verifiable random function
This book explains in-depth the real drivers and workings of Random password generators. It reduces the
risk of your technology, time and resources investment decisions by enabling you to compare your under-
standing of Random password generators with the objectivity of experienced professionals.


Topic relevant selected content from the highest rated entries, typeset, printed and
Combine the advantages of up-to-date and in-depth knowledge with the convenience of
printed books.
A portion of the proceeds of each book will be donated to the Wikimedia Foundation
to support their mission: to empower and engage people around the world to collect
and develop educational content under a free license or in the public domain, and to
disseminate it effectively and globally.
The content within this book was generated collaboratively by volunteers. Please be
advised that nothing found here has necessarily been reviewed by people with the
expertise required to provide you with complete, accurate or reliable information. Some
information in this book maybe misleading or simply wrong. The publisher does not
guarantee the validity of the information found here. If you need specifc advice (for
example, medical, legal, fnancial, or risk management) please seek a professional who is
licensed or knowledgeable in that area.
Sources, licenses and contributors of the articles and images are listed in the section
entitled “References”. Parts of the books may be licensed under the GNU Free
Documentation License. A copy of this license is included in the section entitled “GNU
Free Documentation License”
All used third-party trademarks belong to their respective owners.
Random password generator 1
Password 6
1dl 16
2D Key 17
ATM SafetyPIN software 17
Canonical account 19
Challenge-Handshake Authentication Protocol 20
Challenge-response authentication 21
Cognitive password 23
Default password 25
Diceware 26
Draw a Secret 27
Duress code 27
LM hash 29
Munged password 32
One-time password 33
OpenID 38
Partial Password 49
Passmap 49
PassPattern system 51
Passphrase 51
Password authentication protocol 54
Password cracking 55
Password fatigue 57
Password length parameter 59
Password management 59
Password manager 60
Password notification e-mail 61
Password policy 62
Password strength 65
Password synchronization 75
Password-authenticated key agreement 76
Personal identification number 81
Pre-shared key 84
Privileged password management 85
Risk-based authentication 87
S/KEY 88
Secure Password Authentication 92
Secure Remote Password protocol 92
SecurID 96
Self-service password reset 100
Shadow password 102
Single sign-on 105
Swordfish (password) 107
Windows credentials 109
Zero-knowledge password proof 109
Randomness 110
Algorithmic information theory 118
Algorithmically random sequence 122
Applications of randomness 126
Bernoulli stochastics 130
Biology Monte Carlo method 135
Clock drift 143
Control variates 145
Determinism 147
/dev/random 158
Dice 161
Diehard tests 174
Differential entropy 175
Entropy (information theory) 179
Entropy estimation 191
Fisher–Yates shuffle 193
Global Consciousness Project 200
Hardware random number generator 203
History of randomness 211
Ignorance space 218
Indeterminacy (philosophy) 219
Infinite monkey theorem 226
Edward Kofler 234
Lavarand 237
LavaRnd 238
Linear partial information 239
Mendelian randomization 241
A Million Random Digits with 100,000 Normal Deviates 243
Monte Carlo method 244
Nothing up my sleeve number 255
Philosophical interpretation of classical physics 256
Physical Unclonable Function 258
Random binary tree 262
Random compact set 265
Random number generation 266
Random number table 270
Random permutation 271
Random sample 272
Random sequence 273
Random stimulus 275
Random variable 276
Random variate 282
Randomization 283
Randomizer 284
Randomness extractor 285
Randomness tests 289
Seven states of randomness 290
Shuffle play 291
Shuffling 292
Shuffling machine 297
Spinner (game) 301
Subrandom numbers 302
Two-stage model of free will 305
Yao's principle 314
Bach's algorithm 316
Barrett reduction 317
BB84 317
Beaufort cipher 319
Block cipher modes of operation 320
CDMF 330
Ciphertext stealing 331
Common Scrambling Algorithm 335
CryptGenRandom 338
Crypto++ 342
Cryptographically secure pseudorandom number generator 346
Cycles per byte 349
Feedback with Carry Shift Registers 350
Feige-Fiat-Shamir Identification Scheme 351
Generating primes 352
GGH encryption scheme 353
Hash chain 355
HOTP 356
Industrial-grade prime 358
ISMACryp 359
JOSEKI (cipher) 360
Key schedule 360
Key Wrap 362
Kochanski multiplication 363
KR advantage 365
Linear feedback shift register 366
Mental poker 372
Modular exponentiation 376
Montgomery reduction 380
Pairing-based cryptography 384
RC algorithm 385
Residual block termination 385
Rip van Winkle cipher 386
Schoof's algorithm 386
Secret sharing using the Chinese remainder theorem 391
SecureLog 394
Shamir's Secret Sharing 396
Snuffle 399
Substitution-permutation network 400
Summation generator 401
Symmetric-key algorithm 402
Time-based One-time Password Algorithm 404
Type 1 product 405
Type 2 product 406
Type 3 product 406
Type 4 product 406
Verifiable random function 407
Article Sources and Contributors 408
Image Sources, Licenses and Contributors 417
Article Licenses
License 419
Random password generator
Random password generator
A random password generator is software program or hardware device that takes input from a random or
pseudo-random number generator and automatically generates a password. Random passwords can be generated
manually, using simple sources of randomness such as dice or coins, or they can be generated using a computer
While there are many examples of "random" password generator programs available on the Internet, generating
randomness can be tricky and many programs do not generate random characters in a way that ensures strong
security. A common recommendation is to use open source security tools where possible, since they allow
independent checks on the quality of the methods used. Note that simply generating a password at random does not
ensure the password is a strong password, because it is possible, although highly unlikely, to generate an easily
guessed or cracked password.
A password generator can be part of a password manager. When a password policy enforces complex rules, it can be
easier to use a password generator based on that set of rules than to manually create passwords.
The naive approach
Here are two code samples that a programmer who is not familiar with the limitations of the random number
generators in standard programming libraries might implement:
#include <time.h>
#include <stdio.h>
#include <stdlib.h>
/* Length of the password */
unsigned short int length = 8;
/* Seed number for rand() */
srand((unsigned int) time(0) + getpid());
/* ASCII characters 33 to 126 */
while(length--) {
putchar(rand() % 94 + 33);
In this case, the standard C function rand, which is a pseudo-random number generator, is initially seeded using the
C functions time and getpid, but later iterations use rand instead. According to the ANSI C standard, time returns a
value of ftype time_t, which is implementation defined, but most commonly a 32-bit integer containing the current
Random password generator
number of seconds since January 1, 1970 (see: Unix time), and getpid returns a pid t. There are about 31 million
seconds in a year, so an attacker who knows the year (a simple matter in situations where frequent password changes
are mandated by password policy) and the process ID that the password was generated with, faces a relatively small
number, by cryptographic standards, of choices to test. If the attacker knows more accurately when the password was
generated, he faces an even smaller number of candidates to test – a serious flaw in this implementation.
In situations where the attacker can obtain an encrypted version of the password, such testing can be performed
rapidly enough so that a few million trial passwords can be checked in a matter of seconds. See: password cracking.
The function rand presents another problem. All pseudo-random number generators have an internal memory or
state. The size of that state determines the maximum number of different values it can produce: an n-bit state can
produce at most different values. On many systems rand has a 31 or 32 bit state, which is already a significant
security limitation. Microsoft documentation does not describe the internal state of the Visual C++ implementation
of the C standard library rand, but it has only 32767 possible outputs (15 bits) per call. [1] Microsoft recommends a
different, more secure function, rand_s, be used instead. The output of rand_s is cryptographically secure, according
to Microsoft, and it does not use the seed loaded by the srand function. However its programming interface differs
from rand. [2]
function pass_gen($len) {
$pass = '';
srand((float) microtime() * 10000000);
for ($i = 0; $i < $len; $i++) {
$pass .= chr(rand(32, 126));
return $pass;
In the second case, the PHP function microtime
is used, which returns the current Unix timestamp with
microseconds. This increases the number of possibilities, but someone with a good guess of when the password was
generated, for example the date an employee started work, still has a reasonably small search space. Also some
operating systems do not provide time to microsecond resolution, sharply reducing the number of choices. Finally
the rand
function usually uses the underlying C rand function, and may have a small state space, depending on
how it is implemented. An alternative random number generator, mt_rand, which is based on the Mersenne Twister
pseudo random number generator, is available in PHP, but it also has a 32-bit state. There are proposals for adding
strong random number generation to PHP. [5]
Stronger methods
Some computer operating systems provide much stronger random number generators. One example, common on
most Unix platforms, is /dev/random. The Java programming language includes a class called SecureRandom
Windows programmers can use the Cryptographic Application Programming Interface function CryptGenRandom.
Another possibility, is to derive randomness by measuring some external phenomenon, such as timing user keyboard
input. Using random bytes from any of these sources should prove adequate for most password generation needs.
Random password generator
Here is a code sample that uses /dev/urandom to generate a password with a simple Bash function
function mkpw() { head /dev/urandom | uuencode -m - | sed -n 2p | cut
-c1-${1:-8}; }
The language Python includes a SystemRandom class that obtains cryptographic grade random bits from
/dev/urandom on a UNIX-like system, including Linux and Mac OS-X, while on Windows it uses

Here is a simple Python 2 script that demonstrates the use of this class:
import random, string
myrg = random.SystemRandom()
length = 10
alphabet = string.letters + string.digits
pw = str().join(myrg.choice(alphabet) for _ in range(length))
print pw
A PHP program can open and read from /dev/urandom, if available, or invoke the Microsoft utilities.
A third
option, if OpenSSL is available is to employ the function openssl_random_pseudo_bytes'.'
Mechanical methods
Yet another method is to use physical devices such as dice to generate the randomness. One simple way to do this
uses a 6 by 6 table of characters. The first die roll selects a row in the table and the second a column. So, for
example, a roll of 2 followed by a roll of 4 would select the letter "j" from the table below.
To generate
upper/lower case characters or some symbols a coin flip can be used, heads capital, tails lower case. If a digit was
selected in the dice rolls, a heads coin flip might select the symbol above it on a standard keyboard, such as the '$'
above the '4' instead of '4'.
1 2 3 4 5 6
1 a b c d e f
2 g h i j k l
3 m n o p q r
4 s t u v w x
5 y z 0 1 2 3
6 4 5 6 7 8 9
Random password generator
Type and strength of password generated
Random password generators normally output a string of symbols of specified length. These can be individual
characters from some character set, syllables designed to form pronounceable passwords, or words from some word
list to form a passphrase. The program can be customized to ensure the resulting password complies with the local
password policy, say by always producing a mix of letters, numbers and special characters.
The strength of a random password against a particular attack (brute force search), can be calculated by computing
the information entropy of the random process that produced it. If each symbol in the password is produced
independently, the entropy is just given by the formula
where N is the number of possible symbols and L is the number of symbols in the password. The function log
is the
base-2 logarithm. H is measured in bits.

Entropy per symbol for different symbol sets
Symbol set Symbol count N Entropy per symbol H
Arabic numerals (0-9) (e.g. PIN) 10 3.32 bits
Hexadecimal numerals (0-9, A-F) (e.g. WEP key) 16 4.00 bits
Case insensitive Latin alphabet (a-z or A-Z) 26 4.70 bits
Case insensitive alphanumeric (a-z or A-Z, 0-9) 36 5.17 bits
Case sensitive Latin alphabet (a-z, A-Z) 52 5.70 bits
Case sensitive alphanumeric (a-z, A-Z, 0-9) 62 5.95 bits
All ASCII printable characters 94 6.55 bits
Diceware word list 7776 12.9 bits
Minimum lengths L of randomly generated passwords to achieve desired password entropy
H for symbol sets containing N symbols.
Desired password
entropy H
Case insensitive
Latin alphabet
Case insensitive
Case sensitive
Latin alphabet
Case sensitive
32 bits 10 7 7 6 6 5
40 bits 13 9 8 8 7 7
64 bits 20 14 13 12 11 10
96 bits 29 21 19 17 17 15
128 bits 39 28 25 23 22 20
160 bits 49 35 31 29 27 25
192 bits 58 41 38 34 33 30
224 bits 68 48 44 40 38 35
256 bits 78 55 50 45 43 39
384 bits 116 82 75 68 65 59
512 bits 155 109 100 90 86 78
1024 bits 309 218 199 180 172 156
Random password generator
Any password generator is limited by the state space of the pseudo-random number generator used, if it is based on
one. Thus a password generated using a 32-bit generator is limited to 32 bits entropy, regardless of the number of
characters the password contains.
Note, however, that a different type of attack might succeed against a password evaluated as 'very strong' by the
above calculation.
Password generator programs and Web sites
A large number of password generator programs and Web sites are available on the Internet. Their quality varies and
can be hard to assess if there is no clear description of the source of randomness that is used, and if source code is
not provided to allow claims to be checked. Furthermore, and probably most importantly, transmitting candidate
passwords over the Internet raises obvious security concerns, particularly if the connection to the password
generation site's program is not properly secured or if the site is compromised in some way. Without a secure
channel, it is not possible to prevent eavesdropping, especially over public networks such as the Internet.
[1] http:/ / msdn2. microsoft.com/ en-us/ library/2dfe3bzd.aspx
[2] http:/ / msdn. microsoft.com/ en-us/ library/sxtz2fa8(VS.80). aspx
[3] http:// us3. php. net/ microtime
[4] http:// us3. php. net/ manual/ en/ function.rand.php
[5] http:// www. suspekt. org/ 2008/ 08/ 17/mt_srand-and-not-so-random-numbers/
[6] http:/ / java.sun. com/ j2se/ 1. 4. 2/ docs/ api/ java/ security/ SecureRandom.html
[7] http:/ / mlawire.blogspot. com/ 2009/ 07/ linux-password-generator.html
[8] http:/ / docs. python.org/py3k/ library/random.html
[9] http:/ / docs. python.org/py3k/ library/os. html#os. urandom
[10] a sample PHP secure random program (http://forums. thedailywtf.com/ forums/p/ 16453/ 220289.aspx)
[11] http:// php.net/ manual/ en/ function.openssl-random-pseudo-bytes. php
[12] Levine, John R., Ed.: Internet Secrets, Second edition, page 831 ff. John Wiley and Sons.
[13] Schneier, B: Applied Cryptography, Second edition, page 233 ff. John Wiley and Sons.
[14] "Electronic Authentication Guideline" (http:/ / csrc. nist. gov/ publications/ nistpubs/ 800-63/ SP800-63V1_0_2.pdf) (PDF). NIST. .
Retrieved March 27, 2008.
External links
• Cryptographically Secure Random number on Windows without using CryptoAPI (http:// blogs. msdn. com/
michael_howard/archive/ 2005/ 01/ 14/ 353379.aspx) from MSDN
• RFC 4086 on Randomness Recommendations for Security (http:/ / www.ietf. org/rfc/rfc4086.txt) (Replaces
earlier RFC 1750.)
• Automated Password Generator standard [[FIPS (http:// www.itl. nist. gov/ fipspubs/ fip181.htm)] 181]
A password is a secret word or string of characters that is used for authentication, to prove identity or gain access to
a resource (example: an access code is a type of password). The password should be kept secret from those not
allowed access.
The use of passwords is known to be ancient. Sentries would challenge those wishing to enter an area or approaching
it to supply a password or watchword. Sentries would only allow a person or group to pass if they knew the
password. In modern times, user names and passwords are commonly used by people during a log in process that
controls access to protected computer operating systems, mobile phones, cable TV decoders, automated teller
machines (ATMs), etc. A typical computer user may require passwords for many purposes: logging in to computer
accounts, retrieving e-mail from servers, accessing programs, databases, networks, web sites, and even reading the
morning newspaper online.
Despite the name, there is no need for passwords to be actual words; indeed passwords which are not actual words
may be harder to guess, a desirable property. Some passwords are formed from multiple words and may more
accurately be called a passphrase. The term passcode is sometimes used when the secret information is purely
numeric, such as the personal identification number (PIN) commonly used for ATM access. Passwords are generally
short enough to be easily memorized and typed.
For the purposes of more compellingly authenticating the identity of one computing device to another, passwords
have significant disadvantages (they may be stolen, spoofed, forgotten, etc.) over authentications systems relying on
cryptographic protocols, which are more difficult to circumvent.
Easy to remember, hard to guess
The easier a password is for the owner to remember generally means it will be easier for an attacker to guess.
Passwords which are difficult to remember will reduce the security of a system because (a) users might need to write
down or electronically store the password, (b) users will need frequent password resets and (c) users are more likely
to re-use the same password. Similarly, the more stringent requirements for password strength, e.g. "have a mix of
uppercase and lowercase letters and digits" or "change it monthly", the greater the degree to which users will subvert
the system.
In The Memorability and Security of Passwords,
Jeff Yan et al. examine the effect of advice given to users about a
good choice of password. They found that passwords based on thinking of a phrase and taking the first letter of each
word are just as memorable as naively selected passwords, and just as hard to crack as randomly generated
passwords. Combining two unrelated words is another good method. Having a personally designed "algorithm" for
generating obscure passwords is another good method.
However, asking users to remember a password consisting of a “mix of uppercase and lowercase characters” is
similar to asking them to remember a sequence of bits: hard to remember, and only a little bit harder to crack (e.g.
only 128 times harder to crack for 7-letter passwords, less if the user simply capitalises one of the letters). Asking
users to use "both letters and digits" will often lead to easy-to-guess substitutions such as 'E' --> '3' and 'I' --> '1',
substitutions which are well known to attackers. Similarly typing the password one keyboard row higher is a
common trick known to attackers.
Factors in the security of a password system
The security of a password-protected system depends on several factors. The overall system must, of course, be
designed for sound security, with protection against computer viruses, man-in-the-middle attacks and the like.
Physical security issues are also a concern, from deterring shoulder surfing to more sophisticated physical threats
such as video cameras and keyboard sniffers. And, of course, passwords should be chosen so that they are hard for
an attacker to guess and hard for an attacker to discover using any (and all) of the available automatic attack
schemes. See password strength, computer security, and computer insecurity.
Nowadays it is a common practice for computer systems to hide passwords as they are typed. The purpose of this
measure is to avoid bystanders reading the password. However, some argue that such practice may lead to mistakes
and stress, encouraging users to choose weak passwords. As an alternative, users should have the option to show or
hide passwords as they type them.
Effective access control provisions may force extreme measures on criminals seeking to acquire a password or
biometric token.
Less extreme measures include extortion, rubber hose cryptanalysis, and side channel attack.
Here are some specific password management issues that must be considered in thinking about, choosing, and
handling, a password.
Rate at which an attacker can try guessed passwords
The rate at which an attacker can submit guessed passwords to the system is a key factor in determining system
security. Some systems impose a time-out of several seconds after a small number (e.g., three) of failed password
entry attempts. In the absence of other vulnerabilities, such systems can be effectively secure with relatively simple
passwords, if they have been well chosen and are not easily guessed.
Many systems store or transmit a cryptographic hash of the password in a manner that makes the hash value
accessible to an attacker. When this is done, and it is very common, an attacker can work off-line, rapidly testing
candidate passwords against the true password's hash value. Passwords that are used to generate cryptographic keys
(e.g., for disk encryption or Wi-Fi security) can also be subjected to high rate guessing. Lists of common passwords
are widely available and can make password attacks very efficient. (See Password cracking.) Security in such
situations depends on using passwords or passphrases of adequate complexity, making such an attack
computationally infeasible for the attacker. Some systems, such as PGP and Wi-Fi WPA, apply a
computation-intensive hash to the password to slow such attacks. See key stretching.
Form of stored passwords
Some computer systems store user passwords as cleartext, against which to compare user log on attempts. If an
attacker gains access to such an internal password store, all passwords—and so all user accounts—will be
compromised. If some users employ the same password for accounts on different systems, those will be
compromised as well.
More secure systems store each password in a cryptographically protected form, so access to the actual password
will still be difficult for a snooper who gains internal access to the system, while validation of user access attempts
remains possible.
A common approach stores only a "hashed" form of the plaintext password. When a user types in a password on
such a system, the password handling software runs through a cryptographic hash algorithm, and if the hash value
generated from the user's entry matches the hash stored in the password database, the user is permitted access. The
hash value is created by applying a hash function (for maximum resistance to attack this should be a cryptographic
hash function) to a string consisting of the submitted password and, usually, another value known as a salt. The salt
prevents attackers from easily building a list of hash values for common passwords. MD5 and SHA1 are frequently
used cryptographic hash functions.
A modified version of the DES algorithm was used for this purpose in early Unix systems. The UNIX DES function
was iterated to make the hash function equivalent slow, further frustrating automated guessing attacks, and used the
password candidate as a key to encrypt a fixed value, thus blocking yet another attack on the password shrouding
system. More recent Unix or Unix like systems (e.g., Linux or the various BSD systems) use what most believe to be
still more effective protective mechanisms based on MD5, SHA1, Blowfish, Twofish, or any of several other
algorithms to prevent or frustrate attacks on stored password files.
If the hash function is well designed, it will be computationally infeasible to reverse it to directly find a plaintext
password. However, many systems do not protect their hashed passwords adequately, and if an attacker can gain
access to the hashed values he can use widely available tools which compare the encrypted outcome of every word
from some list, such as a dictionary (many are available on the Internet). Large lists of possible passwords in many
languages are widely available on the Internet, as are software programs to try common variations. The existence of
these dictionary attack tools constrains user password choices which are intended to resist easy attacks; they must not
be findable on such lists. Obviously, words on such lists should be avoided as passwords. Use of a key stretching
hash such as PBKDF2 is designed to reduce this risk.
A poorly designed hash function can make attacks feasible even if a strong password is chosen. See LM hash for a
widely deployed, and insecure, example.
Methods of verifying a password over a network
Various methods have been used to verify submitted passwords in a network setting:
Simple transmission of the password
Passwords are vulnerable to interception (i.e., "snooping") while being transmitted to the authenticating machine or
person. If the password is carried as electrical signals on unsecured physical wiring between the user access point
and the central system controlling the password database, it is subject to snooping by wiretapping methods. If it is
carried as packetized data over the Internet, anyone able to watch the packets containing the logon information can
snoop with a very low probability of detection.
Email is sometimes used to distribute passwords. Since most email is sent as cleartext, it is available without effort
during transport to any eavesdropper. Further, the email will be stored on at least two computers as cleartext—the
sender's and the recipient's. If it passes through intermediate systems during its travels, it will probably be stored on
those as well, at least for some time. Attempts to delete an email from all these vulnerabilities may, or may not,
succeed; backups or history files or caches on any of several systems may still contain the email. Indeed merely
identifying every one of those systems may be difficult. Emailed passwords are generally an insecure method of
An example of cleartext transmission of passwords is the original Wikipedia website. When you logged into your
Wikipedia account, your username and password are sent from your computer's browser through the Internet as
cleartext. In principle, anyone could read them in transit and thereafter log into your account as you; Wikipedia's
servers have no way of distinguishing such an attacker from you. In practice, an unknowably larger number could do
so as well (e.g., employees at your Internet Service Provider, at any of the systems through which the traffic passes,
etc.). More recently, Wikipedia has offered a secure login option, which, like many e-commerce sites, uses the SSL /
(TLS) cryptographically based protocol to eliminate the cleartext transmission. But, because anyone can gain access
to Wikipedia (without logging in at all), and then edit essentially all articles, it can be argued that there is little need
to encrypt these transmissions as there's little being protected. Other websites (e.g., banks and financial institutions)
have quite different security requirements, and cleartext transmission of anything is clearly insecure in those
Using client-side encryption will only protect transmission from the mail handling system server to the client
machine. Previous or subsequent relays of the email will not be protected and the email will probably be stored on
multiple computers, certainly on the originating and receiving computers, most often in cleartext.
Transmission through encrypted channels
The risk of interception of passwords sent over the Internet can be reduced by, among other approaches, using
cryptographic protection. The most widely used is the Transport Layer Security (TLS, previously called SSL) feature
built into most current Internet browsers. Most browsers alert the user of a TLS/SSL protected exchange with a
server by displaying a closed lock icon, or some other sign, when TLS is in use. There are several other techniques in
use; see cryptography.
Hash-based challenge-response methods
Unfortunately, there is a conflict between stored hashed-passwords and hash-based challenge-response
authentication; the latter requires a client to prove to a server that he knows what the shared secret (i.e., password) is,
and to do this, the server must be able to obtain the shared secret from its stored form. On many systems (including
Unix-type systems) doing remote authentication, the shared secret usually becomes the hashed form and has the
serious limitation of exposing passwords to offline guessing attacks. In addition, when the hash is used as a shared
secret, an attacker does not need the original password to authenticate remotely; he only needs the hash.
Zero-knowledge password proofs
Rather than transmitting a password, or transmitting the hash of the password, password-authenticated key
agreement systems can perform a zero-knowledge password proof, which proves knowledge of the password without
exposing it.
Moving a step further, augmented systems for password-authenticated key agreement (e.g., AMP, B-SPEKE,
PAK-Z, SRP-6) avoid both the conflict and limitation of hash-based methods. An augmented system allows a client
to prove knowledge of the password to a server, where the server knows only a (not exactly) hashed password, and
where the unhashed password is required to gain access.
Procedures for changing passwords
Usually, a system must provide a way to change a password, either because a user believes the current password has
been (or might have been) compromised, or as a precautionary measure. If a new password is passed to the system in
unencrypted form, security can be lost (e.g., via wiretapping) even before the new password can even be installed in
the password database. And, of course, if the new password is given to a compromised employee, little is gained.
Some web sites include the user-selected password in an unencrypted confirmation e-mail message, with the obvious
increased vulnerability.
Identity management systems are increasingly used to automate issuance of replacements for lost passwords, a
feature called self service password reset. The user's identity is verified by asking questions and comparing the
answers to ones previously stored (i.e., when the account was opened). Typical questions include: "Where were you
born?," "What is your favorite movie?" or "What is the name of your pet?" In many cases the answers to these
questions can be relatively easily guessed by an attacker, determined by low effort research, or obtained through
social engineering, and so this is less than fully satisfactory as a verification technique. While many users have been
trained never to reveal a password, few consider the name of their pet or favorite movie to require similar care.
Password longevity
"Password aging" is a feature of some operating systems which forces users to change passwords frequently (e.g.,
quarterly, monthly or even more often). Such policies usually provoke user protest and foot-dragging at best and
hostility at worst. There is often an increase in the people who note down the password and leave it where it can
easily be found, as well as helpdesk calls to reset a forgotten password. Users may use simpler passwords or develop
variation patterns on a consistent theme to keep their passwords memorable. Because of these issues, there is some
as to whether password ageing is effective. The intended benefit is mainly that a stolen password will be
made ineffective if it is reset; however in many cases, particularly with administrative or "root" accounts, once an
attacker has gained access, he can make alterations to the operating system that will allow him future access even
after the initial password he used expires. (see rootkit). The other less-frequently cited, and possibly more valid
reason is that in the event of a long brute force attack, the password will be invalid by the time it has been cracked.
Implementing such a policy requires careful consideration of the relevant human factors. It may be required because
of the nature of IT systems the password allows access to; if personal data is involved the EU Data Protection
Directive is in force.
Number of users per password
Sometimes a single password controls access to a device, for example, for a network router, or password-protected
mobile phone. However, in the case of a computer system, a password is usually stored for each user account, thus
making all access traceable (save, of course, in the case of users sharing passwords). A would-be user on most
systems must supply a username as well as a password, almost always at account set up time, and periodically
thereafter. If the user supplies a password matching the one stored for the supplied username, he or she is permitted
further access into the computer system. This is also the case for a cash machine, except that the 'user name' is
typically the account number stored on the bank customer's card, and the PIN is usually quite short (4 to 6 digits).
Allotting separate passwords to each user of a system is preferable to having a single password shared by legitimate
users of the system, certainly from a security viewpoint. This is partly because users are more willing to tell another
person (who may not be authorized) a shared password than one exclusively for their use. Single passwords are also
much less convenient to change because many people need to be told at the same time, and they make removal of a
particular user's access more difficult, as for instance on graduation or resignation. Per-user passwords are also
essential if users are to be held accountable for their activities, such as making financial transactions or viewing
medical records.
Password security architecture
Common techniques used to improve the security of computer systems protected by a password include:
• Not displaying the password on the display screen as it is being entered or obscuring it as it is typed by using
asterisks (*) or bullets (•).
• Allowing passwords of adequate length. (Some legacy operating systems, including early versions of Unix and
Windows, limited passwords to an 8 character maximum,



reducing security.)
• Requiring users to re-enter their password after a period of inactivity (a semi log-off policy).
• Enforcing a password policy to increase password strength and security.
• Requiring periodic password changes.
• Assigning randomly chosen passwords.
• Requiring minimum password lengths.
• Some systems require characters from various character classes in a password—for example, "must have at
least one uppercase and at least one lowercase letter". However, all-lowercase passwords are more secure per
keystroke than mixed capitalization passwords.
• Providing an alternative to keyboard entry (e.g., spoken passwords, or biometric passwords).
• Requiring more than one authentication system, such as 2-factor authentication (something you have and
something you know).
• Using encrypted tunnels or password-authenticated key agreement to prevent access to transmitted passwords via
network attacks
• Limiting the number of allowed failures within a given time period (to prevent repeated password guessing).
After the limit is reached, further attempts will fail (including correct password attempts) until the beginning of
the next time period. However, this is vulnerable to a form of denial of service attack.
• Introducing a delay between password submission attempts to slow down automated password guessing
Some of the more stringent policy enforcement measures can pose a risk of alienating users, possibly decreasing
security as a result.
Write down passwords on paper
Historically, many security experts asked people to memorize their passwords and "Never write down a password".
More recently, many security experts such as Bruce Schneier recommend that people use passwords that are too
complicated to memorize, write them down on paper, and keep them in a wallet.






Password cracking
Attempting to crack passwords by trying as many possibilities as time and money permit is a brute force attack. A
related method, rather more efficient in most cases, is a dictionary attack. In a dictionary attack, all words in one or
more dictionaries are tested. Lists of common passwords are also typically tested.
Password strength is the likelihood that a password cannot be guessed or discovered, and varies with the attack
algorithm used. Passwords easily discovered are termed weak or vulnerable; passwords very difficult or impossible
to discover are considered strong. There are several programs available for password attack (or even auditing and
recovery by systems personnel) such as L0phtCrack, John the Ripper, and Cain; some of which use password design
vulnerabilities (as found in the Microsoft LANManager system) to increase efficiency. These programs are
sometimes used by system administrators to detect weak passwords proposed by users.
Studies of production computer systems have consistently shown that a large fraction of all user-chosen passwords
are readily guessed automatically. For example, Columbia University found 22% of user passwords could be
recovered with little effort.
According to Bruce Schneier, examining data from a 2006 phishing attack, 55% of
MySpace passwords would be crackable in 8 hours using a commercially available Password Recovery Toolkit
capable of testing 200,000 passwords per second in 2006.
He also reported that the single most common
password was password1, confirming yet again the general lack of informed care in choosing passwords among
users. (He nevertheless maintained, based on these data, that the general quality of passwords has improved over the
years—for example, average length was up to eight characters from under seven in previous surveys, and less than
4% were dictionary words.
1998 incident
On July 16, 1998, CERT reported an incident where an attacker had found 186,126 encrypted passwords. By the
time they were discovered, they had already cracked 47,642 passwords.
Alternatives to passwords for authentication
The numerous ways in which permanent or semi-permanent passwords can be compromised has prompted the
development of other techniques. Unfortunately, some are inadequate in practice, and in any case few have become
universally available for users seeking a more secure alternative.
• Single-use passwords. Having passwords which are only valid once makes many potential attacks ineffective.
Most users find single use passwords extremely inconvenient. They have, however, been widely implemented in
personal online banking, where they are known as Transaction Authentication Numbers (TANs). As most home
users only perform a small number of transactions each week, the single use issue has not led to intolerable
customer dissatisfaction in this case.
• Time-synchronized one-time passwords are similar in some ways to single-use passwords, but the value to be
entered is displayed on a small (generally pocketable) item and changes every minute or so.
• PassWindow one-time passwords are used as single-use passwords, but the dynamic characters to be entered are
visible only when a user superimposes a unique printed visual key over a server generated challenge image shown
on the user's screen.
• Access controls based on public key cryptography e.g. ssh. The necessary keys are usually too large to memorize
(but see proposal Passmaze
) and must be stored on a local computer, security token or portable memory
device, such as a USB flash drive or even floppy disk.
• Biometric methods promise authentication based on unalterable personal characteristics, but currently (2008)
have high error rates and require additional hardware to scan, for example, fingerprints, irises, etc. They have
proven easy to spoof in some famous incidents testing commercially available systems, for example, the gummie
fingerprint spoof demonstration,
and, because these characteristics are unalterable, they cannot be changed if
compromised; this is a highly important consideration in access control as a compromised access token is
necessarily insecure.
• Single sign-on technology is claimed to eliminate the need for having multiple passwords. Such schemes do not
relieve user and administrators from choosing reasonable single passwords, nor system designers or
administrators from ensuring that private access control information passed among systems enabling single
sign-on is secure against attack. As yet, no satisfactory standard has been developed.
• Envaulting technology is a password-free way to secure data on e.g. removable storage devices such as USB flash
drives. Instead of user passwords, access control is based on the user's access to a network resource.
• Non-text-based passwords, such as graphical passwords or mouse-movement based passwords.
passwords are an alternative means of authentication for log-in intended to be used in place of conventional
password; they use images, graphics or colours instead of letters, digits or special characters. One system requires
users to select a series of faces as a password, utilizing the human brain's ability to recall faces easily.
In some
implementations the user is required to pick from a series of images in the correct sequence in order to gain
Another graphical password solution creates a one-time password using a randomly-generated grid of
images. Each time the user is required to authenticate, they look for the images that fit their pre-chosen categories
and enter the randomly-generated alphanumeric character that appears in the image to form the one-time

So far, graphical passwords are promising, but are not widely used. Studies on this subject have
been made to determine its usability in the real world. While some believe that graphical passwords would be
harder to crack, others suggest that people will be just as likely to pick common images or sequences as they are
to pick common passwords.
• 2D Key (2-Dimensional Key)
is a 2D matrix-like key input method having the key styles of multiline
passphrase, crossword, ASCII/Unicode art, with optional textual semantic noises, to create big password/key
beyond 128 bits to realize the MePKC (Memorizable Public-Key Cryptography)
using fully memorizable
private key upon the current private key management technologies like encrypted private key, split private key,
and roaming private key.
• Cognitive passwords use question and answer cue/response pairs to verify identity.
Website password systems
Passwords are used on websites to authenticate users and are usually maintained on the Web server, meaning the
browser on a remote system sends a password to the server (by HTTP POST), the server checks the password and
sends back the relevant content (or an access denied message). This process eliminates the possibility of local
reverse engineering as the code used to authenticate the password does not reside on the local machine.
Transmission of the password, via the browser, in plaintext means it can be intercepted along its journey to the
server. Many web authentication systems use SSL to establish an encrypted session between the browser and the
server, and is usually the underlying meaning of claims to have a "secure Web site". This is done automatically by
the browser and increases integrity of the session, assuming neither end has been compromised and that the
SSL/TLS implementations used are high quality ones.
History of passwords
Passwords or watchwords have been used since ancient times. Polybius describes the system for distribution
watchwords in the Roman military as follows:
The way in which they secure the passing round of the watchword for the night is as follows: from the tenth
maniple of each class of infantry and cavalry, the maniple which is encamped at the lower end of the street, a
man is chosen who is relieved from guard duty, and he attends every day at sunset at the tent of the tribune,
and receiving from him the watchword - that is a wooden tablet with the word inscribed on it - takes his leave,
and on returning to his quarters passes on the watchword and tablet before witnesses to the commander of the
next maniple, who in turn passes it to the one next him. All do the same until it reaches the first maniples,
those encamped near the tents of the tribunes. These latter are obliged to deliver the tablet to the tribunes
before dark. So that if all those issued are returned, the tribune knows that the watchword has been given to all
the maniples, and has passed through all on its way back to him. If any one of them is missing, he makes
inquiry at once, as he knows by the marks from what quarter the tablet has not returned, and whoever is
responsible for the stoppage meets with the punishment he merits.
Passwords in military use evolved to include not just a password, but a password and a counterpassword; for
example in the opening days of the Battle of Normandy, paratroopers of the U.S. 101st Airborne Division used a
password - "thunder" - which was presented as a challenge, and answered with the correct response - "flash". The
challenge and response were changed periodically. American paratroopers also famously used a device known as a
"cricket" on D-Day in place of a password system as a temporarily unique method of identification; one metallic
click given by the device in lieu of a password was to be met by two clicks in reply.
Passwords have been used with computers since the earliest days of computing. MIT's CTSS, one of the first time
sharing systems, was introduced in 1961. It had a LOGIN command that requested a user password. "After typing
PASSWORD, the system turns off the printing mechanism, if possible, so that the user may type in his password
with privacy."
In 1978, Robert Morris invented the idea of storing login passwords in a hashed form as part of the
Unix operating system. His algorithm, known as crypt(3), used a 12-bit salt and invoked a modified form of the DES
algorithm 25 times to reduce the risk of pre-computed dictionary attacks.
[1] Vance, Ashlee (January 20, 2010). "If Your Password Is 123456, Just Make It HackMe" (http:/ / www. nytimes. com/ 2010/ 01/ 21/
technology/ 21password. html). The New York Times. .
[2] http:/ / all.net/ journal/netsec/ 1997-09.html Fred Cohen and Associates
[3] The Memorability and Security of Passwords (http:/ / homepages. cs.ncl.ac.uk/ jeff.yan/ jyan_ieee_pwd.pdf)
[4] Lyquix Blog: Do We Need to Hide Passwords? (http:// www.lyquix. com/ blog/ 92-do-we-need-to-hide-passwords)
[5] news.bbc.co.uk: Malaysia car thieves steal finger (http:/ / news.bbc. co. uk/ 2/ hi/ asia-pacific/ 4396831. stm)
[6] Top ten passwords used in the United Kingdom (http:/ / www. modernlifeisrubbish.co. uk/ top-10-most-common-passwords.asp)
[7] Password Protection for Modern Operating Systems (http:/ / www. usenix. org/publications/ login/ 2004-06/ pdfs/ alexander.pdf)
[8] http:// support. microsoft.com/ default. aspx?scid=KB;EN-US;q299656
[9] Schneier on Security discussion on changing passwords (http:// www.schneier.com/ blog/ archives/ 2010/ 11/ changing_passwo. html)
[10] HP-UX security whitepaper (http:/ / www.nasi. com/ docs/ pdfs/ hp-ux_security_whitepaper.pdf) "Passwords are limited to a maximum of
eight significant characters"
[11] "American Express: Strong Credit, Weak Passwords" (http:/ / www. pcmag.com/ article2/ 0,2817,2358985,00.asp)
[12] "Ten Windows Password Myths" (http:/ / www.symantec. com/ connect/ articles/ ten-windows-password-myths): "NT dialog boxes ...
limited passwords to a maximum of 14 characters"
[13] "You must provide a password between 1 and 8 characters in length" (http:/ / jira. codehaus.org/ browse/ REDBACK-87)
[14] "To Capitalize or Not to Capitalize?" (http:/ / world. std.com/ ~reinhold/ dicewarefaq.html#capitalize)
[15] Bruce Schneier : Crypto-Gram Newsletter (http:/ / www. schneier.com/ crypto-gram-0105.html#8) May 15, 2001
[16] "Ten Windows Password Myths" (http:// www.symantec. com/ connect/ articles/ ten-windows-password-myths): Myth #7. You Should
Never Write Down Your Password
[17] "Microsoft security guru: Jot down your passwords" (http:/ / news. cnet.com/Microsoft-security-guru-Jot-down-your-passwords/
[18] "The Strong Password Dilemma" (http:// www.cryptosmith. com/ sanity/ pwdilemma. html) by Richard E. Smith: "we can summarize
classical password selection rules as follows: The password must be impossible to remember and never written down."
[19] "Choosing Random Passwords" (http:/ / www. burtleburtle.net/ bob/ crypto/password. html) by Bob Jenkins
[20] "The Memorability and Security of Passwords -- Some Empirical Results" (http:// www.cl.cam. ac.uk/ TechReports/ UCAM-CL-TR-500.
"your password ... in a secure place, such as the back of your wallet or purse."
[21] "Should I write down my passphrase?" (http:/ / world. std. com/ ~reinhold/ dicewarefaq.html#writeitdown)
[22] Password (http:/ / www1. cs. columbia. edu/ ~crf/howto/ password-howto. html)
[23] Schneier, Real-World Passwords (http:// www. schneier. com/ blog/ archives/ 2006/12/ realworld_passw. html)
[24] MySpace Passwords Aren't So Dumb (http:// www. wired.com/ politics/ security/ commentary/securitymatters/ 2006/ 12/ 72300)
[25] "CERT IN-98.03" (http:/ / www.cert.org/ incident_notes/ IN-98.03. html). . Retrieved 2009-09-09.
[26] http:/ / eprint.iacr. org/2005/ 434
[27] T Matsumoto. H Matsumotot, K Yamada, and S Hoshino, Impact of artificial 'Gummy' Fingers on Fingerprint Systems. Proc SPIE, vol
4677, Optical Security and Counterfeit Deterrence Techniques IV or itu.int/itudoc/itu-t/workshop/security/resent/s5p4.pdf pg 356
[28] http:// waelchatila. com/ 2005/ 09/ 18/ 1127075317148. html
[29] http:/ / mcpmag.com/ reviews/ products/ article.asp?EditorialsID=486
[30] http:/ / searchsecurity. techtarget. com/sDefinition/ 0,,sid14_gci1001829,00. html
[31] Ericka Chickowski (2010-11-03). "Images Could Change the Authentication Picture" (http:/ / www.darkreading.com/ authentication/
security/client/ showArticle. jhtml?articleID=228200140). Dark Reading. .
[32] "Confident Technologies Delivers Image-Based, Multifactor Authentication to Strengthen Passwords on Public-Facing Websites" (http:/ /
www.marketwire.com/ press-release/
Confident-Technologies-Delivers-Image-Based-Multifactor-Authentication-Strengthen-Passwords-1342854. htm). 2010-10-28. .
[33] http:// www. xpreeli.com/ doc/ manual_2DKey_2. 0. pdf
[34] http:/ / www. wipo. int/ pctdb/ en/ wo.jsp?WO=2010010430
[35] Polybius on the Roman Military (http:/ / ancienthistory. about.com/ library/bl/ bl_text_polybius6. htm)
[36] Bando, Mark Screaming Eagles: Tales of the 101st Airborne Division in World War II
[37] CTSS Programmers Guide, 2nd Ed., MIT Press, 1965
[38] Morris, Robert; Thompson, Ken (1978-04-03). "Password Security: A Case History." (http:// cm. bell-labs. com/ cm/ cs/ who/ dmr/passwd.
ps). Bell Laboratories. . Retrieved 2011-05-09.
External links
• Large collection of statistics about passwords (http:/ / www.passwordresearch. com/ stats/ statindex. html)
• Graphical Passwords: A Survey (http:// www. acsac. org/2005/ abstracts/ 89.html)
• PassClicks (http:/ / labs. mininova. org/passclicks/ ), visual passwords
• Centre for Security, Communications and Network Research (http:// www. cscan. org/), University of Plymouth
• Research Papers on Password-based Cryptography (http:/ / www.jablon. org/passwordlinks. html)
• Procedural Advice for Organisations and Administrators (http:// www.emiic. net/ docs/ PasswordGuide. pdf)
• Memorability and Security of Passwords (http:/ / www.ftp. cl.cam. ac. uk/ ftp/users/ rja14/tr500.pdf) -
Cambridge University Computer Laboratory study of password memorability vs. security.
http:/ / 1dl.us
Slogan Your all-in-one tool site.
Type of site web application
Registration No
Launched 2010
Revenue Donations, Advertising
Current status Active
1dl.us is a website that combines several single purpose websites into a single page with a fast loading stripped
down interface. The developers state that it is an ongoing project and more services will be added in the future.
Current services offered as of Wednesday, October 06, 2010 are: URL shortening, Image Hosting, Pastebin,
Disposable Email, Disposable Chat, IRC Chat, Password Strength Meter, and a Random Password Generator. All
services are provided free of charge.
In addition to the services provided, the option is given of creating an account which will allow you to manage all
your links created on the site as well as obtain statistics on each link.
The developers are very active on the site, and have mentioned adding several new features soon including
disposable phone numbers.
June 2010
Development on 1dl.us begins
July 29, 2010
1dl.us is posted on lifehacker.com
August 02, 2010
1dl.us is mentioned on the Internet TV show Ehrensenf by Jeannine Michaelsen
August 19, 2010
1dl.us is posted on about.com
September, 2010
1dl.us is posted on makeuseof.com
September 2, 2010
1dl.us is posted on pcmech.com
September 16, 2010
1dl.us is posted on Kim Komando's website as the "Cool Site of the day"
December 9, 2010
1dl.us obtains its first angel investor.
[1] http:/ / 1dl.us
[2] "1dl.us Blog by founders" (http:/ / blog.1dl.us). .
[3] "1dl.us Is a Swiss Army Knife of Web Utilities" (http:/ / lifehacker.com/ 5599454/ 1dlus-is-a-swiss-army-knife-of-single+use-web-sites). .
[4] "Nachrichten-Stalker und Hasen-Rambo" (http:// www.ehrensenf.de/ 2010/ 08/nachrichten-stalker-und-hasen-rambo/). .
[5] "1dl.us, the Swiss Army Knife of Web-based tools" (http:/ / websearch.about.com/ b/ 2010/ 08/ 19/
1dl-us-the-swiss-army-knife-of-web-based-tools.htm). .
[6] "1dl.us: Quick Access To A Set Of Handy Web Services" (http:// www.makeuseof.com/ dir/1dlus-handy-web-services/ ). .
[7] "An All-In-One Tool Site Which Can Replace Many Single Purpose Sites" (http:/ / www.pcmech.com/ article/
an-all-in-one-tool-site-which-can-replace-many-single-purpose-sites/). .
[8] "A free collection of helpful Web tools" (http:// www.komando.com/ coolsites/ index.aspx?id=9471& utm_medium=nl&
utm_source=csotd& utm_content=2010-09-16-article&utm_campaign=end). .
[9] "1dl.us Blog by founders" (http:// blog.1dl.us). .
External links
• Official site (http:// 1dl.us/ )
2D Key
A 2D key is a special type of password input method proposed by Kok-Wah Lee since year 2005, that is input in a
special grid, instead of a single line. This enables the user to create memorizable (or mnemonic) but long passwords,
such as ASCII art, allowing extreme security.
ATM SafetyPIN software
ATM SafetyPIN software is a software application that would allow users of automated teller machines (ATMs) to
alert the police of a forced cash withdrawal by entering their personal identification number (PIN) in reverse order.
The system was invented and patented by Illinois lawyer Joseph Zingher (U.S. Patent 5731575
The concept of an alternative emergency PIN system, or duress code, for ATM systems has been around since at
least July 30, 1986, when Representative Mario Biaggi, a former police officer, proposed it in the U.S. Congressional
Record, pp. 18232 et seq. Biaggi then proposed House Resolution 785 in 1987 which would have had the FBI track
the problem of express kidnappings and evaluate the idea of an emergency PIN system. HR785 died in committee
without debate.
Zingher has not been successful in marketing his invention.
Police in New York, New Jersey, Ohio, Illinois, and
Kansas have supported the concept.



Police support prompted the Illinois legislature to pass a law making
it mandatory on all ATMs in Illinois. The law was changed shortly after it was passed by a "follow-on" bill that
changed the meaning to the exact opposite of what they were seeking.



In 2006, an e-mail chain letter hoax circulated that claimed a reverse PIN duress code system is in place
American Banker reported on January 2, 2007 that no PIN-reversal duress code is used on any ATM
as of that date. In July 2008 the hoax was still circulating in Australia with the text:
If you should ever be forced by a robber to withdraw money from an ATM, you can notify the police by
entering your PIN in reverse. For example if your PIN is 1234 then you would put in 4321. The ATM
recognizes that your PIN is backwards from the ATM card you placed in the machine. The machine will still
give you the money you requested, but unknown to the robber, the police will be immediately dispatched to
ATM SafetyPIN software
help you. This information was recently broadcasted [sic] on TV and it states that it is seldom used because
people don't know it exists. Please pass this along to everyone possible. Australian Federal Police. AFP Web
site: http:/ / www. afp.gov. au
The same kind of e-mail chain letter hoax is still circulated in India and other parts of the world.
Were the system implemented, PINs that are reversible such as 5555 or 2112 then would be unavailable so that false
alarms would not occur. Moreover, PINs that are semi-reversible such as 5255 or 1241, where the first and last
numbers are the same, would be something to avoid as well so that accidental alarms would not be triggered by
mistakenly switching the middle numbers.
Diebold, a manufacturer of ATMs, states on their website that no such emergency alerting system is currently in use.
They cite an article in the St. Louis Post-Dispatch which claims bankers oppose the reverse-PIN system out of
concerns that "ATM users might hesitate or fumble while trying to enter their PINs backwards under duress, possibly
increasing the chances of violence." Diebold further states that they would be willing to support such technology if
their customers (presumably banks) request it.
2009 bill
A bill making the reverse emergency PIN system mandatory on all ATMs in the state of Illinois was proposed on
February 10, 2009. Subsection (i) is the new bill.
i) A terminal operated in this State must be designed and programmed so that when a consumer enters
his or her personal identification number in reverse order, the terminal automatically sends an alarm to
the local law enforcement agency having jurisdiction over the terminal location. The Commissioner
shall promulgate rules necessary for the implementation of this subsection (i).
Los Angeles City Councilman Greig Smith announced his intention to make the ReversePIN system mandatory on
all ATMs in the city.


[1] ZICUBED ATM SAFETYPIN (http:// www. zicubedatm. com/ ) ATM Safety PIN aka Reverse PIN Web Site.
[2] http:// www. google. com/ patents?vid=5731575
[3] Why Great Ideas Get Shot Down (http:// money. cnn. com/ magazines/ fsb/ fsb_archive/ 2006/ 02/ 01/ 8368177/ index.htm)
CNNMoney.com, 01-27-2006
[4] Asbury Park Press, January 25th, 2006
[5] Are Local Banks Doing All They Can To Protect ATM Users? (http:// www.wlwt.com/ news/ 2896611/detail. html) Jesse Jones, WLWT.
March 3, 2004
[6] St.Louis Post Dispatch, March 27th, 2005
[7] "Wichita Eagle" April 19, 2001
[8] St. Louis Post Dispatch, March 27, 2005
[9] Public Act 093-0898 (http:// ilga. gov/ legislation/ publicacts/ 93/ 093-0898.htm) Illinois General Assembly
[10] FINANCIAL REGULATION (205 ILCS 616/) Electronic Fund Transfer Act (http:// ilga.gov/ legislation/ ilcs/ ilcs3. asp?ActID=1192&
ChapAct=205& nbsp;ILCS& nbsp;616/ & ChapterID=20&ChapterName=FINANCIAL+REGULATION&ActName=Electronic+Fund+
Transfer+ Act.) Illinois General Assembly
[11] Banking on ATM Safety (http:// www.msnbc. msn. com/ id/ 4086277/ ) Forbes, 01-28-2004
[12] Security Update - Reverse PIN Hoax (http:// www. diebold. com/ atmsecurity/securityupdate.htm) Diebold, Inc.
[13] Illinois General Assembly - Full Text of SB1355 (http:/ / www.ilga. gov/ legislation/ fulltext. asp?DocName=& SessionId=76& GA=96&
DocTypeId=SB&DocNum=1355& GAID=10&LegID=42570&SpecSess=& Session=)
[14] http:// www. nbclosangeles. com/ news/ local-beat/ATM-Duress-Code-Good-Idea-Any-Takers-56739192.html
[15] http:/ / www. pasadenastarnews. com/ ci_13263868
[16] http:// www. huffingtonpost.com/ robin-sax/ atmurders-can-be-avoided_b_288666.html
ATM SafetyPIN software
External links
• snopes.com: Reverse PIN Panic Code (http:/ / www.snopes. com/ business/ bank/ pinalert. asp)
• PIN Number Reversal - Use Reverse PIN to Contact Police? - Urban Legends (http:// urbanlegends. about. com/
library/ bl_reverse_pin.htm)
• ATM Security Advise Message : Enter PIN In Reverse to Call Police (http:/ / www.hoax-slayer. com/
• ATM Pin number reverse - Outlook Express Tips (http:// www.outlookexpresstips. com/
Canonical account
A canonical account (or built-in account), in the context of computer software and systems, is an account that is
included by default with a program or firmware. Such accounts usually also have a default password and may have
certain access rights by default.
As such accounts and their password and permissions are usually common knowledge, given that anyone possessing
a copy of the software, the device or their documentation will likely know of the account, a common security
measure is to change the account's password and to double-check or modify the groups (if any) it is included in, or
simply disable or delete it if it is not required.
• Zyxel routers typically have admin as their default firmware administration account and 1234 as the default
password. The password can and should be changed as soon as possible.
• Microsoft Windows 2000 and XP, and possibly other versions, have an account named Guest by default, which
has no password and grants a very basic access to the operating system. Even though it is disabled by default,
some administrators may choose to activate it, change the password and disable it once more for good measure.
This account cannot be deleted.
• If not blank, canonical passwords are usually simple and may often be:
• A simple sequence: 1234, 4321, abcd
• The same as the account: if the account is bob, the password will also be bob
• A word relating to the account or software: support, finance, windows
• Simply password, pass
External links
• Default Router Password List
• Alecto - Default Password List Project
[1] http:/ / www. phenoelit. de/ dpl/ dpl. html
[2] http:/ / www. helith. net/ projects/ alecto/
Challenge-Handshake Authentication Protocol
Challenge-Handshake Authentication Protocol
In computing, the Challenge-Handshake Authentication Protocol (CHAP) authenticates a user or network host to
an authenticating entity. That entity may be, for example, an Internet service provider.
CHAP provides protection against playback attack by the peer through the use of an incrementally changing
identifier and of a variable challenge-value. CHAP requires that both the client and server know the plaintext of the
secret, although it is never sent over the network.
Microsoft has implemented a variant of the Challenge-handshake authentication protocol, called MS-CHAP, which
does not require either peer to know the plaintext.
Working Cycle
CHAP is an authentication scheme used by Point to Point Protocol (PPP) servers to validate the identity of remote
clients. CHAP periodically verifies the identity of the client by using a three-way handshake. This happens at the
time of establishing the initial link, and may happen again at any time afterwards. The verification is based on a
shared secret (such as the client user's password).
1. After the completion of the link establishment phase, the authenticator sends a "challenge" message to the peer.
2. The peer responds with a value calculated using a one-way hash function on the challenge and the secret
3. The authenticator checks the response against its own calculation of the expected hash value. If the values match,
the authenticator acknowledges the authentication; otherwise it should terminate the connection.
4. At random intervals the authenticator sends a new challenge to the peer and repeats steps 1 through 3.
CHAP Packets
Description 1 byte 1 byte 2 bytes 1 byte Variable variable
Challenge Code = 1 ID Length Challenge length Challenge value Name
Response Code = 2 ID Length Response Length Response value Name
Success Code = 3 ID Length Message
Failure Code = 4 ID Length Message
• RFC 1994
Challenge-response authentication
Challenge-response authentication
In computer security, challenge-response authentication is a family of protocols in which one party presents a
question ("challenge") and another party must provide a valid answer ("response") to be authenticated.
The simplest example of a challenge-response protocol is password authentication, where the challenge is asking for
the password and the valid response is the correct password.
Clearly an adversary that can eavesdrop on a password authentication can then authenticate itself in the same way.
One solution is to issue multiple passwords, each of them marked with an identifier. The verifier can pick any of the
identifiers, and the prover must have the correct password for that identifier. Assuming that the passwords are chosen
independently, an adversary who intercepts one challenge-response message pair has no more chance of responding
correctly to a different challenge than an adversary who has intercepted nothing.
For example, when other communications security methods are unavailable, the U.S. military uses the AKAC-1553
DRYAD numeral cipher to authenticate and encrypt some communications. DRYAD includes a list of three-letter
challenge codes, which the verifier is supposed to choose randomly from, and random three-letter responses to them.
For added security, each set of codes is only valid for a particular time period which is ordinarily 24 hours.
Software in the 1980s and 1990s often used a similar method for copy protection: challenges would be questions like
"What is the second word in the third paragraph on page 418 of the manual?". The security assumption was that
copying the manual was more difficult than copying the software disk.
Other non-cryptographic protocols
Challenge-response protocols are also used to assert things other than knowledge of a secret value. CAPTCHAs, for
example, are a sort of variant on the Turing test, meant to determine whether a viewer of a Web application is a real
person. The challenge sent to the viewer is a distorted image of some text, and the viewer responds by typing in that
text. The distortion is designed to make automated optical character recognition (OCR) difficult and preventing a
computer program from passing as a human.
Cryptographic techniques
Non-cryptographic authentication was generally adequate in the days before the Internet, when the user could be sure
that the system asking for the password was really the system they were trying to access, and that nobody was likely
to be eavesdropping on the communication channel to observe the password being entered. To address the insecure
channel problem, a more sophisticated approach is necessary. Many cryptographic solutions involve two-way
authentication, where both the user and the system must each convince the other that they know the shared secret
(the password), without this secret ever being transmitted in the clear over the communication channel, where
eavesdroppers might be lurking.
One way this is done involves using the password as the encryption key to transmit some randomly-generated
information as the challenge, whereupon the other end must return as its response a similarly-encrypted value which
is some predetermined function of the originally-offered information, thus proving that it was able to decrypt the
challenge. For instance, in Kerberos, the challenge is an encrypted integer N, while the response is the encrypted
integer N + 1, proving that the other end was able to decrypt the integer N. In other variations, a hash function
operates on a password and a random challenge value to create a response value.
Such encrypted or hashed exchanges do not directly reveal the password to an eavesdropper. However, they may
supply enough information to allow an eavesdropper to deduce what the password is, using a dictionary attack or
brute-force attack. The use of information which is randomly generated on each exchange (and where the response is
different from the challenge) guards against the possibility of a replay attack, where a malicious intermediary simply
Challenge-response authentication
records the exchanged data and retransmits it at a later time to fool one end into thinking it has authenticated a new
connection attempt from the other.
Authentication protocols usually employ a cryptographic nonce as the challenge to ensure that every
challenge-response sequence is unique. This protects against a replay attack. If it is impractical to implement a true
nonce, a strong cryptographically secure pseudorandom number generator and cryptographic hash function can
generate challenges that are highly unlikely to occur more than once. It is important not to use time-based nonces, as
these can weaken servers in different time zones and servers with inaccurate clocks.
Mutual authentication is performed using a challenge-response handshake in both directions; the server ensures that
the client knows the secret, and the client also ensures that the server knows the secret, which protects against a
rogue server impersonating the real server.
Challenge-response authentication can help solve the problem of exchanging session keys for encryption. Using a
key derivation function, the challenge value and the secret may be combined to generate an unpredictable encryption
key for the session. This is particularly effective against a man-in-the-middle attack, because the attacker will not be
able to derive the session key from the challenge without knowing the secret, and therefore will not be able to
decrypt the data stream.
Simple Example mutual authentication sequence
• Server sends a unique challenge value sc to the client
• Client generates unique challenge value cc
• Client computes cr = hash(cc + sc + secret)
• Client sends cr and cc to the server
• Server calculates the expected value of cr and ensures the client responded correctly
• Server computes sr = hash(sc + cc + secret)
• Server sends sr
• Client calculates the expected value of sr and ensures the server responded correctly
• sc is the server generated challenge
• cc is the client generated challenge
• cr is the client response
• sr is the server response
Password storage
To avoid storage of passwords, some operating systems (e.g. Unix-type) store a hash of the password rather than
storing the password itself. During authentication, the system need only verify that the hash of the password entered
matches the hash stored in the password database. This makes it more difficult for an intruder to get the passwords,
since the password itself is not stored, and it is very difficult to determine a password that matches a given hash.
However, this presents a problem for challenge-response algorithms, which require both the client and the server to
have a shared secret. Since the password itself is not stored, a challenge-response algorithm will usually have to use
the hash of the password as the secret instead of the password itself. In this case, an intruder can use the actual hash,
rather than the password, which makes the stored hashes just as sensitive as the actual passwords.
Often, the hashed password is retrieved from the actual password together with a password salt. Then, the hash
together with the salt are stored. This makes brute-force-attacking harder as the search space is enlarged by the salt.
Challenge-response authentication
Examples of more sophisticated challenge-response algorithms are zero-knowledge password proof and key
agreement systems (such as Secure Remote Password (SRP)), CRAM-MD5, and ssh's challenge-response system
based on RSA [1].
[1] http:/ / www. cag. lcs. mit. edu/ ~rugina/ssh-procedures/
Cognitive password
A cognitive password is a form of knowledge-based authentication that requires a user to answer a question,
presumably something they intrinsically know, to verify their identity. Cognitive password systems have been
researched for many years and are currently commonly used as a form of secondary access. They were developed to
overcome the common memorability vs. strength problem that exists with the traditional password. Cognitive
passwords, when compared to other password systems, can be measured through the usage of a memorability vs.
guessability ratio.
Research on passwords as an authentification method has struggled between memorability and strong security.
Passwords that are easily remembered are easily cracked by attackers. On the other hand strong passwords are
difficult to crack but also difficult to remember.
When passwords are difficult to remember, users may write them
down, and the secrecy of the password is compromised.
. Early research into this tradeoff between security and
usability aimed to develop a password system that utilized easily remembered personal facts and encouraged user
participation. This line of research resulted in the concept of the associative password, a password system based on
user selected cues and responses.
This concept of associative passwords was extended to a pre-specified set of
questions and answers that users would be expected to know and could easily recall.
Cognitive questions
At the core of a cognitive password system lies the questions. These questions were designed to be more memorable
than the standard username/password authentication method. As such, a measure of the strength of a cognitive
password is the memorability/guessability ratio.
Question Development
Questions developed for cognitive password systems are classified as being either fact or opinion based. Fact based
systems have questions with answers that are considered independent of an individual's feelings such as "What is the
name of the high school you attended?". Opinion based questions are the opposite and, as the name implies, have
answers based on personal opinions such as, "What is your favorite color?"
Later research developed a set of
criteria for question selection which included generalized answerability, number of potential answers, and
generalized lack of ambiguity. The first criteria suggested that questions should be answerable by all (i.e. not asking
"When did you purchase your first home?" because not all users may have purchased homes). The second criteria
recommended selecting questions with a sufficiently large set of potential answers (i.e. not asking "How many
children do you have?" because a majority of people would answer 0, 1 or 2). The final criteria looked for questions
that were as unambiguous as possible (i.e. not asking "How many family members do you have?" as there may be
some confusion as to who would be included in that count).
Cognitive password
Memorability vs. guessability
A user's ability to correctly recall their password is expected to decrease as time progresses.
However, the
memorability of cognitive passwords remains relatively stable over time with recall rates significantly higher than
traditional passwords.

When fact and opinion-based questions are compared, the fact-based questions are
more likely to be correctly remembered than opinion-based questions, but still far more likely than traditional
Cognitive questions, with a group averaged as a whole, show relatively high guessability, much
higher than traditional passwords but when analyzed individually, certain questions have been shown to have
acceptable memorability/guessability ratios.
The following are some typical cognitive password questions:
• What is your mother’s maiden name?
• Who is your favorite superhero?
• What is your dog’s name
• What is your car's name?
• What is your favorite movie?
• What city were you born in?
• What is your favorite color?
[1] Harris, Shon (2002). "2" (http:// books. google. com/ books?id=Vp3MEDK0E7sC). Mike Meyers' CISSP(R) Certification Passport. Mike
Meyers' certification passport Passport Series (illustrated ed.). McGraw-Hill Professional. pp. 36. ISBN 9780072225785. .
[2] (Zviran and Haga, 1990a, p. 724)
[3] (Zviran and Elrich, 2006, p. 93)
[4] (Zviran and Haga, 1999, p. 173)
[5] (Smith, 1987)
[6] (Zviran and Haga, 1990a, p.723)
[7] (Bunnell et. al, 1997, p. 631)
[8] (Bunnell et. al, 1997, p. 633)
[9] (Brown et al., 2004, p. 642)
[10] (Bunnell et. al, 1997, p. 635)
[11] (Zviran and Haga, 1990a, p.728)
Works cited
• Brown, Alan S.; al, et. (2004), "Generating and Remembering Passwords", Applied Cognitive Psychology 18 (6):
• Bunnell, Julie; al, et. (1997), "Cognitive, associative and conventional passwords: Recall and guessing rates",
Computers & Security 16 (7): 629–641
• Smith, Sidney L. (1987), "Authenticating Users by Word Association", Human Factors and Ergonomics Society
31 (1): 135–138
• Zviran, Moshe; Haga, William J. (1990a), "Cognitive passwords: The key to easy access control", Computers &
Security 9 (8): 723–736
• Zviran, Moshe; Haga, William J. (1999), "Password Security: An Empirical Study", Journal of Management
Information Systems 15 (4): 161–185
• Zviran, Moshe; Elrich, Zippy (2006), "Identification and Authentication: Technology and Implementation Issues",
Communications of the Association for Information Systems 17 (4): 90–105
Cognitive password
External links
• Visual and Cognitive Password Authentication (http:/ / www.steam.ualberta. ca/ main/ research_areas/
Default password
Where a device needs a username and/or password to login, a default password is usually provided that allows the
device to be accessed during its initial setup. Manufacturers of such equipment typically use a simple password, such
as admin or password on all equipment they ship, in the expectation that users will change the password during
Generally, if the current password of a device is not available it may be necessary to reset the device to factory
defaults to re-enable the default password it was provided with from the factory. This often resets the entire device to
factory defaults and includes all configuration and data.
External links
Examples of default password databases:
• Default Passwords
• Default password list
• Alecto - Default password Database Project
• Default password Database
• Default Password List
• Default OEM password database
[1] http:/ / defaultpasswords. in/
[2] http:/ / www. phenoelit-us. org/dpl/ dpl. html
[3] http:/ / www. helith. net/ projects/ alecto
[4] http:/ / www. defaultpassword. us
[5] http:/ / default-password.info/
[6] http:/ / www. corrupteddatarecovery.com/ pages/ Default-Passwords-Data-Recovery.asp
Diceware is a method for creating passphrases, passwords, and other
cryptographic variables using ordinary dice as a hardware random
number generator. For each word in the passphrase, five dice rolls
are required. The numbers that come up in the rolls are assembled as
a five digit number, e.g. 43146. That number is then used to look up
a word in a word list. In the English list
43146 corresponds to
munch. Lists have been compiled for several languages, including
English, Finnish, German, Italian, Polish, Russian, Spanish and
Swedish. A Diceware word list is any list of unique
words, preferably ones the user will find easy to spell and to remember. The contents of the word list do not have to
be protected or concealed in any way, as the security of a Diceware passphrase is in the number of words selected,
and the number of words each selected word could be taken from.
The level of unpredictability of a Diceware passphrase can be easily calculated: each word adds 12.9 bits of entropy
to the passphrase (that is, bits). Five words (slightly over 64 bits) are considered a minimum length.
This level of unpredictability assumes that a potential attacker knows both that Diceware has been used to generate
the passphrase, the particular word list used, and exactly how many words make up the passphrase. If the attacker
has less information, the entropy can be greater than 12.9 bits per word.
Diceware passphrases can be difficult to remember and some may prefer other methods, such as using the initial
letters of a memorable phrase (for instance, "To be or not to be, that is the question" becomes "Tbontb,titq").
Estimating the entropy of the phrase using the latter approach is more difficult. In this example, the phrase used is
very well-known (being from Shakespeare) and so is easily guessed, as for instance by using a phrase dictionary.
Thus, the entropy of this example is low. Higher entropy can be had from user chosen phrases handled this way, if
the user is careful to avoid guessable phrases.
If the length of Diceware passphrases are assumed to be known to an attacker, then the passphrases yield less
entropy than the ideal 64.62 bits when used with dictionaries containing variable-length words. This is because the
length of the resulting passphrases "leak" information about their composition.
• Internet Secrets, 2nd Edition, John R. Levine, Editor, Chapter 37, IDG Books, 2000, ISBN 0-7645-3239-1
External links
• English diceware page
has the complete description and a word list.
• Dialdice
has a nicely-formatted word list.
[1] http:/ / world.std. com/ ~reinhold/diceware. wordlist. asc
[2] http:/ / world.std. com/ ~reinhold/diceware. html
[3] http:/ / zzzen.com/ dialdice. html
Draw a Secret
Draw a Secret
Draw a Secret (DAS) is a purely graphical password selection and input scheme. The scheme replaces alphanumeric
password strings, with a simple picture drawn on a grid.
Background Draw a Secret (BDAS), a variant of Draw a Secret, is purely graphical password selection and input
scheme. The scheme replaces alphanumeric password strings, with a simple picture drawn on a background image.
[1] [[Newcastle University (http:// www. ncl. ac. uk/ press. office/newslink/ ?ref=1193216061)] NewsLink, October 24, 2007]
Duress code
A duress code is a covert signal used by an individual that is under duress to indicate their state. The term duress
code typically refers to a signal embedded in normal communication, such as a word or phrase used during
conversation to alert other personnel of the duress. Alternatively, the signal may be incorporated into the
authentication process itself, typically in the form of a panic password, distress password, or duress PIN that is
distinct from the user's normal password or PIN. These concepts are related to a panic alarm and often achieve the
same outcome.
Civilian usage
Some home and property alarm systems have duress PINs, where the last two digits of the reset code are switched
around. Entering the code when under duress from an assailant can trigger a silent alarm, alerting police or security
personnel in a covert manner. The implementation of this feature has not been without controversy, as it has been
claimed to lead to false alarms.
A similar mechanism, SafetyPIN, has been proposed for use in ATMs. Note that in
both of these cases, the adversary can request the PIN in advance and ensure the appropriately modified PIN is
entered instead or choose randomly between the two possible codes. This allows the adversary to succeed half of the
In scenarios where a panic password is used to limit access control, instead of triggering an alarm, it is insufficient to
have a single panic password. If the adversary knows the system, a common assumption, then he will simply force
the user to authenticate twice using different passwords and gain access on at least one of the two attempts. More
complex panic password schemes have been proposed to address this problem.

For cases where verbal communication (i.e. via cell phone) is possible with family member or friend, a covert phrase
can be used to signal duress. In the slim chance that a captor allows the person in duress to use their cell phone (i.e.
to obtain a PIN), there is a limited opportunity to use a duress code. Because conversations are often being monitored
by a captor, they must be subtle and short. Ideally, the use of a duress code has been confirmed before the current
situation, so the family member or friend has verifiable evidence that something is wrong, and when the authorities
are notified aren't just limited to speculation. Examples would include asking about someone who does not exists.
For example, "What is Cindy barking at?" when the person on the other side knows that either there is no dog, or
dog's name is Maggie. Covert phrases can be obvious to a captor, when a caller has no reason to state something, like
"How is Mary?". The captor knows that asking about Mary is not related to obtaining an ATM PIN, and therefore
becomes suspicious and possibly agitated.
In addition to a duress code, there is duress activity. This may include the duressed individual withdrawing cash
from an ATM using a specific credit card, instead using their debit card. Many credit card companies allow for email
alerts to be setup when specific activity occurs. There are technical issues that could pose problems, such as a delay
Duress code
in notification, cellular network availability, and the fact that a location is not disclosed, only the activity.
Military Usage
A simple but effective duress code used over the telephone by SOE agents in occupied Europe during World War II
was to give a coded answer when someone checked whether it was convenient to visit a safe-house. If it was
genuinely safe to visit, the answer would be "No, I'm too busy." However, if the safe-house had been compromised
(i.e. the Nazis had captured it, forcing the occupants to answer the phone at gunpoint in order to lure in other
members of the SOE network) the captured agent would say "Yes, come on over." Having been warned that the
safe-house had been compromised, the other agent would hang up the phone and immediately inform his
team-members so that they could take appropriate action. Typically, this meant using escape and evasion procedures,
before the captured agent was tortured by the Gestapo and forced to give incriminating information such names and
addresses etc.
References in popular culture
The concept of duress codes is used in fiction, particularly in spy fiction. In the fourth episode of the fifth season of
24, hostage and CTU agent Jack Bauer used the phrase "flank two" as a duress code. In the 2007 film The Bourne
Ultimatum, a CIA agent Nicky Parsons is given the option of using a panic password when communicating with
headquarters. In the Star Trek episode Bread and Circuses, James T. Kirk gives the Duress Code "Condition Green",
implying to his captors that all is well (compare "Condition Red"), but to his crew that he had been compromised.
[1] http:/ / www. faraonline.org/DuressResolution. pdf
[2] http:// www. cs. uwaterloo.ca/ ~j5clark/papers/ panic. pdf
[3] http:// portal.acm. org/ citation. cfm?id=1866895
LM hash
LM hash
LM hash, LanMan, or LAN Manager hash is the primary hash that Microsoft LAN Manager and Microsoft
Windows versions prior to Windows NT used to store user passwords. Support for the legacy protocol continued in
later versions of Windows for backward compatibility, but was recommended by Microsoft to be turned off by
administrators; as of Windows Vista, the protocol is disabled by default, but continues to be used by some
non-Microsoft CIFS implementations.
The LM hash is computed as follows:

1. The user’s ASCII password is converted to uppercase.
2. This password is null-padded to 14 bytes.

3. The “fixed-length” password is split into two 7-byte halves.
4. These values are used to create two DES keys, one from each 7-byte half, by converting the seven bytes into a bit
stream, and inserting a null bit after every seven bits (so 1010100 becomes 01010100). This generates the 64 bits
needed for a DES key. (A DES key ostensibly consists of 64 bits; however, only 56 of these are actually used by
the algorithm. The null bits added in this step are later discarded.)
5. Each of the two keys is used to DES-encrypt the constant ASCII string “KGS!@#$%”, resulting in two 8-byte
ciphertext values. The DES CipherMode should be set to ECB, and PaddingMode should be set to NONE.
6. These two ciphertext values are concatenated to form a 16-byte value, which is the LM hash.
Security weaknesses
Although it is based on DES, a well-studied block cipher, the LM hash is not a true one-way function as the
password can easily be determined from the hash because of weaknesses in its implementation: Firstly, passwords
longer than 7 characters are divided into two pieces and each piece is hashed separately. Secondly, LM hash does not
use cryptographic salt, a standard technique to prevent pre-computed dictionary attacks. In addition, all lower case
letters in the password are changed to upper case before the password is hashed. The first weakness allows each half
of the password to be attacked separately, at much lower cost. While there are different passwords
made of up to 14 printable ASCII characters, there would be only different 7 character password pieces
using the same character set. Converting lowercase character to uppercase further reduces the key space for each half
to . By mounting a brute force attack on each half separately, modern desktop machines can crack
alphanumeric LM hashes in a few hours.
However, brute force attacks are unnecessary. Because LM hash does not employ salt, a time-memory trade-off
cryptanalysis attack, such as rainbow tables, is also feasible. In 2003, Ophcrack, an implementation of the rainbow
table technique, was published. It specifically targets the weaknesses of LM encryption, and includes pre-computed
data sufficient to crack virtually all alphanumeric LM hashes in a few seconds. Many cracking tools, e.g.
RainbowCrack, L0phtCrack and Cain, now incorporate similar attacks and make cracking of LM hashes trivial.
However, because LM hashing is not used for passwords of 15 characters or longer, such passwords are not subject
to these attacks.
LM hash
To address the security weaknesses inherent in LM encryption, Microsoft introduced the NTLM algorithm with
Windows NT 3.1. NTLM added Unicode support, the RC4 cipher (which does not require any padding or truncating
that would simplify the key). On the negative side, the same DES algorithm is used with only 56-bit encryption.
Furthermore, many Windows clients were configured by default to send both the LM hash and the NTLM hash, so
the use of the NTLM hash provided no additional security while the weaker hash was still present.
While LAN Manager is considered obsolete and current Windows operating systems use the stronger NTLMv2 or
Kerberos hashing methods, Windows systems before Windows Vista/Windows Server 2008 still compute and store
the LAN Manager hash by default for compatibility with LAN Manager and Windows Me or earlier clients, as well
as some 16-bit applications that are still in use on the most current versions of Windows. It is considered good
security practice to disable this feature where it isn't needed.
Microsoft claimed that support for LM would be
completely eliminated in the Windows Vista operating system.
However Windows Vista and Windows Server
2008 still include support for the LM hash, although it is now disabled by default; the feature can be enabled for
local accounts via a security policy setting, and for Active Directory accounts by applying the same setting to
domain controllers. The same method can be used to turn the feature off in Windows 2000, Windows XP and NT.
Users can also prevent a LM hash from being generated for their password by using a password at least 15 characters
in length.
Reasons for continued use
Many legacy third party CIFS implementations have taken considerable time to add support for the stronger
protocols that Microsoft has created to replace LM Hashing because the open source communities supporting these
libraries first had to reverse engineer the newer protocols—Samba took 5 years to add NTLMv2 support, while
JCIFS took 10 years.
Product NTLMv1 support NTLMv2 support
Windows NT 3.1 RTM (1993) Not supported
Windows NT 3.5 RTM (1994) Not supported
Windows NT 3.51 RTM (1995) Not supported
Windows NT 4 RTM (1996)
Service Pack 4
(25 October 1998)
Windows 95 Not supported Directory services client (released with Windows 2000 Server, 17 February 2000)
Windows 98 RTM Directory services client (released with Windows 2000 Server, 17 February 2000)
Windows 2000 RTM (17 February 2000) RTM (17 February 2000)
Windows ME RTM (14 September 2000) Directory services client (released with Windows 2000 Server, 17 February 2000)
Samba ?
Version 3.0
(24 September 2003)
JCIFS Not supported
Version 1.3.0 (25 October 2008)
|+ Availability of NTLM protocols to replace LM
Poor patching regimes subsequent to software releases supporting the feature becoming available have contributed to
some organisations continuing to use LM Hashing in their environments, even though the protocol is easily disabled
in Active Directory itself.
Lastly, prior to the release of Windows Vista, many unattended build processes still used a DOS boot disk (instead of
Windows PE) to start the installation of Windows using WINNT.EXE, something that requires LM hashing to be
enabled for the legacy LAN Manager networking stack to work.
LM hash
[1] "Chapter 3 - Operating System Installation: The LMHash" (http:/ / technet.microsoft.com/ en-us/ library/dd277300.aspx). Microsoft. .
Retrieved 2009-06-21.
[2] Glass, Eric (2003). "The NTLM Authentication Protocol" (http:// davenport. sourceforge.net/ ntlm.html#theLmResponse). . Retrieved
[3] If the password is more than 14 characters long, the LM hash cannot be computed.
[4] "Cluster service account password must be set to 15 or more characters if the NoLMHash policy is enabled" (http:// support. microsoft.com/
kb/828861). Microsoft. 2006-10-30. . Retrieved 2009-06-21.
[5] "How to prevent Windows from storing a LAN manager hash of your password in Active Directory and local SAM databases" (http://
support.microsoft.com/ default. aspx?scid=KB;EN-US;q299656& ). Microsoft Knowledge Base. . Retrieved 2006-06-05.
[6] Johansson, Jesper (August 2006). "The Most Misunderstood Windows Security Setting of All Time" (http:// www.microsoft.com/ technet/
technetmag/issues/ 2006/ 08/ SecurityWatch/). TechNet Magazine. . Retrieved 2007-01-08.
[7] How to prevent Windows from storing a LAN manager hash of your password in Active Directory and local SAM databases (http:// support.
microsoft.com/ default. aspx?scid=KB;EN-US;q299656& )
[8] "Windows NT 4.0 Service Pack 4 Readme.txt File (40-bit)" (http:/ / support. microsoft.com/ kb/ 194507). Microsoft. 1998-10-25. . Retrieved
[9] "The Samba Team announces the first official release of Samba 3.0" (http://www.samba. org/samba/ history/ samba-3. 0. 0. html).
samba.org. 2003-09-24. . Retrieved 2010-05-27.
[10] "The JCIFS library: News" (http://jcifs. samba. org/ ). . Retrieved 2010-05-27.
External links
• Making a Faster Cryptanalytic Time-Memory Trade-Off, Philippe Oechslin, Advances in Cryptology - CRYPTO
2003 (http:// lasecwww. epfl.ch/ ~oechslin/ publications/ crypto03.pdf)
• Ophcrack, the time-memory-trade-off-cracker (http:// ophcrack. sourceforge.net/ )
• The Shmoo Group (http:/ / rainbowtables. shmoo. com/ ) offers pre-computed rainbow tables which are
downloadable via BitTorrent
• Cain and Abel (http:/ / www. oxid. it/ )
• Online LM/NTLM crack using Rainbow tables (http:/ / www.OnlineHashCrack. com/ )
• Plain-Text organization is available for cracking LM hashes (http:// www.plain-text.info/ )
• A Java applet for computing the LM Hash and NT Hash (http:/ / www.arsitech. com/ cryptography/windows/
password/ )
• Java code for LM Hash (http:// forums.sun. com/ thread. jspa?threadID=742735&tstart=2056)
• Mastering Windows Network Forensics (http:/ / books. google. com/ books?id=BhdP2PZy6SoC& pg=PA80&
Munged password
Munged password
In computing, the term munge English pronunciation: /ˈmʌndʒ/ means to attempt to create a strong, secure password
through character substitution. "Munge" is sometimes backronymmed as Modify Until Not Guessed Easily. The
usage differs significantly from Mung (Mash Until No Good), because munging implies destruction of data, while
mungeing implies creation of strong protection for data.
Passwords are used to gain access to computer resources, and computer users generally choose passwords that are
easy to remember, but therefore insecure. Simple passwords are easily hacked by dictionary attacking software.
If a network administrator supplies a password that is too difficult to remember, or requires that passwords be
changed frequently, users tend to write their passwords down to help them remember. Many times passwords can be
found on sticky notes under keyboards, behind pictures, or hidden among other desktop items—another security risk.
Mungeing helps to create a strong password that the user can remember easily. The user may choose any word he or
she likes, then modifies it to make it stronger.
A strong password contains characters from at least 3 of these 4 character sets:
Lower case abcdefghijklmnopqrstuvwxyz
Numbers 01234567890
Special !@#$%^&*()-=_+<>?
Adding a number and/or special character to a password may thwart simple dictionary attacks. For example, the
password "butterfly" could be munged in the following ways:
8uttErfly The b becomes an eight (B8), and any other letter can be capitalized
butt3rfl? The e becomes a three (E3), and the Y becomes a question mark (Y = Why?)
bu2Terfly The two Ts become 2T
8u2T3RfL? a combination of all the above
The substitutions can be anything the user finds easy to remember, such as:
a=@ b=8 c=( d=6 e=3 f=# g=9 h=# i=1 i=! k=< l=1
l=i o=0 q=9 s=5 s=$ t=+ v=> v=< w=uu w=2u x=% y=?
For high-security applications, mungeing may not be very effective, because it only adds 2-3 bits of Entropy,
increasing the time needed to perform a dictionary attack by a factor of 4-8. The increase in search space obtained by
mungeing a few characters of a known word is easily matched by the continuous increase in processing power (ie,
cracking speed), although this can be countered for some applications by limiting password attempts to either one
per few seconds or 5 per longer period of time, usually 5 minutes to an hour.
Munged password
External links
• Jargon File entry for munge (http:// catb. org/jargon/ html/M/ munge.html)
One-time password
A one-time password (OTP) is a password that is valid for only one login session or transaction. OTPs avoid a
number of shortcomings that are associated with traditional (static) passwords. The most important shortcoming that
is addressed by OTPs is that, in contrast to static passwords, they are not vulnerable to replay attacks. This means
that, if a potential intruder manages to record an OTP that was already used to log into a service or to conduct a
transaction, he or she will not be able to abuse it since it will be no longer valid. On the downside, OTPs are difficult
for human beings to memorize. Therefore they require additional technology in order to work.
How OTPs are generated and distributed
OTP generation algorithms typically make use of randomness. This is necessary because otherwise it would be easy
to predict future OTPs from observing previous ones. Concrete OTP algorithms vary greatly in their details. Various
approaches for the generation of OTPs are listed below.
• Based on time-synchronization between the authentication server and the client providing the password (OTPs
are valid only for a short period of time)
• Using a mathematical algorithm to generate a new password based on the previous password (OTPs are,
effectively a chain and must be used in a predefined order).
• Using a mathematical algorithm where the new password is based on a challenge (e.g., a random number
chosen by the authentication server or transaction details) and/or a counter.
There are also different ways to make the user aware of the next OTP to use. Some systems use special electronic
tokens that the user carries and that generate OTPs and show them using a small display. Other systems consist of
software that runs on the user's mobile phone. Yet other systems generate OTPs on the server-side and send them to
the user using an out-of-band channel such as SMS messaging. Finally, in some systems, OTPs are printed on paper
that the user is required to carry with them.
Methods of generating the OTP
RSA SecurID tokens.
A time-synchronized OTP is usually related to a piece of hardware
called a security token (e.g., each user is given a personal token that
generates a one-time password). Inside the token is an accurate clock
that has been synchronized with the clock on the proprietary
authentication server. On these OTP systems, time is an important part
of the password algorithm since the generation of new passwords is
based on the current time rather than, or in addition to, the previous
password or a secret key. This token may be a proprietary device, or a
mobile phone or similar mobile device which runs software that is
proprietary, freeware, or open-source. An example of time-synchronized OTP standard is TOTP.
All of the methods of delivering the OTP below may use time-synchronization instead of algorithms.
One-time password
Mathematical algorithms
Each new OTP may be created from the past OTPs used. An example of this type of algorithm, credited to Leslie
Lamport, uses a one-way function (call it f). The one-time password system works by starting with an initial seed s,
then generating passwords
f(s), f(f(s)), f(f(f(s))), ...
as many times as necessary. If an indefinite series of passwords is wanted, a new seed value can be chosen after the
set for s is exhausted. Each password is then dispensed in reverse, with f(f(...f(s))...) first, to f(s).
If an intruder happens to see a one-time password, he may have access for one time period or login, but it becomes
useless once that period expires. To get the next password in the series from the previous passwords, one needs to
find a way of calculating the inverse function f
. Since f was chosen to be one-way, this is extremely difficult to do.
If f is a cryptographic hash function, which is generally the case, it is (so far as is known) a computationally
infeasible task.
In some mathematical algorithm schemes, it is possible for the user to provide the server with a static key for use as
an encryption key, by only sending a one time password.
The use of challenge-response one-time passwords will require a user to provide a response to a challenge. For
example, this can be done by inputting the value that the token has generated into the token itself. To avoid
duplicates, an additional counter is usually involved, so if one happens to get the same challenge twice, this still
results in different one-time passwords. However, the computation does not usually involve the previous one-time
password; that is, usually this or another algorithm is used, rather than using both algorithms.
The methods of delivering the OTP which are token-based may use either of these types of algorithm instead of
Methods of delivering the OTP
Text Messaging
A common technology used for the delivery of OTPs is text messaging. Because text messaging is a ubiquitous
communication channel, being available in nearly all handsets and with a large customer-base, text messaging has a
great potential to reach all consumers with a low total cost to implement. However, the cost of each text messaging
often for each OTP might not be suitable for some users. OTP over text messaging may also be encrypted using an
A5/x standard which several hacking groups report can be successfully decrypted within minutes or seconds,


or the OTP over SMS might not be encrypted by one's service-provider at all. In addition to threats from hackers,
the mobile phone operator becomes part of the trust chain. In the case of roaming, more than a single mobile phone
operator has to be trusted. Anyone using this information may mount a man-in-the-middle attack.
Mobile Phones
A mobile phone keeps costs low because a large customer-base already owns a mobile phone for purposes other than
generating OTPs. The computing power and storage required for OTPs is usually insignificant compared to that
which modern camera-phones and smartphones typically use. Mobile tokens additionally support any number of
tokens within one installation of the application, allowing a user the ability to authenticate to multiple resources from
one device. This solution also provides model-specific applications to the user's mobile phone. However, a cellphone
used as a token can be lost, damaged, or stolen.
One-time password
Proprietary Tokens
EMV is starting to use a challenge-response algorithm (called "Chip Authentication Program") for credit cards in
Europe. On the other hand, in access control for computer networks, RSA Security's SecurID is one example of a
time-synchronization type of token. Like all tokens, these may be lost, damaged, or stolen; additionally there is an
inconvenience as batteries die (typically cannot plug these into a battery-charger, and this is one more battery that
must be replaced, or in some cases the whole token must be replaced). A variant of the proprietary token was
proposed by RSA in 2006 and was described as "ubiquitous authentication",
in which RSA would partner with
manufacturers to add physical SecurID chips to devices such as mobile phones.
Recently, it has become possible to take the electronic components associated with regular keyfob OTP tokens and
embed them in a credit card form factor. However, because card thickness (.79mm to .84mm) prevents traditional
components or batteries from being used, special polymer-based batteries must be used which have a much lower
battery life than traditional coin (button) cells. Also, extremely low-powered semiconductor components must be
used to conserve the amount of power being used during sleep and/or actual use of the product. Two companies in
particular have led in the production of thin "display card OTP" devices. Those being Identita (http:/ / www.identita.
com) and NagraID Security(http:/ / www. nidsecurity. com/ )
A new version of this technology has been developed that embeds a keypad into a payment card using the same form
and thickness as traditional cards. This card is a credit/debit/ ATM/ Credit ID card with embedded
keypad,display,microprocessor and proximity chip. This new card technology has the potential to leap frog existing
card security options.
Web-based methods
Authentication-as-a-service providers offer various web-based methods for delivering one-time passwords without
the need for tokens. One such method relies on the user’s ability to recognize pre-chosen categories from a
randomly-generated grid of pictures. When first registering on a website, the user chooses several secret categories
of things; such as dogs, cars, boats and flowers. Each time the user logs into the website they are presented with a
randomly-generated grid of pictures. Each picture in the grid has a randomly-generated alphanumeric character
overlaid on it. The user looks for the pictures that fit their pre-chosen categories and enters the associated
alphanumeric characters to form a one-time access code.

Paper-based OTP web-site login
In some countries online banking, the bank sends to the user a
numbered list of OTPs that are printed on paper. For every online
transaction, the user is required to enter a specific OTP from that list.
In Germany, those OTPs are typically called TANs (for 'transaction
authentication numbers'). Some banks even dispatch such TANs to the
user's mobile phone via SMS, in which case they are called mTANs
(for 'mobile TANs').
Comparison of technologies
One-time password
One OTP implementation versus another
In terms of costs, the cheapest OTP solutions are those that deliver OTPs on paper, and those that generate OTPs on
a device that someone already owns. This is because these systems avoid the costs associated with (re-)issuing
proprietary electronic tokens and the cost of SMS messaging.
For systems that rely on electronic tokens, algorithm-based OTP generators must cope with the situation where a
token drifts out-of-sync with its server if the system requires the OTP to be entered by a deadline. This leads to an
additional development cost. Time-synchronized systems, on the other hand, avoid this at the expense of having to
maintain a clock in the electronic tokens (and an offset value to account for clock drift). Whether or not OTPs are
time-synchronized is basically irrelevant for the degree of vulnerability, it but avoids a need to reenter passwords if
the server is expecting the last or next code that the token should be having because the server and token have drifted
Compared to most proprietary hardware tokens, so long as one already carries a phone or another mobile device in
one's pocket, users of mobile devices don't need to carry and protect an extra item (which has no usefulness except
that it generates OTPs). In addition to reducing costs considerably, using a phone as a token offers the convenience
that it is not necessary to deliver devices to each end-user (who typically already own the device). For many users, a
mobile phone may also be trickle-charged to preserve its battery for at least some portion of each day, whereas most
proprietary tokens cannot be trickle-charged. However, most proprietary tokens have tamper-proof features.
OTPs versus other methods of securing data
One-time passwords are vulnerable to social engineering attacks in which phishers steal OTPs by tricking customers
into providing one or more OTPs that they used in the past. In late 2005 customers of a Swedish bank were tricked
into giving up their one-time passwords (The Register article
). In 2006 this type of attack was used on customers
of a US bank (Washington Post Security Blog
). Even time-synchronized OTPs are vulnerable to phishing, by
two methods: The password may be used as quickly by the attacker as the legitimate user must use the OTP, if the
attacker can get the OTP in plaintext quickly enough. The other type of attack—which may be defeated if one's OTP
system implements using the hash chain as discussed above -- is that after the phisher uses this social engineering,
the phisher must then use the information gained (past OTP codes which are no longer valid) to predict what OTP
codes will be used in the future (e.g. an OTP password-generator that is pseudo-random rather than truly random
might or might not be able to be compromised, because pseudo-random numbers are often predictable once one has
the past OTP codes (see also main article); the code implemented in the programming of each Direct OTP or token
will determine whether one is vulnerable to this type of attack.
Although OTPs are in some ways more secure than a memorized password, users of OTP systems are still vulnerable
to man-in-the-middle attacks. OTPs should therefore not be disclosed to any third parties, and using an OTP as one
layer in layered security is safer than using OTP alone; one way to implement layered security is to use an OTP in
combination with a password that is memorized by the user (and never transmitted to the user, like OTPs often are).
An advantage to using layered security is that a single sign-on combined with one master password or password
manager becomes safer than using only 1 layer of security during the sign-on, and thus the inconvenience of
password fatigue is avoided if one usually has long sessions with many passwords that would need to be entered
mid-session (to open different documents, websites, and applications); however, the disadvantage of using many
forms of security all at once during a single sign-on is that one has the inconvenience of more security precautions
during every login—even if one is logging-in only for a brief usage of the computer to access information or an
application that doesn't require as much security as some other top-secret items that computer is used for. See also
Related technologies, below.
One-time password
Related technologies
More often than not, one-time passwords are an embodiment of two-factor authentication (T-FA). T-FA is a form of
layered security where it is unlikely that both layers would be disabled by someone using only one type of attack.
Some single sign-on solutions make use of one-time passwords. One-time password technology is often used with a
security token.
Newer, interactive T-FA approaches, such as Duo Security's Duo Push technology
Technologies' Interactive Transaction Authentication (ITA) system, attempt to close the loop where attackers could
get hold of OTPs, by prompting a user on a paired mobile phone about the transaction taking place. When accepting
the transaction, the message is again relayed (over GPRS or SMS technology) to the authentication server. The
whole transaction is encrypted using standard Public/Private Key Encryption.
Many OTP technologies are patented. This makes standardization in this area more difficult, as each company tries
to push its own technology. Standards do, however, exist, for example RFC 2289
and RFC 4226 (HOTP).
External links
• RSA Labs OTP standardization proposal
• Mobile one time password for windows phone
• Cross-platform and HOTP compliant mobile solution
• Open OTP solution for mobile phones
[1] EOTP - Static Key Transfer (http:// ossbox. com/ index. php?page=eotp)
[2] Barkan, Elad; Eli Biham; Nathan Keller (2003). "Instant Ciphertext-Only Cryptanalysis of GSM Encrypted Communication" (http://
cryptome.org/gsm-crack-bbk.pdf). Crypto 2003: 600–16. .
[3] Barkan, Elad; Eli Biham; Nathan Keller. "Instant Ciphertext-Only Cryptanalysis of GSM Encrypted Communication by Barkan and Biham of
Technion (Full Version)" (http:/ / www.cs. technion. ac. il/ users/ wwwb/ cgi-bin/ tr-get.cgi/ 2006/CS/ CS-2006-07.pdf). .
[4] Gueneysu, Tim; Timo Kasper; Martin Novotný; Christof Paar; Andy Rupp (2008). "Cryptanalysis with COPACOBANA" (http:/ / www.
sciengines. com/ copacobana/ paper/ TC_COPACOBANA. pdf). Transactions on Computers Nov. 2008: 1498–1513. .
[5] Nohl, Karsten; Chris Paget (2009-12-27). "GSM: SRSLY?" (http:// events. ccc.de/ congress/ 2009/ Fahrplan/ events/ 3654. en.html). 26th
Chaos Communication Congress (26C3):. . Retrieved 2009-12-30.
[6] http:/ / www. encyclopedia. com/ doc/ 1G1-142107014.html
[7] Ericka Chickowski (2010-11-03). "Images Could Change the Authentication Picture" (http:/ / www. darkreading.com/ authentication/
security/client/ showArticle. jhtml?articleID=228200140). Dark Reading. .
[8] "Confident Technologies Delivers Image-Based, Multifactor Authentication to Strengthen Passwords on Public-Facing Websites" (http:/ /
www.marketwire.com/ press-release/
Confident-Technologies-Delivers-Image-Based-Multifactor-Authentication-Strengthen-Passwords-1342854. htm). 2010-10-28. .
[9] http:// www. theregister.co. uk/ 2005/10/ 12/ outlaw_phishing/
[10] http:// blog.washingtonpost. com/ securityfix/2006/ 07/ citibank_phish_spoofs_2factor_1. html
[11] Duo Push: SSH keys that call you back (http:// blog. duosecurity.com/ 2011/ 04/ ssh-keys-that-call-you-back/)
[12] http:/ / www. ietf.org/ rfc/rfc2289.txt
[13] http:/ / www. rsasecurity. com/ rsalabs/ node. asp?id=2816
[14] http:/ / social. zune. net/ redirect?type=phoneApp&id=e9d1040a-20f4-df11-9264-00237de2db9e
[15] http:// www. iamboo. it/ en/ prodotti/iambootp
[16] http:/ / motp.sourceforge.net/
The OpenID logo
OpenID is an open standard that describes how users can be
authenticated in a decentralized manner, obviating the need for
services to provide their own ad hoc systems and allowing users to
consolidate their digital identities.
The OpenID protocol does not rely on a central authority to
authenticate a user's identity. Moreover, neither services nor the
OpenID standard may mandate a specific means by which to
authenticate users, allowing for approaches ranging from the common (such as passwords) to the novel (such as
smart cards or biometrics).
The term OpenID may also refer to an ID as specified in the OpenID standard; these IDs take the form of a unique
URL, and are managed by some 'OpenID provider' that handles authentication.
OpenID authentication is now used and provided by several large websites. Providers include AOL, BBC,
IBM, MySpace, Orange, PayPal, VeriSign, LiveJournal, and Yahoo!.



Technical Overview
A basic glossary of the terms used with OpenID:
The entity that wants to assert a particular identity.
Identifier or OpenID
The URL or XRI chosen by the end-user to name the end-user's identity.
Identity provider or OpenID provider
A service that specializes in registering OpenID URLs or XRIs and providing OpenID authentication (and
possibly other identity services). Note that the OpenID specifications use the term "OpenID provider" or "OP".
Relying party
The site that wants to verify the end-user's identifier; other terms include "service provider" or the now
obsolete "consumer".
The program (such as a browser) used by the end-user to communicate with the relying party and OpenID
Logging in
The end-user interacts with a relying party (such as a website) that provides a means by which to specify an OpenID
for the purposes of authentication; an end-user typically has previously registered an OpenID (e.g.
alice.openid.example.org) with an OpenID provider (e.g. openid.example.org).
The relying party typically transforms the OpenID into a canonical URL form (e.g. http://alice.openid.example.org/).
• With OpenID 1.0, the relying party then requests the HTML resource identified by the URL and reads an HTML
link tag to discover the OpenID provider's URL (e.g. http://openid.example.org/openid-auth.php). The relying
party also discovers whether to use a delegated identity (see below).
• With OpenID 2.0, the relying party discovers the OpenID provider URL by requesting the XRDS document (also
called the Yadis document) with the content type application/xrds+xml; this document may be available at the
target URL and is always available for a target XRI.
There are two modes in which the relying party may communicate with the OpenID provider:
• checkid_immediate, in which the relying party requests that the OpenID provider not interact with the end-user.
All communication is relayed through the end-user's user-agent without explicitly notifying the end-user.
• checkid_setup, in which the end-user communicates with the OpenID provider via the same user-agent used to
access the relying party.
The checkid_setup mode is more popular on the Web; also, the checkid_immediate mode can fall back to the
checkid_setup mode if the operation cannot be automated.
First, the relying party and the OpenID provider (optionally) establish a shared secret, referenced by an associate
handle, which the relying party then stores. If using the checkid_setup mode, the relying party redirects the user's
user-agent to the OpenID provider so the end-user can authenticate directly with the OpenID provider.
The method of authentication may vary, but typically, an OpenID provider prompts the end-user for a password or
an InfoCard, and then asks whether the end-user trusts the relying party to receive the necessary identity details.
If the end-user declines the OpenID provider's request to trust the relying party, then the user-agent is redirected back
to the relying party with a message indicating that authentication was rejected; the relying party in turn refuses to
authenticate the end-user.
If the end-user accepts the OpenID provider's request to trust the relying party, then the user-agent is redirected back
to the relying party along with the end-user's credentials. That relying party must then confirm that the credentials
really came from the OpenID provider. If the relying party and OpenID provider had previously established a shared
secret, then the relying party can validate the identity of the OpenID provider by comparing its copy of the shared
secret against the one received along with the end-user's credentials; such a relying party is called stateful because it
stores the shared secret between sessions. In contrast, a stateless or dumb relying party must make one more
background request (check_authentication) to ensure that the data indeed came from the OpenID provider.
After the OpenID has been verified, authentication is considered successful and the end-user is considered logged in
to the relying party under the identity specified by the given OpenID (e.g. alice.openid.example.org). The relying
party typically then stores the end-user's OpenID along with the end-user's other session information.
To obtain an OpenID-enabled URL that can be used to log into OpenID-enabled websites, a user needs to register an
OpenID identifier with an identity provider. Identity providers offer the ability to register a URL (typically a
third-level domain, e.g. username.example.com) that will automatically be configured with OpenID authentication
Once they have registered an OpenID, a user can also use an existing URL under their own control (such as a blog or
home page) as an alias or "delegated identity". They simply insert the appropriate OpenID tags in the HTML
serve a Yadis document.
Starting with OpenID Authentication 2.0 (and some 1.1 implementations), there are two types of identifiers that can
be used with OpenID: URLs and XRIs.
XRIs are a new form of Internet identifier designed specifically for cross-domain digital identity. For example, XRIs
come in two forms—i-names and i-numbers—that are usually registered simultaneously as synonyms. I-names are
reassignable (like domain names), while i-numbers are never reassigned. When an XRI i-name is used as an OpenID
identifier, it is immediately resolved to the synonymous i-number (the CanonicalID element of the XRDS
document). This i-number is the OpenID identifier stored by the relying party. In this way, both the user and the
relying party are protected from the user's OpenID identity ever being taken over by another party as can happen
with a URL based on a reassignable DNS name.
As of December 2009, there are over 1 billion OpenID enabled accounts on the Internet (see below) and
approximately 9 million sites have integrated OpenID consumer support.
OpenID Providers
Site URL Format Comments
Yahoo! me.yahoo.com
Yahoo! began allowing their usernames to be used as openIDs beginning January 31,
Yahoo! does not require the username to be passed in the openID string.
LiveJournal username.livejournal.com LiveJournal supports OpenID as both a provider and a relying party.
MySpace myspace.com/username
WordPress username.wordpress.com
Blogger username.blogger.com
Google Profile
Google does not require the username to be passed in the openID string.
Verisign username.pip.verisignlabs.com Verisign offers a secure OpenID service, with two-factor authentication, which they call
"Personal Identity Provider"
Typepad blogname.typepad.com
ClaimID claimid.com/username
username.clavid.com Strong Authentication OpenID Provider supporting Password, YubiKey, SMS-OTP,
iPhone/Android OTP's, Certificates/Smartcards as well as AXSionics biometric
fingerprint reader.
steamcommunity.com/openid/ Allows one to use OpenID services with their Steam login and password
Orange openid.orange.fr/username or just
Offers OpenIDs to their broadband subscribers, and accepts OpenID to allow non
subscriber users to access a subset of services.
decentralized & private OpenID provider that allows one to use their tonido url as
Launchpad launchpad.net/~username See [23] for details.
seznam.cz username.id.seznam.cz
Offers personas
for easy access on different required/optional contact info details
A Dutch service.
A Japanese service.
Relying parties and other services
• Other services accepting OpenID as an alternative to registration include Wikitravel,
photo sharing host
Zooomr, identity aggregator ClaimID, calendar booking Bookwhen, icon provider IconBuffet, user stylesheet
repository UserStyles.org, Music Xray.
• SourceForge
• Stack Exchange Network
• Luxsci is both an OpenID consumer and provider.
• Facebook supports OpenID 2.0, allowing an existing account to have an OpenID associated as an alternative login
Facebook connect provides an API for other websites to leverage Facebook logins.
• In 2.0 RC1.1, Simple Machines Forum allows the administrator to allow registration using an OpenID.
• RCDevs provides an OpenID 2.0 server, allowing users to authenticate with OpenOTP SMSOTP, MailOTP, Soft
Tokens... alternative login method. Details about RCDevs OpenID and OpenOTP at http:/ / www.rcdevs. com/ .
Some of the companies (especially the biggest ones) which did enable OpenID have been criticized for being a
provider of OpenID identities to third-party websites, without being an OpenID consumer and allowing credentials
of another website to work with their own websites. (For example, logging into Yahoo! through Windows Live
OpenID Foundation
The OpenID Foundation
is a 501(c)(3) non-profit organization incorporated in the United States. The OpenID
Foundation was formed to help manage copyright, trademarks, marketing efforts and other activities related to the
success of the OpenID community.
The OpenID Foundation's board of directors has eight community members and seven corporate members:
Community Members
• John Bradley (Independent)
• Mike Jones (Microsoft)
• Brian Kissel (JanRain)
• Chris Messina (Google)
• Axel Nennker (Deutsche Telekom)
• Nat Sakimura (Nomura Research Institute)
• Allen Tom (Yahoo!)
• Kick Willemse (Evidos)
Corporate Members
• Facebook - David Recordon
• Google - Eric Sachs
• IBM - Nataraj (Raj) Nagaratnam
• Microsoft - Michael B. Jones
• PayPal - Andrew Nash
• Ping Identity - Pamela Dingle
• VeriSign - Nico Popp
• Yahoo! - Raj Mata
Legal issues
The OpenID trademark in the United States was assigned to the OpenID Foundation in March 2008.
It had been
registered by NetMesh Inc. before the OpenID Foundation was operational.

In Europe, as of August 31, 2007,
the OpenID trademark is registered to the OpenID Europe Foundation.
The OpenID logo was designed by Randy "ydnar" Reddig, who in 2005 had expressed plans to transfer the rights to
an OpenID organization.
Since the original announcement of OpenID, the official site has stated:
Nobody should own this. Nobody's planning on making any money from this. The goal is to release
every part of this under the most liberal licenses possible, so there's no money or licensing or registering
required to play. It benefits the community as a whole if something like this exists, and we're all a part
of the community.
Sun Microsystems, VeriSign and a number of smaller companies involved in OpenID have issued patent
non-assertion covenants covering OpenID 1.1 specifications. The covenants state that the companies will not assert
any of their patents against OpenID implementations and will revoke their promises from anyone who threatens, or
asserts, patents against OpenID implementors.

Security and phishing
Some observers have suggested that OpenID has security weaknesses and may prove vulnerable to phishing


For example, a malicious relying party may forward the end-user to a bogus identity provider
authentication page asking that end-user to input their credentials. On completion of this, the malicious party (who in
this case also control the bogus authentication page) could then have access to the end-user's account with the
identity provider, and as such then use that end-user’s OpenID to log into other services.
In an attempt to combat possible phishing attacks some OpenID providers mandate that the end-user needs to be
authenticated with them prior to an attempt to authenticate with the relying party.
This relies on the end-user
knowing the policy of the identity provider. In December 2008, the OpenID Foundation approved version 1.0 of the
Provider Authentication Policy Extension (PAPE), which "enables Relying Parties to request that OpenID Providers
employ specified authentication policies when authenticating users and for OpenID Providers to inform the Relying
Parties which policies were actually used."
Regardless, this issue remains a significant additional vector for
man-in-the-middle phishing attacks.
Other security issues identified with OpenID involve lack of privacy and failure to address the trust problem.
The original OpenID authentication protocol was developed in May 2005
by Brad Fitzpatrick, creator of popular
community website LiveJournal, while working at Six Apart.
Initially referred to as Yadis (an acronym for "Yet
another distributed identity system"),
it was named OpenID after the openid.net domain name was given to Six
Apart to use for the project.
OpenID support was soon implemented on LiveJournal and fellow LiveJournal
engine community DeadJournal for blog post comments and quickly gained attention in the digital identity

Web developer JanRain was an early supporter of OpenID, providing OpenID software libraries
and expanding its business around OpenID-based services.
In late June, discussions started between OpenID users and developers from enterprise software company NetMesh,
leading to collaboration on interoperability between OpenID and NetMesh's similar Light-Weight Identity (LID)
protocol. The direct result of the collaboration was the Yadis discovery protocol, adopting the name originally used
for OpenID. The new Yadis was announced on October 24, 2005.
After a discussion at the 2005 Internet Identity
a few days later, XRI/i-names developers joined the Yadis project,
contributing their Extensible
Resource Descriptor Sequence (XRDS) format for utilization in the protocol.
In December, developers at Sxip Identity began discussions with the OpenID/Yadis community
after announcing
a shift in the development of version 2.0 of its Simple Extensible Identity Protocol (SXIP) to URL-based identities
like LID and OpenID.
In March 2006, JanRain developed a Simple Registration (SREG) extension for OpenID
enabling primitive profile-exchange
and in April submitted a proposal to formalize extensions to OpenID. The
same month, work had also begun on incorporating full XRI support into OpenID.
Around early May, key
OpenID developer David Recordon left Six Apart, joining VeriSign to focus more on digital identity and guidance
for the OpenID spec.

By early June, the major differences between the SXIP 2.0 and OpenID projects were
resolved with the agreement to support multiple personas in OpenID by submission of an identity provider URL
rather than a full identity URL. With this, as well as the addition of extensions and XRI support underway, OpenID
was evolving into a full-fledged digital identity framework, with Recordon proclaiming "We see OpenID as being an
umbrella for the framework that encompasses the layers for identifiers, discovery, authentication and a messaging
services layer that sits atop and this entire thing has sort of been dubbed 'OpenID 2.0'.
" In late July, Sxip began to
merge its Digital Identity Exchange (DIX) protocol into OpenID, submitting initial drafts of the OpenID Attribute
Exchange (AX) extension in August. Late in 2006, a ZDNet opinion piece made the case for OpenID to users, web
site operators and entrepreneurs.
On January 31, 2007, Symantec announced support for OpenID in its Identity Initiative products and services.
week later, on February 6 Microsoft made a joint announcement with JanRain, Sxip, and VeriSign to collaborate on
interoperability between OpenID and Microsoft's Windows CardSpace digital identity platform, with particular focus
on developing a phishing-resistant authentication solution for OpenID. As part of the collaboration, Microsoft
pledged to support OpenID in its future identity server products and JanRain, Sxip, and VeriSign pledged to add
support for Microsoft's Information Card profile to their future identity solutions.
In mid-February, AOL
announced that an experimental OpenID provider service was functional for all AOL and AOL Instant Messenger
(AIM) accounts.
In May, Sun Microsystems began working with the OpenID community, announcing an OpenID program,
as well
as entering a non-assertion covenant with the OpenID community, pledging not to assert any of its patents against
implementations of OpenID.
In June, OpenID leadership formed the OpenID Foundation, an Oregon-based public
benefit corporation for managing the OpenID brand and property.
The same month, an independent OpenID
Europe Foundation was formed in Belgium
by Snorri Giorgetti. By early December, non-assertion agreements
were collected by the major contributors to the protocol and the final OpenID Authentication 2.0 and OpenID
Attribute Exchange 1.0 specifications were ratified on December 5.
In mid-January 2008, Yahoo! announced initial OpenID 2.0 support, both as a provider and as a relying party,
releasing the provider service by the end of the month.
In early February, Google, IBM, Microsoft, VeriSign and
Yahoo! joined the OpenID Foundation as corporate board members.
Around early May, SourceForge, Inc.
introduced OpenID provider and relying party support to leading open source software development website
In late July, popular social network service MySpace announced support for OpenID as a
In late October, Google launched support as an OpenID provider and Microsoft announced that
Windows Live ID would support OpenID.
In November, JanRain announced a free hosted service, RPX Basic,
that allows websites to begin accepting OpenIDs for registration and login without having to install, integrate and
configure the OpenID open source libraries.
In January 2009, PayPal joined the OpenID Foundation as a corporate member, followed shortly by Facebook in
February. The OpenID Foundation formed an executive committee and appointed Don Thibeau as executive
director. In March, MySpace launched their previously announced OpenID provider service, enabling all MySpace
users to use their MySpace URL as an OpenID. In May, Facebook launched their relying party functionality,

letting users use an automatic login-enabled OpenID account (e.g. Google) to log into Facebook.
[1] Eldon, Eric (2009-04-14). "Single sign-on service OpenID getting more usage » VentureBeat" (http:// venturebeat.com/ 2009/ 04/ 14/
single-sign-on-service-openid-getting-more-usage/). venturebeat.com. . Retrieved 2009-04-25.
[2] bashburn, bill (2008-04-22). "BBC Joins OpenID Foundation" (http:// openid.net/ 2008/ 04/ 22/
british-broadcasting-corp-bbc-joins-openid-foundation/ ). .
[3] Riley, Duncan (2008-01-18). "Google Offers OpenID Logins Via Blogger" (http:// www.techcrunch.com/ 2008/ 01/ 18/
google-offers-openid-logins-via-blogger/ ). TechCrunch. . Retrieved 2008-03-20.
[4] "How do I get an OpenID?" (http:// openid. net/ get/ ). OpenID Foundation. . Retrieved 2008-03-20.
[5] "Technology Leaders Join OpenID Foundation to Promote Open Identity Management on the Web" (http:// www-03. ibm.com/ press/ us/
en/pressrelease/ 23461. wss). 008-02-07. .
[6] Bergman, Artur (2008-02-07). "OpenID Foundation - Google, IBM, Microsoft, VeriSign and Yahoo!" (http:/ / radar.oreilly.com/ archives/
2008/02/ openid-foundation-google-ibm-m.html). O'Reilly Media. . Retrieved 2008-03-19.
[7] "OpenID Authentication 1.1#Delegation" (http:/ / openid. net/ specs/ openid-authentication-1_1.html#delegating_authentication). .
[8] Paul Tarjan. "Easy OpenID Delegation with Yadis" (http:// blog.paulisageek. com/ 2009/ 06/ easy-openid-delegation-with-yadis.html). .
Retrieved 2009-06-30.
[9] Kissel, Brian (2009-12-16). "OpenID 2009 Year in Review" (http:// openid.net/2009/ 12/ 16/ openid-2009-year-in-review/). .
[10] AOL Inc.. "OpenID Central" (http:/ / dev. aol. com/ topic/ openid). . Retrieved 2011-05-31.
[11] "Frequently Asked Questions" (http:// clickpass. com/ docs/ faq). . Retrieved 2011-05-31.
[12] Bylund, Anders (17 January 2008). "Yahoo! No More Password Profusion!" (http:// www.fool. com/ investing/ general/ 2008/ 01/ 17/
yahoo-no-more-password-profusion.aspx). The Motley Fool. . Retrieved 2008-02-14.
[13] http:// www. google. com/ profiles
[14] Google, Inc.. "Google OpenID API documentation page" (http:// code.google.com/ apis/ accounts/ docs/ OpenID.html). . Retrieved
[15] https:// www. google. com/ accounts/ o8/ id
[16] Archer, Mike (4 February 2010). "OpenID URL Formatting" (http:/ / digitalenginesoftware.com/ blog/ archives/
24-OpenID-Provider-URL-Formatting.html). Digital Engine Software. . Retrieved 2010-03-23.
[17] http:// myopenid.com
[18] http:/ / clavid.com/
[19] http:/ / steamcommunity. com/ dev
[20] http:// www. tonido. com/ app_open_id_home. html
[21] http:// ''username''.tonidoid. com/ app/ openid
[22] http:// www. downloadsquad. com/ 2009/ 08/ 29/ tonido-now-lets-you-roll-your-own-openid-provider-also-debuts-t/
[23] https:/ / help.launchpad. net/ YourAccount/OpenID
[24] http:// www. xlogon.net/ de/ openid-provider
[25] http:// my.xlogon.net
[26] "OpenID" (http:/ / www.hyves-developers. nl/ documentation/ openid/ specifications). . Retrieved 2011-05-31.
[27] "mixi OpenID" (http:/ / developer.mixi. co. jp/ openid). . Retrieved 2011-05-31.
[28] "WikiTravel OpenID login page" (http:// wikitravel. org/en/ Special:OpenIDLogin). . Retrieved 2009-04-25.
[29] "OpenID Requirements - Facebook Developer Wiki" (http:// wiki. developers.facebook.com/ index. php/ OpenID_Requirements). .
Retrieved 2010-04-28.
[30] John Timmer, OpenID being Balkanized even as Google, Microsoft sign on (http:/ / arstechnica.com/ news. ars/ post/
20081029-openid-being-balkanized-even-as-google-microsoft-sign-on. html).
[31] http:// openid.net/ foundation
[32] OpenID Board of Directors (2007-06-01). "OpenID Foundation" (http:// openid.net/ foundation/ ). OpenID Foundation. . Retrieved
[33] "Trademark Assignment, Serial #: 78899244" (http:// assignments. uspto. gov/assignments/ q?db=tm& sno=78899244). United States
Patent and Trademark Office. 2008-05-06. . Retrieved 2008-05-19. "Exec Dt: 03/27/2008"
[34] "Latest Status Info" (http:/ / tarr.uspto. gov/ servlet/ tarr?regser=serial&entry=78899244). United States Patent and Trademark Office.
2006-03-27. . Retrieved 2008-03-20.
[35] "NetMesh: Company / Management" (http:/ / netmesh. us/ company/ management/ ). NetMesh. . Retrieved 2008-03-20.
[36] "OpenID Europe Trademark & Logo Policy" (http:/ / www.openideurope. eu/policies/ openid-trademark-policy/). OpenID Europe
Foundation. . Retrieved 2008-03-20.
[37] Reddig, Randy (2005-06-29). "OpenID Logo" (http:/ / lists. danga. com/ pipermail/yadis/ 2005-June/ 000990. html). Danga Interactive. .
Retrieved 2008-03-20.
[38] Fitzpatrick, Brad. "Intellectual Property" (http:// openid.net/ intellectual-property/). .
[39] "Sun OpenID: Non-Assertion Covenant" (http:// www.sun. com/ software/ standards/ persistent/ openid/ nac.xml). Sun Microsystems. .
Retrieved 2008-03-20.
[40] "VeriSign's OpenID Non-Assertion Patent Covenant" (http:/ / www.verisign. com/ research/Consumer_Identity_and_Profile_Management/
042160.html). VeriSign. . Retrieved 2008-03-20.
[41] Crowley, Paul (2005-06-01). "Phishing attacks on OpenID" (http:// lists. danga.com/ pipermail/yadis/ 2005-June/ 000470.html). Danga
Interactive. . Retrieved 2008-03-20.
[42] Anderson, Tim (2007-03-05). "OpenID still open to abuse" (http:// www. itweek.co.uk/ 2184695). IT Week. . Retrieved 2007-03-13.
[43] Slot, Marco. "Beginner's guide to OpenID phishing" (http:// openid.marcoslot.net/ ). . Retrieved 2007-07-31.
[44] "Verisign PIP FAQ" (https:// pip. verisignlabs. com/ faq.do#faq5). . Retrieved 2008-11-13.
[45] Jones, Mike. "PAPE Approved as an OpenID Specification" (http:// openid.net/ 2008/ 12/ 31/ pape-approved-as-an-openid-specification/).
OpenID Foundation. .
[46] Stefan Brands (2007-08-22). "The problem(s) with OpenID" (http:// www. untrusted. ca/ cache/ openid. html). . Retrieved 2010-12-12.
(originally published on The Identity Corner at www.idcorner.org/?p=161)
[47] Fitzpatrick, Brad (2005-05-16). "Distributed Identity: Yadis" (http:/ / community. livejournal.com/ lj_dev/ 683939.html). LiveJournal. .
Retrieved 2008-03-20.
[48] Waters, John K (2007-12-01). "OpenID Updates Identity Spec" (http:/ / web. archive.org/ web/ 20080208155322/ http:/ / reddevnews.com/
news/devnews/ article. aspx?editorialsid=913). Redmond Developer News. Archived from the original (http:// reddevnews.com/ news/
devnews/ article.aspx?editorialsid=913) on 2008-02-08. . Retrieved 2008-03-20.
[49] "Glossary" (http:/ / www. livejournal.com/ doc/ server/ appx.glossary.html). LiveJournal Server: Technical Info. . Retrieved 13 October
[50] Lehn, David I. (18 May 2005). "18 May 2005" (http:// www. advogato. org/person/ dlehn/ diary/ 5.html). Advogato blog for dlehn.
Advogato. . Retrieved 13 October 2009. "They were looking for a name and managed to email me about openid.net right before I was going to
offer it to them. So I gave it to them for the new and improved OpenID project."
[51] "OpenID: an actually distributed identity system" (http:// web.archive.org/ web/ 20050924033518/ www. danga.com/ openid/ ). Internet
Archive. 2005-09-24. . Retrieved 2008-03-20.
[52] Fitzpatrick, Brad (2006-05-30). "brad's life - OpenID and SixApart" (http:/ / brad. livejournal.com/ 2226738. html). LiveJournal. . Retrieved
[53] Recordon, David (2005-12-24). "Announcing YADIS...again" (http:// lists. danga. com/ pipermail/ yadis/ 2005-October/001511. html).
Danga Interactive. . Retrieved 2008-03-20.
[54] http:// iiw.idcommons. net
[55] Reed, Dummond (2005-12-31). "Implementing YADIS with no new software" (http:// lists. danga.com/ pipermail/yadis/ 2005-October/
001544. html). Danga Interactive. . Retrieved 2008-03-20.
[56] Reed, Drummond (2008-11-30). "XRD Begins" (http:// www. equalsdrummond. name/ ?p=172). Equals Drummond. . Retrieved 5 January
[57] Hardt, Dick (2005-12-18). "Sxip concerns with YADIS" (http:// lists. danga. com/ pipermail/yadis/ 2005-December/001873.html). Danga
Interactive. . Retrieved 2008-03-20.
[58] Hardt, Dick (2005-12-10). "SXIP 2.0 Teaser" (http:// identity20. com/ ?p=44). Identity 2.0. . Retrieved 2008-03-20.
[59] Hoyt, Josh (2006-03-15). "OpenID + Simple Registration Information Exchange" (http:/ / lists. danga.com/ pipermail/ yadis/ 2006-March/
002304. html). Danga Interactive. . Retrieved 2008-03-20.
[60] Grey, Victor (2006-04-02). "Proposal for an XRI (i-name) profile for OpenID" (http:// lists. danga. com/ pipermail/yadis/ 2006-April/
002388. html). Danga Interactive. . Retrieved 2008-03-20.
[61] Recordon, David (2006-04-29). "Movin' On...." (http:// daveman692.livejournal.com/ 251286. html). LiveJournal. . Retrieved 2008-03-20.
[62] Recordon, David (2006-06-16). "Moving OpenID Forward" (http:// lists. danga. com/ pipermail/yadis/ 2006-June/ 002631. html). Danga
Interactive. . Retrieved 2008-05-19.
[63] Johannes Ernst and David Recordon. Editor:Phil Becker (2006-12-04). "The case for OpenID" (http://www. zdnet.com/ blog/ digitalid/
the-case-for-openid/78). ZDNet. . Retrieved 2010-12-12.
[64] "Symantec Unveils Security 2.0 Identity Initiative at DEMO 07 Conference" (http:// www. symantec. com/ about/ news/ release/ article.
jsp?prid=20070131_01). Symantec. 2007-01-31. . Retrieved 2008-03-20.
[65] Graves, Michael (2007-02-06). "VeriSign, Microsoft & Partners to Work together on OpenID + Cardspace" (http:// blogs. verisign. com/
infrablog/ 2007/ 02/ verisign_microsoft_partners_to_1.php). VeriSign. . Retrieved 2008-03-20.
[66] Panzer, John (2007-02-16). "AOL and 63 Million OpenIDs" (http:/ / dev. aol.com/ aol-and-63-million-openids). AOL Developer Network. .
Retrieved 2008-03-20.
[67] "Sun Microsystems Announces OpenID Program" (http:// www. prnewswire.com/ cgi-bin/ stories. pl?ACCT=104& STORY=/ www/
story/05-07-2007/0004582105& EDATE=). PR Newswire. 2007-05-07. . Retrieved 2008-03-20.
[68] OpenID Europe Foundation (http:/ / www. openideurope.eu/ foundation/)
[69] "OpenID 2.0…Final(ly)!" (http:// openid. net/ 2007/ 12/ 05/ openid-2_0-final-ly/). OpenID Foundation. 2007-12-05. . Retrieved
[70] "Yahoo! Announces Support for OpenID; Users Able to Access Multiple Internet Sites with Their Yahoo! ID" (http:/ / web.archive. org/
web/ 20080304014817/ http:/ / biz. yahoo. com/ bw/ 080117/ 20080117005332. html). Yahoo!. 2008-01-17. Archived from the original (http:/
/biz.yahoo. com/ bw/ 080117/ 20080117005332. html) on 2008-03-04. . Retrieved 2008-03-20.
[71] "Technology Leaders Join OpenID Foundation to Promote Open Identity Management on the Web" (http:/ / www.marketwire.com/ mw/
release.do?id=818650). OpenID Foundation (Marketwire). 2008-02-07. . Retrieved 2008-03-20.
[72] SourceForge, Inc. (May 7, 2008). "SourceForge Implements OpenID Technology" (http:// www.primenewswire. com/ newsroom/ news.
html?d=142213). Press release. . Retrieved 2008-05-21.
[73] "MySpace Announces Support for ‘OpenID’ and Introduces New Data Availability Implementations" (http:// www.businesswire. com/
news/ home/ 20080722006024/ en). Business Wire. MySpace. 2008-07-22. pp. 2. . Retrieved 2008-07-23.
[74] "Microsoft and Google announce OpenID support" (http:/ / openid.net/ 2008/ 10/ 30/ microsoft-and-google-announce-openid-support/).
OpenID Foundation. 2008-10-30. .
[75] JanRain, Inc. (November 14, 2008). "JanRain Releases Free Version of Industry Leading OpenID Solution" (http:/ / www.janrain.com/
press/ 2008/ rpxnow). Press release. . Retrieved 2008-11-14.
[76] "Facebook Developers | Facebook Developers News" (http:/ / developers.facebook.com/ news. php?blog=1&story=246).
Developers.facebook.com. 2009-05-18. . Retrieved 2009-07-28.
[77] "Facebook now accepts Google account logins" (http:/ / www.pocket-lint.com/ news/ news. phtml/ 24185/
facebook-accepting-google-login-openid.phtml). Pocket-lint.com. 2009-05-19. . Retrieved 2009-07-28.
[78] "OpenID Requirements - Facebook Developer Wiki" (http:// wiki. developers.facebook.com/ index. php/ OpenID_Requirements).
Wiki.developers.facebook.com. 2009-06-26. . Retrieved 2009-07-28.
External links
• OpenID official site (http:// openid. net/ )
• OpenID (http:/ / www. dmoz. org/Computers/ Security/Authentication/ Single_Sign-On/OpenID/) at the Open
Directory Project
• OpenID explained (http:/ / openidexplained. com/ )
OTPW is a one-time password system developed for authentication in Unix-like operating systems by Markus
Kuhn. A user's real password is not directly transmitted across the network. Rather, the real password is combined
with a short set of characters (constant secret) and a set of one-time tokens to form a single-use password. As the
single-use password is only used once, passwords intercepted by a password sniffer or key logger are not useful to an
OTPW is supported in Unix and Linux (via Pluggable authentication modules), OpenBSD, NetBSD, and FreeBSD,
and a generic open source implementation can be used to enable its use on other systems.
OTPW, like the other one-time password systems, is sensitive to a man in the middle attack if used by itself. This
could for example be solved by putting SSL, SPKM or similar security protocol "under it" which authenticates the
server and gives point-to-point security between the client and server.
Design and differences from other implementations
Unlike S/KEY, OTPW is not based on the Lamport's scheme in which every one-time password is the one-way hash
value of its successor. Password lists based on the Lamport's scheme have the problem that if the attacker can see
one of the last passwords on the list, then all previous passwords can be calculated from it. We also do not store the
encrypted passwords as suggested by Aviel D. Rubin in Independent One-Time Passwords, in order to keep the host
free of files with secrets. Both proposals aimed to save memory in the host system.
Storing passwords
In OTPW one-way hash value of every single password is stored in a potentially widely readable file in the user’s
home directory. For instance, hash values of 300 passwords (a typical A4 page) require only a four kilobyte long
.otpw file, a typically negligible amount of storage space.
Generating passwords
The passwords are carefully generated random numbers. The random number generator is based on the
RIPEMD-160 secure hash function. The random number generator is seeded by hashing together the output of
various shell commands. These provide unpredictability in the form of a system random number seed, access times
of important system files, usage history of the host, and more. The random state is the 160-bit output of the hash
function. The random state is iterated after each use by concatenating the old state with the current high-resolution
timer output and hashing the result again. The first 72 bits of the hash output are encoded with a modified base64
scheme to produce readable passwords, while the remaining 88 bits represent the undisclosed internal state of the
random number generator.
Form of a password
In many fonts, the characters 0 and O or 1 and l and I are difficult to distinguish, therefore the modified base64
encoding replaces the three characters 01l by corresponding :, = and %. If for instance a zero is confused with a
capital O by the user, the password verification routine will automatically correct for this.
S/KEY uses sequences of short English words as passwords. OTPW uses by default a base64 encoding instead,
because that allows more passwords to be printed on a single page, with the same password entropy. In addition, an
average human spy needs over 30 seconds to write a 12-character random string into short-term memory, which
provides a good protection against brief looks that an attacker might have on a password list. Lists of short words on
the other hand are much faster to memorize. OTPW can handle arbitrary password generation algorithms, as long as
the length of the password is fixed. In the current version, the otpw-gen can generate both base-64 encoded (option
-p) and 4-letter-word encoded (option -p1) passwords with a user-specified entropy (option -e).
The prefix password
The prefix password ensures that neither stealing the password list nor eavesdropping the line alone can provide
unauthorized access. Admittedly, the security obtained by OTPW is not comparable with that of a
challenge-response system in which the user has a PIN protected special calculator that generates the response. On
the other hand, a piece of paper is much more portable, much more robust, and much cheaper than a special
calculator. OTPW was designed for the large user base, for which an extra battery-powered device is inconvenient or
not cost effective and who therefore still use normal Unix passwords everywhere.
Passwords locking
In contrast to the suggestion made in RFC 1938
, OTPW does not lock more than one one-time password at a time.
If we did this, an attacker could easily exhaust our list of unlocked passwords and force us to either not login at all or
use the normal Unix login password. Therefore, OTPW locks only one single password and for all further logins a
triple-challenge is issued. If more than 100 unused passwords remain available, then there are over a million
different challenges and an attacker has very little chance to perform a successful race attack while the authorized
user finishes password entry.
One-time password authentication with the OTPW package is accomplished via a file .otpw located in the user’s
home directory. No state is kept in any system-wide files, therefore OTPW does not introduce any new setuid root
programs. As long as a user does not have .otpw in his home directory, the one-time-password facility has not been
activated for him.
Setting up passwords
A user who wants to set up the one-time-password capability just executes the otpw-gen program. The program will
ask for a prefix password that the user has to select and memorize and it will then write to standard output a
password list. This list can be formatted and printed and the prefix password should be memorized.
Logging in
Where one-time-password authentication is used, the password prompt will be followed by a 3-digit password
number. Enter first the prefix password that was given to otpw-gen, followed directly (without hitting return
between) by the password with the requested number from the printed password list:
login: kuhn
Password 019: geHeimOdAkH62c
In this example, geHeim was the prefix password.
A clever attacker might observe the password being entered and might try to use the fact that computers can send
data much faster than users can finish entering passwords. In the several hundred milliseconds that the user needs to
press the return key after the last character, an attacker could on a parallel connection to the same machine send the
code of the return key faster than the user.
To prevent such a race-for-the-last-key attack, any login attempt that is taking place concurrently with another
attempt will require three one-time passwords to be entered:
login: kuhn
Password 022/000/004: geHeimQ=XK4I7wIZdBbqyHA5z9japt
• One-time passwords
• One-time pad
• Haller, Neil; Metz, Craig; Nesser, Philip J.; Straw, Mike (February 1998). A One-Time Password System. IETF.
STD 61. RFC 2289.
External links
• OTPW Home
[1] http:/ / tools. ietf.org/html/ rfc1938
[2] http:// www. cl. cam. ac. uk/ ~mgk25/ otpw. html
Partial Password
Partial Password
Partial Password is a mode of password authentication.
By asking the user to enter only a few specific characters from their password
, rather than the whole password,
partial passwords help to protect the user from password theft. As only part of the password is revealed at once it
becomes more difficult to obtain the password using techniques such as keystroke logging or shoulder surfing.
Verifying Partial Passwords
It is good practice to not store passwords in cleartext, instead when checking a whole password it is common to store
the result of passing the password to a cryptographic hash function. As the user doesn't supply the whole password it
cannot be verified against a stored digest of the whole password. Some have suggested storing the digest of each
combination of letters that could be requested but they note that this results in generating and storing a large amount
of digests

. A better solution in terms of storage space and security is using a secret sharing scheme
[1] "What is partial password verification?" (http:/ / ask. co-operativebank.co.uk/ help/ customer_services/ partial_password).
Co-operative_Bank. . Retrieved 2011-03-03.
[2] "Partial Passwords and Keystroke Loggers" (http:/ / www. plynt. com/ blog/ 2005/ 08/ partial-passwords-and-keystrok/). . Retrieved
[3] "Partial Passwords - How?" (http:// www. smartarchitects. co.uk/ news/ 9/ 15/ Partial-Passwords---How.html). . Retrieved 2011-03-03.
Passmap (pronounced /ˈpæsmæp/) is an image based method used for authentication, similar to passwords. The word
passmap originates from the word password by substituting word with pass. Passmap is a patented technology of
Hydrabyte, Inc.
Easy to remember, hard to guess
A common way of authentication (e.g.: online services) is using
character based passwords. However, the human brain tends to
remember shapes and structures easier than characters.
substitutes the characters used for passwords with a single image. You
can choose your favorite image (e.g.: shot with your camera) to be
your personal passmap. Your image is divided into at least 64 sub-parts
with a rectangular grid. An example is shown using an image of the
Statue of Liberty on the top of this page. When creating your passmap,
you select at least five sub-parts of the image in a fixed sequence that
forms your secret of authentication. With the selected sub-parts you
can form structures that are easy for you to remember, or you can use
the underlying image to serve as a guide in remembering your
passmap. In a cryptographic sense, passmap is equal to defining a password over an alphabet of at least 64
Effective against phishing
Phishing is the criminally fraudulent process of attempting to acquire sensitive information such as usernames,
passwords and credit card details by masquerading as a trustworthy entity in an electronic communication. The
process consists of creating a fake website of a common web service that mimics the functionality of authenticating
Passmap can protect a website against phishing effectively, because no matter how perfectly an attacker can
fake a website, it is extremely difficult to retrieve all the images of the users that is needed for authentication.
Touchscreen friendly
Touchscreen is becoming a common way of interacting with mobile devices. Such devices have virtual keyboards to
input characters, the keys of which are often small and hard to hit. Moreover, special characters (which are often
required to be part of a password) can often be accessed only by touching multiple keys. Contrary to this,
authenticating with Passmap is as simple as touching sub-parts of an image that you can even zoom into, to better fit
your finger.
[1] http:/ / hydrabyte.com
[2] ScienceDaily.com (2009). (http:/ / www.sciencedaily. com/ releases/ 2009/ 04/ 090429132231. htm). Human Brain Can Recognize Objects
Much Faster Than Some Have Thought
[3] IBM® (2009). (http:// www-935.ibm. com/ services/ us/ iss/ xforce/trendreports/). IBM® Internet Security Systems™ X-Force® 2009
Mid-Year Trend and Risk Report
External links
• (http:// passmap. com) Homepage of Passmap
• (http:// hydrabyte. com) Hydrabyte, Inc.
PassPattern system
PassPattern system
PassPattern System (PPS) is a pseudodynamic password user authentication system, which is based on patterns.
Unlike the other dynamic password-based systems, PassPattern System(PPS)
does not require any extra hardware,
software or mathematical operations. The complete paper is available at Springer
[1] PassPattern System, NETLAB, Indian Institute of Technology Madras (http:// netlab.cs. iitm. ernet.in/ pps)
[2] PassPattern System (PPS): A Pattern-Based User Authentication Scheme (http:/ / www.springerlink.com/ content/ pvmmn1n134033kk3/ )
A passphrase is a sequence of words or other text used to control access to a computer system, program or data. A
passphrase is similar to a password in usage, but is generally longer for added security. Passphrases are often used to
control both access to, and operation of, cryptographic programs and systems. Passphrases are particularly applicable
to systems that use the passphrase as an encryption key. The origin of the term is by analogy with password. The
modern concept of passphrases is believed to have been invented by Sigmund N. Porter
in 1982.
Considering that the entropy of written English is less than 1.1 bits per character,
passphrases can be relatively
weak. NIST has estimated that the 23 character pass phrase "IamtheCapitanofthePina4" contains a 45 bit-strength.
The equation employed here is:
4 bits (1st character) + 14 bits (characters 2–8) + 18 bits (characters 9–20) + 3 bits (characters 21–23) + 6 bits
(bonus for upper case, lower case, and alphanumeric) = 45 bits
Using this guideline, to achieve the 80-bit strength recommended for high security (non-military) by NIST, a
passphrase would need to be 58 characters long, assuming a composition that includes uppercase and alphanumeric.
There is room for debate regarding the applicability of this equation, depending on the number of bits of entropy
assigned. For example, five-letter words each contain 2.3 bits of entropy, which would mean only a 35-character
passphrase is necessary to achieve 80 bit strength.
If the words or components of a passphrase may be found in a language dictionary—especially one available as
electronic input to a software program—the passphrase is rendered more vulnerable to dictionary attack. This is a
particular issue if the entire phrase can be found in a book of quotations or phrase compilations. However, the
required effort (in time and cost) can be made impracticably high if there are enough words in the passphrase and
how randomly they are chosen and ordered in the passphrase. The number of combinations which would have to be
tested under sufficient conditions make a dictionary attack so difficult as to be infeasible. These are difficult
conditions to meet, and selecting at least one word that cannot be found in any dictionary significantly increases
passphrase strength.
For example, the widely used cryptography standard OpenPGP requires that a user make up a passphrase that must
be entered whenever encrypting, decrypting, or signing messages. Internet services like CryptoHeaven and Hushmail
provide free encrypted e-mail or file sharing services, but the security present depends almost entirely on the quality
of the chosen passphrase.
Compared to passwords
Passphrases differ from passwords. A password is usually short—six to ten characters. Such passwords may be
adequate for various applications (if frequently changed, if chosen using an appropriate policy, if not found in
dictionaries, if sufficiently random, and/or if the system prevents online guessing, etc.) such as:
• Logging onto computer systems
• Negotiating keys in an interactive setting (e.g. using password-authenticated key agreement)
• Enabling a smart-card or PIN for an ATM card (e.g. where the password data (hopefully) cannot be extracted)
But passwords are typically not safe to use as keys for standalone security systems (e.g., encryption systems) that
expose data to enable offline password guessing by an attacker. Passphrases are generally stronger, and a clearly
better choice in these cases. First, they usually are (and always should be) much longer—20 to 30 characters or more
is typical—making some kinds of brute force attacks entirely impractical. Second, if well chosen, they will not be
found in any phrase or quote dictionary, so such dictionary attacks will be almost impossible. Third, they can be
structured to be more easily memorable than passwords without being written down, reducing the risk of hardcopy
Passphrase selection
Typical advice about choosing a passphrase includes suggestions that it should be:
• Long enough to be hard to guess (e.g., automatically by a search program, as from a list of famous phrases).
• Not a famous quotation from literature, holy books, et cetera
• Hard to guess by intuition—even by someone who knows the user well
• Easy to remember and type accurately
• For better security, any easily memorable encoding at your own level can be applied.
Example methods
One method to create a strong passphrase is to use dice to select words at random from a long list, a technique often
referred to as diceware. While such a collection of words might appear to violate the "not from any dictionary" rule,
the security is based entirely on the large number of possible ways to choose from the list of words and not from any
secrecy about the words themselves. For example, if there are 7776 words in the list and six words are chosen
randomly, then there are 7776
= 221073919720733357899776 combinations, providing about 78 bits of entropy.
(The number 7776 was chosen to allow words to be selected by throwing five dice. 7776 = 6
Another is to choose two phrases, turn one into an acronym, and include it in the second, making the final
passphrase. For instance, using two English language typing exercises, we have the following. The quick brown fox
jumps over the lazy dog, becomes tqbfjotld. Including it in, Now is the time for all good men to come to the aid of
their country, might produce, Now is the time for all good tqbfjotld to come to the aid of their country as the
There are several points to note here, all relating to why this example pass phrase is not a good one.
• It has appeared in public and so should be avoided by everyone.
• It's long (which is a considerable virtue in theory) and requires a good typist (which is an overwhelming problem
for most people in actual practice). (Whatever software is accepting the passphrase for testing should never echo
it to your display, lest shoulder surfers take advantage.) Typing errors are much more likely under such
conditions, especially for extended phrases.
• It doesn't contain any non-alphabetic characters. Converting, say, the "l" (Latin small letter L) in the acronym to a
"1" (digit one) would be an improvement.
• Individuals and organizations serious about cracking computer security have compiled lists of passwords derived
in this manner from the most common quotations, song lyrics, and so on.
The PGP Passphrase FAQ
suggests a procedure that attempts a better balance between theoretical security and
practicality than this example. All procedures for picking a passphrase involve a tradeoff between security and ease
of use; security should be at least "adequate" while not "too seriously" annoying users. Both criteria should be
evaluated to match particular situations.
Another supplementary approach to frustrating brute-force attacks is to derive the key from the passphrase using a
deliberately-slow hash function, such as PBKDF2 as described in RFC 2898.
Windows support
If backward compatibility with Microsoft LAN Manager is not needed, in versions of Windows NT (including
Windows 2000, Windows XP and later), a passphrase can be used as a substitute for a Windows password. If the
passphrase is longer than 14 characters, this will also cause the very weak LM hash to not be generated.
Unix support
In recent versions of Unix-like operating systems such as Linux, OpenBSD, NetBSD, Solaris and FreeBSD, up to
255 character passphrases can be used.
[1] Sigmund N. Porter. "A password extension for improved human factors". Computers and Security, 1(1):54-56, January 1982.
[2] Matt Mahoney. "Refining the Estimated Entropy of English by Shannon Game Simulation" (http:// cs. fit.edu/ ~mmahoney/ dissertation/
entropy1.html). Florida Institute of Technology. . Retrieved March 27, 2008.
[3] "Electronic Authentication Guideline" (http:// csrc. nist. gov/ publications/ nistpubs/ 800-63/ SP800-63V1_0_2.pdf) (PDF). NIST. .
Retrieved April 7, 2008.
[4] Jesper M. Johansson. "The Great Debates: Pass Phrases vs. Passwords. Part 2 of 3" (http:// www. microsoft. com/ technet/ security/ secnews/
articles/ itproviewpoint100504.mspx). Microsoft Corporation. . Retrieved March 27, 2008.
[5] Randall T. Williams (1997-01-13). "The Passphrase FAQ" (http:// www.iusmentis. com/ security/ passphrasefaq/ ). . Retrieved 2006-12-11.
External links
• Diceware page (http:// www. diceware.com)
• Passkool - A deterministic "pronounceable" password generator (http:/ / passkool. sourceforge.net)
• Passphrase FAQs (http:/ / www. pgpi. org/doc/ faq/passphrase/ )
• Passphrase based password generator browser button (http:// www.pass4all. com)
Password authentication protocol
Password authentication protocol
A password authentication protocol (PAP) is an authentication protocol that uses a password.
PAP is used by Point to Point Protocol to validate users before allowing them access to server resources. Almost all
network operating system remote servers support PAP.
PAP transmits unencrypted ASCII passwords over the network and is therefore considered insecure. It is used as a
last resort when the remote server does not support a stronger authentication protocol, like CHAP or EAP (while the
last is actually a framework).
Password-based authentication is the protocol that two entities share a password in advance and use the password
as the basic of authentication. Existing password authentication scheme can be categorized into two types:
weak-password authentication schemes and strong-password authentication schemes. In general, strong-password
authentication protocols have the advantages over the weak-password authentication schemes in that their
computational overhead are lighter, designs are simpler, and implementation are easier, and therefore are especially
suitable for some constrained environments.
Working cycle
• Client sends username and password
• Server sends authentication-ack (if credentials are OK) or authentication-nak (otherwise)
PAP Packets
Description 1 byte 1 byte 2 bytes 1 byte Variable 1 byte Variable
Authentication-request Code = 1 ID Length Username length Username Password length Password
Authentication-ack Code = 2 ID Length Message length Message
Authentication-nak Code = 3 ID Length Message length Message
PAP packet embedded in a PPP frame. The protocol field has a value of C023 (hex).
Flag Address Control Protocol (C023 (hex)) Payload (table above) FCS Flag
Password cracking
Password cracking
Password cracking is the process of recovering passwords from data that has been stored in or transmitted by a
computer system. A common approach is to repeatedly try guesses for the password. The purpose of password
cracking might be to help a user recover a forgotten password (though installing an entirely new password is less of a
security risk, but involves system administration privileges), to gain unauthorized access to a system, or as a
preventive measure by system administrators to check for easily crackable passwords. On a file-by-file basis,
password cracking is utilized to gain access to digital evidence for which a judge has allowed access but the
particular file's access is restricted.
Time needed for password searches
The time to crack a password is related to bit strength (see password strength), which is a function of the password's
information entropy. Most methods of password cracking require the computer to produce many candidate
passwords, each of which is checked. Brute-force cracking, in which a computer tries every possible key or password
until it succeeds, is the lowest common denominator of password cracking. More common methods of password
cracking, such as dictionary attacks, pattern checking, word list substitution, etc., attempt to reduce the number of
trials required and will usually be attempted before brute force.
The ability to crack passwords using computer programs is a function of the number of possible passwords per
second which can be checked. If a hash of the target password is available to the attacker, this number can be quite
large. If not, the rate depends on whether the authentication software limits how often a password can be tried, either
by time delays, CAPTCHAs, or forced lockouts after some number of failed attempts.
Individual desktop computers can test anywhere between one million to fifteen million passwords per second against
a password hash for weaker algorithms, such as DES or LanManager. See: John the Ripper benchmarks
user-selected eight-character password with numbers, mixed case, and symbols, reaches an estimated 30-bit strength,
according to NIST. 2
is only one billion permutations and would take an average of 16 minutes to crack.
ordinary desktop computers are combined in a cracking effort, as can be done with botnets, the capabilities of
password cracking are considerably extended. In 2002, distributed.net successfully found a 64-bit RC5 key in four
years, in an effort which included over 300,000 different computers at various times, and which generated an average
of over 12 billion keys per second.
Graphics processors can speed up password cracking by a factor of 50 to 100
over general purpose computers. As of 2011, commercial products are available that claim the ability to test up to
2,800,000,000 passwords a second on a standard desktop computer using a high-end graphics processor.
Such a
device can crack a 10 letter single-case password in one day. Note that the work can be distributed over many
computers for an additional speedup proportional to the number of available computers with comparable GPUs.
If a cryptographic salt is not used in the password system, the attacker can pre-compute hash values for common
passwords variants and for all passwords shorter than a certain length, allowing very rapid recovery. Long lists of
pre-computed password hashes can be efficiently stored rainbow tables. Such tables are available on the Internet for
several common password authentication systems.
Another situation where quick guessing is possible is when the password is used to form a cryptographic key. In
such cases, an attacker can quickly check to see if a guessed password successfully decodes encrypted data. For
example, one commercial product claims to test 103,000 WPA PSK passwords per second.
Despite their capabilities, desktop CPUs are slower at cracking passwords than purpose-built password breaking
machines. In 1998, the Electronic Frontier Foundation (EFF) built a dedicated password cracker using FPGAs, as
opposed to general purpose CPUs. Their machine, Deep Crack, broke a DES 56-bit key in 56 hours, testing over 90
billion keys per second.
In 2010, the Georgia Tech Research Institute developed a method of using GPGPU to
crack passwords, coming up with a minimum secure password length of 12 characters.


Password cracking
Perhaps the fastest way to crack passwords is through the use of pre-computed rainbow tables. These encode the
hashes of common passwords based on the most widely used hash functions and can crack passwords in a matter of
seconds. However, they are only effective on systems that do not use a salt, such as Windows LAN Manager and
some application programs.
The best method of preventing password cracking is to ensure that attackers cannot get access even to the encrypted
password. For example, on the Unix operating system, encrypted passwords were originally stored in a publicly
accessible file /etc/passwd. On modern Unix (and similar) systems, on the other hand, they are stored in the file
/etc/shadow, which is accessible only to programs running with enhanced privileges (ie, 'system' privileges).
This makes it harder for a malicious user to obtain the encrypted passwords in the first instance. Unfortunately, many
common network protocols transmit passwords in cleartext or use weak challenge/response schemes.

Modern Unix systems have replaced traditional DES-based password hashing with stronger methods based on MD5
and Blowfish.
Other systems have also begun to adopt these methods. For instance, the Cisco IOS originally used
a reversible Vigenère cipher to encrypt passwords, but now uses md5-crypt with a 24-bit salt when the "enable
secret" command is used.
These newer methods use large salt values which prevent attackers from efficiently
mounting offline attacks against multiple user accounts simultaneously. The algorithms are also much slower to
execute which drastically increases the time required to mount a successful offline attack.
Many hashes used for storing passwords, such as MD5 and the SHA family, are designed for fast computation and
efficient implementation in hardware. Using key stretching algorithms, such as PBKDF2, to form password hashes
can significantly reduce the rate at which passwords can be tested.
Solutions like a security token give a formal proof answer by constantly shifting password. Those solutions abruptly
reduce the timeframe for brute forcing (attacker needs to break and use the password within a single shift) and they
reduce the value of the stolen passwords because of its short time validity.
There are many password cracking software tools, but the most popular
are Cain and Abel, John the Ripper,
Hydra, ElcomSoft and Lastbit. Many litigation support software packages also include password cracking
functionality. Most of these packages employ a mixture of cracking strategies, with brute force and dictionary
attacks proving to be the most productive.
[1] http:/ / openwall.info/wiki/ john/ benchmarks
[2] "Electronic Authentication Guideline" (http:// csrc. nist. gov/ publications/ nistpubs/ 800-63/ SP800-63V1_0_2.pdf) (PDF). NIST. .
Retrieved March 27, 2008.
[3] "64-bit key project status" (http:// stats. distributed. net/ projects.php?project_id=5). Distributed.net. . Retrieved March 27, 2008.
[4] [[ElcomSoft (http:/ / www. elcomsoft. com/ eprb.html#gpu)] Password Recovery Speed table], NTLM passwords, Nvidia Tesla S1070 GPU,
accessed 2011-2-1
[5] http:// www. elcomsoft. com/ ewsa. html Elcomsoft Wireless Security Auditor, HD5970 GPU, accessed 2011-2-11
[6] "EFF DES Cracker machine brings honesty to crypto debate" (http:/ / w2. eff.org/ Privacy/Crypto/ Crypto_misc/ DESCracker/HTML/
19980716_eff_descracker_pressrel.html). EFF. . Retrieved March 27, 2008.
[7] "Teraflop Troubles: The Power of Graphics Processing Units May Threaten the World's Password Security System" (http:// www. gtri.
gatech. edu/ casestudy/ Teraflop-Troubles-Power-Graphics-Processing-Units-GPUs-Password-Security-System). Georgia Tech Research
Institute. . Retrieved 2010-11-07.
[8] "Want to deter hackers? Make your password longer" (http:/ / www.msnbc. msn. com/ id/ 38771772/ ). MSNBC. 2010-08-19. . Retrieved
[9] Walters, Dave (2010-09-02). "The Rise of The Programmable GPU – And The Death Of The Modern Password" (http:// techdrawl.com/
News-Post/ Tech-Transfer/The-Rise-of-The-Programmable-GPU- -And-The-Death-Of-The-Modern-Password). Techdrawl. . Retrieved
Password cracking
[10] No Plaintext Passwords (http:/ / www.usenix. org/ publications/ login/ 2001-11/pdfs/ singer. pdf)
[11] Cryptanalysis of Microsoft's Point-to-Point Tunneling Protocol (http:/ / www.schneier. com/ paper-pptp.html)
[12] A Future-Adaptable Password Scheme (http:// www. usenix. org/events/ usenix99/ provos.html)
[13] MDCrack FAQ 1.8 (http:/ / c3rb3r.openwall. net/ mdcrack/download/ FAQ-18. txt)
[14] Password Protection for Modern Operating Systems (http:/ / www. usenix. org/publications/ login/ 2004-06/ pdfs/ alexander.pdf)
[15] "Top 10 Password Crackers" (http:// sectools. org/ crackers.html). Sectools. . Retrieved 2008-11-01.
External links
• Philippe Oechslin: Making a Faster Cryptanalytic Time-Memory Trade-Off. (http:// lasecwww. epfl.ch/ pub/
lasec/ doc/ Oech03. pdf) CRYPTO 2003: pp617–630
• NIST Special Publication 800-63: Electronic Authentication Guideline (http:// csrc. nist. gov/ publications/
nistpubs/ 800-63/SP800-63V1_0_2.pdf)
Password fatigue
Password fatigue, also known as password chaos or identity chaos, is the feeling experienced by many people
who are required to remember an excessive number of passwords as part of their daily routine, such as to logon to a
computer at work, undo a bicycle lock or conduct banking from an ATM.
The increasing prominence of information technology and the Internet in employment, finance, recreation and other
aspects of people's lives, and the ensuing introduction of secure transaction technology, has led to people
accumulating a proliferation of accounts and passwords. According to a 2002 survey of British online-security
consultant NTA Monitor, the typical intensive computer user has 21 accounts that require a password
Aside from contributing to stress, password fatigue may encourage people to adopt habits that reduce the security of
their protected information. For example, an account holder might use the same password for several different
accounts, deliberately choose easy to remember passwords that are vulnerable to cracking, or rely on written records
of their passwords.
Other factors causing password fatigue are
• unexpected demands that a user create a new password
• unexpected demands that a user create a new password that uses particular pattern of letters, digits, and special
• demand that the user type the new password twice
• blind typing, both when responding to a password prompt and when setting a new password.
Some companies are well organized in this respect, have implemented alternative authentication methods
adopted technologies so that a user's credentials are entered automatically, but others may not focus on ease of use or
even worsen the situation by constantly implementing new applications with their own authentication system.
Password fatigue will typically affect users, but can also affect technical departments who manage user accounts as
they are constantly reinitializing passwords; this situation ends up lowering morale in both cases. In some cases users
end up typing their passwords in cleartext in text files so as to not have to remember them, or even writing them
down on paper notes.
Single sign-on software (SSO) can help mitigate this problem by only requiring users to remember one password to
an application that in turn will automatically give access to several other accounts, with or without the need for agent
software on the user's computer. A potential disadvantage is that loss of a single password will prevent access to all
services using the SSO system, and moreover theft or misuse of such a password presents a criminal or attacker with
many targets.
Password fatigue
Many operating systems provide a mechanism to store and retrieve passwords by using the users login password to
unlock an encrypted password database. Mac OS X has a Keychain feature that provides this functionality, and
similar functionality is present in the GNOME and KDE open source desktops. Microsoft Windows does not have an
explicit function for this, favoring centralized authentication based on the proprietary Microsoft Active Directory
In addition, web browser developers have added similar functionality to all of the major browsers, and password
management software such as KeePass and Password Safe can help mitigate the problem of password fatigue by
storing passwords in a database encrypted with a single password.
Additionally the majority of password protected web services provide a password recovery feature that will allow
users to recover their passwords via the email address (or other information) tied to that account.
These tools pose the problem that if the user's system is corrupted, stolen or compromised, apart from problems of
the data being misused, they can also lose access to sites where they rely on the password store or recovery features
to remember their login data. For this reason it is often advised to keep a separate record of sites, usernames and
passwords that is physically independent of the system.
Many sites in an attempt to block bad passwords also block good password practices such as MD5 and SHA1 hashes
through requiring both lower and uppercase letters or by limiting password length. Some sites also block Unicode or
special characters.
[1] Hayday, Graham. Security nightmare: How do you maintain 21 different passwords? (http:// software. silicon.com/ security/
0,39024655,11036760,00.htm), Silicon.com, 2002-12-11
[2] Such as digital certificates, OTP tokens, fingerprint authentication or password hints.
External links
• Access Denied (http:/ / www. washingtonpost. com/ wp-dyn/ content/ article/2006/ 09/ 22/
AR2006092201612_pf.html) By Yuki Noguchi, Washington Post September 23, 2006.
• Bad Form: 61% Use Same Password for Everything (http:// www.readwriteweb.com/ archives/
majority_use_same_password. php) by Josh Catone, January 17, 2008.
• TheFreeDictionary article on the subject (http:// computing-dictionary.thefreedictionary.com/ password+ chaos)
• identitychaos.com - MIIS & ILM blog (http:/ / www.identitychaos. com)
Password length parameter
Password length parameter
In telecommunication, a password length parameter is a basic parameter the value of which affects password
strength against brute force attack and so is a contributor to computer security.
One use of the password length parameters is in the expression , where is the probability that a
password can be guessed in its lifetime, is the maximum lifetime a password can be used to log in to a system,
is the number of guesses per unit of time, and is the number of unique algorithm-generated passwords (the
'password space').
The degree of password security is determined by the probability that a password can be guessed in its lifetime.
 This article incorporates public domain material from websites or documents of the General Services
Password management
There are several forms of software used to help users or organizations better manage passwords:
• Intended for use by a single user:
• Password manager software is used by individuals to organize and encrypt many personal passwords. This is
also referred to as a password wallet.
• Intended for use by a multiple users/groups of users:
• Password synchronization software is used by organizations to arrange for different passwords, on different
systems, to have the same value when they belong to the same person.
• Self-service password reset software enables users who forgot their password or triggered an intruder lockout
to authenticate using another mechanism and resolve their own problem, without calling an IT help desk.
• Enterprise Single signon software monitors applications launched by a user and automatically populates login
IDs and passwords.
• Web single signon software intercepts user access to web applications and either inserts authentication
information into the HTTP(S) stream or redirects the user to a separate page, where the user is authenticated
and directed back to the original URL.
• Privileged password management software
Password manager
Password manager
A password manager is software that helps a user organize passwords and PIN codes. The software typically has a
local database or files that holds the encrypted password data. Many password managers also work as a form filler,
thus they fill the user and password data automatically into forms. These are usually implemented as a browser
Password managers come in three basic flavors:
• Desktop - desktop software storing passwords on a computer hard drive.
• Portable - portable software storing passwords and program on a mobile device, such as a PDA, smart phone or as
a portable application on a USB stick such as U3 or similar.
• Web based - Online password manager where passwords are stored on a provider's website.
Password managers can also be used as a defense against phishing. Unlike human beings, a password manager
program, which can handle automated login script is not susceptible to visual imitations and look alike websites.
With this built-in advantage, the use of a password manager is beneficial even if the user only has a few passwords
to remember. However not all password managers can automatically handle the more complex login procedures
imposed by many banking websites.
Password managers typically use a user-selected master password or passphrase to form the key used to encrypt the
protected passwords. This master password must be strong enough to resist attack (eg, brute force, dictionary attacks,
A compromised master password renders all of the protected passwords vulnerable. This demonstrates the inverse
relation between usability and security: a single password may be more convenient (usable), but if compromised
would render compromised all of the passwords held.
As with any system which involves the user entering a password, the master password may also be attacked and
discovered using key logging or acoustic cryptanalysis. Some password managers attempt to use virtual keyboards to
reduce this risk - though this again is vulnerable to key loggers which take screenshots as data is entered.
Some password managers include a password generator. Generated passwords may be guessable if the password
manager uses a weak random number generator instead of a cryptographically secure one.
Password managers that do not prevent swapping their memory to hard drive make it then possible to extract
unencrypted passwords from the computers hard drive, though turning off swap, or installing more memory prevents
this risk.
Online password manager
An online password manager is a website that securely stores login details. They are a web-based version of more
conventional desktop-based password manager.
The advantages of online password managers over desktop-based versions are portability (they can generally be used
on any computer with a web browser and a network connection, without having to install software), and a reduced
risk of losing passwords through theft from or damage to a single PC - also the same risk is present for the server that
is used to store the users passwords on. In both cases this risk can be prevented by ensuring secure backups are
The major disadvantage of online password managers is the requirement that you trust the hosting site.
The use of a web-based password manager is an alternative to single sign-on techniques, such as OpenID or
Microsoft's Windows Live ID scheme (formerly Passport), or may serve as a stop-gap measure pending adoption of
Password manager
a better method.
External links
• Password manager
at the Open Directory Project
[1] http:/ / www. dmoz.org/Computers/ Security/Products_and_Tools/ Password_Tools/ /
Password notification e-mail
Password notification e-mail is a common technique used by websites. If a user forgets their password then a
password notification e-mail is sent containing enough information for the user to access their account again. This
method of password retrieval relies on the assumption that only the legitimate owner of the account has access to the
inbox for that particular e-mail address.
The process is often initiated by the user clicking on a forgotten password link on the website where, after entering
their username or e-mail address, the password notification e-mail would be automatically sent to the inbox of the
account holder. Some websites, such as Dating Direct, allow the user to choose to include the password in every
e-mail sent from the website. This has the problem that all of the e-mails received must be treated with the same
security as a password notification e-mail.
The email sent could contain a new, temporary password for the account or a URL that can be followed to enter a
new password for that account. The new password or the URL often contain a randomly generated string of text that
can only be obtained by reading that particular email. This is a very common technique used by websites such as
Another method used is to send all or part of the original password in the email. Sending only a few characters of the
password, a method employed by Friends Reunited, can help the user to remember their original password, without
having to reveal the whole password to them.
Security problems
The main issue is that the contents of the password notification email can be easily discovered by anyone with access
to the inbox of the account owner. This could be as a result of shoulder surfing or if the inbox itself is not password
protected. The contents could then be used to compromise the security of the account. The user would therefore have
the responsibility of either securely deleting the e-mail or ensuring that its contents are not revealed to anyone else.
A partial solution to this problem, employed by websites such as Google Accounts, is to cause any links contained
within the e-mail to expire after a period of time, making the e-mail useless if it is not used quickly after it is sent.
One problem with sending the original password in the e-mail is that the password contained within could be used to
access other accounts used by the user, if that user had chosen to use the same password for two or more accounts.
E-mails are often not secure so, unless the e-mail had been encrypted prior to being sent, the contents could be read
by anyone who eavesdrops on the e-mail.
Password policy
Password policy
A password policy is a set of rules designed to enhance computer security by encouraging users to employ strong
passwords and use them properly. A password policy is often part of an organization's official regulations and may
be taught as part of security awareness training. The password policy may either be advisory or mandated by
technical means.
Aspects of password policy
Typical components of a password policy include:
Password length and formation
Many policies require a minimum password length, typically 8 characters. Some systems impose a maximum length
for compatibility with legacy systems.
Some policies suggest or impose requirements on what type of password a user can choose, such as:
• the use of both upper- and lower-case letters (case sensitivity)
• inclusion of one or more numerical digits
• inclusion of special characters, e.g. @, #, $ etc.
• prohibition of words found in a dictionary or the user's personal information
• prohibition of passwords that match the format of calendar dates, license plate numbers, telephone numbers, or
other common numbers
• prohibition of use of company name or its abbreviation
As of October 2005, employees of the UK Government are advised to use passwords of the following form:
consonant, vowel, consonant, consonant, vowel, consonant, number, number (for example pinray45). This form is
called an Environ password and is case-insensitive. Unfortunately, since the form of this 8-character password is
known to potential attackers, the number of possibilities that need to be tested is actually fewer than a 6-character
password of no form (486,202,500 vs 2,176,782,336).
Other systems create the password for the users or let the user select one of a limited number of displayed choices.
Password duration
Some policies require users to change passwords periodically, e.g. every 90 or 180 days. Systems that implement
such policies sometimes prevent users from picking a password too close to a previous selection.
This policy can often backfire. Since it's hard to come up with 'good' passwords that are also easy to remember, if
people are required to come up with many passwords because they have to change them often, they end up using
much weaker passwords; the policy also encourages users to write passwords down. Also, if the policy prevents a
user from repeating a recent password, this means that there is a database in existence of everyone's recent passwords
(or their hashes) instead of having the old ones erased from memory.
Requiring a very strong password, and not requiring it be changed is often better. However it does have a major
drawback: if someone acquires a password, if it's not changed, they may have long term access.
It is necessary to weigh these factors: the likelihood of someone guessing a password because it is weak, vs the
likelihood of someone managing to steal, or otherwise acquire without guessing, a password.
Password policy
Common password practice
Password policies often include advice on proper password management such as:
• never sharing a computer account
• never using the same password for more than one account
• never telling a password to anyone, including people who claim to be from customer service or security
• never write down a password
• never communicating a password by telephone, e-mail or instant messaging
• being careful to log off before leaving a computer unattended
• changing passwords whenever there is suspicion they may have been compromised
• operating system password and application passwords are different
• password should be alpha-numeric
Password policies may include progressive sanctions beginning with warnings and ending with possible loss of
computer privileges or job termination. Where confidentiality is mandated by law, e.g. with classified information, a
violation of password policy could be a criminal offense. Some consider a convincing explanation of the importance
of security to be more effective than threats of sanctions.
Selection process
The level of password strength required depends, in part, on how easy it is for an attacker to submit multiple guesses.
Some systems limit the number of times a user can enter an incorrect password before some delay is imposed or the
account is frozen. At the other extreme, some systems make available a specially hashed version of the password so
anyone can check its validity. When this is done, an attacker can try passwords very rapidly and much stronger
passwords are necessary for reasonable security. (See password cracking and password length equation.) Stricter
requirements are also appropriate for accounts with higher privileges, such as root or system administrator accounts.
Usability considerations
Password policies are usually a tradeoff between theoretical security and the practicalities of human behavior. For
• Requiring excessively complex passwords and forcing them to be changed frequently can cause users to write
passwords down in places that are easy for an intruder to find, such as a Rolodex or post-it note near the
• Users often have dozens of passwords to manage. It may be more realistic to recommend a single password be
used for all low security applications, such as reading on-line newspapers and accessing entertainment web sites.
• Similarly, demanding that users never write down their passwords may be unrealistic and lead users to choose
weak ones. An alternative is to suggest keeping written passwords in a secure place, such as a safe or an
encrypted master file. The validity of this approach depends on what the most likely threat is deemed to be. While
writing down a password may be problematic if potential attackers have access to the secure store, if the threat is
primarily remote attackers who do not have access to the store, it can be a very secure method.
• Inclusion of special characters can be a problem if a user has to logon a computer in a different country. Some
special characters may be difficult or impossible to find on keyboards designed for another language.
• Some identity management systems allow Self Service Password Reset, where users can bypass password
security by supplying an answer to one or more security questions such as "where were you born?," "what's your
favorite movie?," etc. Often the answers to these questions can easily be obtained by social engineering, phishing
or simple research.
Password policy
Other approaches are available that are generally considered to be more secure than simple passwords. These include
use of a security token or one-time password system, such as S/Key.
Enforcing a Policy
The more complex a password policy the harder it may be to enforce, due to user difficulty in remembering or
choosing a suitable password.
Most companies will require users to familiarise themselves with any password policy, much in the same way a
company would require empoyees to be aware of Health & Safety regulations, or building fire exits, however it is
often difficult to ensure that the relevant policies are actually being followed.
External links
• Choosing good passwords
• Password management best practices
• Changing Passwords for Key User Accounts
• "Is It Just My Imagination?" article by Suzanne Ross
"Inkblots not only help users create a strong password,
but people also seem to enjoy using them."
• "Preventing reuse of passwords" by Matthew Slyman
A method of strongly discouraging password reuse
across systems with different levels of security
[1] http:/ / psynch. com/ docs/ choosing-good-passwords. html
[2] http:/ / psynch. com/ docs/ password-management-best-practices. html
[3] http:/ / www. windowsecurity. com/ articles/ Changing-Passwords-Key-User-Accounts.html
[4] http:/ / research.microsoft.com/ en-us/ news/ features/ inkblots.aspx
[5] http:// www. slyman. org/blog/ 2011/ 02/ preventing-password-reuse/
Password strength
Password strength
Password strength is a measure of the effectiveness of a password in
resisting guessing and brute-force attacks. In its usual form, it
estimates how many trials an attacker who does not have direct access
to the password would need, on average, to guess it correctly. The
strength of a password is a function of length, complexity, and
However, other attacks on passwords can succeed without a brute
search of every possible password. For instance, knowledge about a user may suggest possible passwords (such as
pet names, children's names, etc.). Hence estimates of password strength must also take into account resistance to
other attacks as well.
Using strong passwords lowers overall risk of a security breach, but strong passwords do not replace the need for
other effective security controls. The effectiveness of a password of a given strength is strongly determined by the
design and implementation of the authentication system software, particularly how frequently password guesses can
be tested by an attacker and how securely information on user passwords is stored and transmitted. Risks are also
posed by several means of breaching computer security which are unrelated to password strength. Such means
include wiretapping, phishing, keystroke logging, social engineering, dumpster diving, side-channel attacks, and
software vulnerabilities.
Determining password strength
There are two factors to consider in determining password strength: the ease with which an attacker can check the
validity of a guessed password, and the average number of guesses the attacker must make to find the correct
password. The first factor determined by how the password is stored and what it is used for, while the second factor
is determined by how long the password is, what set of symbols it is drawn from and how it is created.
Password guess validation
The most obvious way to test a guessed password is to attempt to use it to access the resource the password was
meant to protect. However, this can be slow and many systems will delay or block access to an account after several
wrong passwords are entered. On the other hand, systems that use passwords for authentication must store them in
some form to check against entered values. Usually only a cryptographic hash of a password is stored instead of the
password itself. If the hash is strong enough, it is very hard to reverse it, so an attacker that gets hold of the hash
value cannot directly recover the password. However, if the cryptographic hash data files have been stolen,
knowledge of the hash value lets the attacker quickly test guesses. (See Password cracking.)
In 2010, the Georgia Tech Research Institute developed a method of using GPGPU to crack passwords much
As of 2011, commercial products are available that claim the ability to test up to 2,800,000,000 passwords
per second on a standard desktop computer using a high-end graphics processor.
Such a device can crack a 10
letter single-case password in one day. Note that the work can be distributed over many computers for an additional
speedup proportional to the number of available computers with comparable GPUs. Special key stretching hashes are
available that take a relatively long time to compute, reducing the rate at which guessing can take place. Although it
is considered best practice to use key stretching, many common systems do not.
Another situation where quick guessing is possible is when the password is used to form a cryptographic key. In
such cases, an attacker can quickly check to see if a guessed password successfully decodes encrypted data. For
example, one commercial product claims to test 103,000 WPA PSK passwords per second.
Password strength
If a cryptographic salt is not used in the password system, the attacker can pre-compute hash values for common
passwords variants and for all passwords shorter than a certain length, allowing very rapid recovery. Long lists of
pre-computed password hashes can be efficiently stored using rainbow tables. Such tables are available on the
Internet for several common password authentication systems.
Password creation
Passwords are created either automatically (using randomizing equipment) or by a human. The strength of randomly
chosen passwords against a brute force attack can be calculated with precision.
Commonly, passwords are initially created by asking a human to choose a password, sometimes guided by
suggestions or restricted by a set of rules. This typically happens at the time of account creation for computer
systems or Internet Web sites. In this case, only estimates of strength are possible, since humans tend to follow
patterns in such tasks, and those patterns may assist an attacker.
In addition, lists of commonly chosen passwords
are widely available for use by password guessing programs. Any of the numerous online dictionaries for various
languages is such a list. All items in such lists are considered weak, as are passwords that are simple modifications of
them. Either can be quickly tried. For some decades, investigations of passwords on multi-user computer systems
have shown that 40% or more are readily guessed using only computer programs, and more can be found when
information about a particular user is taken into account during the attack.
Automatic password generation, if properly done, can avoid any connection between a password and its user. For
example, one's pet's name is quite unlikely to be generated by such a system. For a password chosen from a
sufficiently large 'password space', brute force search time can be made so long as to be infeasible. However, truly
random passwords can be tricky to generate (see random password generation), and they tend to be difficult for the
user to remember.
Entropy as a measure of password strength
It is usual in the computer industry to estimate password strength in terms of information entropy, measured in bits, a
concept from information theory. Instead of the number of guesses needed to find the password with certainty, the
base-2 logarithm of that number is given, which is the number of "entropy bits" in a password. A password with, say,
42 bits of strength calculated in this way would be as strong as a string of 42 bits chosen randomly, say by a fair coin
toss. Put another way, a password with 42 bits of strength would require 2
attempts to exhaust all possibilities
during a brute force search. Thus, adding one bit of entropy to a password doubles the number of guesses required,
which makes an attacker's task twice as difficult. On average, an attacker will have to try half the possible passwords
before finding the correct one.
(see Law of large numbers)
Random passwords
Random passwords consist of a string of symbols of specified length taken from some set of symbols using a random
selection process in which each symbol is equally likely to be selected. The symbols can be individual characters
from a character set (e.g., the ASCII character set), syllables designed to form pronounceable passwords, or even
words from a word list (thus forming a passphrase).
The strength of random passwords depends on the actual entropy of the underlying number generator; these are often
not truly random, but pseudo random. Many publicly available password generators use random number generators
found in programming libraries that offer limited entropy. However most modern operating systems offer
cryptographically strong random number generators that are suitable for password generation. It is also possible to
use ordinary dice to generate random passwords. See Random password generator#Stronger methods. Random
password programs often have the ability to ensure that the resulting password complies with a local password
policy; for instance, by always producing a mix of letters, numbers and special characters.
Password strength
For passwords generated by a process that randomly selects a string of symbols of length, L, from a set of N possible
symbols, the number of possible passwords can be found by raising the number of symbols to the power L, i.e. N
The strength of a random password as measured by the information entropy is just the base-2 logarithm or log
of the
number of possible passwords, assuming each symbol in the password is produced independently. Thus a random
password's information entropy, H, is given by the formula
where N is the number of possible symbols and L is the number of symbols in the password. H is measured in bits.
Entropy per symbol for different symbol sets
Symbol set Symbol count N Entropy per symbol H
Arabic numerals (0–9) (e.g. PIN) 10 3.3219 bits
hexadecimal numerals (0–9, A-F) (e.g. WEP keys) 16 4.0000 bits
Case insensitive Latin alphabet (a-z or A-Z) 26 4.7004 bits
Case insensitive alphanumeric (a-z or A-Z, 0–9) 36 5.1699 bits
Case sensitive Latin alphabet (a-z, A-Z) 52 5.7004 bits
Case sensitive alphanumeric (a-z, A-Z, 0–9) 62 5.9542 bits
All ASCII printable characters 94 6.5546 bits
Diceware word list 7776 12.9248 bits
Password strength depends on symbol set and length
Increasing the number of possible symbols from which random passwords are chosen will increase the strength of
generated passwords of any given length. For example, the printable characters in the ASCII character set (roughly
those on a standard U.S. English keyboard) include 26 letters (in two case variants), 10 digits, and 33
non-alphanumeric symbols (i.e., punctuation, grouping, etc.), for a total of 94 symbols (95 if space is included).
However the same strength can always be achieved with a smaller symbol set by choosing a longer password. In the
extreme, binary passwords can be very secure, even though they only use two possible symbols. See table below.
Thus a 14 character password consisting of only random lowercase letters has the same strength (4.7×14 = 65.8 bits)
as a ten character password chosen at random from all printable ASCII characters (65.55 bits).
Minimum lengths L of randomly generated passwords to achieve desired password entropy
H for symbol sets containing N symbols.
Desired password
entropy H
Case insensitive
Latin alphabet
Case insensitive
Case sensitive
Latin alphabet
Case sensitive
32 bits 10 7 7 6 6 5
40 bits 13 9 8 8 7 7
64 bits 20 14 13 12 11 10
80 bits 25 18 16 15 14 13
96 bits 29 21 19 17 17 15
128 bits 39 28 25 23 22 20
Password strength
160 bits 49 35 31 29 27 25
192 bits 58 41 38 34 33 30
224 bits 68 48 44 40 38 35
256 bits 78 55 50 45 43 40
384 bits 116 82 75 68 65 59
512 bits 155 109 100 90 86 79
1024 bits 309 218 199 180 172 157
Note that the full strength associated with using the entire ASCII character set (numerals, mixed case letters and
special characters) is only achieved if each character in the password is chosen randomly from that set. Capitalizing a
letter and adding a couple of numbers and a special character to a password will not achieve the same strength. If the
numbers and special character are added in predictable ways, say at the beginning and end of the password,
could even lower password strength compared to an all letter random password of the same length.
Because national keyboard implementations vary, not all 94 ASCII printable characters can be used everywhere.
This can present a problem to an international traveler who wished to log into remote system using a keyboard on a
local computer. See keyboard layout.
Authentication programs (e.g., those which determines access to a computer system) vary in which characters they
allow in passwords. Some do not recognize case differences (e.g., the upper-case "E" is considered equivalent to the
lower-case "e"), others prohibit some of the other symbols. In the past few decades, systems have permitted more
characters in passwords, but limitations still exist. Many hand held devices, such as PDAs and smart phones, require
complex shift sequences to enter special characters. Systems also vary in the maximum length of passwords allowed,
with some older systems limited to eight characters.
Human-generated passwords
People are notoriously remiss at achieving sufficient entropy to produce satisfactory passwords. Some stage
magicians exploit this inability for amusement, in a minor way, by divining supposed random choices (of numbers,
say) made by audience members.
Thus, in one analysis of over 3 million eight-character passwords, the letter "e" was used over 1.5 million times,
while the letter "f" was only used 250,000 times. A uniform distribution would have had each character being used
about 900,000 times. The most common number used is "1", whereas the most common letters are a, e, o, and r.
NIST suggests the following scheme to estimate the entropy of human-generated passwords:
• the entropy of the first character is four bits;
• the entropy of the next seven characters are two bits per character;
• the ninth through the twentieth character has 1.5 bits of entropy per character;
• characters 21 and above have one bit of entropy per character.
This would imply that an eight-character human-selected password has about 18 bits of entropy.
Users rarely make full use of larger characters sets in forming passwords. For example, hacking results obtained
from a MySpace phishing scheme in 2006 revealed 34,000 passwords, of which only 8.3% used mixed case,
numbers, and symbols.
Password strength
Bit strength threshold
As a practical matter, passwords must be both reasonable and functional for the end user as well as strong enough for
the intended purpose. Passwords that are too difficult to remember may be forgotten and so are more likely to be
written on paper, which some consider a security risk.
In contrast, others argue that forcing users to remember
passwords without assistance can only accommodate weak passwords, and thus poses a greater security risk.
According to Bruce Schneier, most people are good at securing their wallets or purses, which is a "great place" to
store a written password.
Some basic benchmarks have been established for brute force searches in the context of attempting to find keys used
in encryption. The problem is not the same since these approaches involve astronomical numbers of trials, but the
results are suggestive for password choice. In 1999, an Electronic Frontier Foundation project broke 56-bit DES
encryption in less than a day using specially designed hardware.
In 2002, distributed.net cracked a 64-bit key in 4
years, 9 months, and 23 days.
As of August 17, 2010, distributed.net estimates that cracking a 72-bit key using
current hardware will take about 48,712 days or 133.5 years.
Due to currently understood limitations from
fundamental physics, there is no expectation that any digital computer (or combination) will be capable of breaking
256-bit encryption via a brute-force attack.
Whether or not quantum computers will be able to do so in practice is
still unknown, though theoretical analysis suggests such possibilities.
As a result, there can be no exact answer to the somewhat different problem of the password strength required to
resist brute force attack in practice. NIST recommends 80-bits for the most secure passwords, which can nearly be
achieved with a 95-character choice (e.g., the original ASCII character set) with a 12-character random password (12
x 6.5 bits = 78).
A 2010 Georgia Tech Research Institute study also recommended a 12-character random
password, but as a minimum length requirement.

Guidelines for strong passwords
Common guidelines
Common guidelines for choosing good passwords are designed to make passwords less easily discovered by
intelligent guessing:



• Password length should be around 12 to 14 characters if permitted, and longer still if possible while remaining
• Use randomly generated passwords where feasible
• Avoid any password based on repetition, dictionary words, letter or number sequences, usernames, relative or pet
names, romantic links (current or past), or biographical information (e.g., ID numbers, ancestors names or dates).
• Include numbers, and symbols in passwords if allowed by the system
• If the system recognizes case as significant, use capital and lower-case letters
• Avoid using the same password for multiple sites or purposes
• If you write your passwords down, keep the list in a safe place, such as a wallet or safe, not attached to a monitor
or in an unlocked desk drawer
Password strength
Additional guidelines
Double a character consecutively, to discourage shoulder surfing, the technique whereby someone observes the
typing over a shoulder. Don't triple a character and don't double more than one character. If the typist is fast, it's hard
to see how many times a key was consecutively pressed.
On email services there is usually a restore password function that a hacker can figure out and by doing so bypass a
password. Its generally a good idea to have a hard to guess restore password question to further secure the password.
As a user might need access from a phone with a small keyboard, consider which nonalphanumerics appear on all
models, if any do.
Individuals and businesses can also choose to use devices or cloud-based applications that generate a one-time
password, which are functional for only one session or expire after a limited amount of time. One-time password
generator solutions are available using cloud-based services, mobile phone applications, a security token and other
Examples of weak passwords
As with any security measure, passwords vary in effectiveness (i.e., strength); some are weaker than others. For
example, the difference in weakness between a dictionary word and a word with obfuscation (i.e., letters in the
password are substituted by, say, numbers— a common approach) may cost a password cracking device a few more
seconds– this adds little strength. The examples below illustrate various ways weak passwords might be constructed,
all of which are based on simple patterns which result in extremely low entropy:
• Default passwords (as supplied by the system vendor and meant to be changed at installation time): password,
default, admin, guest, etc. All are typically very easy to discover.
• Dictionary words: chameleon, RedSox, sandbags, bunnyhop!, IntenseCrabtree, etc., can be automatically tried at
very high speeds.
• Words with numbers appended: password1, deer2000, john1234, etc., can be easily tested automatically with
little lost time.
• Words with simple obfuscation: p@ssw0rd, l33th4x0r, g0ldf1sh, etc., can be easily tested automatically with little
additional effort.
• Doubled words: crabcrab, stopstop, treetree, passpass, etc., can be easily tested automatically.
• Common sequences from a keyboard row: qwerty, 12345, asdfgh, fred, etc., can be easily tested automatically.
• Numeric sequences based on well known numbers such as 911
(9-1-1, 9/11)
, 314159...
, or 27182...
, etc., can
easily be tested automatically.
• Identifiers: jsmith123, 1/1/1970, 555–1234, "your username", etc., can easily be tested automatically.
• Anything personally related to an individual: license plate number, Social Security number, current or past
telephone number, student ID, address, birthday, sports team, relative's or pet's
names/nicknames/birthdays/initials, etc., can easily be tested automatically after a simple investigation of person's
There are many other ways a password can be weak,
corresponding to the strengths of various attack schemes; the
core principle is that a password should have high entropy (usually taken to be equivalent to randomness) and not be
readily derivable by any "clever" pattern, nor should passwords be mixed with information identifying the user.
Password strength
Examples that follow guidelines
The passwords below are examples that follow some of the published guidelines for strong passwords. But note
carefully that, since these example passwords have been published in this article, they should never be used as actual
passwords as their publication have rendered them and derivates of them as weak passwords. The listed passwords
are only for usage as examples.
• 4pRte!ai@3 – mixes uppercase, lowercase, numbers, and punctuation. This suggests a large character set which
increases an attacker's work factor, a desirable property.
• Tp4tci2s4U2g! – builds from a phrase a user can memorize: "The password for (4) this computer is too (2) strong
for you to (4U2) guess!" — mixes types of character. If the phrase is not 'well-known' (e.g., published in any
quotation compendium – online or in print, no matter how obscure to you), this password should have high
entropy for an attacker, and be easier to remember than many passwords.
• tDI"60Hs7Q – are characters selected from two poetry stanzas by different methods from a page selected using an
honest die, but likely to be hard to memorize.
• l52@36291QBs( – represents the serial number of a US currency bill with added elements that should be random,
e.g. chosen via the honest die mentioned above. The bill and its serial number are likely to be hard to connect to
the user and thus present high entropy to an attacker. Note that some currency may use predictable serial numbers
(e.g., adding check digits, padding, type codes and the like, often covertly) and, if so, will have less entropy than
might be expected.
• BBslwys90! – is loosely based on a phrase that a user might memorize: "Big Brother is always right (right angle =
90°)!" – also mixes character classes, which increases an attacker's work factor.
• B1g bRother |$ alw4ys riGHt!? - is an example of an extremely strong, easy to remember yet hard to guess
password created by using a passphrase as a password. Key features are multiple words, length, random
punctuation, random capitalization, and random simple substitutions. If a user can remember these small changes
and chooses a phrase that is obscurely personal (the user associates it with something, but any other person would
not generally make this assumption), the result should be a strong password that needn't be written down.
Password policy
A password policy is a guide to choosing satisfactory passwords. Some are controversial. They are usually intended
• assist users in choosing strong passwords
• ensure the passwords are suited to the target population
• recommendations to users with regard to the handling of their passwords
• a requirement to change any password which has been lost or compromised, and perhaps that no password be
used longer than a limited time
• some policies prescribe the pattern of characters which passwords must contain
For example, password expiration is often covered by password policies. Password expiration serves two
• if the time to crack a password is estimated to be 100 days, password expiration times fewer than 100 days may
help ensure insufficient time for an attacker.
• if a password has been compromised, requiring it to be changed regularly should limit the access time for the
Some argue that password expirations have become obsolete,
• asking users to change passwords frequently encourages simple, weak passwords.
• if one has a truly strong password, there is little point in changing it since the existing password is already strong.
Changing passwords which are already strong introduces risk that the new password may be less strong. Since
Password strength
any compromised password is weak by definition, the possibility of compromise must be considered in estimating
password strength.
Creating and handling passwords
The hardest passwords to crack, for a given length and character set, are random character strings; if long enough
they resist brute force attacks (because there are many characters) and guessing attacks (due to high entropy).
However, such passwords are typically the hardest to remember. The imposition of a requirement for such passwords
in a password policy may encourage users to write them down, store them in PDAs or cellphones, or share them with
others as a safeguard against memory failure. Some people consider each of these user resorts to increase security
risks. Others suggest the absurdity of expecting users to remember distinct complex passwords for each of the
dozens of accounts they access. For example, security expert Bruce Schneier recommends writing down your
Simply, people can no longer remember passwords good enough to reliably defend against dictionary attacks,
and are much more secure if they choose a password too complicated to remember and then write it down.
We're all good at securing small pieces of paper. I recommend that people write their passwords down on a
small piece of paper, and keep it with their other valuable small pieces of paper: in their wallet.
—Bruce Schneier 2005
The following measures may increase acceptance of strong password requirements, if carefully used:
• a training program. Also, updated training for those who fail to follow the password policy (lost passwords,
inadequate passwords, etc.).
• rewarding strong password users by reducing the rate, or eliminating altogether, the need for password changes
(password expiration). The strength of user-chosen passwords can be estimated by automatic programs which
inspect and evaluate proposed passwords, when setting or changing a password.
• displaying to each user the last login date and time in the hope that the user may notice unauthorized access,
suggesting a compromised password.
• allowing users to reset their passwords via an automatic system, which reduces help desk call volume. However,
some systems are themselves insecure; for instance, easily guessed or researched answers to password reset
questions bypass the advantages of a strong password system.
• using randomly generated passwords that do not allow users to choose their own passwords, or at least offering
randomly generated passwords as an option.
Memory techniques
Password policies sometimes suggest memory techniques to assist remembering passwords:
• mnemonic passwords: Some users develop mnemonic phrases and use them to generate high-entropy (more or
less random) passwords which are nevertheless relatively easy for the user to remember. For instance, the first
letter of each word in a memorable phrase. Silly ones are possibly more memorable.
Another way to make
random-appearing passwords more memorable is to use random words (see diceware) or syllables instead of
randomly-chosen letters.
• after-the-fact mnemonics: After the password has been established, invent a mnemonic that fits.
It does not
have to be reasonable or sensible, only memorable. This allows passwords to be random.
• password patterns: Any pattern in a password makes guessing (automated or not) easier and reduces an attacker's
work factor.
• In an example from the UK in October 2005, employees of the British government were advised to use
passwords of the following form: consonant, vowel, consonant, consonant, vowel, consonant, number, number
(for example pinray45). This pattern is called an Environ password and is case-insensitive. The pattern of
alternating vowel and consonant characters was intended to make passwords more likely to be pronounceable
Password strength
and thus more memorable. Unfortunately, such patterns severely reduce the password's information entropy,
making brute force password attacks considerably more efficient.
Protecting passwords
Computer users are generally advised to "never write down a password anywhere, no matter what" and "never use
the same password for more than one account." However, an ordinary computer user may have dozens of
password-protected accounts. Users with multiple accounts needing passwords often give up and use the same
password for every account. When varied password complexity requirements prevent use of the same (memorable)
scheme for producing high-strength passwords, overly simplified passwords will often be created to satisfy irritating
and conflicting password requirements. A Microsoft expert was quoted as saying at a 2005 security conference: "I
claim that password policy should say you should write down your password. I have 68 different passwords. If I am
not allowed to write any of them down, guess what I am going to do? I am going to use the same password on every
one of them."
If passwords are written down, they should never be kept in obvious places such as address books, Rolodex files,
under drawers or keyboards, or behind pictures. Perhaps the worst, but all too common, location is a Post-It note on
the computer monitor. Better locations are a safe deposit box or a locked file approved for information of sensitivity
comparable to that protected by the password. Most locks on office file cabinets are far from adequate. Software is
available for popular hand-held computers that can store passwords for numerous accounts in encrypted form.
Another approach is to encrypt by hand on paper and remember the encryption method and key.
And another
approach is to use a single password or slightly-varying passwords for low-security accounts and select distinctly
separate strong passwords for a smaller number of high-value applications such as for online banking.
Password managers
A reasonable compromise for using large numbers of passwords is to record them in a password manager such as
KeePass. A password manager allows the user to use hundreds of different passwords, and only have to remember a
single password, the one which opens the encrypted password database. Needless to say, this single password should
be strong and well-protected (not recorded anywhere). Most password managers can automatically create strong
passwords using a cryptographically secure random password generator, as well as calculating the entropy of the
generated password. A good password manager will provide resistance against attacks such as key logging, clipboard
logging and various other memory spying techniques.
Password strength advisers
Several web sites, and some standalone programs meant to be run without a network connection on a local machine,
offer automated tests of password strength adequacy. They are problematic. Any network based checking necessarily
involves submitting one's password to a purpose declared system somewhere. Doing so eases an attacker's problem
very considerably; the relevant network traffic is identifiable as passwords saving much sifting effort, authentication
of network connection problems permit authentication problems (eg, site spoofing) which are lessened for equivalent
programs running on local computers.
Even when run on local machines, without network involvement, there are potential problems. Implementational
problems (eg, errors in programming or algorithm choice) are always possible, of course, and many of these exhibit
no discernible clues .for a user or administrator. And, such programs are limited to estimates of brute force attack
vulnerability regardless of where they are run. Passwords which are vulnerable to guessing attacks cannot be
checked automatically, as not everyone's pet is named Rover of Fluffy, nor do all children or relatives have common
names, readily found in lists.
As a result, use of such checking facilities cannot be recommended. The tests they can apply are easy for users to
perform, if they understand some basic principles. And the increased attack opportunities should be avoided as well,
Password strength
on the general principle of not making an attacker's task easier.
[1] "Cyber Security Tip ST04-002" (http:// www.us-cert.gov/ cas/ tips/ ST04-002.html). Choosing and Protecting Passwords. US CERT. .
Retrieved June 20, 2009.
[2] "Teraflop Troubles: The Power of Graphics Processing Units May Threaten the World's Password Security System" (http:/ / www. gtri.
gatech. edu/ casestudy/ Teraflop-Troubles-Power-Graphics-Processing-Units-GPUs-Password-Security-System). Georgia Tech Research
Institute. . Retrieved 2010-11-07.
[3] Elcomsoft.com (http:/ / www.elcomsoft. com/ eprb.html#gpu), ElcomSoft Password Recovery Speed table], NTLM passwords, Nvidia Tesla
S1070 GPU, accessed 2011-2-1
[4] Elcomsoft Wireless Security Auditor, HD5970 GPU (http:// www.elcomsoft.com/ ewsa. html) accessed 2011-2-11
[5] "Electronic Authentication Guideline" (http:// csrc. nist. gov/ publications/ nistpubs/ 800-63/ SP800-63V1_0_2.pdf) (PDF). NIST. .
Retrieved March 27, 2008.
[6] Schneier, B: Applied Cryptography, 2e, page 233 ff. John Wiley and Sons.
[7] Microsoft.com (http:// www.microsoft. com/ protect/fraud/passwords/ create.aspx)
[8] Burnett, Mark (2006). Kleiman, Dave. ed. Perfect Passwords. Rockland, Massachusetts: Syngress Publishing. p. 181. ISBN 1-59749-041-5.
[9] Bruce Schneier. "MySpace Passwords aren't so Dumb" (http:// www.wired. com/politics/ security/ commentary/securitymatters/ 2006/ 12/
72300?currentPage=2). Wired Magazine. . Retrieved April 11, 2008.
[10] A. Allan. "Passwords are Near the Breaking Point" (http:/ /www. indevis.de/ dokumente/ gartner_passwords_breakpoint.pdf) (PDF).
Gartner. . Retrieved April 10, 2008.
[11] Bruce Schneier. "Schneier on Security" (http:/ / www.schneier. com/ blog/ archives/2005/ 06/ write_down_your.html). Write Down Your
Password. . Retrieved April 10, 2008.
[12] "EFF DES Cracker machine brings honesty to crypto debate" (http:// w2.eff.org/ Privacy/Crypto/ Crypto_misc/ DESCracker/HTML/
19980716_eff_descracker_pressrel.html). EFF. . Retrieved March 27, 2008.
[13] "64-bit key project status" (http:// stats. distributed. net/ projects.php?project_id=5). Distributed.net. . Retrieved March 27, 2008.
[14] Bruce Schneier. "Snakeoil: Warning Sign #5: Ridiculous key lengths" (http:/ / www.schneier.com/ crypto-gram-9902.html). . Retrieved
March 27, 2008.
[15] "Want to deter hackers? Make your password longer" (http:// www. msnbc. msn. com/ id/ 38771772/ ). MSNBC. 2010-08-19. . Retrieved
[16] Microsoft Corporation, Strong passwords: How to create and use them (http:// www. microsoft.com/ protect/yourself/ password/ create.
[17] Bruce Schneier, Choosing Secure Passwords (http:/ / www.schneier. com/ blog/ archives/ 2007/ 01/ choosing_secure.html)
[18] Google, Inc., How safe is your password? (https:/ / www.google.com/ accounts/ PasswordHelp)
[19] Bidwell, Teri re (2002). Syngress Publishing. ISBN 1931836515.
[20] Nick, Re: Password Security Tips, November 14, 2009 (topic post) (http:/ / forum.scottmueller. com/ viewtopic.php?p=6100#p6100), as
accessed January 20, 2010
[21] http:// www. offlinetalk.com/ index. php/ topic,12. msg30/
[22] E.g., for a keyboard with only 17 nonalphanumeric characters, see one for a BlackBerry phone in an enlarged image (http:// www.
hardwaresecrets.com/ fullimage.php?image=18705) in support of Sandy Berger, BlackBerry Tour 9630 (Verizon) Cell Phone Review, in
Hardware Secrets (August 31, 2009) (http:/ / www.hardwaresecrets.com/ article/ 795/ 2), both as accessed January 19, 2010. That some
websites don’t allow nonalphanumerics is indicated by Kanhef, Idiots, For Different Reasons (June 30, 2009) (topic post) (http:/ / forums.
theregister. co.uk/post/ 527230), as accessed January 20, 2010.
[23] Bidwell, p. 87
[24] "In Defense of Password Expiration" (http:// lopsa. org/ node/ 295). League of Professional Systems Administrators. . Retrieved April 14,
[25] Eugene Spafford. "Security Myths and Passwords" (http:// www. cerias.purdue.edu/ weblogs/ spaf/ general/post-30/ ). The Center for
Education and Research in Information Assurance and Security. . Retrieved April 14, 2008.
[26] Schneier.com (http:// www.schneier. com/ blog/ archives/ 2005/ 06/ write_down_your.html)
[27] Mnemonic Devices (Indianapolis, Ind.: Bepko Learning Ctr., University College) (http:// uc.iupui. edu/ uploadedFiles/
Learning_Center_Site/Mnemonic Devices. pdf), as accessed January 19, 2010
[28] Remembering Passwords (ChangingMinds.org) (http:/ / changingminds. org/ techniques/ memory/ remembering_passwords.htm), as
accessed January 19, 2010
[29] Microsoft security guru: Jot down your passwords (http:/ / news. com.com/ Microsoft+security+ guru+Jot+ down+ your+passwords/
2100-7355_3-5716590. html?tag=nefd.pop), News.com.com Retrieved on 2007-05-07
[30] Simple methods (e.g., ROT13 and some other old ciphers) may suffice; for more sophisticated hand-methods see Bruce Schneier, The
Solitaire Encryption Algorithm (May 26, 1999) (ver. 1.2) (http:// www. schneier.com/ solitaire. html), as accessed January 19, 2010, and Sam
Siewert, Big Iron Lessons, Part 5: Introduction to Cryptography, From Egypt Through Enigma (IBM, July 26, 2005) (http:// www.ibm.com/
developerworks/power/ library/pa-bigiron5/), as accessed January 19, 2010.
Password strength
External links
• Choosing Good Passwords — A User Guide (http:/ / psynch. com/ docs/ choosing-good-passwords. html).
• Password Policy Guidelines (http:// psynch. com/ docs/ password-policy-guidelines. html).
• Examples of common (and hence weak) passwords (http:// www.modernlifeisrubbish.co. uk/
top-10-most-common-passwords. asp)
• Bruce Schneier (December 14, 2006). MySpace Passwords Aren't So Dumb (http:// www.schneier. com/
essay-144. html).
• How to Write Better Passwords (http:// www. csoonline. com/ read/120105/ ht_passwords. html)
• RFC 4086: Randomness Requirements for Security (http:/ / tools. ietf. org/html/ rfc4086)
• Frequently used passwords to avoid (http:/ / www.searchlores. org/commonpass1. htm)
• Steve Gibson (June 2011). GRC's | Password Haystacks: How well Hidden is Your Needle? (https:/ / www.grc.
com/haystack. htm)
• Microsoft's Password Strength Checker (https:// www.microsoft.com/ security/ pc-security/ password-checker.
aspx?WT. mc_id=Site_Link)
• passwordmeter.com's Password Strength Checker (http:// www.passwordmeter. com/ )
• pwsecurity.de Password Security (http:// www.pwsecurity. de/ )
Password synchronization
Password synchronization is defined as any process or technology that helps users to maintain a single password
that is subject to a single security policy, and changes on a single schedule across multiple systems.
It's a type of Identity management software and it's considered as easier to implement than enterprise single sign-on
(SSO), as there is no client software deployment, and user enrollment can be automated.
Password synchronization is an effective mechanism for addressing password management problems on an
enterprise network:
• Users with synchronized passwords tend to remember their passwords.
• Simpler password management means that users make significantly fewer password-related calls to the help desk.
• Users with just one or two passwords are much less likely to write down their passwords.
Some (in particular those who sell single signon systems) claim that password synchronization is less secure than
single signon, since compromise of one password means compromise of all. The counter-argument is that, with
single signon, compromise of the primary password (from which an encryption key is derived and used to protect all
other, stored passwords) also compromises all, so the security of password synchronization and single signon is
similar -- i.e., both systems depend strongly on the security of a single password, and that password must be well
defended, regardless of such academic arguments.
Password synchronization
Two types of password synchronization processes are commonly available in commercial software:
• Transparent password synchronization, triggered by a password change on an existing system. The new password
is automatically forwarded to other user objects that belong to the same user, on other systems (of the same or
different types).
• Web-based password synchronization, initiated by the user with a web browser, in place of the existing native
password change process. The web-based process allows the user to set multiple passwords at once.
The best form of password synchronization is one that securely synchronizes only the stored representations of the
original passwords -- not by sharing the clear text password itself. For this, however, both parties must share the
same password storage and verification scheme. Therefore, this feature is typically only found in proprietary forms
where the password scheme is controlled by a single vendor on both ends. As standards for password storage evolve,
password synchronization between vendors may begin to utilize this third and more secure synchronization type.
External links
Password Management Project Roadmap
vendor-neutral white paper about how to run a project to deploy this
type of software
[1] http:/ / password-manager. hitachi-id.com/ docs/ password-management-project-roadmap.html
Password-authenticated key agreement
In cryptography, a password-authenticated key agreement method is an interactive method for two or more parties
to establish cryptographic keys based on one or more party's knowledge of a password.
Password-authenticated key agreement generally encompasses methods such as:
• Balanced password-authenticated key exchange
• Augmented password-authenticated key exchange
• Password-authenticated key retrieval
• Multi-server methods
• Multi-party methods
In the most stringent password-only security models, there is no requirement for the user of the method to remember
any secret or public data other than the password.
Password authenticated key exchange (PAKE) is where two or more parties, based only on their knowledge of a
password, establish a cryptographic key using an exchange of messages, such that an unauthorized party (one who
controls the communication channel but does not possess the password) cannot participate in the method and is
constrained as much as possible from guessing the password. (The optimal case yields exactly one guess per run
exchange.) Two forms of PAKE are Balanced and Augmented methods.
Balanced PAKE allows parties that use the same password to negotiate and authenticate a shared key. Examples of
these are:
• Encrypted Key Exchange (EKE)
• PAK and PPK
Password-authenticated key agreement
• SPEKE (Simple password exponential key exchange)
• J-PAKE (Password Authenticated Key Exchange by Juggling)
Augmented PAKE is a variation applicable to client/server scenarios, in which an attacker must perform a
successful brute-force attack in order to masquerade as the client using stolen server data. Examples of these are:
• Augmented-EKE
Password-authenticated key retrieval is a process in which a client obtains a static key in a password-based
negotiation with a server that knows data associated with the password, such as the Ford and Kaliski methods. In the
most stringent setting, one party uses only a password in conjunction with two or more (N) servers to retrieve a static
key, in a way that protects the password (and key) even if any N-1 of the servers are completely compromised.
Brief history
The first successful password-authenticated key agreement methods were Encrypted Key Exchange methods
described by Steven M. Bellovin and Michael Merritt in 1992. Although several of the first methods were flawed,
the surviving and enhanced forms of EKE effectively amplify a shared password into a shared key, which can then
be used for encryption and/or message authentication.
The first provably-secure PAKE protocols were given in work by M. Bellare, D. Pointcheval, and P. Rogaway
(Eurocrypt 2000) and V. Boyko, P. MacKenzie, and S. Patel (Eurocrypt 2000). These protocols were proven secure
in the so-called random oracle model (or even stronger variants), and the first protocols proven secure under standard
assumptions were those of O. Goldreich and Y. Lindell (Crypto 2001) and J. Katz, R. Ostrovsky, and M. Yung
(Eurocrypt 2001).
The first password-authenticated key retrieval methods were described by Ford and Kaliski in 2000.
A considerable number of refinements, alternatives, variations, and security proofs have been proposed in this
growing class of password-authenticated key agreement methods. Current standards for these methods include IETF
RFC 2945 and RFC 5054, IEEE Std 1363.2-2008, ITU-T X.1035 and ISO-IEC 11770-4:2006.
• Bellare, M.; D. Pointcheval; P. Rogaway (2000). "Authenticated Key Exchange Secure against Dictionary
Attacks". Advances in Cryptology -- Eurocrypt 2000 LNCS (Springer-Verlag) 1807: 139.
• Bellovin, S. M.; M. Merritt (May 1992). "Encrypted Key Exchange: Password-Based Protocols Secure Against
Dictionary Attacks". Proceedings of the I.E.E.E. Symposium on Research in Security and Privacy (Oakland): 72.
• Boyko, V.; P. MacKenzie; S. Patel (2000). "Provably Secure Password-Authenticated Key Exchange Using
Diffie-Hellman". Advances in Cryptology -- Eurocrypt 2000, LNCS (Springer-Verlag) 1807: 156.
• Ford, W.; B. Kaliski (14–16 June 2000). "Server-Assisted Generation of a Strong Secret from a Password".
Proceedings of the IEEE 9th International Workshops on Enabling Technologies: Infrastructure for
Collaborative Enterprises (Gaithersburg MD: NIST): 176. doi:10.1109/ENABL.2000.883724.
• Goldreich, O.; Y. Lindell (2001). "Session-Key Generation Using Human Passwords Only". Advances in
Cryptology -- Crypto 2001 LNCS (Springer-Verlag) 2139.
Password-authenticated key agreement
• IEEE Std 1363.2-2008: IEEE Standard Specifications for Password-Based Public-Key Cryptographic
Techniques. IEEE. 2009
• Katz, J.; R. Ostrovsky; M. Yung (2001). Efficient Password-Authenticated Key Exchange Using
Human-Memorable Passwords. 2045. Springer-Vergal.
• T. Wu. The SRP-3 Secure Remote Password Protocol. IETF RFC 2945.
• D. Taylor, T. Wu, N. Mavrogiannopoulos, T. Perrin. Using the Secure Remote Password (SRP) Protocol for TLS
Authentication. IETF RFC 5054.
• Y. Sheffer, G. Zorn, H. Tschofenig, S. Fluhrer. An EAP Authentication Method Based on the Encrypted Key
Exchange (EKE) Protocol. IETF RFC 6124.
• ISO/IEC 11770-4:2006 Information technology—Security techniques—Key management—Part 4: Mechanisms
based on weak secrets.
External links
• IEEE P1363 Working Group
• IEEE Std 1363.2-2008: IEEE Standard Specifications for Password-Based Public-Key Cryptographic Techniques
• David Jablon's links for password-based cryptography
• Simple Password-Based Encrypted Key Exchange Protocols Abdalla et al 2005
[1] http:/ / grouper.ieee. org/groups/ 1363/ index. html
[2] http:/ / ieeexplore.ieee. org/servlet/ opac?punumber=4773328
[3] http:// jablon.org/passwordlinks. html
[4] http:// www. di. ens. fr/~mabdalla/ papers/AbPo05a-letter.pdf
PBKDF2 (Password-Based Key Derivation Function) is a key derivation function that is part of RSA
Laboratories' Public-Key Cryptography Standards (PKCS) series, specifically PKCS #5 v2.0, also published as
Internet Engineering Task Force's RFC 2898. It replaces an earlier standard, PBKDF1, which could only produce
derived keys up to 160 bits long.
PBKDF2 applies a pseudorandom function, such as a cryptographic hash, cipher, or HMAC to the input password or
passphrase along with a salt value and repeats the process many times to produce a derived key, which can then be
used as a cryptographic key in subsequent operations. The added computational work makes password cracking
much more difficult, and is known as key stretching. When the standard was written in 2000, the recommended
minimum number of iterations was 1000, but the parameter is intended to be increased over time as CPU speeds
increase. Having a salt added to the password reduces the ability to use a precomputed dictionary to attack a
password (such as rainbow tables) and means that multiple passwords have to be tested individually, not all at once.
The standard recommends a salt length of at least 64 bits.
Key derivation process
Function is defined as
DK = PBKDF2(PRF, P,S,c,dkLen)
• PRF is a parameter of PBKDF2 - it is a pseudorandom function of two parameters with output length hLen (e.g.
keyed HMAC)
• P is the master password for which a derivation is generated
• S is a salt
• c number of iterations, positive integer
• dkLen is a length of derived key
• DK is a generated derived key
For each hLen-bit block T
of derived key DK, computing is as follows:
DK = T
|| T
|| ... || T
= F(P,S,c,i)
Where F is an xor of c iterations of chained PRF. First iteration of PRF uses master password P as PRF key and salt
concatenated to i. Second and greater PRF uses P and output of previous PRF computation:
F(P,S,c,i) = U
^ U
^ ... ^ U
U1 = PRF(P,S || INT_msb(i))
U2 = PRF(P,U
Uc = PRF(P,U
− 1)
For example, WPA2 uses
DK = PBKDF2(HMAC−SHA1, passphrase, ssid, 4096, 256)
Systems that use PBKDF2
• Wi-Fi Protected Access (WPA and WPA2) used to secure Wi-Fi wireless networks
• Microsoft Windows Data Protection API (DPAPI)
• OpenDocument encryption used in OpenOffice.org
• WinZip's AES Encryption scheme.

• LastPass for password hashing.
Disk encryption software
• FileVault (Mac OS X) from Apple Computer
• FreeOTFE (Windows and Pocket PC PDAs); also supports mounting Linux (e.g. LUKS) volumes under Windows
• LUKS (Linux Unified Key Setup) (Linux)
• TrueCrypt (Windows, Linux, and Mac OS X)
• DiskCryptor (Windows)
• Cryptographic disk
• GEOM ELI module for FreeBSD
• softraid crypto for OpenBSD
• EncFS (Linux) since v1.5.0
BlackBerry vulnerability
In September 2010, ElcomSoft announced a password cracking utility for Research In Motion BlackBerry mobile
devices that takes advantage of what Vladimir Katalov, ElcomSoft's CEO, described as the "very strange way, to say
the least" in which the BlackBerry uses PBKDF2. The BlackBerry encrypts backup files with AES-256. In turn, the
AES key is derived from the user's password using PBKDF2. However the BlackBerry software uses only one
PBKDF2 iteration. By contrast, according to Katalov, Apple's iOS 3 uses 2000 iterations and iOS 4 uses 10,000.
[1] http:/ / msdn. microsoft.com/ library/default.asp?url=/ library/en-us/ dnsecure/ html/ windataprotection-dpapi.asp
[2] http:// www. winzip. com/ aes_tips. htm
[3] http:/ / www. winzip. com/ gladman. cgi
[4] http:/ / blog.lastpass. com/ 2011/ 05/ lastpass-security-notification. html
[5] http:// crypto.nsa. org/vilefault/ 23C3-VileFault. pdf
[6] http:/ / netbsd. org/guide/ en/ chap-cgd.html
[7] http:/ / web.nvd. nist. gov/ view/ vuln/ detail?vulnId=CVE-2010-3741
[8] http:// www. infoworld.com/ t/ mobile-device-management/you-can-no-longer-rely-encryption-protect-blackberry-436
External links
• RSA PKCS #5 (http:/ / www.rsa. com/ rsalabs/ node.asp?id=2127) – RSA Laboratories PKCS #5 v2.0 -
Multiple Formats, and test vectors.
• RFC 2898 – Specification of PKCS #5 v2.0.
• RFC 6070 – Test vectors for PBKDF2 with HMAC-SHA1.
• ActionScript 3.0 implementation (http:// code. google. com/ p/as3-pbkdf2/ )
• .NET's built-in function (http:// msdn. microsoft.com/ en-us/ library/system. security. cryptography.
rfc2898derivebytes. aspx)
• C# implementation (http:/ / msdn. microsoft.com/ en-us/ magazine/ cc163913.aspx)
• JavaScript implementation (slow) (http:// anandam. name/ pbkdf2) JavaScript implementation (fast) (http://
code. google. com/ p/ crypto-js/)
• Python implementation (http:/ / www. dlitz.net/ software/python-pbkdf2)
• Perl implementation (http:/ / search. cpan. org/dist/ Crypt-PBKDF2/)
• Ruby implementation (http:/ / github. com/ emerose/ pbkdf2-ruby/tree/master)
• C implementation (http:/ / www. openbsd. org/cgi-bin/cvsweb/ src/ sbin/ bioctl/ pbkdf2.c?rev=HEAD&
• PHP implementation (http:/ / www. itnewb.com/ v/
Personal identification number
A personal identification number (PIN, pronounced "pin") is a secret numeric password shared between a user and
a system that can be used to authenticate the user to the system. Typically, the user is required to provide a
non-confidential user identifier or token (the user ID) and a confidential PIN to gain access to the system. Upon
receiving the user ID and PIN, the system looks up the PIN based upon the user ID and compares the looked-up PIN
with the received PIN. The user is granted access only when the number entered matches with the number stored in
the system. Hence, despite the name, a PIN does not personally identify the user.
PINs are most often used for automated teller machines (ATMs) but are increasingly used at the point of sale, for
debit cards and credit cards. Throughout Europe and Canada the traditional in-store credit card signing process is
increasingly being replaced with a system where the customer is asked to enter their PIN instead of signing. In the
UK and Ireland this goes under the term 'Chip and PIN', since PINs were introduced at the same time as EMV chips
on the cards. In other parts of the world, PINs have been used before the introduction of EMV. Apart from financial
uses, GSM mobile phones usually allow the user to enter a PIN of between 4 and 8 digits. The PIN is recorded in the
SIM card.
In 2006, James Goodfellow, the inventor of the personal identification number, was awarded an OBE in the Queen's
Birthday Honours List.
Personal identification number
PIN length
The concept of a PIN originates with the inventor of the ATM, John Shepherd-Barron. One day in 1967, while
thinking about more efficient ways banks could disburse cash to their customers, it occurred to him that the vending
machine model was a proven fit. For authentication Shepherd-Barron at first envisioned a six-digit numeric code,
given what he could reliably remember. His wife however preferred four digits, which became the most commonly
used length.
ISO 9564-1, the international standard for PIN management and security, allows for PINs from 4 up
to 12 digits, but also notes that "For usability reasons, an assigned numeric PIN should not exceed six digits in
PIN validation
There are several main methods of validating PINs. The operations discussed below are usually performed within a
hardware security module (HSM).
IBM 3624
The IBM method is used to generate what is termed a natural PIN. The natural PIN is generated by encrypting the
primary account number (PAN), using an encryption key generated specifically for the purpose.
This key is
sometimes referred to as the PIN generation key (PGK). This PIN is directly related to the primary account number.
To validate the PIN, the issuing bank regenerates the PIN using the above method, and compares this with the
entered PIN.
Natural PINs can not be user selectable because they are derived from the PAN. If the card is reissued with a new
PAN, a new PIN must be generated.
Natural PINs allow banks to issue PIN reminder letters as the PIN can be generated.
IBM 3624 + offset
To allow user selectable PINs it is possible to store a PIN offset value. The Offset is found by subtracting natural
PIN from the customer selected PIN using modulo 10.
For example, if the natural PIN is 1234, and the user wishes
to have a PIN of 2345, the offset is 1111.
The offset can be stored either on the card track data,
or in a database at the card issuer.
To validate the PIN, the issuing bank calculates the natural PIN as in the above method, then adds the offset and
compares this value to the entered PIN.
VISA method
The VISA method is used by many card schemes and is not VISA-specific. The VISA method generates a PIN
verification value (PVV). Similar to the offset value, it can be stored on the card's track data, or in a database at the
card issuer. This is called the reference PVV.
The VISA method takes the right most 11 digits of the PAN excluding the checksum value, a PIN validation key
index (PVKI) and the required PIN value encrypted with the PIN validation key (PVK) referenced by the PVKI.
From this encrypted value, the PVV is found.
To validate the PIN, the issuing bank calculates a PVV value from the entered PIN and PAN and compares this value
to the reference PVV. If the reference PVV and the calculated PVV match, the correct PIN was entered.
Unlike the IBM method, the VISA method doesn't derive a PIN. The PVV value is used to confirm the PIN entered
at the terminal, was also used to generate the reference PVV. The PIN used to generate a PVV can be randomly
generated or user selected or even derived using the IBM method.
Personal identification number
PIN security
Financial PINs are often 4-digit numbers in the range 0000-9999, resulting in 10,000 possible numbers. Switzerland
is a notable exception with 6 digit pins being given by default. However, some banks do not give out numbers where
all digits are identical (such as 1111, 2222, ...), consecutive (1234, 2345, ...), numbers that start with one or more
zeroes, or the last 4 digits of your social security number. Many PIN verification systems allow three attempts,
thereby giving a card thief a 0.06% probability of guessing the correct PIN before the card is blocked. This holds
only if all PINs are equally likely and the attacker has no further information available, which has not been the case
with some of the many PIN generation and verification algorithms that banks and ATM manufacturers have used in
the past.
In 2002 two PhD students at Cambridge University, Piotr Zieliński and Mike Bond, discovered a security flaw in the
PIN generation system of the IBM 3624, which was duplicated in most later hardware. Known as the decimalization
table attack, the flaw would allow someone who has access to a bank's computer system to determine the PIN for an
ATM card in an average of 15 guesses.

If a mobile phone PIN is entered incorrectly three times, the SIM card is blocked until a Personal Unblocking Code
(PUC), provided by the service operator, is entered. If the PUC is entered incorrectly ten times, the SIM card is
permanently blocked, requiring a new SIM card.
Safety practices for PIN:
• Limit PIN usage.
• Use the link key instead of the PIN.
• Use in secure environments.
"PIN number"
The term "PIN number" (hence "personal identification number number") is commonly used. This is an example of
RAS syndrome (Redundant Acronym Syndrome syndrome).
Reverse PIN hoax
Rumours have been in e-mail circulation claiming that in the event of entering a PIN into an ATM backwards, police
will be instantly alerted as well as money being ordinarily issued as if the PIN had been entered correctly.
intention of this scheme would be to protect victims of muggings; however, despite the system being proposed for
use in some US states, there are no ATMs currently in existence that employ this software.
Related pages
• ATM SafetyPIN software
• ISO 9564, international standard for PIN management and security in retail banking
• Personal Unblocking Code
• PIN pad
• Point of sales
• Transaction authentication number
Personal identification number
[1] Your ID number is not a password (http:// webb-site. com/ articles/ identity. asp), Webb-site.com, 8 November 2010
[2] "Royal honour for inventor of Pin" (http:// news. bbc. co.uk/ 1/ hi/ scotland/ glasgow_and_west/ 5087984. stm). BBC. 2006-06-16. .
Retrieved 2007-11-05.
[3] "The Man Who Invented The CASH Machine" (http:/ / news.bbc.co.uk/ 2/ hi/ business/ 6230194.stm). BBC. 2007-06-25. . Retrieved
[4] ISO 9564-1:2002 Banking -- Personal Identification Number (PIN) management and security -- Part 1: Basic principles and requirements for
online PIN handling in ATM and POS systems (http:/ / www.iso.org/ iso/ iso_catalogue/ catalogue_tc/ catalogue_detail.
htm?csnumber=29374), clause 7.1
[5] "3624 PIN Generation Algorithm" (http:// publib. boulder.ibm.com/ infocenter/zos/ v1r9/index. jsp?topic=/ com.ibm.zos.r9. csfb400/
csfb4z80539. htm). IBM. .
[6] "PIN Offset Generation Algorithm" (http:// publib. boulder.ibm. com/ infocenter/zos/ v1r9/index.jsp?topic=/ com.ibm. zos. r9.csfb400/
csfb4z80541. htm). IBM. .
[7] "Track format of magnetic stripe cards" (http:// www. gae.ucm.es/ ~padilla/ extrawork/tracks.html). .
[8] "PVV Generation Algorithm" (http:// publib. boulder.ibm. com/ infocenter/zos/ v1r9/index. jsp?topic=/ com. ibm. zos. r9.csfb400/
csfb4z80545. htm). IBM. .
[9] Kuhn, Markus (July 1997) (PDF). Probability theory for pickpockets — ec-PIN guessing (http:// www.cl. cam.ac. uk/ ~mgk25/ ec-pin-prob.
pdf). . Retrieved 2006-11-24.
[10] Zieliński, P & Bond, M (February 2003) (PDF). Decimalisation table attacks for PIN cracking (http:// www.cl.cam.ac.uk/ TechReports/
UCAM-CL-TR-560. pdf). University of Cambridge Computer Laboratory. . Retrieved 2006-11-24.
[11] "Media coverage" (http:// www.cl. cam. ac. uk/ ~mkb23/ media-coverage.html). University of Cambridge Computer Laboratory. .
Retrieved 2006-11-24.
[12] MySecureCyberSpace (http:// www.mysecurecyberspace. com/ encyclopedia/ index/ pin-cracking.html#msc. encyclopedia.pincracking)
[13] "Reverse PIN Panic Code" (http:// www.snopes. com/ business/ bank/ pinalert.asp). . Retrieved 2007-03-02.
Pre-shared key
In cryptography, a pre-shared key or PSK is a shared secret which was previously shared between the two parties
using some secure channel before it needs to be used. Such systems almost always use symmetric key cryptographic
algorithms. The term PSK is used in WiFi encryption such as WEP or WPA, where both the wireless access points
(AP) and all clients share the same key.
The characteristics of this secret or key are determined by the system which uses it; some system designs require that
such keys be in a particular format. It can be a password like 'bret13i', a passphrase like 'Idaho hung gear id gene', or
a hexadecimal string like '65E4 E556 8622 EEE1'. The secret is used by all systems involved in the cryptographic
processes used to secure the traffic between the systems.
As in all cases of crypto systems relying on one or more keys for confidentiality, the key or keys used must be
sufficiently difficult to attack. One particular attack is always possible against keys, the brute force key space search
attack. A sufficiently long, randomly chosen, key can resist any practical brute force attack, though not in principle if
an attacker has sufficient computer power. (See password strength and password cracking for more discussion).
Unavoidably, however, pre-shared keys are held by both parties to the communication, and so can be compromised
at one end, without the knowledge of anyone at the other. There are several tools available to help one choose strong
passwords, though doing so over any network connection is inherently unsafe as one can't in general know who, if
anyone, may be eavesdropping on the interaction. Choosing keys used by cryptographic algorithms is somewhat
different in that any pattern whatsoever should be avoided, as any such pattern may provide an attacker with a lower
effort attack than brute force search. This implies random key choice to force attackers to spend as much effort as
possible; this is very difficult in principle and in practice as well. As a general rule, any software program except a
Cryptographically secure pseudorandom number generator should be avoided.
Privileged password management
Privileged password management
Privileged password management software may be deployed by organizations to secure the passwords for login
IDs that have elevated security privileges. This is most often done by periodically changing every such password to a
new, random value. Since users and automated software processes need these passwords to function, privileged
password management systems must also store these passwords and provide various mechanisms to disclose these
passwords in a secure and appropriate manner.
Please also see Privileged Identity Management -- another name for the same type of software.
Examples of privileged passwords
There are three main types of privileged passwords. They are used to authenticate:
• Local administrator accounts.
• Service accounts.
• Connections by one application to another.
Local administrator accounts and passwords
On Unix and Linux systems, the root user is a privileged login account. On Windows, the equivalent is
Administrator. On SQL databases, the equivalent is sa. In general, most operating systems, databases, applications
and network devices include an administrative login, used to install software, configure the system, manage users,
apply patches, etc. On some systems, different privileged functions are assigned to different users, which means that
there are more privileged login accounts, but each of them is less powerful.
Service accounts and passwords
On the Windows operating system, service programs execute in the context of either SYSTEM (very privileged, but
has no password), or of a user account. When services run as a non-SYSTEM user, the service control manager must
provide a login ID and password to run the service program -- so service accounts have passwords. On Unix and
Linux systems, init and inetd can launch service programs as non-privileged users without knowing their passwords,
so services do not normally have passwords.
Embedded application accounts and passwords
Often, one application needs to be able to connect to another, to access a service. A common example of this pattern
is when a web application must log into a database to retrieve some information. These inter-application connections
normally require a login ID and password and this password.
Securing privileged passwords
A privileged password management system secures privileged passwords by:
• Periodically changing each password to a new, random value.
• Storing these values.
• Protecting the stored values (e.g., using encryption and replicated storage).
• Providing mechanisms to disclose these passwords to various types of participants in the system:
• IT administrators.
• Programs that launch services (e.g., service control manager on Windows).
• Applications that must connect to other applications.
Privileged password management
Required infrastructure
A privileged password management system requires extensive infrastructure:
• A mechanism to schedule password changes.
• Connectors to various kinds of systems.
• Mechanisms to update various participants with new password values.
• Extensive auditing.
• Encrypted storage.
• Authentication for parties that wish to retrieve password values.
• Access controls / authorization to decide whether password disclosure is appropriate.
• Replicated storage, to ensure that hardware failure or a site disaster does not lead to loss of data.
Other kinds of password management
There is software to manage other kinds of passwords - typically for end users. See Password management to learn
about single signon, password wallets, password reset, password synchronization and more.
• Best Practices for Managing Privileged Passwords
(from Hitachi ID Systems, Inc.)
• Best Practices Guide to Privileged Password Management
(from Irdeto)
• Privileged Identity Management, IDC Defines an Identity and Access Management Submarket
(from IDC)
[1] http:/ / privileged-password-manager.hitachi-id. com/ docs/ privileged-password-management-best-practices.html
[2] http:/ / datacenter.cloakware.com/ google/ wp-password-management/wp-privileged-password-management.pdf
[3] http:/ / www. cyber-ark.com/ pdf/ IDC_White_Paper_PIM_GRC.pdf
Risk-based authentication
Risk-based authentication
Risk-based Authentication process flow. The
process highlighted in green is what gets added
by the Risk-based Authentication systems.
Risk-based authentication is a non-static authentication system which
takes into account the profile of the agent requesting access to the
system to determine the risk profile associated with that transaction.
The risk profile is then used to determine the complexity of the
challenge. Higher risk profiles leads to stronger challenges, whereas a
static username/password may suffice for lower risk profiles. Risk
based implementation allows the application to only challenge the user
for additional credentials when the risk level is appropriate. "Machine
authentication is often used in a risk based authentication set up. The
machine authentication will run in the background and only ask the
customer for additional authentication if the computer is not
recognized. In a risk based authentication system, the institution
decides if additional authentication is necessary. If the risk is deemed appropriate, enhanced authentication will be
triggered, such as a one time password delivered via an out of band communication. Risk based authentication can
also be used during the session to prompt for additional authentication when the customer performs a certain high
risk transaction, such as a money transfer or an address change. Risk based authentication is very beneficial to the
customer because additional steps are only required if something is out of the ordinary, such as the login attempt is
from a new machine."
1. Better UI and UX compared to blanket 2-Factor authentication
2. Easier subsequent login process, thus reducing the chances of the user leaving the session open unnecessarily.
1. The system that computes the risk profile has to be diligently maintained and updated as new threat vectors
emerge. Improper configuration may lead to unauthorized access.
2. The user's connection profile (e.g. IP Geolocation, connection type, keystroke dynamics, user behaviour) has to
be detected and used to compute the risk profile. Lack of proper detection may lead to unauthorized access.
[1] Williamson, G. "Enhanced Authentication In Online Banking" (http:/ / utica.edu/ academic/ institutes/ ecii/ publications/ articles/
51D6D996-90F2-F468-AC09C4E8071575AE.pdf) Journal of Economic Crime Management 4.2 (2006): 18–19. Print
S/KEY is a one-time password system developed for authentication to Unix-like operating systems, especially from
dumb terminals or untrusted public computers on which one does not want to type a long-term password. A user's
real password is combined in an offline device with a short set of characters and a decrementing counter to form a
single-use password. Because each password is only used once, they are useless to password sniffers.
Because the short set of characters does not change until the counter reaches zero, it is possible to prepare a list of
single-use passwords, in order, that can be carried by the user. Alternatively, the user can present the password,
characters and desired counter value to a local calculator to generate the appropriate one-time password that can then
be transmitted over the network in the clear. The latter form is more common and practically amounts to
challenge-response authentication.
S/KEY is supported in Linux (via Pluggable authentication modules), OpenBSD, NetBSD, and FreeBSD, and a
generic open source implementation can be used to enable its use on other systems. One common implementation is
called OPIE. S/KEY is a trademark of Telcordia Technologies, formerly known as Bell Communications Research
S/KEY is also sometimes referred to as Lamport's scheme, after its author, Leslie Lamport. It was developed by
Neil Haller, Phil Karn and John Walden at Bellcore in the late 1980s. With the expiration of the basic patents on
public key cryptography and the widespread use of laptop computers running SSH and other cryptographic protocols
that can secure an entire session, not just the password, S/KEY is falling into disuse. SecurID is a related one-time
password scheme that still sees widespread use because, unlike S/KEY, it provides two-factor authentication by
requiring a physical token that cannot be easily reproduced.
Password generation
The server is the computer that will perform the authentication.
S/KEY password generation
1. This step begins with a secret key . This
secret can either be provided by the user, or
can be generated by a computer. Either way,
if this secret is disclosed then the security of
S/KEY is compromised
2. A cryptographic hash function is applied times to , thereby producing a hash chain of one-time
passwords. The passwords are the results of the application of the cryptographic hash function:
3. The initial secret is discarded
4. The user is provided with the passwords, printed out in reverse order:
5. The passwords are discarded from the server. Only the password
, the one at the top of the user's list, is stored on the server.
S/KEY authentication
After password generation, the user has
a sheet of paper with passwords on
it. The first password is the same
password that the server has stored.
This first password will not be used for
authentication (the user should scratch
this password on the sheet of paper), the
second one will be used instead:
• The user provides the server with the
second password on the list
and scratches that password.
• The server attempts to compute
where is the
password supplied. If
produces the first password (the one
the server has stored), then the
authentication is successful. The
server will then store as the current reference.
For subsequent authentications, the user will provide password . (The last password on the printed list, password
, is the first password generated by the server, , where is the initial secret). The server will compute
password and will compare the result to password , which is stored as reference on the server.
The security of S/KEY relies on the difficulty of reversing cryptographic hash functions. Assume an attacker
manages to get hold of a password that was used for a successful authentication. Supposing this is password i, this
password is already useless for subsequent authentications, because each password can only be used once. It would
be interesting for the attacker to find out password i-1, because this password is the one that will be used for the next
However this would require inverting the hash function that produced password i-1 using password i (password i-1 =
H(password i)), which is extremely difficult to do with current cryptographic hash functions.
S/KEY is however vulnerable to a man in the middle attack if used by itself. It is also vulnerable to certain race
conditions, such as where an attacker's software sniffs the network to learn the first N-1 characters in the password
(where N equals the password length), establishes its own TCP session to the server, and in rapid succession tries all
valid characters in the Nth position until one succeeds. These types of vulnerabilities can be avoided by using ssh,
SSL, SPKM or other encrypted transport layer.
Internally, S/KEY uses 64 bit numbers. For human usability purposes, each number is mapped to 6 short words of 1
to 4 characters each from a publicly accessible 2048-word dictionary. For example, one 64 bit number maps to
• The handbook of applied cryptography, Alfred J. Menezes, Paul C. van Oorschot and Scott A. Vanstone
Chapter 10 describes Lamport's scheme on page 396
• RFC 1760 - The S/KEY One-Time Password System
• RFC 2289 - A One-Time Password System
External links
• jsotp: JavaScript OTP & S/Key Calculator
• Introduction to the system
• Java Micro Edition S/key calculator for cell phones
[1] http:/ / citeseer.ist. psu. edu/ cache/ papers/ cs/ 5967/ ftp:zSzzSzftp.mcc.ac. ukzSzpubzSzsecurityzSzPAPERSzSzPASSWORDzSzSKEY.
pdf/haller94skey. pdf
[2] http:// www. cacr. math. uwaterloo.ca/ hac/
[3] http:/ / www. ocf.berkeley. edu/ ~jjlin/ jsotp/
[4] http:/ / www. orange-carb.org/ SkeyCalc/ documentation. html#about
[5] http:/ / tanso. net/ j2me-otp
Secure Password Authentication
Secure Password Authentication
Secure Password Authentication (SPA) is a proprietary Microsoft protocol used to authenticate Microsoft email
clients with an electronic mail server when using the Simple Mail Transfer Protocol (SMTP), Post Office Protocol
(POP), or Internet Message Access Protocol (IMAP).
The protocol was based on the Integrated Windows
Authentication (NTLM) authentication scheme.
[1] http:/ / www. kuro5hin.org/ ?op=displaystory;sid=2002/ 4/ 28/ 1436/ 66154
Secure Remote Password protocol
The Secure Remote Password protocol (SRP) is a password-authenticated key agreement protocol.
The SRP protocol has a number of desirable properties: it allows a user to authenticate himself to a server, it is
resistant to dictionary attacks mounted by an eavesdropper, and it does not require a trusted third party. It effectively
conveys a zero-knowledge password proof from the user to the server. Only one password can be guessed at per
attempt in revision 6 of the protocol. One of the interesting properties of the protocol is that even if one or two of the
cryptographic primitives it uses are attacked, it is still secure. The SRP protocol has been revised several times, and
is currently at revision six.
The SRP protocol creates a large private key shared between the two parties in a manner similar to Diffie–Hellman,
then verifies to both parties that the two keys are identical and that both sides have the user's password. In cases
where encrypted communications as well as authentication are required, the SRP protocol is more secure than the
alternative SSH protocol and faster than using Diffie–Hellman with signed messages. It is also independent of third
parties, unlike Kerberos. The SRP protocol, version 3 is described in RFC 2945. SRP version 6 is also used for
strong password authentication in SSL/TLS
(in TLS-SRP) and other standards such as EAP
and SAML, and is
being standardized in IEEE P1363 and ISO/IEC 11770-4.
The following notation is used in this description of the protocol, version 6:
• q and N = 2q + 1 are chosen such that both are prime (N is a safe prime and q is a Sophie Germain prime). N must
be large enough so that computing discrete logarithms modulo N is infeasible.
• All arithmetic is performed in the field of integers modulo N, .
• g is a generator of the multiplicative group.
• k is a parameter derived by both sides; for example, k = H(N, g).
• s is a small salt.
• I is an identifying username.
• p is the user's password.
• H() is a hash function; e.g., SHA-256.
• v is the host's password verifier, v = g
, x = H(s,p).
• u, a and b are random.
• | denotes concatenation.
All other variables are defined in terms of these.
Secure Remote Password protocol
First, to establish a password p with Steve, Carol picks a small random salt s, and computes x = H(s, p), v = g
. Steve
stores v and s, indexed by I, as Carol's password verifier and salt. x is discarded because it is equivalent to the
plaintext password p. This step is completed before the system is used.
1. Carol → Steve: I | A, with A = g
2. Steve → Carol: s | B, with B = kv + g
3. Both: u = H(A, B)
4. Carol: S
= (B - kg
(a + ux)
5. Carol: K
= H(S
6. Steve: S
= (Av
7. Steve: K
= H(S
Now the two parties have a shared, strong session key K. To complete authentication, they need to prove to each
other that their keys match. One possible way is as follows:
1. Carol → Steve: M
= H(H(N) XOR H(g) | H(I) | s | A | B | K
). Steve verifies M
2. Steve → Carol: M
= H(A | M
| K
). Carol verifies M
This method requires guessing more of the shared state to be successful in impersonation than just the key. While
most of the additional state is public, private information could safely be added to the inputs to the hash function,
like the server private key. The two parties also employ the following safeguards:
1. Carol will abort if she receives B == 0 (mod N) or u == 0.
2. Steve will abort if he receives A == 0 (mod N).
3. Carol must show her proof of K first. If Steve detects that Carol's proof is incorrect, he must abort without
showing his own proof of K.
Implementation example in Python
# An example SRP-6a authentication
# WARNING: Do not use for real cryptographic purposes beyond testing.
# based on http://srp.stanford.edu/design.html
import hashlib
import random
def global_print(*names):
x = lambda s: ["%s", "0x%x"][isinstance(s, long)] % s
print "".join("%s = %s\n" % (name, x(globals()[name])) for name in
def H(*a): # a one-way hash function
return int(hashlib.sha256(str(a)).hexdigest(), 16) % N
def cryptrand(n=1024):
return random.SystemRandom().getrandbits(n) % N
# A large safe prime (N = 2q+1, where q is prime)
# All arithmetic is done modulo N
# (generated using "openssl dhparam -text 1024")
N = '''00:c0:37:c3:75:88:b4:32:98:87:e6:1c:2d:a3:32:
Secure Remote Password protocol
N = int(''.join(N.split()).replace(':', ''), 16)
g = 2 # A generator modulo N
k = H(N, g) # Multiplier parameter (k=3 in legacy SRP-6)
print "#. H, N, g, and k are known beforehand to both client and
global_print("H", "N", "g", "k")
print "0. server stores (I, s, v) in its password database"
# the server must first generate the password verifier
I = "person" # Username
p = "password1234" # Password
s = cryptrand(64) # Salt for the user
x = H(s, p) # Private key
v = pow(g, x, N) # Password verifier
global_print("I", "p", "s", "x", "v")
print "1. client sends username I and public ephemeral value A to the
a = cryptrand()
A = pow(g, a, N)
global_print("a", "A") # client->server (I, A)
print "2. server sends user's salt s and public ephemeral value B to
b = cryptrand()
B = (k * v + pow(g, b, N)) % N
global_print("b", "B") # server->client (s, B)
print "3. client and server calculate the random scrambling parameter"
u = H(A, B) # Random scrambling parameter
print "4. client computes session key"
x = H(s, p)
S_c = pow(B - k * pow(g, x, N), a + u * x, N)
K_c = H(S_c)
global_print("S_c", "K_c")
Secure Remote Password protocol
print "5. server computes session key"
S_s = pow(A * pow(v, u, N), b, N)
K_s = H(S_s)
global_print("S_s", "K_s")
print "6. client sends proof of session key to server"
M_c = H(H(N) ^ H(g), H(I), s, A, B, K_c)
# client->server (M_c) ; server verifies M_c
print "7. server sends proof of session key to client"
M_s = H(A, M_c, K_s)
# server->client (M_s) ; client verifies M_s
Real world implementations
• TLS-SRP is a set of ciphersuites for transport layer security that uses SRP.
• The Javascript Crypto Library
includes a Javascript implementation of the SRP protocol, open source, AGPL
licensed. Used in Clipperz online password manager
• The Srp-Hermetic Library
uses SRP as part of the process to establish a secure AJAX channel. Srp-Hermetic is
released under the MIT open source license.
• Gnu Crypto
provide a Java implementation licensed under the GNU General Public License with the "library
exception", which permits its use as a library in conjunction with non-Free software.
• The Legion of the Bouncy Castle
provide Java and C# implementations licensed under the MIT License.
• srplibcpp
is a C++ implement base on Miracl.
• csrp
is a C implementation depend on OpenSSL
[1] Taylor, David; Tom Wu; Nikos Mavrogiannopoulos; Trevor Perrin (November 2007). "Using the Secure Remote Password (SRP) Protocol
for TLS Authentication" (http:// tools. ietf.org/ html/ rfc5054). . RFC 5054
[2] Carlson, James; Bernard Aboba; Henry Haverinen (July 2001). "EAP SRP-SHA1 Authentication Protocol" (http:// tools. ietf.org/ html/
draft-ietf-pppext-eap-srp-03). IETF. . Draft.
[3] http:// sourceforge.net/ projects/ clipperz
[4] http:// www. clipperz.com
[5] http:/ / blog.denksoft. com/ ?page_id=26
[6] http:// www. gnu. org/software/ gnu-crypto/
[7] http:// www. bouncycastle. org/
[8] http:// code.google. com/ p/ srplibcpp/
[9] http:/ / code.google. com/ p/ csrp/
Secure Remote Password protocol
External links
• Official website (http:// http:// srp. stanford.edu)
Manual pages
• pppd(8) (http:// linux. die.net/ man/ 8/ pppd): Point-to-Point Protocol Daemon – Linux Administration and
Privileged Commands Manual
• srptool(1) (http:/ / linux. die. net/ man/ 1/ srptool): Simple SRP password tool – Linux User Commands Manual
• RFC 2944 - Telnet Authentication: SRP
• RFC 2945 - The SRP Authentication and Key Exchange System
• RFC 3720 - Internet Small Computer Systems Interface (iSCSI)
• RFC 3723 - Securing Block Storage Protocols over IP
• RFC 3669 - Guidelines for Working Groups on Intellectual Property Issues
• RFC 5054 - Using the Secure Remote Password (SRP) Protocol for TLS Authentication
Other links
• IEEE 1363 (http:// grouper.ieee. org/groups/ 1363/ )
• SRP Intellectual Property Slides (http:/ / www.pdl. cmu. edu/ mailinglists/ ips/ mail/ msg08027. html)
SecurID, now known as RSA SecurID, is a mechanism developed by Security Dynamics (later RSA Security and
now RSA, The Security Division of EMC) for performing two-factor authentication for a user to a network resource.
RSA SecurID token (older style, model SD600)
The RSA SecurID authentication mechanism consists of a "token"—a
piece of hardware (e.g. a token or USB) or software (e.g. a "soft token"
for a computer, PDA or cell phone)—assigned to a computer user that
generates an authentication code at fixed intervals (usually 30 or 60
seconds) using a built-in clock and the card's factory-encoded random
key (known as the "seed" and often provided as an ASCII file). The
seed is different for each token, and is loaded into the corresponding
RSA SecurID server (RSA Authentication Manager, formerly
ACE/Server) as the tokens are purchased. The seed is typically 128 bits
long. Some RSA SecurID deployments may use varied second rotations, such as 30-second increments.
The token hardware is designed to be tamper-resistant to deter reverse engineering. Despite this, public code has
been developed by the security community allowing a user to emulate RSA SecurID in software, but only if they
have access to a current RSA SecurID code, and the original RSA SecurID seed file introduced to the server.
RSA SecurID token
RSA SecurID (new style, SID800 model with
smartcard functionality)
the RSA SecurID authentication scheme, the seed record is the secret
key used to generate one-time passwords. "Soft tokens" are merely
commercial software implementations of the same algorithms
implemented in the tamper-resistant hardware, only the soft tokens
require the seed record to be distributed to clients so that the seed
record may be used as input in the one-time password generation.
Newer versions also feature a USB connector, which allows the token
to be used as a smart card-like device for securely storing
A user authenticating to a network resource—say, a dial-in server or a
firewall—needs to enter both a personal identification number and the
number being displayed at that moment on their RSA SecurID token.
Some systems using RSA SecurID disregard PIN implementation
altogether, and rely on password/RSA SecurID code combinations.
The server, which also has a real-time clock and a database of valid
cards with the associated seed records, computes what number the
token is supposed to be showing at that moment in time, checks it
against what the user entered, and makes the decision to allow or deny
On systems implementing PINs, a "duress PIN" may be used—an alternate code which creates a security event log
showing that a user was forced to enter their PIN, while still providing transparent authentication.
While the RSA SecurID system adds a strong layer of security to a network, difficulty can occur if the authentication
server's clock becomes out of sync with the clock built in to the authentication tokens. However, typically the RSA
Authentication Manager automatically corrects for this without affecting the user. It is also possible to resync a token
manually in the RSA Authentication Manager. Providing authentication tokens to everyone who might need to
access a resource can be expensive (about $15 per year + licencing costs), particularly since tokens are programmed
to "expire" at a fixed time, usually three years, requiring purchase of a new token.
RSA SecurID currently commands over 70% of the two-factor authentication market (source: IDC) and 25 million
devices have been produced to date. A number of competitors, such as VASCO, make similar security tokens,
mostly based on the open OATH HOTP standard. A study on OTP published by Gartner in 2010 mentions OATH
and SecurID as the only competitors.
RSA Security has pushed forth an initiative called "Ubiquitous Authentication", partnering with device
manufacturers such as IronKey, SanDisk, Motorola, Freescale Semiconductor, Redcannon, Broadcom and
BlackBerry to embed the SecurID software into everyday devices such as USB flash drives and cell phones, to
reduce cost and the number of objects that the user must carry.
Other network authentication systems, such as OPIE and S/Key (sometimes more generally known as OTP, as S/Key
is a trademark of Telcordia Technologies, formerly Bellcore) attempt to provide the "something you have" level of
authentication without requiring a hardware token.
Theoretical vulnerabilities
The most simple practical vulnerability with any password containers is just losing the special key device or the
activated smart phone with the integrated key function. Such vulnerability cannot be healed with any single token
container device within the pre-set time span of activation. All further consideration presumes performant loss
prevention, e.g. by additional electronic leash or body sensor and alarm.
While RSA SecurID tokens offer a level of protection against password replay attacks, they might fail to provide
adequate protection against man in the middle type attacks. In the attack model where an attacker is able to
manipulate the authentication data flow between a user and the server, the attacker will be able to then forward this
authentication information on to the server themselves, effectively masquerading as the given user. If the attacker
manages to block the authorised user from authenticating to the server until the next token code will be valid, he will
be able to log in to the server. RSA SecurID does not prevent Man in the Browser (MitB) based attacks.
SecurID authentication server tries to prevent password sniffing and simultaneous login by declining both
authentication requests, if two valid credentials are presented within a given time frame. See an unverified John G.
Brainard post
for more information. If the attacker removes from the user the ability to authenticate however, the
SecurID server will assume that it is the user who is actually authenticating and hence will allow the authentication
through. Under this attack model, the system security can be improved using encryption/authentication mechanisms
such as SSL.
Although soft tokens may be more convenient, critics indicate that the tamper-resistant property of hard tokens is
unmatched in soft token implementations
, which could potentially allow seed record secret keys to be duplicated
and user impersonation to occur.
Hard tokens on the other hand can be physically stolen (or acquired via social engineering) from end users. The
small form factor makes hard token theft much more viable than laptop/desktop scanning. A user will typically wait
more than one day before reporting the device as missing , giving the attacker plenty of time to breach the protected
March 2011 system compromise
On March 17, 2011, RSA announced that they had been victims of "an extremely sophisticated cyber attack".
Concerns were raised specifically in reference to the SecurID system, saying that "this information could potentially
be used to reduce the effectiveness of a current two-factor authentication implementation." However, their formal
SEC 8K submission
indicates that they don't believe the breach will have a "material impact on its financial
results." The extent of the compromise and the associated risk to customers will not be known until further details
have been released.
There are some hints that the breach involved the theft of RSA's database mapping token serial numbers to the secret
token "seeds" that were injected to make each one unique.
Reports of RSA executives telling customers to
"ensure that they protect the serial numbers on their tokens"
lend credibility to this hypothesis.
In a March 21 email to customers, RSA essentially admitted that the information stolen from their internal network
would allow an attacker to compromise a SecurID-protected system without having physical possession of the token:
"7. Have my SecurID token records been taken?
For the security of our customers, we are not releasing any additional information about what was taken. It is
more important to understand all the critical components of the RSA SecurID solution.
To compromise any RSA SecurID deployment, the attacker needs to possess multiple pieces of information
about the token, the customer, the individual users and their PINs. Some of this information is never held by
RSA and is controlled only by the customer. In order to mount a successful attack, someone would need to
have possession of all this information."
Barring a fatal weakness in the cryptographic implementation of the tokencode generation algorithm (which is
unlikely, since it involves the simple and direct application of the extensively scrutinized AES-128 block cipher), the
only circumstance under which an attacker could mount a successful attack having only information about (but not
physical possession of) the token, is if the token seed records had been leaked.
May 2011, Lockheed Martin compromise
In May 2011 this information seems to have been used to crack into the Lockheed Martin systems.

Lockheed Martin claims that due to "aggressive actions" by the company's information security team, "No customer,
program or employee personal data" was compromised by this "significant and tenacious attack".
The Department
of Homeland Security and the Defense Department have offered to help determine the scope of the attack.
External links
Technical details
• Sample SecurID Token Emulator with token Secret Import
I.C.Wiener, Bugtraq post.
• Apparent Weaknesses in the Security Dynamics Client/Server Protocol
Adam Shostack, 1996.
• Usenet thread discussing new SecurID details
Vin McLellan, et al., comp.security.misc.
• Unofficial SecurID information and some reverse-engineering attempts
Yahoo Groups securid-users.
• Inside a hardware token
• Analysis of possible risks from 2011 compromise
• Understanding the Impact of the RSA SecurID Breach (2011)
Published attacks against the SecurID hash function
• Cryptanalysis of the Alleged SecurID Hash Function
(PDF) Alex Biryukov, Joseph Lano, and Bart Preneel.
• Improved Cryptanalysis of SecurID
(PDF) Scott Contini and Yiqun Lisa Yin.
• Fast Software-Based Attacks on SecurID
(PDF) Scott Contini and Yiqun Lisa Yin.
Versatile Authentication Server
• Bank-Grade Token Agnostic (RSA,Vasco,OATH) Life Cycle Management System with Support for HSMs.
[1] Sample SecurID Token Emulator with Token Secret Import (http:// seclists. org/bugtraq/ 2000/ Dec/ 459)
[2] RSA SecurID SID800 Hardware Authenticator (http:/ / www. rsa.com/ products/securid/ datasheets/ 9651_SID800_DS_0908-lowres.pdf)
[3] Diodati, Mark (2010). "Road Map: Replacing Passwords with OTP Authentication" (http:// www.burtongroup.com/ Research/
PublicDocument.aspx?cid=2107). Burton Group. . "Gartner's expectation is that the hardware OTP form factor will continue to enjoy modest
growth while smartphone OTPs will grow and become the default hardware platform over time. ... If the organization does not need the
extensive platform support, then OATH-based technology is likely a more cost-effective choice."
[4] http:// www. encyclopedia. com/ doc/ 1G1-142107014.html
[5] "Testing Multiple Factors Authentication (OWASP-AT-009)" (http:/ / www. owasp.org/ index.php/
Testing_Multiple_Factors_Authentication_(OWASP-AT-009)). .
[6] http:// malpaso. ru/securid/ brainard.htm
[7] http:// securology.blogspot. com/ 2007/11/ soft-tokens-arent-tokens-at-all.html
[8] "Open Letter to RSA Customers" (http:/ / www. rsa. com/ node.aspx?id=3872). .
[9] "EMC / RSA 8K filing" (http:/ / www.sec. gov/ Archives/ edgar/ data/ 790070/ 000119312511070159/ d8k. htm). .
[10] "RSA won't talk? Assume SecurID is broken" (http:/ / www.theregister.co.uk/ 2011/ 03/ 24/ rsa_securid_news_blackout/ ). .
[11] "Did hackers nab RSA SecurID's secret sauce?" (http:/ / www. networkworld.com/ news/ 2011/ 031811-rsa-breach-reassure.html). .
[12] Leyden, John. "Lockheed Martin suspends remote access after network 'intrusion'." (http:/ / www.channelregister.co. uk/ 2011/ 05/ 27/
lockheed_securid_hack_flap/ ) The Register, 27 May 2011.
[13] "Lockheed Network Reportedly Suffers Security Breach." (http:// www.foxnews.com/ scitech/ 2011/ 05/ 28/
lockheed-network-suffers-security-breach/?test=latestnews) Fox News quoting WSJ, 28 May 2011.
[14] "Lockheed Martin confirms attack on its IT network." (http:// www.google. com/ hostednews/ afp/article/
ALeqM5hO0TYWRsxt1CKUUEXKd04BQwsdGQ?docId=CNG. 377fe057126251044306fe73e1e5ae83.401) AFP, 28 May 2011.
[15] Wolf, Jim. "Lockheed Martin hit by cyber incident, U.S. says." (http:/ / uk. reuters.com/ article/2011/ 05/ 28/
us-usa-defense-hackers-idUKTRE74Q6VY20110528) Reuters, 28 May 2011.
[16] http:/ / seclists. org/lists/ bugtraq/ 2000/ Dec/ 0459. html
[17] http:// www. homeport.org/ ~adam/ dimacs. html
[18] http:/ / groups.google. ca/ group/comp. security. misc/ browse_frm/thread/e00fa564dc6aba5a/
[19] http:/ / groups.yahoo. com/ group/securid-users/
[20] http:// www. rxndxm.com/ 2009/ 04/ what-is-random.html
[21] http:/ / intrepidusgroup.com/ insight/ 2011/ 03/ risk-posed-by-securid-hack
[22] http:/ / www. hbarel.com/ blog?itemid=62
[23] http:// eprint.iacr. org/2003/ 162. pdf
[24] http:/ / eprint.iacr. org/2003/ 205. pdf
[25] http:/ / palms. ee. princeton.edu/ PALMSopen/ contini04fast. pdf
[26] http:/ / www. i-sprint.com/ products_uas. htm
Self-service password reset
Self-service password reset is defined as any process or technology that allows users who have either forgotten
their password or triggered an intruder lockout to authenticate with an alternate factor, and repair their own problem,
without calling the help desk. It is a common feature in identity management software and often bundled in the same
software package as a password synchronization capability.
Typically users who have forgotten their password launch a self-service application from an extension to their
workstation login prompt, using their own or another user's web browser, or through a telephone call. Users establish
their identity, without using their forgotten or disabled password, by answering a series of personal questions, using
a hardware authentication token, responding to a password notification e-mail or, less often, by providing a biometric
sample. Users can then either specify a new, unlocked password, or ask that a randomly generated one be provided.
Self-service password reset expedites problem resolution for users "after the fact," and thus reduces help desk call
volume. It can also be used to ensure that password problems are only resolved after adequate user authentication,
eliminating an important weakness of many help desks: social engineering attacks, where an intruder calls the help
desk, pretends to be the intended victim user, claims that he has forgotten his password, and asks for a new
There are many software products available to allow employees to self-reset passwords.
On the other hand, self-service password reset that relies solely on answers to personal questions can introduce new

since the answers to such questions can often be obtained by social engineering, phishing
techniques or simple research. While users are frequently reminded never to reveal their password, they are less
likely to treat as sensitive the answers to many commonly used security questions, such as pet names, place of birth
or favorite movie. Much of this information may be publicly available on some users' personal home pages. Other
answers can be elicited by someone pretending to conduct an opinion survey or offering a free dating service. Since
many organizations have standard ways of determining login names from real names, an attacker who knows the
names of several employees at such an organization can choose one whose security answers are most readily
This vulnerability is not strictly due to self-service password reset -- it often exists in the help desk prior to
deployment of automation. Self-service password reset technology is often used to reduce this type of vulnerability,
Self-service password reset
by introducing stronger caller authentication factors than the human-operated help desk had been using prior to
deployment of automation.
In September 2008, the Yahoo e-mail account of Governor of Alaska and Vice President of the United States
nominee Sarah Palin was accessed without authorization by someone who was able to research answers to two of her
security questions, her zip code and date of birth and was able to guess the third, where she met her husband.[3] This
incident clearly highlighted that the choice of security questions is very important to prevent social engineering
attacks on password systems.
Preference-based Authentication
Jakobsson, Stolterman, Wetzel, and Yang proposed to use preferences to authenticate users for password reset.

The underlying insights are that preferences are stable over a long period of time,
and are not publicly recorded.
Their approach includes two phases---setup and authentication. During the setup, a user is asked to select items that
they either like or dislike from several categories of items which are dynamically selected from a big candidate set
and are presented to the user in a random order. During the authentication phase, a user is asked to classify his
preferences (like or dislike) for the selected items displayed to him in a random order. See [7] for a live system. They
evaluated the security of their approach by user experiments, user emulations, and attacker simulations.
A major problem with self-service password reset inside corporations and similar organizations is enabling users to
access the system if they forgot their primary password. Since SSPR systems are typically web-based, a user must
launch a web browser to fix his problem -- but the user cannot log into his workstation until the problem is solved.
There are various approaches to addressing this Catch-22, all of which are compromises (e.g., desktop software
deployment, domain-wide password reset account, telephone access, visiting a neighbour, continuing to call the help
desk, etc.).
There are two additional problems related to the one of locked out users:
• Mobile users, physically away from the corporate network, who forgot their PC's login password.
• Passwords cached by the operating system or browser, which might continue to be offered to servers after a
password change that was initiated on another computer (help desk, password management web server, etc.) and
therefore trigger an intruder lockout.
The vouching option
In conjunction with preference-based authentication, self-service password reset procedures could also rely on the
network of existing human relations among users. In this scenario, the user who forgot his password asks a colleague
for assistance. The "helper" colleague authenticates with the password reset application and vouches for user's

In this scenario, the problem changes from one of authenticating the user who forgot his password to one of
understanding which users should have the ability to vouch for which other users.
Self-service password reset
[1] Griffith, Virgil. "Messin' with Texas, Deriving Mother's Maiden Names Using Public Records" (http:/ / www. rsa.com/ rsalabs/ cryptobytes/
CryptoBytes-Winter07. pdf). .
[2] Rabkin, Ariel. "Personal Knowledge Questions for Fallback Authentication: Security Questions in the Era of Facebook." (http:// cups. cs.
cmu. edu/ soups/ 2008/ proceedings/ p13Rabkin. pdf). .
[3] http:/ / news. yahoo. com/ s/ ap/ 20080918/ ap_on_el_pr/ palin_hacked
[4] Jakobsson, Markus et al.. "Love and Authentication" (http:// www.ravenwhite.com/ files/ chi08JSWY. pdf). .
[5] Jakobsson, Markus et al.. "Quantifying the Security of preference-based Authentication" (http:/ / www.cs. stevens. edu/ ~lyang/ lyangpage/
dim20-yang. pdf). .
[6] Crawford, Duane et al. (1986). "The Stability of Leisure Preferences". Journal of Leisure Research 18.
[7] http:// www. blue-moon-authentication.com
[8] Finetti, Mario. "Self service password reset in large organisations" (http:/ / www.scmagazineuk. com/
Self-service-password-reset-in-large-organisations/article/128175/ ). .
[9] RSA Laboratories. "Fourth-factor authentication: somebody you know" (http:/ / doi. acm.org/10. 1145/ 1180405. 1180427). .
External links
• Ariel Rabkin. " Personal knowledge questions for fallback authentication: Security questions in the era of
Facebook. (http:// cups. cs. cmu. edu/ soups/ 2008/ proceedings/ p13Rabkin. pdf)" SOUPS 2008.
• Self service password reset in large organisations (http:// www.scmagazineuk. com/
Self-service-password-reset-in-large-organisations/article/128175/ ) on password reset procedures based on
Shadow password
In computing, Unix-like operating systems use the shadow password database mechanism to increase the security
level of passwords by restricting all but highly privileged users' access to encrypted password data. Typically, that
data is kept in files owned by and accessible only by, the super user (i.e., on Unix-like systems, the root user, and on
many others, the administrator account).
Unshadowed passwords
On a system without shadowed passwords (typically older Unix systems dating from before 1990 or so), the passwd
file holds the following user information for each user account:
• Username
• Salt combined with the current hash of the user's password (usually produced from a cryptographic hash function)
• Password expiration information
• User ID (UID)
• Default group ID (GID)
• Full name
• Home directory path
• Login shell
The passwd file is readable by all users so that name service switch can work (e.g., to ensure that user names are
shown when the user lists the contents of a folder), but only the root user can write to it. This means that an attacker
with unprivileged access to the system can obtain the hashed form of every user's password. Those values can be
used to mount a brute force attack offline, testing possible passwords against the hashed passwords relatively quickly
without alerting system security arrangements designed to detect an abnormal number of failed login attempts. Users
often select passwords vulnerable to such password cracking techniques.
Shadow password
Shadowed passwords
Systems administrators can reduce the likelihood of such brute force attacks by making the list of hashed passwords
unreadable by unprivileged users. The obvious way to do this is to make the passwd database itself readable only by
the root user. However, this would restrict access to other data in the file such as username-to-userid mappings,
which would break many existing utilities and provisions. One solution is a "shadow" password file to hold the
password hashes separate from the other data in the world-readable passwd file. For local files, this is usually
/etc/shadow on Linux and Unix systems, or /etc/master.passwd on BSD systems; each is readable only
by root. (Root access to the data is considered acceptable since on systems with the traditional "all-powerful root"
security model, the root user would be able to obtain the information in other ways in any case). Virtually all recent
Unix-like operating systems use shadowed passwords.
With a shadowed password scheme in use, the /etc/passwd file typically shows a character such as '*', or 'x' in
the password field for each user instead of the hashed password, and /etc/shadow usually contains the following
user information:
• User login name
• salt and hashed password OR a status exception value e.g.:
• "$id$salt$encrypted", where "$id" is the hashing algorithm used (On linux, "$1$" stands for MD5, "$2$" is
Blowfish, "$5$" is SHA-256 and "$6$" is SHA-512, crypt(3) manpage
, other Unix may have different
values, like NetBSD
• "NP" or "!" or null - No password, the account has no password.
• "LK" or "*" - the account is Locked, user will be unable to log-in
• "!!" - the password has expired
• Days since epoch of last password change
• Days until change allowed
• Days before change required
• Days warning for expiration
• Days before account inactive
• Days since Epoch when account expires
• Reserved
While the most important information in the shadow file consists of the salt and the hashed password, the file usually
contains other fields as well, such as the last time the password was changed, when the password will expire, and
whether the account is disabled. The format of the shadow file is simple, and basically identical to that of the
password file, to wit, one line per user, ordered fields on each line, and fields separated by colons. Many systems
require the order of user lines in the shadow file be identical to the order of the corresponding users in the password
To modify the contents of the shadow file on most systems, users generally invoke the passwd program, which in
turn largely depends on PAM. For example, the type of hash used is dictated by the configuration of the
pam_unix.so module. By default, the MD5 hash has been used, while current modules are also capable of
stronger hashes such as blowfish, SHA256 and SHA512.
Note that the shadow password file does not entirely solve the problem of attacker access to hashed passwords, as
some network authentication schemes operate by transmitting the encrypted password over the network (sometimes
in cleartext), making it vulnerable to interception. Copies of system data, such as system backups written to tape or
optical media, can also become a means for illicitly obtaining hashed passwords. In addition, the functions used by
legitimate password-checking programs need to be written in such a way that malicious programs cannot make large
numbers of authentication checks at high rates of speed.
Shadow password
Password shadowing first appeared in UNIX systems with the development of System V Release 3.2 in 1988 and
BSD4.3 Reno in 1990. But, vendors who had performed ports from earlier UNIX releases did not always include the
new password shadowing features in their releases, leaving users of those systems exposed to password file attacks.
In 1987 the author of the original Shadow Password Suite, Julie Haugh, experienced a computer break-in and wrote
the initial release of the Shadow Suite containing the login, passwd and su commands. The original release, written
for the SCO Xenix operating system, quickly got ported to other platforms. The Shadow Suite was ported to Linux in
1992 one year after the original announcement of the Linux project, and was included in many early distributions.
System administrators may also arrange for the storage of passwords in distributed databases such as NIS and
LDAP, rather than in files on each connected system. In the case of NIS, the shadow password mechanism is often
still used on the NIS servers; in other distributed mechanisms the problem of access to the various user
authentication components is handled by the security mechanisms of the underlying data repository.
External links
• authconfig
, a command-line tool for controlling the use of shadow passwords
• An example shadow file
, showing the general layout of the file
[1] Rob Lemos (2002-05-22). "Passwords: the weakest link?" (http:/ / www.news. com/ 2009-1001-916719.html). CNET News.com. . Retrieved
[2] http:// www. kernel.org/ doc/ man-pages/ online/ pages/ man3/ crypt. 3. html
[3] http:/ / netbsd. gw. com/ cgi-bin/man-cgi?crypt+3+ NetBSD-current
[4] http:/ / linux.die. net/ man/ 8/ authconfig
[5] http:/ / configuration.logfish. net/ index. php/ etc/ shadow
Single sign-on
Single sign-on
Single sign-on (SSO) is a property of access control of multiple related, but independent software systems. With this
property a user logs in once and gains access to all systems without being prompted to log in again at each of them.
Single sign-off is the reverse property whereby a single action of signing out terminates access to multiple software
As different applications and resources support different authentication mechanisms, single sign-on has to internally
translate to and store different credentials compared to what is used for initial authentication.
Benefits include:
• Reduces phishing success, because users are not trained to enter password everywhere without thinking.
• Reducing password fatigue from different user name and password combinations
• Reducing time spent re-entering passwords for the same identity
• Can support conventional authentication such as Windows credentials (i.e., username/password)
• Reducing IT costs due to lower number of IT help desk calls about passwords
• Security on all levels of entry/exit/access to systems without the inconvenience of re-prompting users
• Centralized reporting for compliance adherence.
SSO uses centralized authentication servers that all other applications and systems utilize for authentication
purposes, and combines this with techniques to ensure that users do not actively have to enter their credentials more
than once.
SSO users need not remember so many passwords to login to different systems or applications.
The term enterprise reduced sign-on is preferred by some authors who believe single sign-on to be impossible in real
use cases.
As single sign-on provides access to many resources once the user is initially authenticated ("keys to the castle"), it
increases the negative impact in case the credentials are available to other persons and misused. Therefore, single
sign-on requires an increased focus on the protection of the user credentials, and should ideally be combined with
strong authentication methods like smart cards and one-time password tokens.
Single sign-on also makes the authentication systems highly critical; a loss of their availability can result in denial of
access to all systems unified under the SSO. SSO can thus be undesirable for systems to which access must be
guaranteed at all times, such as security or plant-floor systems.
Single sign-on
Common Single Sign-On Configurations
Kerberos based
• Initial sign-on prompts the user for credentials, and gets a Kerberos ticket-granting ticket (TGT).
• Additional software applications requiring authentication, such as email clients, wikis, revision control systems,
etc., use the ticket-granting ticket to acquire service tickets, proving the user's identity to the mailserver / wiki
server / etc. without prompting the user to re-enter credentials.
Windows environment - Windows login fetches TGT. Active Directory-aware apps fetch service tickets, so user is
not prompted to re-authenticate.
UNIX/Linux environment - Login via Kerberos PAM modules fetches TGT. Kerberized client applications such as
Evolution, Firefox, and SVN use service tickets, so user is not prompted to re-authenticate.
Smart card based
Initial sign on prompts the user for the smart card. Additional software applications also use the smart card, without
prompting the user to re-enter credentials. Smart card-based single sign-on can either use certificates or passwords
stored on the smart card.
OTP Token
Also referred to as one-time password token. Two-factor authentication with OTP tokens
follows industry best
practices for authenticating users.
This OTP token method is more secure and effective at prohibiting unauthorized
access than other authentication methods.
Integrated Windows Authentication
Integrated Windows Authentication is a term associated with Microsoft products and refers to the SPNEGO,
Kerberos, and NTLMSSP authentication protocols with respect to SSPI functionality introduced with Microsoft
Windows 2000 and included with later Windows NT-based operating systems. The term is used more commonly for
the automatically authenticated connections between Microsoft Internet Information Services and Internet Explorer.
Cross-platform Active Directory integration vendors have extended the Integrated Windows Authentication
paradigm to UNIX, Linux and Mac systems.
Shared authentication schemes which are not single sign-on
Single sign on requires that users literally sign in once to establish their credentials. Systems which require the user
to log in multiple times to the same identity are inherently not single sign on. For example, an environment where
users are prompted to log in to their desktop, then log in to their email using the same credentials, is not single sign
[1] Examples are tokens by RSA Data Security, Vasco, Actividentity or Aladdin
[2] OTP use meets the guidelines in DOE Order 205.1 as well
[3] FAQ on OTP Tokens - One Time Password Tokens (https:// access. llnl. gov/ otp_access/ cgi-bin/ faq.cgi#OTP_acronym)
Single sign-on
External links
• Single Sign-on Intro with Diagrams (http:// www.opengroup.org/security/ sso/ sso_intro. htm)
• SPNEGO Http Servlet Filter - Open Source SSO Library (http:// spnego. sourceforge.net)
• CampusEAI Consortium myCampus QuickLaunch Single Sign-On and Central Authentication Service (http:/ /
www. campuseai. org)
Swordfish (password)
The use of the word "Swordfish" refers to a password which originated in the 1932 Marx Brothers movie Horse
Feathers. The password has since been used in films, TV series, books and videogames.
Original apparition
The password "Swordfish" was first used in the 1932 Marx Brothers movie Horse Feathers in a scene where
Groucho Marx, as Professor Wagstaff, attempts to gain access to a speakeasy guarded by Baravelli (Chico). The
original dialogue occurred as follows:
Baravelli: ...you can't come in unless you give the password.
Professor Wagstaff: Well, what is the password?
Baravelli: Aw, no. You gotta tell me. Hey, I tell what I do. I give you three guesses. It's the name of a fish.
Professor Wagstaff: Is it "Mary?"
Baravelli: [laughing] 'At's-a no fish!
Professor Wagstaff: She isn't? Well, she drinks like one! ...Let me see... Is it "Sturgeon"?
Baravelli: Aw, you-a craze. A "sturgeon", he's a doctor cuts you open when-a you sick. Now I give you one
more chance.
Wagstaff: I got it! "Haddock".
Baravelli: 'At's a-funny, I got a "haddock" too.
Wagstaff: What do you take for a "haddock"?
Baravelli: Sometimes I take an aspirin, sometimes I take a calomel.
Wagstaff: Y'know, I'd walk a mile for a calomel.
Baravelli: You mean chocolate calomel? I like-a that too, but you no guess it. [Slams door. Wagstaff knocks
again. Baravelli opens peephole again.] Hey, what's-a matter, you no understand English? You can't come in
here unless you say, "Swordfish." Now I'll give you one more guess.
Professor Wagstaff: ...swordfish, swordfish... I think I got it. Is it "swordfish"?
Baravelli: Hah. That's-a it. I guess it.
Professor Wagstaff: Pretty good, eh?
Harpo Marx ("Pinky"), whose characters operated only in pantomime, is still able to get into the speakeasy by
pulling a fish and a small sword out of his trench coat and showing them to the doorman.
Swordfish (password)
Uses in other works
It was referenced in the movie Swordfish, the Robot Chicken Episode Password: Swordfish, the Terry Pratchett
novel Night Watch, The Mad Men episode "Six Month Leave", the book The Sword of the Samurai Cat, the movie
Meet the Applegates, the movie Arena, the computer games Discworld, Return to Zork and Quest for Glory, a
Commodore 64 computer game Impossible Mission, and the online game Kingdom of Loathing as part of the quest
for the Holy Macguffin. In the premiere episode of the television show Sam & Max, a character says "What's the
password...? And if you say 'Swordfish' I'll lose it!"
, and in the later Sam & Max Season One, in a scene during
the third episode where the player is given a dialogue selection to guess random passwords, one of the passwords
guessed is swordfish.
The password "Swordfish" was also used in...
• Flaming Carrot
• Arena (1989)
• Hackers (1995)
• The Net (1995)
• Swordfish (2001)
• Return to Zork (1993)
• Meet the Applegates (1991)
• Enter the Matrix (2003)
"Swordfish" is also used as a name in the P. G. Wodehouse novel How Right You Are, Jeeves. In the story, a
recurring Wodehouse character named Sir Roderick Glossop poses as a butler in order to secretly determine the
sanity of another character. The various residents in the pastoral manor take to calling Glossop "Swordfish", most
likely a tribute to the movie scene described above.
In the TV sitcom Night Court, judge Harry, disillusioned about his life, goes into an It's a Wonderful Life dream and
in the secret club in the courthouse basement, Moose gains entrance by saying "Swordfish".
In the TV sitcom "Too Close for Comfort" the Marx Bros film and password "Swordfish" is mentioned.
This reference was also used in the Disney Show Recess, as a password to get into a performing arts club.
In the ninth episode of the second season of the AMC show Mad Men, Roger Sterling jokes about the password for
an illegal casino in New York being "swordfish".
In the second season episode "That Old Gang of Mine" of the TV series Lois & Clark: The New Adventures of
Superman, Lois Lane and Clark Kent enter a secret gambling parlor. Lois tries naming several passwords, including
"Swordfish" to the parlor's doorman, but none worked. The doorman said he had seen the Marx Brother's movie and
sarcastically said it was a nice try. Clark interjects and says, "the fat lady sings", the correct password which he
overheard when a previous person tried to get in.
[1] Memorable quotes from Horse Feathers (http:/ / www.imdb.com/ title/ tt0023027/ quotes)
[2] Marx Brothers' references (http:/ / www.marx-brothers.org/info/ reference.htm)
Windows credentials
Windows credentials
Windows Credentials refers to the conventional username and password process when trying to access the Windows
operating system.
Zero-knowledge password proof
In cryptography, a zero-knowledge password proof (ZKPP) is an interactive method for one party (the prover) to
prove to another party (the verifier) that it knows a value of a password, without revealing anything other than the
fact that it knows that password to the verifier. The term is defined in IEEE P1363.2, in reference to one of the
benefits of using a password-authenticated key agreement (PAKE) protocol that is secure against off-line dictionary
attacks. A ZKPP prevents any party from verifying guesses for the password without interacting with a party that
knows it and, in the optimal case, provides exactly one guess in each interaction.
Technically speaking, a ZKPP is different from a zero-knowledge proof.
A common use of a zero-knowledge password proof is in authentication systems where one party wants to prove its
identity to a second party using a password but doesn't want the second party or anybody else to learn anything about
the password.
The first methods to demonstrate a ZKPP were the Encrypted key exchange methods (EKE) described by Steven M.
Bellovin and Michael Merritt in 1992. A considerable number of refinements, alternatives, and variations in the
growing class of password-authenticated key agreement methods were developed in subsequent years. Standards for
these methods include IETF RFC 2945, IEEE P1363.2, and ISO-IEC 11770-4.
• S. M. Bellovin and M. Merritt. Encrypted Key Exchange: Password-Based Protocols Secure Against Dictionary
Attacks. Proceedings of the IEEE Symposium on Research in Security and Privacy, Oakland, May 1992.
• IEEE P1363.2: Proposed Standard for Password-Based Public-Key Cryptography.
External links
• IEEE P1363.2: Proposed Standard for Password-Based Public-Key Cryptography
• David Jablon's links for password-based cryptography
[1] http:/ / grouper.ieee. org/groups/ 1363/ passwdPK/ index. html
Randomness has somewhat disparate meanings as used in several different fields. It also has common meanings
which may have loose connections with some of those more definite meanings. The Oxford English Dictionary
defines "random" thus:
Having no definite aim or purpose; not sent or guided in a particular direction; made, done, occurring,
etc., without method or conscious choice; haphazard.
Closely connected, therefore, with the concepts of chance, probability, and information entropy, randomness implies
a lack of predictability. Randomness is a concept of non-order or non-coherence in a sequence of symbols or steps,
such that there is no intelligible pattern or combination.
The fields of mathematics, probability, and statistics use formal definitions of randomness. In mathematics, a random
variable is a way to assign a value to each possible outcome of an event. In probability and statistics, a random
process is a repeating process whose outcomes follow no describable deterministic pattern, but follow a probability
distribution, such that the relative probability of the occurrence of each outcome can be approximated or calculated.
For example, the rolling of a fair six-sided die in neutral conditions may be said to produce random results, because
one cannot know, before a roll, what number will show up. However, the probability of rolling any one of the six
rollable numbers can be calculated.
The term is often used in statistics to signify well-defined statistical properties, such as a lack of bias or correlation.
Monte Carlo Methods, which rely on random input, are important techniques in science, as, for instance, in
computational science.
Random selection is an official method to resolve tied elections in some jurisdictions
is even an ancient method of divination, as in tarot, the I Ching, and bibliomancy. Its use in politics is very old, as
office holders in Ancient Athens were chosen by lot, there being no voting.
Ancient fresco of dice players in Pompei.
In ancient history, the concepts of chance and randomness were
intertwined with that of fate. Many ancient peoples threw dice to
determine fate, and this later evolved into games of chance. Most
ancient cultures used various methods of divination to attempt to
circumvent randomness and fate.

The Chinese were perhaps the earliest people to formalize odds and
chance 3,000 years ago. The Greek philosophers discussed randomness
at length, but only in non-quantitative forms. It was only in the
sixteenth century that Italian mathematicians began to formalize the
odds associated with various games of chance. The invention of the
calculus had a positive impact on the formal study of randomness. In
the 1888 edition of his book The Logic of Chance John Venn wrote a chapter on "The conception of randomness"
which included his view of the randomness of the digits of the number Pi by using them to construct a random walk
in two dimensions.
The early part of the twentieth century saw a rapid growth in the formal analysis of randomness, as various
approaches to the mathematical foundations of probability were introduced. In the mid- to late-twentieth century,
ideas of algorithmic information theory introduced new dimensions to the field via the concept of algorithmic
Although randomness had often been viewed as an obstacle and a nuisance for many centuries, in the twentieth
century computer scientists began to realize that the deliberate introduction of randomness into computations can be
an effective tool for designing better algorithms. In some cases such randomized algorithms outperform the best
deterministic methods.
Randomness in science
Many scientific fields are concerned with randomness:
• Algorithmic probability
• Chaos theory
• Cryptography
• Game theory
• Information theory
• Pattern recognition
• Probability theory
• Quantum mechanics
• Statistical mechanics
• Statistics
In the physical sciences
In the 19th century, scientists used the idea of random motions of molecules in the development of statistical
mechanics in order to explain phenomena in thermodynamics and the properties of gases.
According to several standard interpretations of quantum mechanics, microscopic phenomena are objectively
That is, in an experiment where all causally relevant parameters are controlled, there will still be some
aspects of the outcome which vary randomly. An example of such an experiment is placing a single unstable atom in
a controlled environment; it cannot be predicted how long it will take for the atom to decay; only the probability of
decay within a given time can be calculated.
Thus, quantum mechanics does not specify the outcome of individual
experiments but only the probabilities. Hidden variable theories are inconsistent with the view that nature contains
irreducible randomness: such theories posit that in the processes that appear random, properties with a certain
statistical distribution are somehow at work "behind the scenes" determining the outcome in each case.
In biology
The modern evolutionary synthesis ascribes the observed diversity of life to natural selection, in which some random
genetic mutations are retained in the gene pool due to the non-random improved chance for survival and
reproduction that those mutated genes confer on individuals who possess them.
The characteristics of an organism arise to some extent deterministically (e.g., under the influence of genes and the
environment) and to some extent randomly. For example, the density of freckles that appear on a person's skin is
controlled by genes and exposure to light; whereas the exact location of individual freckles seems to be random.
Randomness is important if an animal is to behave in a way that is unpredictable to others. For instance, insects in
flight tend to move about with random changes in direction, making it difficult for pursuing predators to predict their
In mathematics
The mathematical theory of probability arose from attempts to formulate mathematical descriptions of chance events,
originally in the context of gambling, but later in connection with physics. Statistics is used to infer the underlying
probability distribution of a collection of empirical observations. For the purposes of simulation, it is necessary to
have a large supply of random numbers or means to generate them on demand.
Algorithmic information theory studies, among other topics, what constitutes a random sequence. The central idea is
that a string of bits is random if and only if it is shorter than any computer program that can produce that string
(Kolmogorov randomness)—this means that random strings are those that cannot be compressed. Pioneers of this
field include Andrey Kolmogorov and his student Per Martin-Löf, Ray Solomonoff, and Gregory Chaitin.
In mathematics, there must be an infinite expansion of information for randomness to exist. This can best be seen
with an example. Given a random sequence of three-bit numbers, each number can have one of only eight possible
000, 001, 010, 011, 100, 101, 110, 111
Therefore, as the random sequence progresses, it must recycle the values it previously used. In order to increase the
information space, another bit may be added to each possible number, giving 16 possible values from which to pick a
random number. It could be said that the random four-bit number sequence is more random than the three-bit one.
This suggests that in order to have true randomness, there must be an infinite expansion of the information space.
Randomness is said to occur in numbers such as log (2) and Pi. The decimal digits of Pi constitute an infinite
sequence and "never repeat in a cyclical fashion". Numbers like pi are also thought to be normal, which means that
their digits are random in a certain statistical sense.
Pi certainly seems to behave this way. In the first six billion decimal places of pi, each of the digits from
0 through 9 shows up about six hundred million times. Yet such results, conceivably accidental, do not
prove normality even in base 10, much less normality in other number bases.
In statistics
In statistics, randomness is commonly used to create simple random samples. This allows surveys to be done with
completely random groups of people to allow realistic data. Common methods of doing this are "drawing names out
of a hat" or using a random digit chart. A random digit chart is simply a large table of random digits.
In information science
In information science, irrelevant or meaningless data is considered to be noise. Noise consists of a large number of
transient disturbances with a statistically randomized time distribution.
In communication theory, randomness in a signal is called "noise" and is opposed to that component of its variation
that is causally attributable to the source, the signal.
In terms of the development of random networks, for communication randomness rests on the two simple
assumptions of Paul Erdős and Alfréd Rényi who said that there were a fixed number of nodes and this number
remained fixed for the life of the network, and that all nodes were equal and linked randomly to each other.
In finance
The random walk hypothesis considers that asset prices in an organized market evolve at random.
Other so-called random factors intervene in trends and patterns to do with supply-and-demand distributions. As well
as this, the random factor of the environment itself results in fluctuations in stock and broker markets.
Randomness versus unpredictability
Randomness, as opposed to unpredictability, is held to be an objective property - determinists believe it is an
objective fact that randomness does not in fact exist. Also, what appears random to one observer may not appear
random to another. Consider two observers of a sequence of bits, when only one of whom has the cryptographic key
needed to turn the sequence of bits into a readable message. For that observer the message is not random, but it is
unpredictable for the other.
One of the intriguing aspects of random processes is that it is hard to know whether a process is truly random. An
observer may suspect that there is some "key" that unlocks the message. This is one of the foundations of
superstition, but also a motivation for discovery in science and mathematics.
Under the cosmological hypothesis of determinism, there is no randomness in the universe, only unpredictability,
since there is only one possible outcome to all events in the universe. A follower of the narrow frequency
interpretation of probability could assert that no event can be said to have probability, since there is only one
universal outcome. On the other hand, under the rival Bayesian interpretation of probability there is no objection to
the use of probabilities in order to represent a lack of complete knowledge of the outcomes.
Some mathematically defined sequences, such as the decimals of pi mentioned above, exhibit some of the same
characteristics as random sequences, but because they are generated by a describable mechanism, they are called
pseudorandom. To an observer who does not know the mechanism, a pseudorandom sequence is unpredictable.
Chaotic systems are unpredictable in practice due to their extreme sensitivity to initial conditions. Whether or not
they are unpredictable in terms of computability theory is a subject of current research. At least in some disciplines
of computability theory, the notion of randomness is identified with computational unpredictability.
Individual events that are random may still be precisely described en masse, usually in terms of probability or
expected value. For instance, quantum mechanics allows a very precise calculation of the half-lives of atoms even
though the process of atomic decay is random. More simply, although a single toss of a fair coin cannot be predicted,
its general behavior can be described by saying that if a large number of tosses are made, roughly half of them will
show up heads. Ohm's law and the kinetic theory of gases are non-random macroscopic phenomena that are assumed
to be random at the microscopic level.
Randomness and religion
Some theologians have attempted to resolve the apparent contradiction between an omniscient deity, or a first cause,
and free will using randomness. Discordians have a strong belief in randomness and unpredictability. Hindu and
Buddhist philosophies state that any event is the result of previous events (karma), and as such, there is no such thing
as a random event or a first event.
Martin Luther, the forefather of Protestantism, believed that there was nothing random based on his understanding of
the Bible. As an outcome of his understanding of randomness, he strongly felt that free will was limited to low-level
decision making by humans. Therefore, when someone sins against another, decision making is only limited to how
one responds, preferably through forgiveness and loving actions. He believed, based on Biblical scripture, that
humans cannot will themselves faith, salvation, sanctification, or other gifts from God. Additionally, the best people
could do, according to his understanding, was not sin, but they fall short, and free will cannot achieve this objective.
Thus, in his view, absolute free will and unbounded randomness are severely limited to the point that behaviors may
even be patterned or ordered and not random. This is a point emphasized by the field of behavioral psychology.
These notions and more in Christianity often lend to a highly deterministic worldview and that the concept of
random events is not possible. Especially, if purpose is part of this universe, then randomness, by definition, is not
possible. This is also one of the rationales for religious opposition to evolution, where, according to theory,
(non-random) selection is applied to the results of random genetic variation.
Donald Knuth, a Stanford computer scientist and Christian commentator, remarks that he finds pseudorandom
numbers useful and applies them with purpose. He then extends this thought to God who may use randomness with
purpose to allow free will to certain degrees. Knuth believes that God is interested in people's decisions and limited
free will allows a certain degree of decision making. Knuth, based on his understanding of quantum computing and
entanglement, comments that God exerts dynamic control over the world without violating any laws of physics,
suggesting that what appears to be random to humans may not, in fact, be so random.
C. S. Lewis, a 20th-century Christian philosopher, discussed free will at length. On the matter of human will, Lewis
wrote: "God willed the free will of men and angels in spite of His knowledge that it could lead in some cases to sin
and thence to suffering: i.e., He thought freedom worth creating even at that price." In his radio broadcast, Lewis
indicated that God "gave [humans] free will. He gave them free will because a world of mere automata could never
In some contexts, procedures that are commonly perceived as randomizers—drawing lots or the like —are used for
divination, e.g., to reveal the will of the gods; see e.g. Cleromancy.
Applications and use of randomness
In most of its mathematical, political, social and religious use, randomness is used for its innate "fairness" and lack
of bias.
Political: Athenian democracy was based on the concept of isonomia (equality of political rights) and used complex
allotment machines to ensure that the positions on the ruling committees that ran Athens were fairly allocated.
Allotment is now restricted to selecting jurors in Anglo-Saxon legal systems and in situations where "fairness" is
approximated by randomization, such as selecting jurors and military draft lotteries.
Social: Random numbers were first investigated in the context of gambling, and many randomizing devices, such as
dice, shuffling playing cards, and roulette wheels, were first developed for use in gambling. The ability to produce
random numbers fairly is vital to electronic gambling, and, as such, the methods used to create them are usually
regulated by government Gaming Control Boards. Random drawings are also used to determine lottery winners.
Throughout history, randomness has been used for games of chance and to select out individuals for an unwanted
task in a fair way (see drawing straws).
Sports: Some sports, including American Football, use coin tosses to randomly select starting conditions for games
or seed tied teams for postseason play. The National Basketball Association uses a weighted lottery to order teams in
its draft.
Mathematical: Random numbers are also used where their use is mathematically important, such as sampling for
opinion polls and for statistical sampling in quality control systems. Computational solutions for some types of
problems use random numbers extensively, such as in the Monte Carlo method and in genetic algorithms.
Medicine: Random allocation of a clinical intervention is used to reduce bias in controlled trials (e.g., randomized
controlled trials).
Religious: Although not intended to be random, various forms of divination such as cleromancy see what appears to
be a random event as a means for a divine being to communicate their will. (See also Free will and Determinism).
Generating randomness
The ball in a roulette can be used as a source of
apparent randomness, because its behavior is very
sensitive to the initial conditions.
It is generally accepted that there exist three mechanisms
responsible for (apparently) random behavior in systems:
1. Randomness coming from the environment (for example,
Brownian motion, but also hardware random number
2. Randomness coming from the initial conditions. This aspect is
studied by chaos theory and is observed in systems whose
behavior is very sensitive to small variations in initial
conditions (such as pachinko machines, dice ...).
3. Randomness intrinsically generated by the system. This is also
called pseudorandomness and is the kind used in
pseudo-random number generators. There are many algorithms
(based on arithmetics or cellular automaton) to generate pseudorandom numbers. The behavior of the system can
be determined by knowing the seed state and the algorithm used. These methods are often quicker than getting
"true" randomness from the environment.
The many applications of randomness have led to many different methods for generating random data. These
methods may vary as to how unpredictable or statistically random they are, and how quickly they can generate
random numbers.
Before the advent of computational random number generators, generating large amounts of sufficiently random
numbers (important in statistics) required a lot of work. Results would sometimes be collected and distributed as
random number tables.
Randomness measures and tests
There are many practical measures of randomness for a binary sequence. These include measures based on
frequency, discrete transforms, and complexity, or a mixture of these. These include tests by Kak, Phillips, Yuen,
Hopkins, Beth and Dai, Mund, and Marsaglia and Zaman.
Misconceptions/logical fallacies
Popular perceptions of randomness are frequently mistaken, based on fallacious reasoning or intuitions.
A number is "due"
see also Coupon collector's problem
This argument is that "in a random selection of numbers, since all numbers will eventually appear, those that have
not come up yet are 'due', and thus more likely to come up soon." This logic is only correct if applied to a system
where numbers that come up are removed from the system, such as when playing cards are drawn and not returned to
the deck. In this case, once a jack is removed from the deck, the next draw is less likely to be a jack and more likely
to be some other card. However, if the jack is returned to the deck, and the deck is thoroughly reshuffled, a jack is as
likely to be drawn as any other card. The same applies in any other process where objects are selected independently,
and none are removed after each event, such as the roll of a die, a coin toss, or most lottery number selection
schemes. Truly random processes such as these do not have memory, making it impossible for past outcomes to
affect future outcomes.
A number is "cursed" or "blessed"
In a random sequence of numbers, a number may be said to be cursed because it has come up less often in the past,
and so it is thought that it will occur less often in the future. A number may be assumed to be blessed because it has
occurred more often than others in the past, and so it is thought to be likely to come up more often in the future. This
logic is valid only if the randomisation is biased, for example with a loaded die. If the die is fair, then previous rolls
give no indication of future events.
In nature, events rarely occur with perfectly equal frequency. So observing outcomes to determine which events are
likely to have a higher probability, makes sense. It is fallacious to apply this logic to systems which are designed so
that all outcomes are equally likely, such as shuffled cards, dice and roulette wheels.
Odds are never dynamic
In the beginning of a scenario, one might calculate the odds of a certain event. The fact is, as soon as one gains more
information about that situation, they may need to re-calculate the odds.
If we are told that a woman has two children, and one of them is a girl, what are the odds that the other child is also a
girl? Considering this new child independently, one might expect the odds that the other child is female are 1/2
(50%). By using mathematician Gerolamo Cardano's method of building a Probability space (illustrating all possible
outcomes), we see that the odds are actually only 1/3 (33%). This is because, for starters, the possibility space
illustrates 4 ways of having these two children: boy-boy, girl-boy, boy-girl, and girl-girl. But we were given more
information. Once we are told that one of the children is a female, we use this new information to eliminate the
boy-boy scenario. Thus the probability space reveals that there are still 3 ways to have two children where one is a
female: boy-girl, girl-boy, girl-girl. Only 1/3 of these scenarios would have the other child also be a girl.
Using a
probability space, we are less likely to miss one of the possible scenarios, or to neglect the importance of new
When the host reveals that
one door only contained a
goat, this is new
This technique provides insights in other situations such as the Monty Hall problem, a
game show scenario in which a car is hidden behind one of three doors, and two goats
are hidden as booby prizes behind the others. Once the contestant has chosen a door, the
host opens one of the remaining doors to reveal a goat, eliminating that door as an
option. With only two doors left (one with the car, the other with another goat), the host
then asks the player whether they would like to keep the decision they made, or switch
and select the other door. Intuitively, one might think the contestant is simply choosing
between two doors with equal probability, and the opportunity provided by the host
makes no difference. Probability spaces reveal that the contestant has received new information, and can increase
their chances of winning by changing to the other door.
Ignoring variance
Whether it is a career in poker, as a salesperson, or even searching for the right partner to marry, variance and
randomness play an important role. Variance sometimes prevents us from drawing causal relationships, even after
we have performed multiple experiments - if the experiment is too complex (as it usually is, in day-to-day life). Put
simply, in a popular game, some bad players are likely to have winning streaks and good players are likely to have
losing streaks. This also explains why Coincidences should be considered skeptically; rare things, by definition,
occasionally happen (e.g. the sudden death of hundreds of animals).

• Randomness by Deborah J. Bennett. Harvard University Press, 1998. ISBN 0-674-10745-4.
• Random Measures, 4th ed. by Olav Kallenberg. Academic Press, New York, London; Akademie-Verlag, Berlin,
1986. MR0854102.
• The Art of Computer Programming. Vol. 2: Seminumerical Algorithms, 3rd ed. by Donald E. Knuth. Reading,
MA: Addison-Wesley, 1997. ISBN 0-201-89684-2.
• Fooled by Randomness, 2nd ed. by Nassim Nicholas Taleb. Thomson Texere, 2004. ISBN 1-58799-190-X.
• Exploring Randomness by Gregory Chaitin. Springer-Verlag London, 2001. ISBN 1-85233-417-7.
• Random by Kenneth Chan includes a "Random Scale" for grading the level of randomness.
[1] Third Workshop on Monte Carlo Methods (http:/ / www.people.fas.harvard.edu/ ~junliu/ Workshops/ workshop2007/ ), Jun Liu, Professor
of Statistics, Harvard University
[2] Municipal Elections Act (Ontario, Canada) 1996, c. 32, Sched., s. 62 (3) : "If the recount indicates that two or more candidates who cannot
both or all be declared elected to an office have received the same number of votes, the clerk shall choose the successful candidate or
candidates by lot."
[3] Handbook to life in ancient Rome by Lesley Adkins 1998 ISBN 0195123328 page 279
[4] Religions of the ancient world by Sarah Iles Johnston 2004 ISBN 0674015177 page 370
[5] Annotated readings in the history of statistics by Herbert Aron David, 2001 ISBN 0387988440 page 115. Note that the 1866 edition of Venn's
book (on Google books) does not include this chapter.
[6] Nature.com (http:// www.nature.com/ nature/ journal/v446/ n7138/ abs/ nature05677.html) in Bell's aspect experiment: Nature
[7] "Each nucleus decays spontaneously, at random, in accordance with the blind workings of chance". Q for Quantum, John Gribbin
[8] Breathnach, A. S. (1982). "A long-term hypopigmentary effect of thorium-X on freckled skin". British Journal of Dermatology 106 (1):
19–25. doi:10.1111/j.1365-2133.1982.tb00897.x. PMID 7059501. "The distribution of freckles seems to be entirely random, and not
associated with any other obviously punctuate anatomical or physiological feature of skin.".
[9] Are the digits of pi random? researcher may hold the key. (http:/ / www. lbl. gov/ Science-Articles/ Archive/pi-random.html)
[10] Laszso Barabasi, (2003), Linked, Rich Gets Richer, P81
[11] Donald Knuth, "Things A Computer Scientist Rarely Talks About", Pg 185, 190-191, CSLI
[12] Terry Ritter, Randomness tests: a literature survey. ciphersbyritter.com (http:/ / www.ciphersbyritter.com/ RES/ RANDTEST.HTM)
[13] Johnson, George (8 June 2008). "Playing the Odds" (http:/ / www.nytimes. com/2008/ 06/ 08/ books/ review/Johnson-G-t.html?_r=1). The
New York Times. .
[14] Stanovich 2007 pg 173
External links
• An 8-foot-tall (2.4 m) Probability Machine (named Sir Francis) comparing stock market returns to the
randomness of the beans dropping through the quincunx pattern. (http:// www.youtube. com/
watch?v=AUSKTk9ENzg) from Index Funds Advisors IFA.com (http:// www.ifa.com)
• QuantumLab (http:/ /www. quantumlab. de) Quantum random number generator with single photons as
interactive experiment.
• Random.org (http:/ / www. random. org) generates random numbers using atmospheric noises (see also
• HotBits (http:/ / www. fourmilab.ch/ hotbits/ ) generates random numbers from radioactive decay.
• QRBG (http:/ / random.irb.hr) Quantum Random Bit Generator
• QRNG (http:/ / qrng.physik. hu-berlin.de/ ) Fast Quantum Random Bit Generator
• Chaitin: Randomness and Mathematical Proof (http:/ / www.cs. auckland. ac. nz/ CDMTCS/ chaitin/ sciamer.
• A Pseudorandom Number Sequence Test Program (Public Domain) (http:// www.fourmilab.ch/ random/)
• Dictionary of the History of Ideas: (http:/ / etext. lib. virginia.edu/ cgi-local/DHI/ dhi.cgi?id=dv1-46) Chance
• Philosophy: Free Will vs. Determinism (http:// www.spaceandmotion. com/ Philosophy-Free-Will-Determinism.
• RAHM Nation Institute (http:/ / www. rahmnation.org)
• History of randomness definitions (http:// www.wolframscience. com/ nksonline/ page-1067b-text), in Stephen
Wolfram's A New Kind of Science
• Computing a Glimpse of Randomness (http:/ / www.cs. auckland.ac.nz/ ~cristian/ Calude361_370. pdf)
• Chance versus Randomness (http:// plato. stanford.edu/ entries/ chance-randomness/ ), from the Stanford
Encyclopedia of Philosophy
Algorithmic information theory
Algorithmic information theory is a subfield of information theory and computer science that concerns itself with
the relationship between computation and information. According to Gregory Chaitin, it is "the result of putting
Shannon's information theory and Turing's computability theory into a cocktail shaker and shaking vigorously."
Algorithmic information theory principally studies complexity measures on strings (or other data structures).
Because most mathematical objects can be described in terms of strings, or as the limit of a sequence of strings, it
can be used to study a wide variety of mathematical objects, including integers and real numbers.
This use of the term "information" might be a bit misleading, as it depends upon the concept of compressibility.
Informally, from the point of view of algorithmic information theory, the information content of a string is
equivalent to the length of the shortest possible self-contained representation of that string. A self-contained
representation is essentially a program – in some fixed but otherwise irrelevant universal programming language –
that, when run, outputs the original string.
From this point of view, a 3000 page encyclopedia actually contains less information than 3000 pages of completely
random letters, despite the fact that the encyclopedia is much more useful. This is because to reconstruct the entire
sequence of random letters, one must know, more or less, what every single letter is. On the other hand, if every
vowel were removed from the encyclopedia, someone with reasonable knowledge of the English language could
reconstruct it, just as one could likely reconstruct the sentence "Ths sntnc hs lw nfrmtn cntnt" from the context and
consonants present. For this reason, high-information strings and sequences are sometimes called "random"; people
also sometimes attempt to distinguish between "information" and "useful information" and attempt to provide
rigorous definitions for the latter, with the idea that the random letters may have more information than the
encyclopedia, but the encyclopedia has more "useful" information.
Unlike classical information theory, algorithmic information theory gives formal, rigorous definitions of a random
string and a random infinite sequence that do not depend on physical or philosophical intuitions about
nondeterminism or likelihood. (The set of random strings depends on the choice of the universal Turing machine
used to define Kolmogorov complexity, but any choice gives identical asymptotic results because the Kolmogorov
complexity of a string is invariant up to an additive constant depending only on the choice of universal Turing
machine. For this reason the set of random infinite sequences is independent of the choice of universal machine.)
Some of the results of algorithmic information theory, such as Chaitin's incompleteness theorem, appear to challenge
common mathematical and philosophical intuitions. Most notable among these is the construction of Chaitin's
constant Ω, a real number which expresses the probability that a self-delimiting universal Turing machine will halt
when its input is supplied by flips of a fair coin (sometimes thought of as the probability that a random computer
program will eventually halt). Although Ω is easily defined, in any consistent axiomatizable theory one can only
compute finitely many digits of Ω, so it is in some sense unknowable, providing an absolute limit on knowledge that
is reminiscent of Gödel's Incompleteness Theorem. Although the digits of Ω cannot be determined, many properties
of Ω are known; for example, it is an algorithmically random sequence and thus its binary digits are evenly
Algorithmic information theory
distributed (in fact it is normal).
Algorithmic information theory was founded by Ray Solomonoff
, who published the basic ideas on which the
field is based as part of his invention of algorithmic probability - a way to overcome serious problems associated
with the application of Bayes rules in statistics. He first described his results at a Conference at Caltech in 1960,
and in a report, Feb. 1960, "A Preliminary Report on a General Theory of Inductive Inference."
information theory was later developed independently by Andrey Kolmogorov, in 1965 and Gregory Chaitin, around
There are several variants of Kolmogorov complexity or algorithmic information; the most widely used one is based
on self-delimiting programs and is mainly due to Leonid Levin (1974). Per Martin-Löf also contributed significantly
to the information theory of infinite sequences. An axiomatic approach to algorithmic information theory based on
Blum axioms (Blum 1967) was introduced by Mark Burgin in a paper presented for publication by Andrey
Kolmogorov (Burgin 1982). The axiomatic approach encompasses other approaches in the algorithmic information
theory. It is possible to treat different measures of algorithmic information as particular cases of axiomatically
defined measures of algorithmic information. Instead of proving similar theorems, such as the basic invariance
theorem, for each particular measure, it is possible to easily deduce all such results from one corresponding theorem
proved in the axiomatic setting. This is a general advantage of the axiomatic approach in mathematics. The
axiomatic approach to algorithmic information theory was further developed in the book (Burgin 2005) and applied
to software metrics (Burgin and Debnath, 2003; Debnath and Burgin, 2003).
Precise definitions
A binary string is said to be random if the Kolmogorov complexity of the string is at least the length of the string. A
simple counting argument shows that some strings of any given length are random, and almost all strings are very
close to being random. Since Kolmogorov complexity depends on a fixed choice of universal Turing machine
(informally, a fixed "description language" in which the "descriptions" are given), the collection of random strings
does depend on the choice of fixed universal machine. Nevertheless, the collection of random strings, as a whole, has
similar properties regardless of the fixed machine, so one can (and often does) talk about the properties of random
strings as a group without having to first specify a universal machine.
An infinite binary sequence is said to be random if, for some constant c, for all n, the Kolmogorov complexity of the
initial segment of length n of the sequence is at least n − c. Importantly, the complexity used here is prefix-free
complexity; if plain complexity were used, there would be no random sequences. However, with this definition, it can
be shown that almost every sequence (from the point of view of the standard measure - "fair coin" or Lebesgue
measure – on the space of infinite binary sequences) is random. Also, since it can be shown that the Kolmogorov
complexity relative to two different universal machines differs by at most a constant, the collection of random infinite
sequences does not depend on the choice of universal machine (in contrast to finite strings). This definition of
randomness is usually called Martin-Löf randomness, after Per Martin-Löf, to distinguish it from other similar
notions of randomness. It is also sometimes called 1-randomness to distinguish it from other stronger notions of
randomness (2-randomness, 3-randomness, etc.).
(Related definitions can be made for alphabets other than the set .)
Algorithmic information theory
Specific sequence
Algorithmic information theory (AIT) is the information theory of individual objects, using computer science, and
concerns itself with the relationship between computation, information, and randomness.
The information content or complexity of an object can be measured by the length of its shortest description. For
instance the string
has the short description "32 repetitions of '01'", while
presumably has no simple description other than writing down the string itself.
More formally, the Algorithmic Complexity (AC) of a string x is defined as the length of the shortest program
computes or outputs x, where the program is run on some fixed reference universal computer.
A closely related notion is the probability that a universal computer outputs some string x when fed with a program
chosen at random. This Algorithmic "Solomon-off" Probability (AP) is key in addressing the old philosophical
problem of induction in a formal way.
The major drawback of AC and AP are their incomputability. Time-bounded "Levin" complexity penalizes a slow
program by adding the logarithm of its running time to its length. This leads to computable variants of AC and AP,
and Universal "Levin" Search (US) solves all inversion problems in optimal (apart from some unrealistically large
multiplicative constant) time.
AC and AP also allow a formal and rigorous definition of randomness of individual strings do not depend on
physical or philosophical intuitions about non-determinism or likelihood. Roughly, a string is Algorithmic
"Martin-Loef" Random (AR) if it is incompressible in the sense that its algorithmic complexity is equal to its length.
AC, AP, and AR are the core sub-disciplines of AIT, but AIT spawns into many other areas. It serves as the
foundation of the Minimum Description Length (MDL) principle, can simplify proofs in computational complexity
theory, has been used to define a universal similarity metric between objects, solves the Maxwell daemon problem,
and many others.
[1] Algorithmic Information Theory (http:// www.cs. auckland.ac.nz/ research/ groups/ CDMTCS/ docs/ ait.php)
[2] Vitanyi, P. " Obituary: Ray Solomonoff, Founding Father of Algorithmic Information Theory" (http:// homepages. cwi.nl/ ~paulv/ obituary.
[3] Paper from conference on "Cerebral Systems and Computers", California Institute of Technology, Feb 8-11, 1960, cited in "A Formal Theory
of Inductive Inference, Part 1, 1964, p. 1
[4] Solomonoff, R., " A Preliminary Report on a General Theory of Inductive Inference (http:// world.std. com/ ~rjs/ z138.pdf)", Report V-131,
Zator Co., Cambridge, Ma., (November Revision of Feb 4, 1960 report.)
Algorithmic information theory
External links
• Algorithmic Information Theory (Scholarpedia) (http:// www.scholarpedia. org/article/
• Chaitin's account of the history of AIT (http:/ / www. cs. auckland. ac. nz/ CDMTCS/ chaitin/ unknowable/ ch6.
Further reading
• Blum, M. (1967) On the Size of Machines, Information and Control, v. 11, pp. 257–265
• Blum M. (1967a) A Machine-independent Theory of Complexity of Recursive Functions, Journal of the ACM, v.
14, No.2, pp. 322–336
• Burgin, M. (1982) Generalized Kolmogorov complexity and duality in theory of computations, Soviet Math.
Dokl., v.25, No. 3, pp. 19–23
• Burgin, M. (1990) Generalized Kolmogorov Complexity and other Dual Complexity Measures, Cybernetics, No.
4, pp. 21–29
• Burgin, M. Super-recursive algorithms, Monographs in computer science, Springer, 2005
• Calude, C.S. (1996) Algorithmic information theory: Open problems, J. UCS, v. 2, pp. 439–441
• Calude, C.S. Information and Randomness: An Algorithmic Perspective, (Texts in Theoretical Computer Science.
An EATCS Series), Springer-Verlag, Berlin, 2002
• Chaitin, G.J. (1966) On the Length of Programs for Computing Finite Binary Sequences, J. Association for
Computing Machinery, v. 13, No. 4, pp. 547–569
• Chaitin, G.J. (1969) On the Simplicity and Speed of Programs for Computing Definite Sets of Natural Numbers,
J. Association for Computing Machinery, v. 16, pp. 407–412
• Chaitin, G.J. (1975) A Theory of Program Size Formally Identical to Information Theory, J. Association for
Computing Machinery, v. 22, No. 3, pp. 329–340
• Chaitin, G.J. (1977) Algorithmic information theory, IBM Journal of Research and Development, v.21, No. 4,
• Chaitin, G.J. Algorithmic Information Theory, Cambridge University Press, Cambridge, 1987
• Kolmogorov, A.N. (1965) Three approaches to the definition of the quantity of information, Problems of
Information Transmission, No. 1, pp. 3–11
• Kolmogorov, A.N. (1968) Logical basis for information theory and probability theory, IEEE Trans. Inform.
Theory, vol. IT-14, pp. 662–664
• Levin, L. A. (1974) Laws of information (nongrowth) and aspects of the foundation of probability theory,
Problems of Information Transmission, v. 10, No. 3, pp. 206–210
• Levin, L.A. (1976) Various Measures of Complexity for Finite Objects (Axiomatic Description), Soviet Math.
Dokl., v. 17, pp. 522–526
• Li, M., and Vitanyi, P. An Introduction to Kolmogorov Complexity and its Applications, Springer-Verlag, New
York, 1997
• Solomonoff, R.J. (1960) A Preliminary Report on a General Theory of Inductive Inference, Technical Report
ZTB-138, Zator Company, Cambridge, Mass.
• Solomonoff, R.J. (1964) A Formal Theory of Inductive Inference, Information and Control, v. 7, No. 1, pp. 1–22;
No.2, pp. 224–254
• Solomonoff, R.J. (2009) Algorithmic Probability: Theory and Applications, Information Theory and Statistical
Learning, Springer NY, Emmert-Streib, F. and Dehmer, M. (Eds), ISBN 978-0-387-84815-0.
• Van Lambagen, (1989) Algorithmic Information Theory, Journal for Symbolic Logic, v. 54, pp. 1389–1400
• Zurek, W.H. (1991) Algorithmic Information Content, Church-Turing Thesis, physical entropy, and Maxwell’s
demon, in Complexity, Entropy and the Physics of Information, (Zurek, W.H., Ed.) Addison-Wesley, pp. 73–89
Algorithmic information theory
• Zvonkin, A.K. and Levin, L. A. (1970) The Complexity of Finite Objects and the Development of the Concepts of
Information and Randomness by Means of the Theory of Algorithms, Russian Mathematics Surveys, v. 256,
pp. 83–124
Algorithmically random sequence
Intuitively, an algorithmically random sequence (or random sequence) is an infinite sequence of binary digits that
appears random to any algorithm. The definition applies equally well to sequences on any finite set of characters.
Random sequences are key objects of study in algorithmic information theory.
As different types of algorithms are sometimes considered, ranging from algorithms with specific bounds on their
running time to algorithms which may ask questions of an oracle, there are different notions of randomness. The
most common of these is known as Martin-Löf randomness (or 1-randomness), but stronger and weaker forms of
randomness also exist. The term "random" used to refer to a sequence without clarification is usually taken to mean
"Martin-Löf random".
Because infinite sequences of binary digits can be identified with real numbers in the unit interval, random binary
sequences are often called random real numbers. Additionally, infinite binary sequences correspond to
characteristic functions of sets of natural numbers; therefore those sequences might be seen as sets of natural
The class of all Martin-Löf random (binary) sequences is denoted by RAND or MLR.
The first suitable definition of a random sequence was given by Per Martin-Löf in 1966. Earlier researchers such as
Richard von Mises had attempted to formalize the notion of a test for randomness in order to define a random
sequence as one that passed all tests for randomness; however, the precise notion of a randomness test was left
vague. Martin-Löf's key insight was to use the theory of computation to formally define the notion of a test for
randomness. This contrasts with the idea of randomness in probability; in that theory, no particular element of a
sample space can be said to be random.
Martin-Löf randomness has since been shown to admit many equivalent characterizations — in terms of
compression, randomness tests, and gambling — that bear little outward resemblance to the original definition, but
each of which satisfy our intuitive notion of properties that random sequences ought to have: random sequences
should be incompressible, they should pass statistical tests for randomness, and it should be difficult to make money
betting on them. The existence of these multiple definitions of Martin-Löf randomness, and the stability of these
definitions under different models of computation, give evidence that Martin-Löf randomness is a fundamental
property of mathematics and not an accident of Martin-Löf's particular model. The thesis that the definition of
Martin-Löf randomness "correctly" captures the intuitive notion of randomness has been called the
Martin-Löf–Chaitin Thesis; it is somewhat similar to the Church–Turing thesis.
Three equivalent definitions
Martin-Löf's original definition of a random sequence was in terms of constructive null covers; he defined a
sequence to be random if it is not contained in any such cover. Leonid Levin and Claus-Peter Schnorr proved a
characterization in terms of Kolmogorov complexity: a sequence is random if there is a uniform bound on the
compressibility of its initial segments. Schnorr gave a third equivalent definition in terms of martingales (a type of
betting strategy). Li and Vitanyi's book An Introduction to Kolmogorov Complexity and Its Applications
is an
excellent introduction to these ideas.
Algorithmically random sequence
• Kolmogorov complexity (Schnorr 1973, Levin 1973): Kolmogorov complexity can be thought of as a lower
bound on the algorithmic compressibility of a finite sequence (of characters or binary digits). It assigns to each
such sequence w a natural number K(w) that, intuitively, measures the minimum length of a computer program
(written in some fixed programming language) that takes no input and will output w when run. Given a natural
number c and a sequence w, we say that w is c-incompressible if .
An infinite sequence S is Martin-Löf random if and only if there is a constant c such that all of S's finite
prefixes are c-incompressible.
• Constructive null covers (Martin-Löf 1966): This is Martin-Löf's original definition. For a finite binary string w
we let C
denote the cylinder generated by w. This is the set of all infinite sequences beginning with w, which is
a basic open set in Cantor space. The product measure μ(C
) of the cylinder generated by w is defined to be
. Every open subset of Cantor space is the union of a countable sequence of disjoint basic open sets, and the
measure of an open set is the sum of the measures of any such sequence. An effective open set is an open set that
is the union of the sequence of basic open sets determined by a recursively enumerable sequence of binary strings.
A constructive null cover or effective measure 0 set is a recursively enumerable sequence of effective open
sets such that and for each natural number i. Every effective null cover determines a
set of measure 0, namely the intersection of the sets .
A sequence is defined to be Martin-Löf random if it is not contained in any set determined by a
constructive null cover.
• Constructive martingales (Schnorr 1971): A martingale is a function such that, for all
finite strings w, , where is the concatenation of the strings a and b.
This is called the "fairness condition"; a martingale is viewed as a betting strategy, and the above condition
requires that the better plays against fair odds. A martingale d is said to succeed on a sequence S if
where is the first n bits of S. A martingale d is constructive (also known as
weakly computable, lower semi-computable, subcomputable) if there exists a computable function
such that, for all finite binary strings w
1. for all positive integers t,
A sequence is Martin-Löf random if and only if no constructive martingale succeeds on it.
(Note that the definition of martingale used here differs slightly from the one used in probability theory.
That definition of martingale has a similar fairness condition, which also states that the expected value after
some observation is the same as the value before the observation, given the prior history of observations. The
difference is that in probability theory, the prior history of observations just refers to the capital history,
whereas here the history refers to the exact sequence of 0s and 1s in the string.)
Interpretations of the definitions
The Kolmogorov complexity characterization conveys the intuition that a random sequence is incompressible: no
prefix can be produced by a program much shorter than the prefix.
The null cover characterization conveys the intuition that a random real number should not have any property that is
“uncommon”. Each measure 0 set can be thought of as an uncommon property. It is not possible for a sequence to lie
in no measure 0 sets, because each one-point set has measure 0. Martin-Löf's idea was to limit the definition to
measure 0 sets that are effectively describable; the definition of an effective null cover determines a countable
collection of effectively describable measure 0 sets and defines a sequence to be random if it does not lie in any of
these particular measure 0 sets. Since the union of a countable collection of measure 0 sets has measure 0, this
definition immediately leads to the theorem that there is a measure 1 set of random sequences. Note that if we
Algorithmically random sequence
identify the Cantor space of binary sequences with the interval [0,1] of real numbers, the measure on Cantor space
agrees with Lebesgue measure.
The martingale characterization conveys the intuition that no effective procedure should be able to make money
betting against a random sequence. A martingale d is a betting strategy. d reads a finite string w and bets money on
the next bit. It bets some fraction of its money that the next bit will be 0, and then remainder of its money that the
next bit will be 1. d doubles the money it placed on the bit that actually occurred, and it loses the rest. d(w) is the
amount of money it has after seeing the string w. Since the bet placed after seeing the string w can be calculated from
the values d(w), d(w0), and d(w1), calculating the amount of money it has is equivalent to calculating the bet. The
martingale characterization says that no betting strategy implementable by any computer (even in the weak sense of
constructive strategies, which are not necessarily computable) can make money betting on a random sequence.
Properties and examples of Martin-Löf random sequences
• Chaitin's halting probability Ω is an example of a random sequence.
(the complement of RAND) is a measure 0 subset of the set of all infinite sequences. This is implied by
the fact that each constructive null cover covers a measure 0 set, there are only countably many constructive null
covers, and a countable union of measure 0 sets has measure 0. This implies that RAND is a measure 1 subset of
the set of all infinite sequences.
• Every random sequence is normal.
• There is a constructive null cover of RAND
. This means that all effective tests for randomness (that is,
constructive null covers) are, in a sense, subsumed by this universal test for randomness, since any sequence that
passes this single test for randomness will pass all tests for randomness. (Martin-Löf 1966)
• There is a universal constructive martingale d. This martingale is universal in the sense that, given any
constructive martingale d, if d succeeds on a sequence, then d succeeds on that sequence as well. Thus, d
succeeds on every sequence in RAND
(but, since d is constructive, it succeeds on no sequence in RAND).
(Schnorr 1971)
• The class RAND is a subset of Cantor space, where refers to the second level of the arithmetical
hierarchy. This is because a sequence S is in RAND if and only if there is some open set in the universal effective
null cover that does not contain S; this property can be seen to be definable by a formula.
• There is a random sequence which is , that is, computable relative to an oracle for the Halting problem.
(Schnorr 1971) Chaitin's Ω is an example of such a sequence.
• No random sequence is decidable, computably enumerable, or co-computably-enumerable. Since these
correspond to the , , and levels of the arithmetical hierarchy, this means that is the lowest level
in the arithmetical hierarchy where random sequences can be found.
• Every sequence is Turing reducible to some random sequence. (Kučera 1985/1989, Gács 1986). Thus there are
random sequences of arbitrarily high Turing degree.
Relative randomness
As each of the equivalent definitions of a Martin-Löf random sequence is based on what is computable by some
Turing machine, one can naturally ask what is computable by a Turing oracle machine. For a fixed oracle A, a
sequence B which is not only random but in fact satisfies the equivalent definitions for computability relative to A
(e.g., no martingale which is constructive relative to the oracle A succeeds on B) is said to be random relative to A.
Two sequences, while themselves random, may contain very similar information, and therefore neither will be
random relative to the other. Any time there is a Turing reduction from one sequence to another, the second sequence
cannot be random relative to the first, just as computable sequences are themselves nonrandom; in particular, this
Algorithmically random sequence
means that Chaitin's Ω is not random relative to the halting problem.
An important result relating to relative randomness is van Lambalgen's theorem, which states that if C is the
sequence composed from A and B by interleaving the first bit of A, the first bit of B, the second bit of A, the second
bit of B, and so on, then C is algorithmically random if and only if A is algorithmically random, and B is
algorithmically random relative to A. A closely related consequence is that if A and B are both random themselves,
then A is random relative to B if and only if B is random relative to A.
Stronger than Martin-Löf randomness
Relative randomness gives us the first notion which is stronger than Martin-Löf randomness, which is randomness
relative to some fixed oracle A. For any oracle, this is at least as strong, and for most oracles, it is strictly stronger,
since there will be Martin-Löf random sequences which are not random relative to the oracle A. Important oracles
often considered are the halting problem, , and the nth jump oracle, , as these oracles are able to answer
specific questions which naturally arise. A sequence which is random relative to the oracle is called
n-random; a sequence is 1-random, therefore, if and only if it is Martin-Löf random. A sequence which is n-random
for every n is called arithmetically random. The n-random sequences sometimes arise when considering more
complicated properties. For example, there are only countably many sets, so one might think that these should
be non-random. However, the halting probability Ω is and 1-random; it is only after 2-randomness is reached
that it is impossible for a random set to be .
Weaker than Martin-Löf randomness
Additionally, there are several notions of randomness which are weaker than Martin-Löf randomness. Some of these
are weak 1-randomness, Schnorr randomness, computable randomness, partial computable randomness.
Additionally, Kolmogorov-Loveland randomness is known to be no stronger than Martin-Löf randomness, but it is
not known whether it is actually weaker.
[1] Jean-Paul Delahaye, Randomness, Unpredictability and Absence of Order (http:/ / books.google.com/ books?id=EDoXdoz-qYQC&
pg=PA145&source=gbs_toc_r&cad=0_0), in Philosophy of Probability, p. 145-167, Springer 1993.
[2] http:// homepages. cwi. nl/ ~paulv/ kolmogorov.html
[3] John M. Hitchcock and Jack H. Lutz (2006). "Why computational complexity requires stricter martingales". Theory of Computing Systems.
• Rod Downey, Denis R. Hirschfeldt, Andre Nies, Sebastiaan A. Terwijn (2006). "Calibrating Randomness". The
Bulletin of Symbolic Logic 12 (3/4): 411–491. doi:10.2178/bsl/1154698741.
• Gács, P. (1986). "Every sequence is reducible to a random one". Information and Control 70 (2/3): 186–192.
• Kučera, A. (1985). "Measure, Π
-classes and complete extensions of PA". Recursion Theory Week. Lecture
Notes in Mathematics 1141, Springer-Verlag. pp. 245–259.
• Kučera, A. (1989). "On the use of diagonally nonrecursive functions". Studies in Logic and the Foundations of
Mathematics. 129. North-Holland. pp. 219–239.
• Levin, L. (1973). "On the notion of a random sequence". Soviet Mathematics Doklady 14: 1413–1416.
• Li, M.; Vitanyi, P. M. B. (1997). An Introduction to Kolmogorov Complexity and its Applications (Second ed.).
Berlin: Springer-Verlag.
• Martin-Löf, P. (1966). "The definition of random sequences". Information and Control 9: 602–619.
• Schnorr, C. P. (1971). "A unified approach to the definition of a random sequence". Mathematical Systems Theory
5 (3): 246–258. doi:10.1007/BF01694181.
Algorithmically random sequence
• Schnorr, C. P. (1973). "Process complexity and effective random tests". Journal of Computer and System
Sciences 7: 376–388. doi:10.1016/S0022-0000(73)80030-3.
• Ville, J. (1939). Etude critique de la notion de collectif. Paris: Gauthier-Villars.
Applications of randomness
Randomness has many uses in gambling, statistics, cryptography, art, etc.
These uses have different randomness requirements, which leads to the use of different randomization methods. For
example, applications in cryptography have strict requirements, whereas other uses (such as generating a "quote of
the day") can use a looser standard of randomness.
Early uses
Unpredictable (by the humans involved) numbers (usually taken to be random numbers) were first formally
investigated in the context of gambling. Many randomizing devices such as dice, shuffling playing cards, and
roulette wheels, seem to have been developed for use in games of chance. Electronic gambling equipment cannot use
these and so theoretical problems are less easy to avoid; methods of creating them are sometimes regulated by
governmental gaming commissions.
Modern electronic casino games contain often one or more random number generators which decide the outcome of
a trial in the game. Even in modern slot machines, where mechanical reels seem to spin on the screen, the reels are
actually spinning for entertainment value only. They eventually stop exactly where the machine's software decided
they would stop when the handle was first pulled. (It has been alleged that some gaming machines' software is
deliberately biased to prevent true randomness, in the interests of maximizing their owners' revenue; the history of
biased machines in the gambling industry is the reason government inspectors attempt to supervise the
machines—electronic equipment has extended the range of supervision. Some thefts from casinos have used clever
modifications of internal software to bias the outcomes of the machines—at least in those which have been
discovered. Gambling establishments keep close track of machine payouts in an attempt to detect such alterations.
Random draws are often used to make a decision where no rational or fair basis exists for making a deterministic
decision, or to make unpredictable moves.
Many ancient cultures saw natural events as signs from the gods; many attempted to discover the intentions of the
gods through various sorts of divination. The underlying theory was that the condition of a, say, a chicken's liver,
was connected with, perhaps, the dangerous storms or military or political fortune. Divination is still practiced and
on much the same basis as formerly.
Political use
Athenian democracy
Fifth century BC Athenian democracy developed out of a notion of isonomia (equality of political rights), and
random selection was a principal way of achieving this fairness.
Greek "democracy" (literally meaning "rule by the
people") was actually run by the people: administration was in the hands of committees allotted from the people and
regularly changed. Although it may seem strange to those used to modern liberal democracy, the Athenian Greeks
Applications of randomness
considered elections to be essentially undemocratic.

This was because citizens chosen on merit or popularity
contradicted the democratic equality of all citizenry. In addition, allotment prevented the corrupt practice of buying
votes as no one could know who would be selected as a magistrate, or to sit on a jury.
Modern use
Allotment is today restricted mainly to the selection of jurors in Anglo-Saxon legal systems like the UK and US.
Proposals have been made for its use in government such as a new constitution for Iraq
and various proposals for
Upper Houses chosen by allotment. (See Lords reform.)
Random numbers have uses in physics (such as noise resonance studies), engineering, and operations research. Many
methods of statistical analysis, such as the bootstrap method, require random numbers. Monte Carlo methods in
physics and computer science require random numbers.
Random numbers are often used in parapsychology as a test of precognition.
Statistical sampling
Statistical practice is based on statistical theory which is, itself, founded on the concept of randomness. Many
elements of statistical practice depend on randomness via random numbers. Where those random numbers fail to be
actually random, any subsequent statistical analysis may suffer from systematic bias. Elements of statistical practice
that depend on randomness include: choosing a representative sample, disguising the protocol of a study from a
participant (see randomized controlled trial) and Monte Carlo simulation.
These applications are useful in auditing (for determining samples - such as invoices) and experimental design (for
example in the creation of double-blind trials).
Many experiments in physics rely on a statistical analysis of their output. For example, an experiment might collect
X-rays from an astronomical source and then analyze the result for periodic signals. Since random noise can be
expected to appear to have faint periodic signals embedded in it, statistical analysis is required to determine the
likelihood that a detected signal actually represents a genuine signal. Such analysis methods requires the generation
of random numbers. If the statistical method is extremely sensitive to patterns in the data (such as those used to
search for binary pulsars), very large amounts of data with no recognizable pattern are needed.
In many scientific and engineering fields, computer simulations of real phenomena are commonly used. When the
real phenomena are affected by unpredictable processes, such as radio noise or day-to-day weather, these processes
can be simulated using random or pseudo-random numbers.
Automatic random number generators were first constructed to carry out computer simulation of physical
phenomena, notably simulation of neutron transport in nuclear fission.
Pseudo-random numbers are frequently used in simulation of statistical events, a very simple example being the
outcome of tossing a coin. More complicated situations are simulation of population genetics, or the behaviour of
sub-atomic particles. Such simulation methods, often called stochastic methods, have many applications in computer
simulation of real-world processes.
Some more speculative projects, such as the Global Consciousness Project, monitor fluctuations in the randomness
of numbers generated by many hardware random number generators in an attempt to predict the scope of an event in
near future. The intent is to prove that large scale events that are about to happen build up a "pressure" which affects
Applications of randomness
the RNGs.
A ubiquitous use of unpredictable random numbers is in cryptography which underlies most of the schemes which
attempt to provide security in modern communications (e.g., confidentiality, authentication, electronic commerce,
For example, if a user wants to use an encryption algorithm, it is best that they select a random number as the key.
These numbers must have high entropy for any attacker, thus increasing attack difficulty. With low entropy numbers
used as keys (ie, relatively easily guessable by attackers), security is likely to be compromised. For example, if a
simple 32 bit linear congruential pseudo-random number generator of the type supplied with most programming
languages is used as a source of keys, then there will only be some four billion possible values produced before the
generator repeats itself. A suitably motivated adversary could simply test them all; this is practical using as of 2010,
using readily available computers. Even if a better random number generator is used, it might be insecure (ie, its
starting value, the seed) might be guessable, producing predictable keys and reducing security to nil. (A vulnerability
of this sort was famously discovered in an early release of Netscape Navigator, forcing the authors to quickly find a
source of "more random" random numbers). For these applications, truly random numbers are ideal, and very high
quality pseudo-random numbers are necessary if truly random numbers are unavailable.
Truly random numbers are absolutely required to be assured of the theoretical security provided by the one-time pad
— the only provably unbreakable encryption algorithm. Furthermore, those random sequences cannot be reused and
must never become available to any attacker, which implies a continuously operable generator. See Venona for an
example of what happens when these requirements are violated when using a one-time pad.
For cryptographic purposes, one normally assumes some upper limit on the work an adversary can do (usually this
limit is astronomically sized). If one has a pseudo-random number generator whose output is "sufficiently difficult"
to predict, one can generate true random numbers to use as the initial value (ie, the seed), and then use the
pseudo-random number generator to produce numbers for use in cryptographic applications. Such random number
generators are called cryptographically secure pseudo-random number generators, and several have been
implemented (for example, the /dev/urandom device available on most Unixes, the Yarrow and Fortuna designs,
server, and AT&T Bell Labs "truerand"). As with all cryptographic software, there are subtle issues beyond those
discussed here, so care is certainly indicated in actual practice. In any case, it is sometimes impossible to avoid the
need for true (i.e., hardware) random number generators.
Since a requirement in cryptography is high entropy (ie, unpredictability to an attacker), any published random
sequence is a poor choice, as are such sequences as the digits in an irrational number such as the φ or even in
transcendental numbers such as π, or e. All are available to an enterprising attacker. Put another way, in
cryptography, random bit streams need to be not only random, but also secret and hence unpredictable. Public or
third-party sources of random values, or random values computed from publicly observable phenomena (weather,
sports game results, stock prices), are almost never cryptographically acceptable, though often tempting and too
often used by the unwary. They permit easier attacks than attacking the cryptography.
Since most cryptographic applications require a few thousand bits at most, slow random number generators serve
well—if they are actually random. This use of random generators is important; many informed observers believe
every computer should have a way to generate true random numbers.
Applications of randomness
Literature, music and art
Some aesthetic theories claim to be based on randomness in one way or another. Little testing is done in these
situations, and so claims of reliance on and use of randomness are generally abstract.
An example of a need for randomness sometimes occurs in arranging items in an art exhibit. Usually this is avoided
by using a theme. As John Cage pointed out, "While there are many ways that sounds might be produced [i.e., in
terms of patterns], few are attempted". Similarly, the arrangement of art in exhibits is often deliberately non-random.
One case of this was Hitler's attempt to portray modern art in the worst possible light by arranging works in worst
possible manner. A case can be made for trying to make art in the worst possible way; i.e., either as anti-art, or as
actually random art.
Dadaism, as well as many other movements in art and letters, has attempted to accommodate and acknowledge
randomness in various ways. Often people mistake order for randomness based on lack of information; e.g., Jackson
Pollock's drip paintings, Helen Frankenthaler's abstractions (e.g., "For E.M."). Thus, in some theories of art, all art is
random in that it's "just paint and canvas" (the explanation of Frank Stella's work).
Similarly, the "unexpected" ending is part of the nature of interesting literature. An example of this is Denis
Diderot's novel Jacques le fataliste (literally: James the Fatalist; sometimes referred to as Jacques the Fatalist or
Jacques the Servant and his Master). At one point in the novel, Diderot speaks directly to the reader:
Now I, as the author of this novel might have them set upon by thieves, or I might have them rest by a
tree until the rain stops, but in fact they kept on walking and then near night-fall they could see the light
of an inn in the distance.
(not an exact quote). Diderot was making the point that the novel (then a recent introduction to European literature)
seemed random (in the sense of being invented out of thin air by the author). See also Eugenio Montale, Theatre of
the Absurd.
Randomness in music is generally thought to be postmodern, including John Cage's chance derived Music of
Changes, Iannis Xenakis' stochastic music, aleatoric music, indeterminate music, or generative music.
Other uses
Random numbers are also used in situations where "fairness" is approximated by randomization, such as selecting
jurors and military draft lotteries. In the Book of Numbers (33:54), Moses commands the Israelites to apportion the
land by lot.
Other examples include selecting, or generating, a "Random Quote of the Day" for a website, or determining which
way a villain might move in a computer game.
Weaker forms of randomness are also closely associated with hash algorithms and in creating amortized searching
and sorting algorithms.
[1] Herodotus 3.80
[2] The Athenian Democracy in the Age of Demosthenes", Mogens Herman Hansen, ISBN 1-85399-585-1
[3] “it is thought to be democratic for the offices to be assigned by lot, for them to be elected is oligarchic,” [Aristotle, Politics 4.1294b]
[4] http:/ / www. sortition. org.uk
External links
• http:/ / random.org
Bernoulli stochastics
Bernoulli stochastics
Bernoulli stochastics is a new branch of science and deals with human uncertainty of future developments
. It
aims at developing quantitative models of the transition from past to future for making reliable and accurate
predictions. Bernoulli stochastics should not be confused with stochastics which is a special branch of mathematics
covering probability theory, the theory of stochastic processes and mathematical statistics.
Bernoulli stochastics is based on Jakob Bernoulli's quantification of randomness and it was mainly developed by
Elart von Collani

during the last two decades. Since uncertainty of the future constitutes one of the main
problems of mankind, Bernoulli stochastics adopts an exceptional position as it provides the means to define and
measure uncertainty and thus enables to handle risks adequately for preventing catastrophic developments.
Uncertainty of the future constitutes not only the main problem for individuals but also for societies and science.
Therefore, Bernoulli stochastics which develops models of uncertainty can be considered as a universal approach for
solving problems since it provides the rules how to deal with uncertainty and the indeterminate future.
The quantitative models of uncertainty developed according to the rules of Bernoulli stochastics include the two
sources of human uncertainty of future development. These are human ignorance about the past or the initial
conditions and randomness which affects the future. Ignorance represents the internal source of human uncertainty,
while randomness is the external source. Ignorance is characteristic for man, while randomness is characteristic for
“For decisions on the most important projects, like the safety of space travel or the prospects for global warming, it is still the custom
to rely on the opinion of experts to evaluate risks. They are known to make mistakes, but no method has been found to replace them.”
James Franklin, The Science of Conjecturing, The JohnsHopkins University Press, Baltimore (2000), p. 369.
There are two types of methods in Bernoulli stochastics. The first type of method enables a glance into the future,
while the second type enables to look into the past. The first type is called stochastic prediction procedure, the
second type stochastic measurement procedure. These stochastic procedures which are based on an model that
objectively reflects reality aim at replacing belief and opinion which are still the prevailing means to overcome the
problems generated by uncertainty and risks.
For understanding and applying Bernoulli stochastics the prevailing causal thinking must be abandoned in favor of
stochastic thinking
. Actually, adopting stochastic thinking constitutes a major difficulty in understanding and
applying Bernoulli stochastics.
The development of Bernoulli stochastics started more than 300 years ago with the theologian and mathematician
Jakob Bernoulli (1655–1705)
from Basel in Switzerland. Jakob Bernoulli succeeded to quantify randomness of
future events
. Due to randomness a future event may or may not occur. Randomness can be observed by repeating
the same experiment several times. Then some events will occur often and others more seldom. Bernoulli explained
randomness of a future event by "the degree of certainty of the occurrence of the event" and called this degree
"probability of the event". He planned to develop a science based on the concept of probability and named this
science in Latin "Ars conjectandi" or in Greek "stochastike", i.e. "science of prediction.". Unfortunately, he died too
early and his masterpiece Ars conjectandi was only published posthumously in 1713
. His proposal was not taken
up by science and instead "probability theory" as a branch of mathematics, and "statistics" as a branch of empirical
science were developed
Bernoulli stochastics
Bernoulli stochastics was introduced in 2000 during the BS Symposium
on "Defining the Science Stochastics, in
Memoriam Jakob Bernoulli". The revised and updated versions of the lectures delivered at the symposium were
published in 2004. Since then Bernoulli stochastics has been further developed and its methods have been
successfully applied in various areas of science and technology, for example in metrology
, quality control
wind energy
and nuclear technology

In 2002 the company Stochastikon GmbH was founded and started to further develop Bernoulli stochastics. In 2008
the first PhD-thesis
by Andreas Binder was published dealing with Bernoulli stochastics and two subsystems of a
web-based information and application system for its establishment. In April 2011, the second PhD-thesis
Bernoulli stochastics by Xiaomin Zhai was completed about design, development and evaluation of a virtual
classroom and teaching contents for Bernoulli stochastics.
“... it is only the manipulation of uncertainty that interests us. We are not concerned with the matter that is uncertain. Thus we do not
study the mechanism of rain; only whether it will rain.”
Dennis Lindley, "The Philosophy of Statistics", The Statistician (2000).
The models in Bernoulli stochastics describe the change from the past to the future including the entire uncertainty
as good as the available information permit. When developing a model of uncertainty, then one should have always
in mind that not the mechanism of the considered process is of major interest, but the future events as expressed by
Dennis Lindley.
1. The first step consists of identifying the aspect of the future development which is of interest. Since the future
development is subject to randomness it is quantified by a variable X which is called random variable. The future
value of a random variable is indeterminate and generally varies when an experiment is repeated.
2. In a second step the relevant aspects of the past must be identified and represented by a variable D. Since the past
is determinate, the variable D is called deterministic variable.
The model refers to the stochastic relation between the deterministic variable D and the random variable X and in
order to describe the uncertainty of the future development realistically it must necessarily cover the two sources of
uncertainty, i.e., ignorance about the past and randomness of the future.
Stochastic model
The stochastic model of uncertainty specifies what is known about the past, i.e., what is known about the value of the
deterministic variable D, and what can occur in the future with respect to the random variable X. A stochastic model
describes quantitatively the relation between past and future by considering the entire uncertainty generated by
ignorance and randomness. The model is called Bernoulli space and is denoted by . It consists of three
• The ignorance space denoted is a bounded set that contains all those values of the deterministic variable D
which according the available knowledge cannot be excluded. The ignorance space thus describes quantitatively
the existing ignorance about the initial conditions. Each subset of the ignorance space represents a certain level of
knowledge or equivalently of ignorance, where the singletons represent complete knowledge.
• The variability function denoted assigns to each level of knowledge (= subset of the ignorance space) a
corresponding range of variability of the random variable X. In other words, each image of the variability function
consists of those values of X which might occur in the future, under the condition that the true, but unknown value
of D is an element of the considered subset of the ignorance space.
• The random structure function denoted assigns to each level of knowledge (= subset of the ignorance space) a
corresponding probability distribution over the corresponding image of the variability function.
Bernoulli stochastics
A Bernoulli Space refers to the pair of variables where X represents the future and
D the past. The ignorance space specifies the available knowledge about the past, the variability function
gives the amount of variability in the future as function of the available knowledge, and finally the random structure
function specifies the probabilities of future events again as a function of the available knowledge about the
initial conditions.
Learning theory
Knowledge or equivalently ignorance refers to facts, i.e., the past, since the future does not exist so far and which of
the many future developments will actually occur is subject to randomness and it is therefore in principle impossible
to know it.
Learning means to increase knowledge or reduce ignorance about facts implying that modelling a learning process is
possible only, if ignorance and randomness are explicitly incorporated into the model. In case of a Bernoulli Space,
ignorance is modelled by the ignorance space and randomness by the variability function and the random
structure function . It follows that the stochastic model given by the Bernoulli Space may be used as basis for
developing a theory of learning.
Natural laws
Natural laws in physics are quantitative models of the transition from the past to the future, and therefore competitors
to the Bernoulli Space. Thus, it is of interest to compare the two approaches.
A natural law is a function which maps the initial conditions represented by the variable D on the future outcome of
the variable X. For any natural law it is assumed that the initial conditions are known exactly, i.e., the ignorance
space is assumed to be a singleton . Moreover most natural laws assume that the future is a mere
transformation of the past, i.e., it is assumed that for given initial conditions d there is only exactly one possible
future outcome implying that the range of variability of X is given by a singleton . The image of the
random structure function is a probability distribution. In case of a natural law it degenerates to a one-point
distribution. Thus, natural laws prove to be degenerate limiting cases of the Bernoulli Space.
None of the natural laws invented in physics incorporates the always existing human ignorance. Contrary, the natural
laws assume complete knowledge and do therefore not admit improving by learning. Consequently, the approach
that results in natural laws turns out to be one of the most serious obstacles for any learning process.
Procedures of Bernoulli stochastics
There are two main types of procedures in Bernoulli stochastics referring to the two main types of problems mankind
is confronted with. The first type consists of prediction procedures and the second type of measurement procedures.
Prediction procedures allow to look into the future, while measurement procedures allow to look into the past. The
future is characterized by indeterminate events, while the past is characterized by determinate facts.
Stochastic prediction procedure
A stochastic prediction procedure aims at reducing the uncertainty about the future development of interest
represented by the random variable X. A Bernoulli Space is developed in order to enable reliable and accurate
prediction about the future development, i.e., about the indeterminate outcome of the random variable X. A
stochastic prediction procedure is a function denoted which assigns to each level of knowledge (= subset of the
ignorance space) a prediction, i.e., a subset of the corresponding range of variability of $X$. The quality of a
prediction is determined by its reliability and its accuracy. The reliability of a prediction is defined by the probability
of its occurrence, and the accuracy of a prediction is defined by its size. A stochastic prediction procedure is
derived in a way that it meets the following two requirements:
Bernoulli stochastics
• Reliability requirement: A stochastic prediction procedure yields predictions that will occur with a probability of
at least where the lower bound is called reliability level of the prediction procedure.
• Accuracy requirement: The size of the predictions obtained by a stochastic prediction procedure is minimum.
The first condition guarantees a sufficient large reliability of the predictions, while the second condition ensures that
the accuracy of the obtained predictions is optimal. A prediction procedure meeting the reliability requirement given
by the reliability level is called -prediction procedure denoted .
Stochastic measurement procedure
A stochastic measurement procedure aims at reducing the ignorance about the true but unknown value of the
deterministic variable D. Reducing ignorance is equivalent with learning and learning is possible only by a learning
process, which is called here a measurement process. The measurement process has an indeterminate outcome which
is represented by a random variable X and for deriving a suitable stochastic measurement procedure the uncertainty
related to the measurement process must be described by a Bernoulli Space . The Bernoulli Space allows to
predict for any possible value of D which is an element of the ignorance space and any reliability level a
prediction .
A stochastic measurement procedure assigns to each outcome of the measurement process, i.e., a subset of the range
of variability of X, a measurement result, i.e., a subset of the ignorance space. Thus, a measurement procedure is a
function denoted by . A stochastic measurement procedure meets the following three requirements:
• Reliability requirement: The probability to obtain a correct result when applying a stochastic measurement
procedure is not smaller than a prescribed reliability level , where a result is called correct if it contains the
true value of the deterministic variable. If a measurement procedure meets this condition it is called
-measurement procedure denoted .
• Completeness requirement: Each possible, i.e., observable result of the measurement process yields a meaningful
measurement result, i.e., a nonempty subset of the ignorance space.
• Accuracy requirement: The measurement results of a stochastic -measurement procedure are on average most
accurate, i.e., have on average minimum size.
Any stochastic -measurement procedure is based on a suitable stochastic -prediction procedure by
the following relation:
where is the observed result of the measurement process. The above relation means that every value d of the
deterministic variable D is considered in the measurement result for which the observation had
been predicted.
The reliability requirement of the measurement procedures is met by the reliability level of the involved
prediction procedure. The completeness and accuracy requirements are met by rather complicated mathematical
optimization procedures. Because of these two requirements, the prediction procedures for measurement procedures
are different from those obtained for prediction procedures.
Bernoulli stochastics
Stochastic thinking
Bernoulli stochastics explicitly admits randomness as a characteristic feature of real world. However, as shown in
subsection "Natural Law", the stochastic model also covers deterministic relations, however, as degenerate limiting
cases. The stochastic approach emanates from the almost obvious fact that everything in the universe is connected
with everything. This universal connectivity excludes causal relations since any change is simultaneously cause and
Furthermore, the universal connectivity is a property of the entire universe and not of any part of it. It follows that
the whole cannot be understood by investigating parts of it and in particular not by investigating elementary
particles, i.e., the smallest parts of the universe. Bernoulli stochastics therefore represents not only a stochastic, but
also a holistic approach in contrast to physics which is based on determinism and reductionism.
As already mentioned applying Bernoulli stochastics requires to abandon causal thinking in favor of stochastic
thinking. The difficulty is that almost everybody seems to understand causal thinking, but only very few can explain
stochastic thinking. Therefore, the main differences are listed below.
• Causal thinking means to trace back the occurrence of a problem to a culprit, i.e., a part of the system. In contrast,
according to stochastic thinking the design of the system yields a positive probability for the problem.
• The solution of a problem based on causal thinking consists of eliminating the culprit but maintaining the system.
In contrast, according to stochastic thinking the situation can only be improved by changing the design of the
system to reduce the probability of the problem.
• Causal thinking means to explain developments by cause and effect chains which refer to isolated parts of the
system. Stochastic thinking does not explain certain partial developments, but looks at the entire system and its
stochastic evolution rules.
Similar as in subsection Learning Theory the above list illustrates that the stochastic approach represents a learning
approach while the causal approach appears as an unsurmountable obstacle for learning.
[1] Elart von Collani, State of the Art and Future of Stochastics, in: Ivor Grattan-Guinness and B.S. Yadav (eds): History of Mathematical
Sciences, Hindustan Book Agency, New Delhi, pp. 171–190, 2004.
[2] Elart von Collani (ed), Defining the Science of Stochastics, Helderman, Lemgo,2004.
[3] Elart von Collani and Xiaomin Zhai, Stochastics, Beijing Publisher Group, Beijing, 2005 (in Chinese).
[4] Elart von Collani, "Response to ‘Desired and Feared—What Do We Do Now and Over the Next 50 Years’ by Xiao-Li Meng" (http:// pubs.
amstat.org/doi/ pdfplus/ 10. 1198/ tast. 2010. 09190), The American Statistician, 2010, 64(1): 23–25.
[5] Usually the year of Jakob Bernoulli's birth is given as 1654, however, this is not correct if the nowadays valid Gregorian calendar is applied.
[6] Elart von Collani, "Jacob Bernoulli Deciphered" (http:// isi.cbs. nl/ bnews/ 06b/index.html), Bernoulli News, 2006, Vol. 13/2.
[7] Jacob Bernoulli, The Art of Conjecturing, translated by Edith Dudley Sykka, 2006, John Hopkins University Press, Baltimore.
[8] Elart von Collani, The forgotton science of prediction, in: V. Nithyanantha Bhat, T. Thrivikraman, V. Madhikar Mallayya, and S. Madhavan
(eds.), History and Heritage of Mathematical Sciences, Sukrtindra Oriental Research Institute, Kerala, pp. 54–70, 2009.
[9] During the World Mathematical Year 2000 a number of international conferences and workshop were organized under the aegus of the
Bernoulli Society, (http:// isi. cbs. nl/ bnews/ 00b/ bn_4. html).
[10] (http:// ib.ptb. de/ 8/ 84/ MATHMET2010/ VORTRAEGE/MathMet2010_Collani. pdf)
[11] Elart von Collani and Karl Baur: Was zum Teufel ist Qualität? Heldermann Verlag, Lemgo, 2007.
[12] Elart von Collani, A. Binder, W. Sans, A. Heitmann, K. Al-Ghazali: Design Load Definition by LEXPOL. Wind Energy 11, 637–653, 2008.
[13] Elart von Collani and Karl Baur, Brennstabauslegung und Brennstabmodellierung – Teil 1 (Fuel rod design and modeling of fuel rods – Part
I), 'Kerntechnik', Vol. 64, 253–260, 2004.
[14] Elart von Collani and Karl Baur, Brennstabauslegung und Brennstabmodellierung – Teil 2 (Fuel rod design and modeling of fuel rods – Part
II), 'Kerntechnik', Vol. 70, 158–167, 2005.
[15] Andreas Binder, Die stochastische Wissenschyaft und zwei Teilsysteme eines Web-basierten Informations- und Anwendungesystems zu ihrer
Etablierung, Ph.D. Thesis, Faculty of Mathematics and Computer Science, University Würzburg, 2006, (http:/ / www.opus-bayern.de/
uni-wuerzburg/volltexte/ 2008/ 2614/ ).
[16] Xiaomin Zhai, Design, Development and Evaluation of a Virtual Classroom and Teaching Contents for Bernoulli Stochastics, Ph.D. Thesis,
Faculty of Mathematics and Computer Science, University Würzburg, 2011, (http:/ / www.opus-bayern.de/ uni-wuerzburg/volltexte/ 2011/
Bernoulli stochastics
5610/ )
External links
• Stochastikon Ecyclopedia, (http:// www. encyclopedia. stochastikon. com)
• E-Learning Programme Stochastikon Magister, (http:// www.magister. stochastikon. com)
• Homepage of Stochastikon GmbH, (http:/ / www. stochastikon. com/ )
• Economic Quality Control, (http:// www. heldermann-verlag.de/ eqc/ eqc23/ eqc23003. pdf)
• Journal of Uncertain Systems, (http:/ / www. worldacademicunion. com/ journal/jus/ jusVol02No3paper05. pdf)
Biology Monte Carlo method
Biology Monte Carlo methods (BioMOCA) have been developed at the University of Illinois at
Urbana-Champaign to simulate ion transport in an electrolyte environment through ion channels or nano-pores
embedded in membranes
. It is a 3-D particle-based Monte Carlo simulator for analyzing and studying the ion
transport problem in ion channel systems or similar nanopores in wet/biological environments. The system simulated
consists of a protein forming an ion channel (or an artificial nanopores like a Carbon Nano Tube, CNT), with a
membrane (i.e. lipid bilayer) that separates two ion baths on either side. BioMOCA is based on two methodologies,
namely the Boltzmann transport Monte Carlo (BTMC)
and particle-particle-particle-mesh (P
. The first one
uses Monte Carlo method to solve the Boltzmann equation, while the later splits the electrostatic forces into
short-range and long-range components.
In full-atomic molecular dynamics simulations of ion channels, most of the computational cost is for following the
trajectory of water molecules in the system. However, in BioMOCA the water is treated as a continuum dielectric
background media. In addition to that, the protein atoms of the ion channel are also modeled as static point charges
embedded in a finite volume with a given dielectric coefficient. So is the lipid membrane, which is treated as a static
dielectric region inaccessible to ions. In fact the only non-static particles in the system are ions. Their motion is
assumed classical, interacting with other ions through electrostatic interactions and pairwise Lennard–Jones
potential. They also interact with the water background media, which is modeled using a scattering mechanism.
The ensemble of ions in the simulation region, are propagated synchronously in time and 3-D space by integrating
the equations of motion using the second-order accurate leap-frog scheme. Ion positions r and forces F are defined at
time steps t, and t + dt. The ion velocities are defined at t – dt/2, t + dt/2. The governing finite difference equations of
motion are
where F is the sum of electrostatic and pairwise ion-ion interaction forces.
Biology Monte Carlo method
Electrostatic field solution
The electrostatic potential is computed at regular time intervals by solving the Poisson’s equation
where and are the charge density of ions and permanent charges on the protein, respectively.
is the local dielectric constant or permittivity, and is the local electrostatic potential. Solving this
equation provides a self-consistent way to include applied bias and the effects of image charges induced at dielectric
The ion and partial charges on protein residues are assigned to a finite rectangular grid using the cloud-in-cell (CIC)
. Solving the Poisson equation on the grid counts for the particlemesh component of the P
M scheme.
However, this discretization leads to an unavoidable truncation of the short-range component of electrostatic force,
which can be corrected by computing the short-range charge-charge Coulombic interactions.
Dielectric coefficient
Assigning the appropriate values for dielectric permittivity of the protein, membrane, and aqueous regions is of great
importance. The dielectric coefficient determines the strength of the interactions between charged particles and also
the dielectric boundary forces (DBF) on ions approaching a boundary between two regions of different permittivity.
However, in nano scales the task of assigning specific permittivity is problematic and not straightforward.
The protein or membrane environment could respond to an external field in a number of different ways



Field induced dipoles, reorientation of permanent dipoles, protonation and deprotonation of protein residues, larger
scale reorganization of ionized side-chains and water molecules, both within the interior and on the surface of the
protein, are all examples of how complicated the assignment of permittivity is. In MD simulations, where all the
charges, dipoles, and field induced atomic dipoles are treated explicitly then it is suggested that a dielectric value of
1 is appropriate. However, in reduced-particle ion simulation programs, such as ours, where the protein, membrane,
and water are continuum backgrounds and treated implicitly, and on top of that, the ion motion takes place on the
same time-scale as the protein’s response to its presence, it is very difficult to assign the dielectric coefficients. In
fact, changing the dielectric coefficients could easily alter the channel characteristics, such as ion permeation and
selectivity The assignment of dielectric coefficient for water is another key issue. The water molecules inside ion
channels could be very ordered due to tapered size of the pore, which is often lined with highly charged residues, or
hydrogen bond formation between water molecules and protein
. As a result, the dielectric constant of water inside
an ion channel could be quite different from the value under bulk conditions. To make the matter even more
complicated, the dielectric coefficients of water inside nanopores is not necessarily an isotropic scalar value, but an
anisotropic tensor having different values in different directions.
Anisotropic permittivity
It has become evident that the macroscopic properties of a system do not necessarily extend to the molecular length
scales. In a recent research study carried by R. Jay Mashl, and Eric Jakobsson at the University of Illinois,
Urbana-Champaign [personal communications], they used Molecular Dynamics simulations to study the properties
of water in featureless hydrophobic cylinders with diameters ranging from 1 to 12 nm. This study showed that water
undergoes distinct transitions in structure, dielectric properties, and mobility as the tube diameter is varied. In
particular they found that the dielectric properties in the range of 1 to 10 nm is quite different from bulk water and is
in fact anisotropic in nature. Though, such featureless hydrophobic channels do not represent actual ion channels and
more research has to be done in this area before one could use such data for ion channels, it is evident that water
properties like permittivity inside an ion channel or nano-pore could be much more complicated that it has been
thought before. While a high axial dielectric constant shields ion’s electrostatic charges in the axial direction (along
the channel), low radial dielectric constant increases the interaction between the mobile ion and the partial charges,
or the dielectric charge images on the channel, conveying stronger selectivity in ion channels.
Biology Monte Carlo method
Solving the Poisson equation based on an anisotropic permittivity has been incorporated into BioMOCA using the
box integration discretization method
, which has been briefly described below.
Box integration discretization
In order to use box integration for discretizing a D-dimensional Poisson equation
with being a diagonal D × D tensor, this differential equation is reformulated as an integral equation. Integration
the above equation over a D-dimensional region , and using Gauss theorem, then the integral formulation is
In this appendix it is assumed to be a two-dimensional case. Upgrading to a three-dimensional system would be
straightforward and legitimate as the Gauss theorem is also valid for the one and three dimensions. is assumed to
be given on the rectangular regions between nodes, while is defined on the grid nodes (as illustrated on figure at
the right).
Box integration for a two-dimensional tensor
product grid. The integration region is indicated
by the dashed rectangle. Charges are assumed to
be given on the same nodes as potential
The integration regions are then chosen as rectangles centered
around node and extending to the 4 nearest neighbor nodes. The
gradient is then approximated using centered difference normal
to the boundary of the integration region , and average over the
integration surface . This approach allows us to approximate the
left hand side of the Poisson equation above in first order as
where and are the two components of the diagonal of the tensor . Discretizing the right-hand side of the
Poisson equation is fairly simple. is discretized on the same grid nodes, as it's been done for .
Biology Monte Carlo method
Ion size
The finite size of ions is accounted for in BioMOCA using pairwise repulsive forces derived from the 6–12
Lennard–Jones potential. A truncated-shifted form of the Lennard–Jones potential is used in the simulator to mimic
ionic core repulsion. The modified form of the Lennard–Jones pairwise potential that retains only the repulsive
component is given by
Here, is the Lennard–Jones energy parameter and is the average of the individual
Lennard–Jones distance parameters for particles i and j. Using a truncated form of the potential is computationally
efficient while preventing the ions from overlapping or coalescing, something that would be clearly unphysical.
Ion-protein interaction
Availability of high-resolution X-ray crystallographic measurements of complete molecular structures provides
information about the type and location of all atoms that forms the protein. In BioMOCA the protein atoms are
modeled as static point charges embedded in a finite volume inaccessible to the ions and associated with a
user-defined dielectric coefficient. Moreover, a number of force-field parameters are available that provide
information about the charge and radii of atoms in different amino-acid groups. The conjunction of the molecular
structure and force fields provide the coordinates, radii, and charge of each atom in the protein channel. BioMOCA
uses such information in the standard PQR (Position-Charge-Radius) format to map the protein system onto a
rectangular grid.
Ideally, the steric interactions between protein atoms and the ions in the aqueous medium are to use a repulsive
potential like Lennard–Jones to prevent ions from penetrating the protein. As this approach could add a significant
load to the amount of calculations, a simpler approach is chosen that treats the protein surfaces as predetermined
hard wall boundaries. Many recent open source molecular biology packages have built-in facilities that determine the
volume accessible to ions in a protein system. The Adaptive Poisson Boltzmann Solver (APBS) scheme
has been
incorporated to BioMOCA to obtain the accessible volume region and therefore partition the simulation domain into
continuous regions.
Ions are deemed to have access to protein and lipid regions and if any point within the finite-size of ionic sphere
crosses the protein or membrane boundary, a collision is assumed and the ion is reflected diffusively.
Ion-water interactions
As a reduced particle approach, BioMOCA replaces the explicit water molecules with continuum background and
handles the ion-water interactions using BTMC method, in which, appropriate scattering rates should be chosen. In
other words, ion trajectories are randomly interrupted by scattering events that account for the ions’ diffusive motion
in water
. In between these scattering events, ions follow the Newtonian forces. The free flight times, T
, are
generated statistically from the total scattering rate according to
where r is a random number uniformly distributed on the unit interval. , a function of momentum, is the total
scattering rate for all collision mechanisms. At the end of each free flight, the ion’s velocity is reselected randomly
from a Maxwellian distribution. As the correct scattering mechanism for ion-water interactions in nonbulk
electrolyte solutions has yet to be developed, a position dependent scattering rate linked to the local diffusivity is
used in our model. This dependency on position comes from the fact that water molecules can have different order of
organization in different regions, which will affect the scattering rate.
Biology Monte Carlo method
Position-dependent diffusivity
It is widely accepted that the ions and water molecules do not have the same mobility or diffusivity in confined
regions as in bulk

. In fact, it is more likely to have a lessening in the effective mobility of ions in ion channels
. In reduced particle methods where the channel water is assumed as implicit continuum background, a mean ion
mobility is needed to reveal how ions could diffuse due to local electrostatic forces and random events. In Transport
Monte Carlo simulations, the total scattering rate ( ), is assumed to only result from ion-water interactions; it is
related to ion diffusivity with the expression
where m is the mass of the ion and D is its diffusion constant. As the equation indicates, reduced diffusivity of ions
inside the lumen of the channel renders to increased incidence of scattering events.
Hydration shells
In addition to having a diffusive effect on ion transport, water molecules also form hydration shells around
individual ions due to their polar nature. The hydration shell not only shields the charge on ions from other ions but
also modulates the ion radial distribution function causing the formation of peaks and troughs. The average
minimum distance between two ions is increased as there is always at least one layer of water molecules present
between them, acting as a physical deterrent preventing two ions from getting too close to each other, in a manner
that is similar to the short-range repulsive component of the Lennard–Jones potential.
The theory of hydration shells is well developed in the physical chemistry literature however a simple model is
required that captures the essential effects with as little computational overhead as possible. For this purpose the
same pairwise potential discussed by Im and Roux
is implemented to include the effect of hydration shells.
The coefficients c
were determined empirically for a 1 M KCl solution, using MD simulations to benchmark the ion
radial distribution functions against Equilibrium Monte Carlo simulations. The effect of hydration shells was found
to be important in simulations at higher salt concentrations where the conductance of many ion channels, porin
among them, is observed to saturate as the salt concentration in the electrolyte baths is further increased. Earlier
simulations that did not include a model of hydration shells did not reproduce the conductance saturation behavior.
This suggests an additional repulsive potential acting to prevent ion crowding, and hence limiting the concentration
of ions and current density in the confined space of the pore even at high bath salt concentration. When the repulsive
potential was included moderate channel conductance was observed.
Conditions and methods
Boundary conditions
The electrical and physiological properties of ion channels are experimentally measured by inserting the channel into
a lipid membrane separating two baths containing solutions of specific concentrations. A constant electrostatic bias
is applied across the channel by immersing the electrodes in the two baths. Formulating boundary conditions that
accurately represent these contact regions may require enormously large bath regions and is a challenging task.
Beyond a Debye length from the membrane the electrostatic potential and ion densities do not vary appreciably. This
assumption has been supported by the results of continuum results presented earlier
. For typical salt
concentrations used in ion channel simulations, the Debye length is of the order of 10 Å. Using the assumption,
Dirichlet boundary conditions are imposed on the potential at the two domain boundary planes that are transverse to
the channel, taking care that these planes are sufficiently far from the membrane.
Biology Monte Carlo method
The other problem in duplicating the experimental conditions is the problem of maintaining fixed charge density in
the two baths. This problem is treated by maintaining the specified density in two buffer regions extending from the
boundary plane toward the membrane. The number of ions needed to maintain the density in the two buffer regions
is calculated at the start of the simulations. The count of the ions in these buffers is sampled throughout the
simulation and an ion is injected whenever a deficit is observed. The initial velocity of the injected particle is
decided according to Maxwellian distribution. It should be noted that the ions can leave the system only by exiting
through the two Dirichlet boundary planes and an ion is not removed artificially from these buffer regions. The
reflections from the Neumann boundary planes are treated as elastic reflections.
Multi-grids and grid focusing method
In all most any of the methods in simulation of ion channels, the major computational cost comes from the
calculation of electrostatic forces acting on the ions. In continuum models, for instance, where ionic density exist
rather than explicit ions, the electrostatic potential is calculated in a self-consistent manner by solving the Poisson
equation. In MD simulations, on the other hand, the electrostatic forces acting on the particles are calculated by
explicit evaluation of the Coulombic force term, often splitting the short-range and long-range electrostatic forces so
they could be computed with different methods. In our model as a reduced particle method, the longrange
electrostatic forces are evaluated by solving the Poisson equation and augmenting the forces so obtained wit a
short-range component. By solving the Poisson equation it is possible to self-consistently include the forces arising
from the bias to the system, while this is a difficult issue to be addressed in MD simulations.
Currently there are two Poisson solvers implemented in BioMOCA based on the finite difference method. One uses
the pre-conditioned Conjugate Gradient scheme (pCG) and is used by default. The later is borrowed from an APBS
solver, which uses a V-multi-grid scheme. Other than the numerical approach to solve the Poisson equation, the main
difference between the two solvers is on how they address the permittivity in the system. In the first solver, a
dielectric value is assigned to each cell in the grid, while in the APBS solver the dielectric coefficients are defined on
the grid nodes. As discussed earlier box integration method is used in the pCG solver, which allows us to treat the
Poisson equation in the most accurate way. Even though a full multigrid solver based on box-integration method has
been under development, there is a neat way to reuse the already exiting code and treat the ion channel systems.
Ion channel simulations require the presence of large bath regions for accurate treatment of screening
. There
being of such bath regions make the mesh domain of Poisson equation large and leads to either a large number of
grid points with fine mesh resolution or a small number of grid points with very coarse discretization. From bulk
simulations a coarse mesh is sufficient for describing the baths using the P
M scheme. However, a fine resolution is
required in the channel domain because of the highly charged nature of these regions and the presence of spatially
varying dielectric regions. Besides we are ultimately interested to study the channel behavior in terms of ion
permeability, selectivity, gating, density, etc… In other words, we are better off putting more computational
resources in the channel region, and bare minimum in the baths to reduce the overall computational cost and speed
up our simulations from weeks to perhaps days instead. A scheme based on the grid focusing method has been
developed that makes it possible to satisfy the requirement of large bath region and a fine grid resolution in channel
at the same time in a computationally effective way. This methodology also allows us to have multiple fine mesh
domains, which may be needed to describe multiple pore channels like OmpF porin, or an array of ion channels
sharing the same bath regions or even having yet finer meshes inside a fine mesh for relatively large channels with
narrow ion passages like Nicotine receptor channel.
The first grid is coarse mesh spanning the entire problem domain including the bath regions and the channel region.
The second grid (and so on for any other grids, 3
, 4
, etc) is a relatively much finer mesh that spans a sub-domain
of the system containing the region that requires fine resolution like the channel pore. The Poisson equation is first
solved on the coarse mesh with all the Dirichlet and Neumann boundary conditions, taking into account the applied
bias. Next the boundary conditions for the secondary meshes are obtained by interpolating from the first or previous
Biology Monte Carlo method
solutions of the Poisson equation. The Poisson equation is solved again for the finer meshes using the new boundary
conditions. In this way, electrostatic fields with different mesh discretization for different regions can be generated.
The electro-motive-force (EMF) is the measurement of the energy needed for a charged particle like ion to cross the
ion channel embedded in a membrane. Part of this potential energy barrier is due the interaction between the crossing
ion and the permanent/partial charges on the protein residues. The other part comes from the induced dipoles in the
protein/membrane dielectric medium, and is referred as dielectric-boundary-force (DBF). To compute the DBF
alone, one may turn off all the static charges on the protein residues and drag the ion through the pore and compute
the energy barrier using
It is important to note that EMF or DBF measurements are just qualitative measurements, as an ion does not
necessarily cross the channel through the center of its lumen in a straight line and it is often accompanied by other
ions moving in the same or opposite directions, which dramatically changes the dynamics of the system. Moreover,
unlike steered MD calculations where the protein residues dynamically reposition themselves as an ion or ions are
bouncing across the channel, in our EMF or DBF calculations protein is modeled as a static continuum, which
further affects the energy calculations in a more quantitative way. Another issue that additionally impacts the
measurements is absence of water hydration molecules, which move with the ion and shield part of its charge.
Having said all of above, still computing EMF or DBF is valuable to address channel selectivity or gating.
Computing either of these two energy barriers is available as an option in BioMOCA.
Visualization using VMD
VMD visualization of Gramicidin 1MAG
molecule along with the structure generated by
BioMOCA, where green represents protein, red
addresses the membrane (i.e. lipid), and purple is
the channel and left and right baths
was equipped with the option of loading BioMOCA
structures. This is a very useful feature as one could load both the
protein structure (i.e. PDB or PQR file) along with the structures
generated by BioMOCA to make comparisons. Figure at the right
shows how BioMOCA has generated a structure for Gramicidin
channel with a membrane wrapped around it. Furthermore, BioMOCA
also dumps the ion trajectories in standard formats so they could be
later loaded to molecular visualization tools such as VMD and watched
frame by frame in a movie format.
Recording trajectories in binary
Other than counting the number of ions crossing the channel, sometimes it is desirable to study their behavior at
different regions of the channel. Such examples would be the average occupancy of ions or their average moving
velocity inside the channel or a nanopore. BioMOCA has been equipped with the option of dumping every ions
position, average and instantaneous velocities, potential and kinetic energies, average and instantaneous
displacements and other info at every step (or few steps) of the simulations in ASCII format, so such trajectory
information could be studied later on to gather further statistics. From a technical point of view however, dumping
such information for tens of ions, even at every few hundreds of time steps, could slow down the simulations and
end up with huge files accumulating to tens of gigabytes. Loading such files later on from disk storage is also a very
time consuming and computationally inefficient procedure. Over and above that, recoding the numerical information
in ASCII format does not hold its machine precision and has loss of accuracy.
Biology Monte Carlo method
Solving such problems is actually an easy task and it is simply to avoid using ASCII format and use binary format
instead. Not only it preserves the machine accuracy but also writing and reading to file system is a lot faster. The
computational overhead to dump the trajectories becomes negligible and the trajectory files become about two orders
of magnitude smaller in size. The downside might be that programming and decoding the data could become very
tricky, but once it’s done correctly and with care, the advantages of using binary format are well worth the extra
effort. BioMOCA is now equipped with the tools to record the trajectory information in binary format.
Simulation tool
BioMOCA Suite
BioMOCA has been wrapped in a GUI, and it is available at nanoHUB.org as the BioMOCA Suite
. The
BioMOCA Suite can perform ion channel flow simulations on any user-supplied channel. The suite includes: a map
generator subtool, which produces protein maps for BioMOCA from the supplied PQR file; a lipid wrapper subtool,
which allows the user to embed their channel in a membrane; and the boundary force potential calculator, which
determines the potential energy barrier presented by the channel. The user can also download the acc and charge files
produced by the map generator and lipid wrapper.
Finally, the suite contains the biology Monte Carlo simulator, which simulates ion channel flow through the user
provided channel. The user has the ability to change a number of parameters, including the transmembrane voltage,
intra- and extra-cellular concentrations of Na
, Cl

, K
, Ca
, and Mg
, and the run time.

Biology Monte Carlo method
[1] T.A. van der Straaten, G. Kathawala, A. Trellakis, R.S. Eisenberg, and U. Ravaioli, Molecular Simulation, 31, 151 (2005)
[2] C. Jacoboni, P. Lugli, The Monte Carlo Method for Semiconductor Device Simulation, Springer Verlag, New York (1989)
[3] R. Hockney, J. Eastwood, Computer Simulation Using Particles, McGraw-Hill, New York (1981)
[4] A. Warshel, S.T. Russell, Q. Rev. Biol., 17, 283 (1984)
[5] C.N. Schutz, A. Warshel, Proteins, 44, 400 (2001)
[6] A. Warshel, A. Papazyan, Curr Opin Struct Biol., 8, 211 (1998)
[7] B. Roux, T. Allen, S. Berneche, W. Im, Q. Rev. Biophys., 37, 15 (2004)
[8] S. Selberherr, Analysis and Simulation of Semiconductor Devices, New York, Springer-Verlag Wein, (1984)
[9] N.A. Baker, D. Sept, M.J. Holst, J.A. McCammon, IBM J. Res. Develop., 45, 427 (2001)
[10] W. Im, B. Roux, J. Mol. Biol., 322, 851 (2002)
[11] T. A. van der Straaten, J. M. Tang, U. Ravaioli, R. S. Eisenberg and N. Aluru, J. Comp. Elect. 2, 29 (2003)
[12] http:/ / www. ks. uiuc. edu/ Research/ vmd
[13] http:/ / www. nanohub. org/tools/ BMCsuite
External links
• Nano Hub (http:// nanohub.org/ )
• BioMOCA Suite (http:/ / nanohub. org/ resources/ BMCsuite/ )
• Fundamentals of cell and molecular biology (http:// nanohub. org/resources/ 8536/ )
• NCN Nano-Devices for Medicine and Biology (http:/ / nanohub. org/topics/
NCNNano-DevicesforMedicineandBiology/ )
Clock drift
Clock drift refers to several related phenomena where a clock does not run at the exact right speed compared to
another clock. That is, after some time the clock "drifts apart" from the other clock. This phenomenon is also used
for instance in computers to build random number generators. On the negative side, clock drift can be exploited by
timing attacks.
Clock drift in normal clocks
Normal clocks such as clocks at home and wristwatches usually drift compared to the actual time. This is why it is
necessary to reset them occasionally. Clocks often drift differently depending on their quality, the exact power they
get from the battery, the surrounding temperature and other environmental variables. Thus the same clock can have
different clock drift rates at different occasions.
Mechanical watches drift much more than quartz ones, but they are designed to drift ahead rather than behind, so that
the watch gains time, making it easier to set the time to the second with the hack (stop mechanism) function.
More advanced clocks and old mechanical clocks often have some kind of speed trimmer where one can adjust the
speed of the clock and thus reduce the clock drift. For instance, in pendulum clocks the clock drift can be
manipulated by slightly changing the length of the pendulum.
Clock drift
Atomic clocks
Atomic clocks are very precise and have nearly no clock drift. The rotation of the Earth itself actually has much
more clock drift (less accuracy) than modern atomic clocks. Thus to keep the Coordinated Universal Time (UTC) in
line with the Earth's rotation, a leap second is added to some years.
As Einstein predicted, relativistic effects can also cause clock drift due to time dilation. This is because there is no
fixed universal time, time being relative to the observer. Special relativity describes how two clocks held by people
in different inertial frames (i.e. moving with respect to each other but not accelerating or decelerating) will each
appear to tick more slowly to the other person.
In addition to this, general relativity gives us gravitational time dilation. Briefly, a clock in a higher gravitational
field (e.g. closer to a planet) will appear to tick more slowly. People holding these clocks would agree on which
clock appeared to be going faster.
Note that it is time itself rather than the function of the clock which is affected. Both effects have been
experimentally observed.
Time dilation is of practical importance. For instance, the clocks in GPS satellites experience this effect due to the
reduced gravity they experience (making their clocks appear to run more quickly than those on Earth) and must
therefore incorporate relativistically corrected calculations when reporting locations to users. If general relativity
were not accounted for, a navigational fix based on the GPS satellites would be false after only 2 minutes, and errors
in global positions would continue to accumulate at a rate of about 10 kilometers each day.
Random number generators
Computer programs often need high quality random numbers, especially for cryptography. There are several similar
ways clock drift can be used to build random number generators (RNGs).
One way to build a hardware random number generator is to use two independent clock crystals, one that for
instance ticks 100 times per second and one that ticks 1 million times per second. On average the faster crystal will
then tick 10,000 times for each time the slower one ticks. But since clock crystals are not precise, the exact number
of ticks will vary. That variation can be used to create random bits. For instance, if the number of fast ticks is even, a
0 is chosen, and if the number of ticks is odd, a 1 is chosen. Thus such a 100/1000000 RNG circuit can produce 100
somewhat random bits per second. Typically such a system is biased—it might for instance produce more zeros than
ones—and so hundreds of somewhat random bits are "whitened" to produce a few unbiased bits.
There is also a similar way to build a kind of "software random number generator". This involves comparing the
timer tick of the operating system (the tick that usually is 100–1000 times per second) and the speed of the CPU. If
the OS timer and the CPU run on two independent clock crystals the situation is ideal and more or less the same as
the previous example. But even if they both use the same clock crystal the process/program that does the clock drift
measurement is "disturbed" by many more or less unpredictable events in the CPU such as interrupts and other
processes and programs that runs at the same time. Thus the measurement will still produce fairly good random
numbers. Some argue they are then not true random numbers but they seem to be good enough for most needs.
Note that most hardware random number generators such as the ones described above are fairly slow. Therefore most
programs only use them to create a good seed that they then feed to a pseudorandom number generator or a
cryptographically secure pseudorandom number generator to produce many random numbers fast.
Clock drift
Timing attack
In 2006, a side channel attack was published
that exploited clock skew based on CPU heating. The attacker causes
heavy CPU load on a pseudonymous server, causing CPU heating. CPU heating is correlated with clock skew, which
can be detected by observing timestamps (under the server's real identity).
[1] Pogge, Richard W.; “Real-World Relativity: The GPS Navigation System” (http:// www. astronomy. ohio-state. edu/ ~pogge/ Ast162/ Unit5/
gps. html) Accessed 4 January 2010.
[2] Steven J. Murdoch. Hot or Not: Revealing Hidden Services by their Clock Skew, ACM CCS 2006. (pdf) (http:// www. cl.cam. ac.uk/
~sjm217/ papers/ ccs06hotornot. pdf)
Control variates
The control variates method is a variance reduction technique used in Monte Carlo methods. It exploits information
about the errors in estimates of known quantities to reduce the error of an estimate of an unknown quantity.
Underlying principle
Let the parameter of interest be , and assume we have a statistic such that . Suppose we
calculate another statistic such that is a known value. Then
is also an unbiased estimator for for any choice of the coefficient . The variance of the resulting estimator
It can be shown that choosing the optimal coefficient
minimizes the variance of , and that with this choice,
hence, the term variance reduction. The greater the value of , the greater the variance reduction achieved.
In the case that , , and/or are unknown, they can be estimated across the Monte Carlo
replicates. This is equivalent to solving a certain least squares system; therefore this technique is also known as
regression sampling.
Control variates
We would like to estimate
The exact result is . Using Monte Carlo integration, this integral can be seen as the
expected value of , where
and U follows a uniform distribution [0, 1]. Using a sample of size n denote the points in the sample as
. Then the estimate is given by
If we introduce as a control variate with a known expected value
Using realizations and an estimated optimal coefficient we obtain the following results
Estimate Variance
Classical estimate 0.69475 0.01947
Control variates 0.69295 0.00060
The variance was significantly reduced after using the control variates technique.
[1] Glasserman, P. (2004). Monte Carlo Methods in Financial Engineering (Stochastic Modelling and Applied Probability) (1 ed.). New York:
Springer., p. 185.
• Ross, Sheldon M. (2002) Simulation 3rd edition ISBN 978-0125980531
• Averill M. Law & W. David Kelton (2000), Simulation Modeling and Analysis, 3rd edition. ISBN 0-07-116537-1
• S. P. Meyn (2007) Control Techniques for Complex Networks, Cambridge University Press. ISBN
9780521884419. Downloadable draft (https:// netfiles. uiuc.edu/ meyn/ www/ spm_files/ CTCN/ CTCN. html)
(Section 11.4: Control variates and shadow functions)
Determinism is the general philosophical thesis that states that for everything that happens there are conditions such
that, given them, nothing else could happen. The several versions of this thesis rest upon various alleged
connections, and interdependencies of things and events, asserting that these hold without exception. There have
been many versions of deterministic theories in the history of philosophy, springing from diverse motives and
considerations; some of which overlap considerably. These should be considered in the light of their historical
significance, together with certain alternative theories that philosophers have proposed. There are five theories of
determinism to be considered, which can for convenience be called ethical determinism, logical determinism,
theological determinism, physical determinism, and psychological. Specifically, causal determinism should be
noted as the concept that events within a given paradigm are bound by causality in such a way that any state (of an
object or event) is completely, or at least to some large degree, determined by prior states. In physics, this principle
is known as cause-and-effect.
Determinism is also the name of a broader philosophical view, which conjectures that every type of event, including
human cognition (behaviour, decision, and action) is causally determined by previous events.
In philosophical arguments, the concept of determinism in the domain of human action is often contrasted with free
will. The argument called indeterminism (otherwise "nondeterminism") negates deterministic causality as a factor
and opposes the deterministic argument.
Determinists believe any determined system is fully governed by causal laws resulting in only one possible state at
any point in time. A debate within determinism exists about the scope of determined systems, with some maintaining
that the entire universe is a single determinate system and others identifying other more limited determinate systems.
Within numerous historical debates, many varieties and philosophical positions on the subject of determinism exist,
most prominently the free will debates involving compatibilism and incompatibilism.
Predeterminism proposes there is an unbroken chain of prior occurrences stretching back to the origin of the
Determinism should not be confused with self-determination of human actions by reasons, motives, and desires, nor
with predestination, which specifically factors the possible existence of God into its tenets; moreover, determinism
explicitly does not suggest that prediction is possible, whatever the means- this is a separate, epistemological
Causal (or Nomological) determinism
generally assumes that every event has an antecedent cause in an infinite
causal chain going back to Aristotle's Prime Mover or the beginning of the universe. Determinists believe that there
is nothing uncaused or self-caused (causa sui). Research in quantum mechanics complicates this position further (see
'Arguments' section below). Such determinism is sometimes illustrated by the thought experiment of Laplace's
Many philosophical theories of determinism frame
themselves with the idea that reality follows a sort of
predetermined path
Logical determinism or Determinateness is the notion that all
propositions, whether about the past, present, or future, are either
true or false. Note that one can support Causal Determinism
without necessarily supporting Logical Determinism (depending
on one's views on the nature of time) and vice versa. The problem
of free will is especially salient now with Logical Determinism:
how can choices be free, given that propositions about the future
already have a truth value in the present (i.e. it is already
determined as either true or false)? This is referred to as the
problem of future contingents.
"Determinism" is often used to describe these two beliefs together:
a belief in Causal Determinism (that the universe follows natural,
causal laws) along with Logical Determinism (belief that the
future is fixed).
This is sometimes called Hard determinism. Unlike Fatalism (described below) which explicitly
denies that human choices affect their future, Hard Determinism allows that the future is fixed because of the choices
humans will make.
Often synonymous with Logical Determinism are the ideas behind Spatio-temporal Determinism or Eternalism: the
view of special relativity. J. J. C. Smart, a proponent of this view, uses the term "tenselessness" to describe the
simultaneous existence of past, present, and future. In physics, the "block universe" of Hermann Minkowski and
Albert Einstein assumes that time is simply a fourth dimension that already exists (like the three spatial dimensions).
In other words, all the other parts of time are real, just like the city blocks up and down one's street, although we only
ever perceive one part of time.
Historical determinism is the stance in explaining history, or advocating a political position, that events are
historically predetermined (and/or currently constrained) by various forces. Historical Determinism is synonymous
with Causal Determinism. It is associated with the dialectical idealism of G.W.F. Hegel.
Necessitarianism is a metaphysical principle that denies all mere possibility; there is exactly one way for the world to
be. Leucippus claimed there were no uncaused events, and that everything occurs for a reason and by necessity. The
focus, then, is on a sort of teleology. This view is similar, and often synonymous, with fatalism.
Fatalism is the
simple idea that everything is fated to happen, so that humans have no control over their future. Notice that fate has
arbitrary power. Fate also need not follow any causal or otherwise deterministic laws.
Even without a full understanding of
microscopic physics, we can predict
the distribution of 1000 coin tosses
Theological determinism or predestination is the concept that there is a God
who determines all that humans will do, either by knowing their actions in
advance, via some form of omniscience
or by decreeing their actions in
The problem of free will, in this context, is the problem of how our
actions can be free if there is a being who has determined them for us ahead of
Adequate determinism is the thesis that quantum indeterminacy can be ignored
for most macroscopic events, since random quantum events "average out" in the
limit of large numbers of particles, where the laws of quantum mechanics
asymptotically approach the laws of classical mechanics.
Determined by nature or nurture
Looking at a sculpture after some time, one does
not ask whether we are seeing the effects of the
starting materials OR environmental influences -
the two interact
Some deterministic theories frame themselves as an answer to the
Nature or Nurture debate, suggesting that one will entirely determine
behaviour. As scientific understanding has grown, the strongest
versions of these theories have been widely rejected as a single cause
In other words, the modern deterministic theories attempt to explain
how the interaction of both nature and nurture is entirely predictable.
The concept of heritability has been helpful to make this distinction.
Biological determinism, sometimes called Genetic determinism, is the
idea that each of our behaviors, beliefs, and desires are fixed by our
genetic nature.
Behaviorism is the idea that all behavior can be traced to specific
causes—either environmental or reflexive. This Nurture-focused
determinism was developed by John B. Watson and B. F. Skinner.
Cultural determinism or social determinism is the nurture-focused
theory that it is the culture in which we are raised that determines who
we are.
Environmental determinism is also known as climatic or geographical
determinism. It holds the view that the physical environment, rather than social conditions, determines culture.
Supporters often also support Behavioral determinism. Key proponents of this notion have included Ellen Churchill
Semple, Ellsworth Huntington, Thomas Griffith Taylor and possibly Jared Diamond, although his status as an
environmental determinist is debated.
Factor priority
A technological determinist might
suggest that technology, like the
mobile phone, is the greatest factor
shaping human civilization.
Other 'deterministic' theories actually seek only to highlight the importance of a
particular factor in predicting the future. These theories often use the factor as a
sort of guide or constraint on the future. They need not suppose that complete
knowledge of that one factor would allow us to make perfect predictions.
Psychological determinism can mean that humans must act according to reason,
but it can also be synonymous with some sort of Psychological egoism. The latter
is the view that humans will always act according to their perceived best interest.
Linguistic determinism claims that our language determines (at least limits) the
things we can think and say and thus know. The Sapir–Whorf hypothesis argues
that individuals experience the world based on the grammatical structures they
habitually use.
Economic determinism is the theory which attributes primacy to the economic
structure over politics in the development of human history. It is associated with
the dialectical materialism of Karl Marx.
Technological determinism is a reductionist theory that presumes that a society's technology drives the development
of its social structure and cultural values. Media determinism, a subset of technological determinism, is a
philosophical and sociological position which posits the power of the media to impact society. Two leading media
determinists are the Canadian scholars Harold Innis and Marshall McLuhan.
Free will and determinism
A table showing the different positions
related to free will and determinism
Philosophers have argued that either Determinism is true or Indeterminism
is true, but also that 'Free will' either exists or it does not. This creates four
possible positions. Compatibilism refers to the view that free will is, in
some sense, compatible with Determinism. The three 'Incompatibilist'
positions, on the other hand, deny this possibility. They instead suggest
there is a dichotomy between determinism and free will (only one can be
To the Incompatibilists, one must choose either free will or Determinism,
and maybe even reject both. The result is one of three positions:
• Metaphysical Libertarianism (free will, and no determinism) a position not to be confused with the more
commonly cited Political Libertarianism
• Hard Determinism (Determinism, and no free will)
• Hard Indeterminism (No Determinism, and no free will either).
Thus, although many Determinists are Compatibilists, calling someone a 'Determinist' is often done to denote the
'Hard Determinist' position.
The Standard argument against free will, according to philosopher J. J. C. Smart focuses on the implications of
Determinism for 'free will'.
He suggests that, if determinism is true, all our actions are predicted and we are not
free; if determinism is false, our actions are random and still we do not seem free.
In his book, The Moral Landscape, author and neuroscientist Sam Harris mentions some ways that determinism and
modern scientific understanding might challenge the idea of a contra-causal free will. He offers one thought
experiment where a mad scientist represents determinism. In Harris' example, the mad scientist uses a machine to
control all the desires, and thus all the behaviour, of a particular human. Harris believes that it is no longer as
tempting, in this case, to say the victim has "free will". Harris says nothing changes if the machine controls desires at
random - the victim still seems to lack free will. Harris then argues that we are also the victims of such unpredictable
desires (but due to the unconscious machinations of our brain, rather than those of a mad scientist). Based on this
introspection, he writes "This discloses the real mysery of free will: if our experience is compatible with its utter
abscence, how can we say that we see any evidence for it in the first place?"
adding that "Whether they are
predictable or not, we do not cause our causes."
That is, he believes there is compelling evidence of absence of
free will.
Research has found that reducing a person's belief in free will risks making them less helpful and more aggressive
; this could occur because the individual's sense of Self-efficacy suffers.
Some determinists argue that materialism does not present a complete understanding of the universe, because while
it can describe determinate interactions among material things, it ignores the minds or souls of conscious beings.
A number of positions can be delineated:
1. Immaterial souls are all that exist (Idealism).
2. Immaterial souls exist and exert a non-deterministic causal influence on bodies. (Traditional free-will,
interactionist dualism).

3. Immaterial souls exist, but are part of deterministic framework.
4. Immaterial souls exist, but exert no causal influence, free or determined (epiphenomenalism, occasionalism)
5. Immaterial souls do not exist — there is no mind-body dichotomy, and there is a Materialistic explanation for
intuitions to the contrary.
Another topic of debate is the implication that Determinism has on morality. Hard determinism is particularly
criticized for making moral judgements impossible.
Some of the main philosophers who have dealt with this issue are Marcus Aurelius, Omar Khayyám, Thomas
Hobbes, Baruch Spinoza, Gottfried Leibniz, David Hume, Baron d'Holbach (Paul Heinrich Dietrich), Pierre-Simon
Laplace, Arthur Schopenhauer, William James, Friedrich Nietzsche, Albert Einstein, Niels Bohr, and, more recently,
John Searle, Ted Honderich, and Daniel Dennett.
Mecca Chiesa notes that the probabilistic or selectionistic determinism of B.F. Skinner comprised a wholly separate
conception of determinism that was not mechanistic at all. Mechanistic determinism assumes that every event has an
unbroken chain of prior occurrences, but a selectionistic or probabilistic model does not.

Eastern tradition
The idea that the entire universe is a deterministic system has been articulated in both Eastern and non-Eastern
religion, philosophy, and literature.
A shifting flow of probabilities for futures lies at the heart of theories associated with the Yi Jing (or I Ching, the
Book of Changes). Probabilities take the center of the stage away from things and people. A kind of divine volition
sets the fundamental rules for the working out of probabilities in the universe, and human volitions are always a
factor in the ways that humans can deal with the real world situations one encounters.
The followers of the philosopher Mozi made some early discoveries in optics and other areas of physics, ideas that
were consonant with deterministic ideas.
In the philosophical schools of India, the concept of precise and continual effect of laws of Karma on the existence
of all sentient beings is analogous to western deterministic concept. Karma is the concept of "action" or "deed" in
Indian religions. It is understood as that which causes the entire cycle of cause and effect (i.e., the cycle called
saṃsāra) originating in ancient India and treated in Hindu, Jain, Sikh and Buddhist philosophies. Karma is
considered predetermined and deterministic in the universe, with the exception of a human, who through free will
can influence the future. See Karma in Hinduism.
Western tradition
In the West, the Ancient Greek atomists Leucippus and Democritus were the first to anticipate determinism when
they theorized that all processes in the world were due to the mechanical interplay of atoms, but this theory did not
gain much support at the time. Determinism in the West is often associated with Newtonian physics, which depicts
the physical matter of the universe as operating according to a set of fixed, knowable laws. The "billiard ball"
hypothesis, a product of Newtonian physics, argues that once the initial conditions of the universe have been
established, the rest of the history of the universe follows inevitably. If it were actually possible to have complete
knowledge of physical matter and all of the laws governing that matter at any one time, then it would be theoretically
possible to compute the time and place of every event that will ever occur (Laplace's demon). In this sense, the basic
particles of the universe operate in the same fashion as the rolling balls on a billiard table, moving and striking each
other in predictable ways to produce predictable results.
Whether or not it is all-encompassing in so doing, Newtonian mechanics deals only with caused events, e.g.: If an
object begins in a known position and is hit dead on by an object with some known velocity, then it will be pushed
straight toward another predictable point. If it goes somewhere else, the Newtonians argue, one must question one's
measurements of the original position of the object, the exact direction of the striking object, gravitational or other
fields that were inadvertently ignored, etc. Then, they maintain, repeated experiments and improvements in accuracy
will always bring one's observations closer to the theoretically predicted results. When dealing with situations on an
ordinary human scale, Newtonian physics has been so enormously successful that it has no competition. But it fails
spectacularly as velocities become some substantial fraction of the speed of light and when interactions at the atomic
scale are studied. Before the discovery of quantum effects and other challenges to Newtonian physics, "uncertainty"
was always a term that applied to the accuracy of human knowledge about causes and effects, and not to the causes
and effects themselves.
Newtonian mechanics as well as any following physical theories are results of observations and experiments, and so
they describe "how it all works" within a tolerance. However, old western scientists believed if there are any logical
connections found between an observed cause and effect, there must be also some absolute natural laws behind.
Belief in perfect natural laws driving everything, instead of just describing what we should expect, led to searching
for a set of universal simple laws that rule the world. This movement significantly encouraged deterministic views in
western philosophy.
Modern perspectives
Cause and effect
Since the early twentieth century when astronomer Edwin Hubble first hypothesized that redshift shows the universe
is expanding, prevailing scientific opinion has been that the current state of the universe is the result of a process
described by the Big Bang. Many theists and deists claim that it therefore has a finite age, pointing out that
something cannot come from nothing (the definition of nothing, however, is problematic at the most arcane level of
physics). The big bang does not describe from where the compressed universe came; instead it leaves the question
open. Different astrophysicists hold different views about precisely how the universe originated (Cosmogony). The
philosophical argument here would be that the big bang triggered every single action, and possibly mental thought,
through the system of cause and effect.
Generative processes
Although it was once thought by scientists that any indeterminism in quantum mechanics occurred at too small a
scale to influence biological or neurological systems, there is evidence that nervous systems are indeterministic. It is
unclear what implications this has for free will given various possible reactions to the standard problem in the first
Certainly not all biologists grant determinism: Christof Koch argues against it, but in favour of free will
(making him a sort of libertarian) by appealing to Generative processes (emergence).
Some proponents of emergentist or generative philosophy, cognitive sciences and evolutionary psychology, argue
that free will does not exist.

They suggest instead that an illusion of free will is experienced due to the
generation of infinite behaviour from the interaction of finite-deterministic set of rules and parameters. Thus the
unpredictability of the emerging behaviour from deterministic processes leads to a perception of free will, even
though free will as an ontological entity does not exist.

Certain experiments looking at the Neuroscience of
free will can be said to support this possibility.
In Conway's Game of Life, the
interaction of just 4 simple rules
creates patterns that seem
somehow "alive".
As an illustration, the strategy board-games chess and Go have rigorous rules in
which no information (such as cards' face-values) is hidden from either player and
no random events (such as dice-rolling) happen within the game. Yet, chess and
especially Go with its extremely simple deterministic rules, can still have an
extremely large number of unpredictable moves. By this analogy, it is suggested, the
experience of free will emerges from the interaction of finite rules and deterministic
parameters that generate nearly infinite and practically unpredictable behaviour. In
theory, if all these events were accounted for, and there were a known way to
evaluate these events, the seemingly unpredictable behaviour would become

Another hands on example of generative processes is John
Horton Conway's playable Game of Life.
Mathematical models
Many mathematical models of physical systems are deterministic. This is true of most models involving differential
equations (notably, those measuring rate of change over time). Mathematical models that are not deterministic
because they involve randomness are called stochastic. Because of sensitive dependence on initial conditions, some
deterministic models may appear to behave non-deterministically; in such cases, a deterministic interpretation of the
model may not be useful due to numerical instability and a finite amount of precision in measurement. Such
considerations can motivate the consideration of a stochastic model even though the underlying system is governed
by deterministic equations.


Compatibilism is the acceptance of both Free Will and Determinism. The negation of determinism is called
Quantum mechanics and classical physics
Since the beginning of the 20th century, quantum mechanics has revealed previously concealed aspects of events.
Newtonian physics, taken in isolation rather than as an approximation to quantum mechanics, depicts a universe in
which objects move in perfectly determinative ways. At human scale levels of interaction, Newtonian mechanics
makes predictions that are agreed with, within the accuracy of measurement. Poorly designed and fabricated guns
and ammunition scatter their shots rather widely around the center of a target, and better guns produce tighter
patterns. Absolute knowledge of the forces accelerating a bullet should produce absolutely reliable predictions of its
path, or so it was thought. However, knowledge is never absolute in practice and the equations of Newtonian
mechanics can exhibit sensitive dependence on initial conditions, meaning small errors in knowledge of initial
conditions can result in arbitrarily large deviations from predicted behavior.
At atomic scales the paths of objects can only be predicted in a probabilistic way. The paths may not be exactly
specified in a full quantum description of the particles; "path" is a classical concept which quantum particles do not
exactly possess. The probability arises from the measurement of the perceived path of the particle. In some cases, a
quantum particle may trace an exact path, and the probability of finding the particles in that path is one. The
quantum development is at least as predictable as the classical motion, but it describes wave functions that cannot be
easily expressed in ordinary language. In double-slit experiments, photons are fired one-by-one through a double-slit
apparatus at a distant screen and do not arrive at any single point, nor do the photons arrive in a scattered pattern
analogous to bullets fired by a fixed gun at a distant target. Instead, the light arrives in varying concentrations at
widely separated points, and the distribution of its collisions with the target can be calculated reliably. In that sense
the behavior of light in this apparatus is deterministic, but there is no way to predict where in the resulting
interference pattern an individual photon will make its contribution (see Heisenberg Uncertainty Principle).
Some have argued
that, in addition to the conditions humans can observe and the laws we can deduce, there are
hidden factors or "hidden variables" that determine absolutely in which order photons reach the detector screen.
They argue that the course of the universe is absolutely determined, but that humans are screened from knowledge of
the determinative factors. So, they say, it only appears that things proceed in a merely probabilistically determinative
way. In actuality, they proceed in an absolutely deterministic way. Although matters are still subject to some
measure of dispute, quantum mechanics makes statistical predictions which would be violated if some local hidden
variables existed. There have been a number of experiments to verify those predictions, and so far they do not appear
to be violated, though many physicists believe better experiments are needed to conclusively settle the question. (See
Bell test experiments.) It is possible, however, to augment quantum mechanics with non-local hidden variables to
achieve a deterministic theory that is in agreement with experiment. An example is the Bohm interpretation of
quantum mechanics.
On the macro scale it can matter very much whether a bullet arrives at a specific point at a specific time; there are
analogous quantum events that have macro- as well as quantum-level consequences. It is easy to contrive situations
in which the arrival of an electron at a screen at a certain point and time would trigger one event and its arrival at
another point would trigger an entirely different event. (See Schrödinger's cat.)
Chaotic radioactivity is the next
explanatory challenge for physicists
supporting determinism
All uranium found on earth is thought to have been synthesized during a
supernova explosion that occurred roughly 5 billion years ago. Even before
the laws of quantum mechanics were developed to their present level, the
radioactivity of such elements has posed a challenge to determinism due to its
unpredictability. One gram of uranium-238, a commonly occurring
radioactive substance, contains some 2.5 x 10
atoms. Each of these atoms
are identical and indistinguishable according to all tests known to modern
science. Yet about 12600 times a second, one of the atoms in that gram will
decay, giving off an alpha particle. The challenge for determinism is to
explain why and when decay occurs, since it does not seem to depend on
external stimulus. Indeed, no extant theory of physics makes testable
predictions of exactly when any given atom will decay.
The time dependent Schrödinger equation gives the first time derivative of the quantum state. That is, it explicitly
and uniquely predicts the development of the wave function with time.
So if the wave function itself is reality (rather than probability of classical coordinates), quantum mechanics can be
said to be deterministic. Since we have no practical way of knowing the exact magnitudes, and especially the phases,
in a full quantum mechanical description of the causes of an observable event, this turns out to be philosophically
similar to the "hidden variable" doctrine.
According to some, quantum mechanics is more strongly ordered than Classical Mechanics, because while Classical
Mechanics is chaotic, quantum mechanics is not. For example, the classical problem of three bodies under a force
such as gravity is not integrable, while the quantum mechanical three body problem is tractable and integrable, using
the Faddeev Equations. This does not mean that quantum mechanics describes the world as more deterministic,
unless one already considers the wave function to be the true reality. Even so, this does not get rid of the
probabilities, because we can't do anything without using classical descriptions, but it assigns the probabilities to the
classical approximation, rather than to the quantum reality.
Asserting that quantum mechanics is deterministic by treating the wave function itself as reality implies a single
wave function for the entire universe, starting at the origin of the universe. Such a "wave function of everything"
would carry the probabilities of not just the world we know, but every other possible world that could have evolved.
For example, large voids in the distributions of galaxies are believed by many cosmologists to have originated in
quantum fluctuations during the big bang. (See cosmic inflation and primordial fluctuations.) If so, the "wave
function of everything" would carry the possibility that the region where our Milky Way galaxy is located could
have been a void and the Earth never existed at all. (See large-scale structure of the cosmos.)
First cause
Intrinsic to the debate concerning determinism is the issue of first cause. Deism, a philosophy articulated in the
seventeenth century, holds that the universe has been deterministic since creation, but ascribes the creation to a
metaphysical God or first cause outside of the chain of determinism. God may have begun the process, Deism
argues, but God has not influenced its progression. This perspective illustrates a puzzle underlying any conception of
Assume: All events have causes, and their causes are all prior events. There is no cycle of events such that an event
(possibly indirectly) causes itself.
The picture this gives us is that Event A
is preceded by A
, which is preceded by A
, and so forth.
Under these assumptions, two possibilities seem clear, and both of them question the validity of the original
(1) There is an event A
prior to which there was no other event that could serve as its cause.
(2) There is no event A
prior to which there was no other event, which means that we are presented with an
infinite series of causally related events, which is itself an event, and yet there is no cause for this infinite
series of events.
Under this analysis the original assumption must have something wrong with it. It can be fixed by admitting one
exception, a creation event (either the creation of the original event or events, or the creation of the infinite series of
events) that is itself not a caused event in the sense of the word "caused" used in the formulation of the original
assumption. Some agency, which many systems of thought call God, creates space, time, and the entities found in the
universe by means of some process that is analogous to causation but is not causation as we know it. This solution to
the original difficulty has led people to question whether there is any reason for there only being one divine
quasi-causal act, whether there have not been a number of events that have occurred outside the ordinary sequence of
events. Others might argue that this is simply redefining the question.
Another possibility is that the "last event" loops back to the "first event" causing an infinite loop. If you were to call
the Big Bang the first event, you would see the end of the Universe as the "last event". In theory, the end of the
Universe would be the cause of the beginning of the Universe. You would be left with an infinite loop of time with
no real beginning or end. This theory eliminates the need for a first cause, but does not explain why there should be a
loop in time.
Immanuel Kant carried forth this idea of Leibniz in his idea of transcendental relations, and as a result, this had
profound effects on later philosophical attempts to sort these issues out. His most influential immediate successor, a
strong critic whose ideas were yet strongly influenced by Kant, was Edmund Husserl, the developer of the school of
philosophy called phenomenology. But the central concern of that school was to elucidate not physics but the
grounding of information that physicists and others regard as empirical. In an indirect way, this train of investigation
appears to have contributed much to the philosophy of science called logical positivism and particularly to the
thought of members of the Vienna Circle, all of which have had much to say, at least indirectly, about ideas of
[1] http:/ / plato.stanford.edu/ entries/ incompatibilism-arguments/
[2] Causal Determinism (http:// plato. stanford.edu/ entries/ determinism-causal/ SEP,)
[3] Leucippus, Fragment 569 - from Fr. 2 Actius I, 25, 4
[4] Fischer, John Martin (1989) God, Foreknowledge and Freedom. Stanford, CA: Stanford University Press. ISBN 1-55786-857-3
[5] Watt, Montgomery (1948) Free-Will and Predestination in Early Islam. London:Luzac & Co.
[6] de Melo-Martín I (2005). "Firing up the nature/nurture controversy: bioethics and genetic determinism". J Med Ethics 31 (9): 526–30.
doi:10.1136/jme.2004.008417. PMC 1734214. PMID 16131554.
[7] Andrew, Sluyter. "Neo-Environmental Determinism, Intellectual Damage Control, and Nature/Society Science". Antipode 4 (35).
[8] J. J. C. Smart, "Free-Will, Praise and Blame,"Mind, July 1961, p.293-4.
[9] Sam Harris, The Moral Landscape (2010), pg.216, note102
[10] Sam Harris, The Moral Landscape (2010), pg.217, note109
[11] Baumeister RF, Masicampo EJ, Dewall CN. (2009). Prosocial benefits of feeling free: disbelief in free will increases aggression and reduces
helpfulness. Pers Soc Psychol Bull. 35(2):260-8. PMID 19141628 doi:10.1177/0146167208327217
[12] By 'soul' in the context of (1) is meant an autonomous immaterial agent that has the power to control the body but not to be controlled by the
body (this theory of determinism thus conceives of conscious agents in dualistic terms). Therefore the soul stands to the activities of the
individual agent's body as does the creator of the universe to the universe. The creator of the universe put in motion a deterministic system of
material entities that would, if left to themselves, carry out the chain of events determined by ordinary causation. But the creator also provided
for souls that could exert a causal force analogous to the primordial causal force and alter outcomes in the physical universe via the acts of
their bodies. Thus, it emerges that no events in the physical universe are uncaused. Some are caused entirely by the original creative act and
the way it plays itself out through time, and some are caused by the acts of created souls. But those created souls were not created by means of
physical processes involving ordinary causation. They are another order of being entirely, gifted with the power to modify the original
creation. However, determinism is not necessarily limited to matter; it can encompass energy as well. The question of how these immaterial
entities can act upon material entities is deeply involved in what is generally known as the mind-body problem. It is a significant problem
which philosophers have not reached agreement about
[13] Free Will (Stanford Encyclopedia of Philosophy) (http:/ / plato. stanford.edu/ entries/ freewill/)
[14] Chiesa, Mecca (2004) Radical Behaviorism: The Philosophy & The Science.
[15] Ringen, J. D. (1993). Adaptation, teleology, and selection by consequences. Journal of Applied Behavior Analysis. 60,3–15. (http:/ / www.
pubmedcentral. nih.gov/ articlerender.fcgi?artid=1322142)
[16] Swartz, Norman (2003) The Concept of Physical Law / Chapter 10: Free Will and Determinism ( http:// www.sfu. ca/ philosophy/
[17] Lewis, E.R.; MacGregor, R.J. (2006). "On Indeterminism, Chaos, and Small Number Particle Systems in the Brain" (http:// www.eecs.
berkeley. edu/ ~lewis/ LewisMacGregor. pdf). Journal of Integrative Neuroscience 5 (2): 223–247. doi:10.1142/S0219635206001112. .
[18] Koch, Christof (September 2009). "Free Will, Physics, Biology and the Brain". In Murphy, Nancy; Ellis, George; O'Connor, Timothy.
Downward Causation and the Neurobiology of Free Will. New York, USA: Springer. ISBN 978-3642032042.
[19] Kenrick, D. T., Li, N. P., & Butner, J. 2003; Nowak A., Vallacher R.R., Tesser A., Borkowski W., 2000;
[20] Epstein J.M. and Axtell R. 1996; Epstein J.M. 1999
[21] John Conway's Game of Life (http:// www. bitstorm. org/gameoflife/)
[22] Werndl, Charlotte (2009). Are Deterministic Descriptions and Indeterministic Descriptions Observationally Equivalent? (http:// dx. doi.
org/ 10. 1016/ j.shpsb. 2009. 06. 004). Studies in History and Philosophy of Modern Physics 40, 232-242.
[23] Werndl, Charlotte (2009). Deterministic Versus Indeterministic Descriptions: Not That Different After All? (http:/ / philsci-archive.pitt. edu/
archive/00004775/). In: A. Hieke and H. Leitgeb (eds), Reduction, Abstraction, Analysis, Proceedings of the 31st International Ludwig
Wittgenstein-Symposium. Ontos, 63-78.
[24] J. Glimm, D. Sharp, Stochastic Differential Equations: Selected Applications in Continuum Physics, in: R.A. Carmona, B. Rozovskii (ed.)
Stochastic Partial Differential Equations: Six Perspectives, American Mathematical Society (October 1998) (ISBN 0-8218-0806-0).
[25] Albert Einstein insisted that, "I am convinced God does not play dice" in a private letter to Max Born, 4 December 1926, Albert Einstein
Archives (http:/ /www.alberteinstein. info/ db/ ViewDetails. do?DocumentID=38009) reel 8, item 180
References and bibliography
• Daniel Dennett (2003) Freedom Evolves. Viking Penguin.
• John Earman (2007) "Aspects of Determinism in Modern Physics" in Butterfield, J., and Earman, J., eds.,
Philosophy of Physics, Part B. North Holland: 1369-1434.
• George Ellis (2005) "Physics and the Real World," Physics Today.
• Epstein J.M. (1999) "Agent Based Models and Generative Social Science," Complexity IV (5).
• -------- and Axtell R. (1996) Growing Artificial Societies — Social Science from the Bottom. MIT Press.
• Kenrick, D. T., Li, N. P., & Butner, J. (2003) "Dynamical evolutionary psychology: Individual decision rules and
emergent social norms," Psychological Review 110: 3–28.
• Albert Messiah, Quantum Mechanics, English translation by G. M. Temmer of Mécanique Quantique, 1966, John
Wiley and Sons, vol. I, chapter IV, section III.
• Nowak A., Vallacher R.R., Tesser A., Borkowski W., (2000) "Society of Self: The emergence of collective
properties in self-structure," Psychological Review 107.
• Schimbera, Jürgen / Schimbera, Peter: Determination des Indeterminierten. Kritische Anmerkungen zur
Determinismus- und Freiheitskontroverse. Verlag Dr. Kovac, Hamburg , ISBN 978-3-8300-5099-5.
External links
• Hard determinism proved (http:// www. proofoffate.com)
• Stanford Encyclopedia of Philosophy entry on Causal Determinism (http:/ / plato. stanford.edu/ entries/
determinism-causal/ )
• Determinism in History (http:// etext. lib. virginia.edu/ cgi-local/ DHI/dhi. cgi?id=dv2-02) from the Dictionary
of the History of Ideas
• Philosopher Ted Honderich's Determinism web resource (http:/ / www.ucl.ac. uk/ ~uctytho/ dfwIntroIndex.htm)
• Determinism on Information Philosopher (http:/ / www.informationphilosopher.com/ freedom/determinism.
• An Introduction to Free Will and Determinism (http:/ / www.galilean-library.org/int13. html) by Paul Newall,
aimed at beginners.
• The Society of Natural Science (http:/ / www. determinism. com)
• Determinism and Free Will in Judaism (http:/ / www.chabad.org/article. asp?AID=3017)
• Snooker, Pool, and Determinism (http:// www. jottings. ca/ john/ cogitations. html)
In Unix-like operating systems, /dev/random is a special file that serves as a random number generator or as a
pseudorandom number generator. It allows access to environmental noise collected from device drivers and other
sources. Not all operating systems implement the same semantics for /dev/random. Linux was the first operating
system to implement a true random number generator in this way.
Random number generation from kernel space was implemented for the first time for Linux
in 1994 by Theodore
The implementation uses secure hashes rather than ciphers, as required to avoid legal restrictions that were in
place when the generator was originally designed. The implementation was also designed with the assumption that
any given hash or cipher might eventually be found to be weak, and so the design is durable in the face of any such
weaknesses. Fast recovery from pool compromise is not considered a requirement, because the requirements for pool
compromise are sufficient for much easier and more direct attacks on unrelated parts of the operating system.
In this implementation, the generator keeps an estimate of the number of bits of noise in the entropy pool. From this
entropy pool random numbers are created. When read, the /dev/random device will only return random bytes within
the estimated number of bits of noise in the entropy pool. /dev/random should be suitable for uses that need very
high quality randomness such as one-time pad or key generation. When the entropy pool is empty, reads from
/dev/random will block until additional environmental noise is gathered.
The intent is to serve as a
cryptographically secure pseudorandom number generator, delivering output with entropy as large as possible. This
is suggested for use in generating cryptographic keys for high-value or long-term protection.
A counterpart to /dev/random is /dev/urandom ("unlocked"/non-blocking random source
) which reuses the
internal pool to produce more pseudo-random bits. This means that the call will not block, but the output may
contain less entropy than the corresponding read from /dev/random. While it is still intended as a pseudorandom
number generator suitable for most cryptographic purposes, it is not recommended for the generation of long-term
cryptographic keys.
It is also possible to write to /dev/random. This allows any user to mix random data into the pool. Non-random data
is harmless, because only a privileged user can issue the ioctl needed to increase the entropy estimate. The current
amount of entropy and the size of the Linux kernel entropy pool are available in /proc/sys/kernel/random/.
Gutterman, Pinkas, & Reinman in March 2006 published a detailed cryptographic analysis of the Linux random
number generator
in which they describe several weaknesses. Perhaps the most severe issue they report is with
embedded or Live CD systems such as routers and diskless clients, for which the bootup state is predictable and the
available supply of entropy from the environment may be limited. For a system with non-volatile memory, they
recommend saving some state from the RNG at shutdown so that it can be included in the RNG state on the next
reboot. In the case of a router for which network traffic represents the primary available source of entropy, they note
that saving state across reboots "would require potential attackers to either eavesdrop on all network traffic" from
when the router is first put into service, or obtain direct access to the router's internal state. This issue, they note, is
particularly critical in the case of a wireless router whose network traffic can be captured from a distance, and which
may be using the RNG to generate keys for data encryption.
The FreeBSD operating system implements a 256-bit variant of the Yarrow algorithm, intended to provide a
cryptographically secure pseudorandom stream—this replaced a previous Linux style random device. Unlike the
Linux /dev/random, the FreeBSD /dev/random device never blocks. Its behavior is similar to the Linux
/dev/urandom, and /dev/urandom on FreeBSD is linked to /dev/random.
Yarrow is based on the assumptions that modern PRNGs are very secure if their internal state is unknown to an
attacker, and that they are better understood than the estimation of entropy. Whilst entropy pool based methods are
completely secure if implemented correctly, if they overestimate their entropy they may become less secure than
well-seeded PRNGs. In some cases an attacker may have a considerable amount of control over the entropy, for
example a diskless server may get almost all of it from the network—rendering it potentially vulnerable to
man-in-the-middle attacks. Yarrow places a lot of emphasis on avoiding any pool compromise and on recovering
from it as quickly as possible. It is regularly reseeded; on a system with small amount of network and disk activity,
this is done after a fraction of a second.
In 2004, Landon Curt Noll tested the FreeBSD 5.2.1 version of /dev/random and suggested that it was not
cryptographically secure because its output had multiple uniformity flaws.
Similar flaws were found in the Linux
2.4.21-20, Solaris 8 patch 108528-18, and Mac OS X 10.3.5 implementations of /dev/random.
FreeBSD also provides support for hardware random number generators, which will replace Yarrow when present.
Other operating systems
/dev/random and /dev/urandom are also available on Solaris
, Mac OS X
, NetBSD
, OpenBSD
, Tru64
, AIX 5.2
, and HP-UX 11i v2
. As with FreeBSD, AIX implements its own Yarrow-based
design, however AIX uses considerably fewer entropy sources than the standard /dev/random implementation and
stops refilling the pool when it thinks it contains enough entropy.
In Windows NT, similar functionality is delivered by ksecdd.sys, but reading the special file \Device\KsecDD does
not work as in UNIX. The documented methods to generate cryptographically random bytes are CryptGenRandom
and RtlGenRandom.
While DOS doesn't naturally provide such functionality there is an open source third-party driver called Noise.sys
which functions similarly in that it creates 2 devices, RANDOM$ and URANDOM$, which are also accessible
as /DEV/RANDOM$ and /DEV/URANDOM$, that programs can access for random data.
EGD as an alternative
A software program called EGD (entropy gathering daemon) is a common alternative for Unix systems which do not
support the /dev/random device. It is a user space daemon which provides high quality cryptographic random data.
Some cryptographic software such as OpenSSL, GNU Privacy Guard, and the Apache HTTP Server support using
EGD when a /dev/random device is not available.
, or a compatible alternative such as prngd
, gather pseudo-random entropy from various sources,
process it to remove bias and improve cryptographic quality, and then make it available over a Unix domain socket
(with /dev/egd-pool being a common choice), or over a TCP socket. The entropy gathering usually entails
periodically forking subprocesses to query attributes of the system that are likely to be frequently changing and
unpredictable, such as monitoring CPU, I/O, and network usage as well as the contents of various log files and
temporary directories.
EGD communicates with other programs which need random data using a simple protocol. The client connects to an
EGD socket and sends a command, identified by the value of the first octet:
• command 0: query the amount of entropy currently available. The EGD daemon returns a 4-byte number in big
endian format representing the number of random bytes that can currently be satisfied without delay.
• command 1: get random bytes, no blocking. The second byte in the request tells EGD how many random bytes of
output it should return, from 1 to 255. If EGD does not have enough entropy to immediately satisfy the request,
fewer bytes, or perhaps no bytes may be returned. The first octet of the reply indicates how many additional bytes,
those containing the random data, immediately follow in the reply.
• command 2: get random bytes, blocking. The second byte tells EGD how many random bytes of output it should
return. If EGD does not have enough entropy, it will wait until it has gathered enough before responding. Unlike
command 1, the reply starts immediately with the random bytes rather than a length octet, as the total length of
returned data will not vary from the amount requested.
• command 3: update entropy. This command allows the client to provide additional entropy to be added to EGD's
internal pool. The next two bytes, interpreted as a 16-bit big endian integer indicate how many bits of randomness
the caller is claiming to be supplying. The fourth byte indicates how many additional bytes of source data follow
in the request. The EGD daemon may mix in the received entropy and will return nothing back.
[1] Jack Lloyd (December 9, 2008). "On Syllable's /dev/random" (http:// www.webcitation.org/5gOzG0qvc). Archived from the original (http:/
/www. randombit.net/ bitbashing/ security/ syllable_dev_random. html) on 2009-04-29. . Retrieved 2009-04-27.
[2] "/dev/random" (http:/ / www.webcitation. org/5gOzGLECz). everything2.com. June 8, 2003. Archived from the original (http://
everything2.com/title/ %2Fdev%2Frandom) on 2009-04-29. . Retrieved 2009-04-27.
[3] urandom(4) (http:/ / linux. die. net/ man/4/ urandom) – Linux Special Files Manual
[4] http:/ / www. kernel.org/ doc/ man-pages/ online/ pages/ man4/ random. 4.html
[5] Zvi Gutterman; Benny Pinkas, Tzachy Reinman (March 6, 2006). "Analysis of the Linux Random Number Generator" (http:// www. pinkas.
net/ PAPERS/ gpr06.pdf) (PDF). . Retrieved 2008-09-18.
[6] "How good is LavaRnd?: Detailed Description of Test Results and Conclusions" (http:/ / www.lavarnd.org/ what/ nist-test.html), LavaRnd
(LavaRnd), 22 Sep 2004, , retrieved 22 Dec. 2010
[7] http:/ / blogs. sun. com/ yenduri/ entry/dev_random_in_solaris
[8] http:/ / developer.apple. com/ mac/ library/documentation/ Darwin/ Reference/ManPages/ man4/ random. 4. html
[9] http:// netbsd. gw. com/ cgi-bin/man-cgi?rnd++ NetBSD-current
[10] http:/ / www. openbsd. org/cgi-bin/man. cgi?query=srandom&apropos=0& sektion=4& manpath=OpenBSD+ Current& arch=i386&
[11] http:/ / h30097. www3.hp. com/ docs/ base_doc/ DOCUMENTATION/V51B_HTML/ MAN/ MAN4/ 0199____. HTM
[12] http:// publib.boulder.ibm.com/ infocenter/pseries/ v5r3/topic/ com. ibm.aix.files/ doc/ aixfiles/ random. htm#idx927
[13] http:// software.hp. com/ portal/ swdepot/ displayProductInfo.do?productNumber=KRNG11I
[14] Iain Roberts (April 25, 2003). "AIX 5.2 /dev/random and /dev/urandom devices" (http:// lists. gnupg. org/ pipermail/gnupg-devel/
2003-April/019954. html). Lists.gnupg.org. . Retrieved 2008-09-18.
[15] http:// www. rahul. net/ dkaufman/ index. html
[16] http:/ / egd.sourceforge.net/
[17] http:/ / prngd.sourceforge.net/
• CryptGenRandom (http:/ / msdn2. microsoft.com/ en-us/ library/aa379942. aspx)
• RtlGenRandom (http:/ / msdn2. microsoft. com/ en-us/ library/aa387694. aspx)
• Biege, Thomas; Analysis of a strong Random Number Generator (http:/ / www.suse. de/ ~thomas/ papers/
random-analysis.pdf) Slides (http:// www.suse. de/ ~thomas/ papers/ 23c3-random-analysis.pdf)
Four coloured dice showing all six possible sides (on a standard
6-sided die) with pips
A die (plural dice, from Old French dé, from Latin
datum "something which is given or played"
) is a
small throwable object with multiple resting attitudes,
used for generating random numbers or other symbols.
This makes dice suitable as gambling devices,
especially for craps or sic bo, or for use in
non-gambling tabletop games.
A traditional die is a cube (often with corners slightly
rounded), with each of its six faces showing a different
number. The design as a whole is aimed at each die
providing one randomly determined integer, in the
range from one to six, with each of those values being
equally likely.
More generally, a variety of similar devices are often described as dice. Such specialized dice may have polyhedral
or irregular shapes, and may have faces marked with various symbols instead of numbers. They may be used to
produce random results other than one through six. There are also "loaded" or "crooked" dice, designed to favor
some results over others, for purposes of deception or amusement.
Bone die found at Cantonment Clinch (1823 –
1834), an American fort used in the American
Civil War by both Confederate and Union forces
at separate times. The fort was also used in 1898
in the Spanish-American War.
Dice have been used throughout Asia since before recorded history.
The oldest known dice were excavated as part of a 5000-year-old
backgammon set, at Shahr-i Sokhta, the Burnt City, an archeological
site in south-eastern Iran.
Excavations from ancient tombs in the Harappan civilization,
to further indicate a South Asian origin. Dicing is mentioned as an
Indian game in the Rig Veda, Atharva Veda
and Buddha games list.
It also plays a critical role in the great Hindu epic, the Mahabharata,
where Yudhisthira plays a game of dice against the Kauravas for the
northern kingdom of Hastinapura, which becomes the trigger for a
great war. There are several biblical references to "casting lots", as in
Psalm 22, indicating that it had become commonplace in the region as
of the time of King David. In its primitive form knucklebones was
essentially a game of skill played by women and children. In a derivative form of knucklebones, the four sides of the
bones received different values and were counted as with modern dice. Gambling with three or sometimes two dice
was a very popular form of amusement in Greece, especially with the upper classes, and was an almost invariable
accompaniment to symposia.
(However, the "dice game" mentioned in Rigveda book 10 hymn 34, which is about the harm that gambling
addiction causes, is played with about 150 nuts from a tree called Vibhīdaka (Terminalia bellerica).)
Knucklebones die, made of Steatite
A collection of historical dice from Asia
Dice were probably originally made from the ankle bones (specifically
the talus or "astragalus") of hoofed animals (such as oxen), colloquially
known as "knucklebones", which are approximately tetrahedral (hence
the slang term "bones" used for dice). Modern Mongolians still use
such bones, known as shagai, for games and fortunetelling. In addition
to bone, ivory, wood, metal, and stone materials have been commonly
used. Recently, the use of plastics, including cellulose acetate and
Bakelite, is nearly universal. It is almost impossible to trace clearly the
development of dice as distinguished from knucklebones, because
ancient writers confused the two. It is certain, however, that both were
used in prehistoric times.
The Romans were passionate gamblers, especially in the luxurious
days of the Roman Empire, and dicing was a favorite form, though it
was forbidden except during the Saturnalia. Horace derided what he
presented as a typical youth of the period, who wasted his time amid
the dangers of dicing instead of taming his charger and giving himself
up to the hardships of the chase. Throwing dice for money was the
cause of many special laws in Rome. One of these stated that no suit
could be brought by a person who allowed gambling in his house, even
if he had been cheated or assaulted. Professional gamblers were
common, and some of their loaded dice are preserved in museums. The
common public-houses were the resorts of gamblers, and a fresco is
extant showing two quarrelling dicers being ejected by the indignant
host. Twenty-sided dice date back to Roman times, as far back as 2nd
century AD.
Tacitus states that the Germans were passionately fond of dicing, so
much so, indeed, that, having lost everything, they would even stake
their personal liberty. Centuries later, during the Middle Ages, dicing became the favorite pastime of the knights, and
both dicing schools and guilds of dicers existed. After the downfall of feudalism the famous German mercenaries
called landsknechts established a reputation as the most notorious dicing gamblers of their time. Many of the dice of
the period were curiously carved in the images of men and beasts. In France both knights and ladies were given to
dicing. This persisted through repeated legislation, including interdictions on the part of St. Louis in 1254 and 1256.
In China, India, Japan, Korea, and other Asiatic countries, dice have always been popular and are so still. The
markings on Chinese dominoes evolved from the markings on dice, taken two at a time.
Ordinary dice
Western-style, Asian-style, and casino dice
Common dice are small cubes 1 to 3 cm along an edge (16 mm being
most common), whose faces are numbered from one to six (usually by
patterns of round dots called pips). Since classical antiquity
it is
traditional to arrange the numbers so that opposite faces add to seven; this
implies that the faces 1, 2 and 3 meet at a common vertex. This constraint
leaves one more abstract design choice: the faces representing 1, 2 and 3
respectively can be placed in either clockwise or counterclockwise order
about this vertex. If the 1, 2 and 3 faces run counterclockwise around their
common vertex, the die is called "right-handed"; if they run clockwise it is
called "left-handed". Standard modern Western dice are right-handed,
whereas Chinese dice are often left-handed.
The pips on traditional European dice are arranged in specific patterns.
These can be conveniently described by pairs: one pair of pips in opposite
corners; the next pair in the remaining corners; and a last pair in the
middles of two opposite sides. For odd numbers, a dot is added in the
center of the face; thus "three" is a diagonal row, and "five" a quincunx.
Asian-style dice bear similar patterns, but the pips are typically closer to the centre of the face; the "one" pip is larger
than the others; and the "one" and "four" pips are coloured red. In some older sets the "one" pip is a colorless
depression. It is suggested that an entirely black and white color combination on the "one" side would be unlucky
and red (a lucky color in Chinese culture) would counteract this. In some Asian languages, the word "four" sounds
similar to the word "death" and is considered unlucky. Several legends also mention that the "four" side is colored
red because a Chinese emperor (one legend said it was a Ming dynasty emperor, while another stated it was Chung
Tsung) ordered it as "fours" helped him win a dice game (sugoroku) against his empress. This story, however, is
questionable at best, as it is also probable that "red fours" are also of common Indian origin.

Another reason
why the "four" side might be colored red is because in Asian cultures, the number four is seen as unlucky, like the
number 13 in Western culture, and as mentioned before, it is colored red so that the luckiness of the red counteracts
the unluckiness of the four.
Typical facets of an Asian-style (top) and a
Western-style die (bottom). Note the
compactedness of the pips in the Asian-style die
compared to those of the Western-style one.
Dice are thrown to provide random numbers for gambling and other
games, and thus are a type of hardware random number generator. The
result of a die roll is random in the sense of lacking predictability, not
lacking cause. Exactly how dice are thrown determines how they will
land according to the laws of classical mechanics. However, dice also
can exhibit sensitive dependence on initial conditions, making it
difficult to predict the outcome of a die roll even with good
information about exactly how it is thrown. Some people claim that the
pips on the face of certain styles of dice can cause a small bias, but
there is no research to support this claim. The supposed bias is reduced somewhat in the Japanese die with its
oversized single pip (pictured). Casino dice have markings that are flush, offering the assurance that this brings them
very close to providing true uniformly distributed random numbers.
Dice are thrown, singly or in groups, from the hand or from a cup or box designed for the purpose, onto a flat
surface. The face of each die that is uppermost when it comes to rest provides the value of the throw. A typical dice
game today is craps, wherein two dice are thrown at a time, and wagers are made on the total value of up-facing pips
on the two dice. They are also frequently used to randomize allowable moves in board games, usually by deciding
the distance through which a piece will move along the board; examples of this are ludo and backgammon.
Precision dice
Precision casino dice, used for the game of craps, may have a polished finish, making them transparent, or a sand
finish, making them translucent. Casino dice have their pips drilled, and then filled flush with a paint of the same
density as the acetate used for the dice, such that the dice remain in balance. In casino play, a stick of 5 dice is used,
all stamped with a matching serial number to prevent potential cheaters from substituting a die.
Precision backgammon dice are also made with the pips filled in as with casino dice. While casino dice are
noticeably larger than common dice, with sharp edges and corners, precision backgammon dice tend to be slightly
smaller. Their corners and edges are rounded to allow better movement inside the dice cup and to stop chaotic rolls
from damaging the playing surface.
Computer generated dice
Some computer games, such as clones of board games, must use computer generated dice. The values are usually
determined by a pseudorandom number generator, then displayed as a visual representation of a die. The reverse is
also possible with bar coded dice shuffling as a source of true random data for computers.
While the terms ace, deuce, trey, cater, cinque and sice are hardly common today having been replaced with the
ordinary names of the numbers one to six, they are still used by some professional gamblers to describe the different
sides of the dice. Ace is from the Latin as, meaning "a unit";
the others are the numbers 2–6 in old French. (The
dice game marketed as Kismet uses ace, deuce, and trey.)
In many modern gaming contexts, most notably Tabletop RPGs, the count and number of sides of dice to be rolled at
any given time is reduced to a common set of notations. Typically this involves the lower-case letter "d", preceded
by a die count and followed by (optionally) the number of sides of the dice. For example, 6d8 or 2d6; the former
meaning "six eight-sided dice," and the latter meaning "two six-sided dice." Addition or various other arithmetic
operations are often added at the end as well, for example, 3d6+4 "three six-sided dice plus four to the outcome
Crooked dice
"Crooked dice" refers to dice that have been altered in some way to change the statistical distribution of their
Loaded dice
A loaded (also gaffed, cogged, weighted, crooked, gag or fixed) die is one that has been tampered with to land
with a selected side facing upwards more often than it otherwise would simply by chance. There are several methods
for creating loaded dice, including having some edges round and other sharp and slightly off square faces. If the dice
are not transparent, weights can be added to one side or the other. They can be modified to produce winners
("passers") or losers ("miss-outs"). "Tappers" have a drop of mercury in a reservoir at the center of the cube, with a
capillary tube leading to another mercury reservoir at the side of the cube. The load is activated by tapping the die on
the table so that the mercury leaves the center and travels to the side. Often one can see the circle of the cut used to
remove the face and bury the weight. In a professional die, the weight is inserted in manufacture; in the case of a
wooden die, this can be done by carving the die around a heavy inclusion, like a pebble around which a tree has
A variable loaded die is hollow with a small weight and a semi-solid substance inside, usually wax, whose melting
point is just lower than the temperature of the human body. This allows the cheater to change the loading of the die
by breathing on it or holding it firmly in hand, causing the wax to melt and the weight to drift down, making the
chosen opposite face more likely to land up. A less common type of variable die can be made by inserting a magnet
into the die and embedding a coil of wire in the game table. Then, either leave the current off and let the die roll
unchanged or run current through the coil to increase the likelihood that the north side or the south side will land on
the bottom depending on the direction of the current.
Transparent acetate dice, used in all reputable casinos, are harder to tamper with.
Cheat dice
Cheat dice (see below) are often sold as loaded dice but usually are not technically loaded.
Shaved dice
A die can be "shaved" on one side, that is, slightly shorter in one dimension, making it slightly off-square and thus
affecting its outcome. One countermeasure employed by casinos against shaved dice is to measure the dice with a
Iced dice
Iced dice have lead in them, making them land on the 6 side more often. The "ice" refers to the lead in the dice.
Heated dice
If a plastic die is heated in an oven for 10–15 minutes, with the desired face upward, the plastic will soften slightly
and "pool" at the opposite (bottom) side without showing much, if any, visible distortion.
Dice with faces other than digit sequences
As noted, the faces of most dice are labeled using an unbroken series of whole numbers, starting at one (rarely zero),
expressed with either pips or digits. Common exceptions include:
• color dice (for example, with the colors of the playing pieces used in a game)
• Poker dice, with labels reminiscent of playing cards. Several varieties exist, but the most common contain the
following pattern: 9♣, 10♦, Jack, Queen, King, A♠; the face cards have only colors attached to them and not
• dice with letters (for example, in Boggle)
• average dice (2, 3, 3, 4, 4, 5) (In some war games, units are identified as regulars or irregulars. Because regulars
are more predictable, the strength of a regular unit is multiplied by an average die. For this reason, average dice
are jocularly called regular dice.)
• extreme dice (1, 1, 2, 5, 6, 6) These are the opposite of average dice and can be used to represent chaotic or
berserker type troops in some wargames.
• cheat dice, such as:
• With its faces numbered 2 3 4 5 6 6
• for craps, a pair of dice in which one die has five on each face , and its mate has a mixture of twos and sixes
(or, in another scenario, an all-six die with another die that rolls one and five), guaranteeing rolls of seven or
• dice with a single sequence of markings repeated multiple times, for example:
• a cubical die numbered 1 2 3 1 2 3, or 1 2 1 2 1 2
• cubical dice numbered 0 1 2 0 1 2. Dice rolls with these dice have the same expected value as the number of
dice thrown.
• icosahedral dice numbered twice from 1 to 10 (commonly used in Dungeons & Dragons before the
popularization of ten-sided dice).
• Fudge dice with an equal amount of −, blank and +.
Backgammon Doubling
• random direction dice, also known as scatter dice. The dice have arrows on each side;
the outcome of a roll is a random direction. Scatter dice are used in tabletop
wargames such as Warhammer Fantasy Battle to determine random movements of
troops, wind direction or direction of misfired arms. Note that this is an unusual case
where the majority of the time the die is read not according to which symbol is shown
on its uppermost face, but its compass orientation.
• A doubling cube with the numbers 2, 4, 8, 16, 32, and 64 is used in backgammon and
some other boardgames. This die is not actually rolled; it is used to denote the current
stakes of the game. There is also a doubling octahedron with 1, 2, 4, 8, 16, 32, 64, and
• Some board games use dice with positive and negative numbers for use in gain or loss of something.
• Sicherman dice, a pair having the same odds of rolling a given sum as a pair of standard six-sided dice, but with
different markings: one die has 1, 3, 4, 5, 6, and 8, and the other has 1, 2, 2, 3, 3, and 4. Sicherman dice are the
only such alternative arrangement if positive whole numbers are used.
• I Ching dice such as
• Eight-sided dice bearing the eight trigrams
• Six-sided dice bearing yin and yang twice each, and old yin and old yang once each
• "Projector dice" which are clear and marked only on one of each pair of opposing faces. For a "six"-sided die, for
example, a clear 12-sided shape is used. Rolled on an overhead projector such a die will have the top or bottom
marking equally readable.
• Sex dice. The game generally uses two die, one for actions and one for body part. Instead of numbers, each face
on the die contains a word. When rolled, a random combination of words are formed to create a sexual instruction
such as "kiss lips" which a player must perform.
• Money Dice. Six-sided dice used for randomly generating coin values with facings: 1¢, 5¢, 10¢, 25¢, 50¢ and $,
representing the Cent, Nickel, Dime, Quarter, Half dollar and Dollar coins of the United States, Canada and other
countries with dollar-based economies and similar coinage denominations.
Non-cubical dice
Barrel Dice
10-sided die
Some dice are polyhedral other than cubical in shape. Both seven– and
eight-sided dice of modern format are stated in the 13th century Libro
de los juegos to have been invented by Alfonso X in order to speed up
play in chess variants.

In more recent times around the early 1950s, they have become
popular among players of wargames and have since been employed
extensively in role-playing games, German-style board games, and
trading card games. Although polyhedral dice are a relative novelty
during modern times, some ancient cultures appear to have used them
in games (as evidenced by the discovery of two icosahedral dice dating
from the days of ancient Rome, currently on display in the British
Museum). In modern times, such dice are typically plastic, and have
faces bearing numerals rather than patterns of dots. Reciprocally
symmetric numerals are distinguished with a dot in the lower right
corner (6. vs 9.) or by being underlined (6 vs 9).
The platonic solids are commonly used to make dice of 4, 6, 8, 12, and
20 faces. Other shapes can be found to make dice with other numbers
of faces but, other than the 10-sided, they are rarely used. (See
Zocchihedron.) The 4-sided platonic solid is difficult to roll, and a few
games like Dayakattai and Daldøs use a 4-sided rolling pin instead.
A large number of different probability distributions can be obtained
using these dice in various ways. For example, 10-sided dice (or
20-sided dice labeled with single digits) are often used in pairs to
produce a uniform distribution of random percentages; they avoid
number base conversions and are more convenient. Summing multiple dice produces approximations to normal
distributions ("bell curves"), while eliminating high or low throws can be used to skew the distribution in various
Using these techniques, games can closely approximate the real probability distributions of the events they simulate.
There is some controversy over whether manufacturing processes create genuinely "fair" or "honest" dice (dice that
roll with even distributions over their number span). Casino dice are legally required to be fair; those used by others
are not subject to legally required standards.
Spherical dice also exist; these function like the plain cubic dice, but have an octahedral internal cavity in which a
weight moves which causes them to settle in one of six orientations when rolled. However, these dice are somewhat
awkward in use because they require a flat and level surface to roll properly — an uneven surface often causes them
to stop partway between two numbers, while a sloped surface will cause the dice to keep rolling.
Cowry shells, Yut sticks or coins may be used as a kind of two-sided dice. Because of their lack of symmetry, cowry
shells and Yut sticks are not likely to yield a uniform distribution, and the angle and speed of the throw may possibly
affect the result.
Standard variations
A matched Platonic solids set of five dice, (from
left) tetrahedron (4 sides), cube (6), octahedron
(8), dodecahedron (12), and icosahedron (20).
Dice are often sold in sets, matching in color, of five or six different shapes: the five Platonic solids, whose faces are
regular polygons, and optionally the pentagonal trapezohedron, whose faces are ten kites, each with two different
edge lengths and three different angles; the die's vertices also are of two different kinds.
Normally, the faces on a die will be numbered sequentially beginning with 1, and opposite faces will thus add up to
one more than the number of faces (but in the case of the d4 and dice with an odd-number of faces, this is simply not
possible). Some dice, such as d10, are usually numbered sequentially beginning with 0, in which case the opposite
faces will add to one less than the number of faces.
Sides Shape Notes
4 tetrahedron Each face has three numbers: they are arranged such that the upright number (which counts) is the same on all
three visible faces. Alternatively, all of the sides have the same number in the lowest edge and no number on the
top. This die does not roll well and thus it is usually thrown into the air instead.
6 cube A common die. The sum of the numbers on opposite faces is seven.
8 octahedron Each face is triangular; looks like two square pyramids attached base-to-base. Usually, the sum of the opposite
faces is 9.
10 pentagonal
Each face is a kite. The die has two sharp corners, where five kites meet, and ten blunter corners, where three
kites meet. The ten faces usually bear numbers from zero to nine, rather than one to ten (zero being read as "ten"
in many applications). Often all odd numbered faces converge at one sharp corner, and the even ones at the other.
The sum of the numbers on opposite faces is usually 9 (numbered 0-9) or 11 (number 1-10).
12 dodecahedron Each face is a regular pentagon. The sum of the numbers on opposite faces is usually 13.
20 icosahedron
Faces are equilateral triangles. A 2nd century AD Roman icosahedron die is in the collection of the British
Museum, though the game for which it was used is not known.
The sum of the numbers on opposite faces is
Rarer variations
Sides Shape Notes
1 sphere Most commonly a joke die , this is just a sphere with a 1 marked on it. About spherical dice that may produce more
than one result, see the section non-cubical dice above. See also Monostatic polytope, Gömböc.
2 cylinder This is nothing more than a coin shape with 1 marked on one side and 2 on the other. While some tasks in
roleplaying require flipping a coin, the game rules usually simply call for the use of a coin rather than requiring the
use of a two-sided die. It is possible, however, to find dice of this sort for purchase, but they are rare, and can
typically be found among other joke dice.
3 Rounded-off
triangular prism
This is a rounded-off triangular prism, intended to be rolled like a rolling-pin style die. The die is rounded-off at the
edges to make it impossible for it to somehow land on the triangular sides, which makes it look a bit like a jewel.
When the die is rolled, one edge (rather than a side) appears facing upwards. On either side of each edge the same
number is printed (from 1 to 3). The numbers on either side of the up-facing edge are read as the result of the die roll.
Another possible shape is the "American Football" or "Rugby ball" shape, where the ends are pointed (with rounded
points) rather than just rounded. A third variety features faces that resemble warped squares.
5 Triangular prism This is a prism that is thin enough to land either on its "edge" or "face". When landing on an edge, the result is
displayed by digits (2–4) close to the prism's top edge. The triangular faces are labeled with the digits 1 and 5.
7 Pentagonal prism Similar in constitution to the 5-sided die. When landing on an edge, the topmost edge has pips for 1–5. The
pentagonal faces are labeled with the digits 6 and 7. This kind of die is particularly odd since it has pips for five of its
results and digits for two of them. Seven-sided dice are used in a seven-player variant of backgammon. Some
variants have heptagonal ends and rectangular faces.
12 rhombic
Each face is a rhombus.
14 heptagonal
Each face is a kite.
16 octagonal dipyramid Each face is an isosceles triangle.
24 tetrakis hexahedron Each face is an isosceles triangle.
24 deltoidal
Each face is a kite.
30 rhombic
Each face is a rhombus. Although not included in most dice kits, it can be found in most hobby and game stores.
34 heptadecagonal
Each face is a kite.
50 icosakaipentagonal
The faces of the 50-sided die are kites, although very narrow.
100 Zocchihedron 100-sided dice can be found in hobby and game stores, and such a die is used in some narrative role-playing games
such as Dungeons & Dragons. They are not, however, a true polyhedron. A 100-sided die is made by flattening 100
facets on a sphere. The name Zocchihedron was taken from its creator, Lou Zocchi. A typical d100 will be hollow
and filled with small plastic objects to dampen the die's momentum when rolled (lest it take off like a golf ball). A
100-sided die is equivalent to a pair of ten-sided dice, and so, even in role-playing games, the Zocchihedron is rarely
seen and is generally more of a novelty item.
The full geometric set of "uniform fair dice" (face-transitive) are:
• Platonic solids, the five regular polyhedra: 4, 6, 8, 12, 20 sides
• Catalan solids, the duals of the 13 Archimedean solids: 12, 24, 30, 48, 60, 120 sides
• Bipyramids, the duals of the infinite set of prism, with triangle faces: any even number above 4
• Trapezohedrons, the duals of the infinite set of antiprisms, with kite faces: any even number above 4
• Disphenoids, an infinite set of tetrahedra made from congruent non-regular triangles: 4 sides
• "Rolling-pin style dice" (also called "rolling logs"
) are the only way to make dice with an odd number of flat
They are based on an infinite set of prisms. All the (rectangular) faces they may actually land on are
congruent, so they are equally fair. (The other 2 sides of the prism are rounded or capped with a pyramid,
designed so that the die never actually rests on those faces.)
Probability distribution for the sum of two six-sided dice
For a single roll of a fair s-sided die, the
probability of rolling each value, 1 through
s, is exactly
. This is an example of a
discrete uniform distribution. For a double
roll, however, the total of both rolls is not
evenly distributed, but is distributed in a
triangular curve. For two six-sided dice, for
example, the probability distribution is as
2 3 4 5 6 7 8 9 10 11 12











Probability (simplified)











For three or more die rolls, the curve becomes more bell-shaped with each additional die (according to the central
limit theorem). The exact probability distribution of a sum of n s-sided dice can be calculated as the repeated
convolution of the single-die probability distribution with itself.
where F
(k) = 1/s for 1 ≤ k ≤ s and 0 otherwise.
A fastest algorithm would adapt the exponentiation by squaring algorithm, using
For example, in the triangular curve described above,
Equivalently, one can calculate the probability using combinations:
The probability of rolling any exact sequence of numbers is simply 1/s
. For example, the chance of rolling 1, 2, and
3 in that order with three rolls of a six-sided die is 1/6
, or 1/216.
Application in role-playing games
Full set of matching dice used in role-playing games: a d4, d6, d8, d12, d20, and two d10s
for percentile: ones and tens.
While polyhedral dice had previously
been used in teaching basic arithmetic,
the fantasy role-playing game
Dungeons & Dragons is largely
credited with popularizing their use in
roleplaying games. Some games use
only one type, such as Exalted which
uses only ten-sided dice, while others use numerous types for different game purposes, such as Dungeons &
Dragons, which make use of 20-, 12-, 10-, 8- and 4-sided dice in addition to the traditional 6-sided die. Unlike the
common six-sided die, these dice often have the numbers engraved on them rather than a series of dots.
Typical role-playing dice, showing a variety of
colors and styles. Note the older hand-inked
green 12-sided die (showing an 11),
manufactured before pre-inked dice were
common. Many players collect or acquire a large
number of mixed and unmatching dice.
Roleplaying games generally use dice to determine the outcome of
events, such as the success or failure of actions which are difficult to
perform. A player may have to roll dice for combat, skill use, or magic
use, amongst other things, generally referred to as a "check". This is
generally considered fairer than decision by game master fiat, since
success and failure are decided randomly based on a flat probability.
Games typically determine success as either a total on one or more dice
above (Dungeons & Dragons third edition) or below (Call of Cthulhu)
a target number, or a certain number of rolls above a certain number
(such as 8 or higher on a d10) on one or more dice (White Wolf's
World of Darkness series). The player may gain a bonus or penalty due
to circumstances or character skill, usually either by a number added to
or subtracted from the final result, or by having the player roll extra or
fewer dice. For example, a character trying to climb a sheer wall may
subtract from their dice roll (known as a penalty) if the wall is slippery,
which simulates the increased difficulty of climbing a slickened surface, while a character using a rope may add to
the roll (known as a bonus) to simulate that the rope makes the act of climbing easier.
Dice can also be used by a game master for other purposes, such as to randomly generate game content or to make
arbitrary decisions. Some games use dice to determine what attributes the player's character has when created, such
as how strong he or she is.
In Dungeons & Dragons and some other roleplaying games which use more than one kind of die, dice notation is
used for clarity and conciseness. For example, a six-sided die is referred to as a d6, and the notation for rolling two
such dice is 2d6. A constant bias may be added or subtracted by ordinary arithmetic: for example, 2d6+4 adds a 4
point bonus, while 2d6-2 subtracts a 2 point penalty. Games which use only one type of dice rarely require complex
dice notation.
A common special case is percentile rolls, referred to in dice notation as 1d100 or 1d%. Since actual hundred-sided
dice are large, almost spherical, and difficult to read, percentile rolls are usually handled by rolling two ten-sided
dice together, using one as the "tens" and the other as the "units". A roll of ten or zero on either die is taken as a zero,
unless both are zeros or tens, in which case this is 100 (rather than zero). To avoid this confusion, some sets of
percentile dice exist where one is marked in tens (00, 10, 20... up to 90) and the other from 0 to 9. White and black
percentile dice are also used, and are commonly found in color-coded dice sets, or sold separately.
Dice for role-playing games are usually made of plastic, though infrequently metal, wood, and semi-precious stone
dice can be found. Early polyhedral dice from the 1970s and 1980s were made of a soft plastic that would easily
wear as the die was used. Typical wear and tear would gradually round the corners and edges of the die until it was
unusable. Many early dice were unmarked and players took great care in painting their sets of dice. Some
twenty-sided dice of this era were numbered zero through nine twice; half of the numbers had to be painted a
contrasting color to signify the "high" faces. Such a die could also double as a ten-sided die by ignoring the
distinguishing coloring.
Use for divination
Dice can be used for divination. Using dice for such a purpose is called cleromancy. A pair of standard 6-sided dice
is usual though other forms of polyhedra can be used. Tibetan Buddhists sometimes use this method of divination.
It is uncertain if the Pythagoreans used the platonic solids as dice, but it is highly likely. They referred to these
perfect geometries as "the dice of the gods". Julia E. Diggins writes that the Pythagorean Brotherhood sought to
understand the mysteries of the Universe through an understanding of geometry in polyhedra. (Guthrie: The
Pythagorean Sourcebook)
Astrological dice are a specialized set of three 12-sided dice for divination, using the concepts of astrology and
containing astrological symbols for the planets, the zodiac signs and the astrological houses. The first die represents
planets, the Sun, the Moon, and the nodes of the Moon; the second die represents the 12 zodiac signs; and the third
represents the 12 houses. In simplified terms, the planets and quasi-planets could represent the 'actor'; the zodiac
signs could represent the 'role' being played by the actor; and the house could represent the 'scene' in which the actor
Rune dice are a specialized set of dice for divination (runecasting), using the symbols of the runes printed on the
An icosahedron is used to provide the answers of a Magic 8-Ball, which is conventionally used to provide advice on
yes-or-no questions.
[1] AskOxford: die (http:// www.askoxford.com/ concise_oed/ die_2?view=uk)
[2] http:// www. presstv. ir/detail. aspx?id=5668& sectionid=351020108
[3] Possehl, Gregory. "Meluhha". In: J. Reade (ed.) The Indian Ocean in Antiquity. London: Kegan Paul Intl. 1996a, 133–208
[4] 2.3, 4.38, 6.118, 7.52, 7.109
[5] http:/ / www. christies. com/ Lotfinder/lot_details. aspx?intObjectID=4205385
[6] Cf. Greek Anthology (http:/ /www. archive.org/details/ greekanthology05newyuoft) Book 14, §8: "The Opposite Pairs of Numbers on a Die.
The numbers on a die run so: six one, five two, three four."
[7] Standard Dice (http:// homepage. ntlworld.com/ dice-play/ DiceStandard.htm) from dice-play
[8] Chinese Dice (http:/ / www.gamesmuseum. uwaterloo.ca/ Archives/ Culin/ Dice1893/ dice.html) from the Elliott Avedon Museum &
Archive of Games
[9] Bar Coded Dice for Digital Entropy Collection (http:/ / pudec.connotech. com/)
[10] AskOxford: ace (http:// www.askoxford.com/ concise_oed/ ace)
[11] http:// www. fullbooks. com/ The-Art-of-Iugling-or-Legerdemaine.html
[12] http:// games. rengeekcentral.com/ tc4.html
[13] http:// wwmat.mat. fc.ul. pt/ ~jnsilva/ HJT2k9/ AlfonsoX. pdf
[14] Thompson, Clive (December 2, 2003). "Ancient Roman dungeonmastering" (http:// www. collisiondetection.net/ mt/ archives/ 2003/ 12/
ancient_roman_d.html). Collision Detection. . Retrieved 2006-06-26.
[15] The International Bone Rollers' Guild (http:// hometown. aol.com/ dicetalk/ polymor2. htm)
[16] Properties of Dice (http:/ / hjem. get2net. dk/ Klaudius/ Dice.htm)
• Persi Diaconis and Joseph B. Keller. " Fair Dice (http:/ / www-stat.stanford. edu/ ~cgates/ PERSI/papers/
fairdice.pdf)". The American Mathematical Monthly, 96(4):337–339, 1989. (Discussion of dice that are fair "by
symmetry" and "by continuity".)
• Bias and Runs in Dice Throwing and Recording: A Few Million Throws. G. R. Iverson. W. H. Longcour, and
others. Psychometrika, Vol. 36, No. 1, March 1971
• Knizia, Reiner (1999). Dice Games Properly Explained. Elliot Right Way Books. ISBN 0-7160-2112-9.
External links
• Weisstein, Eric W., " Dice (http:/ / mathworld.wolfram.com/ Dice.html)" from MathWorld. Analysis of dice
probabilities, also features Uspenski's work on rolling multiple dice.
• mathematically "Fair Dice"
• Fair Dice (http:/ / www. maa. org/editorial/mathgames/ mathgames_05_16_05. html) is an illustrated Math
Games (http:/ / www. maa. org/news/ mathgames. html) column about all the possible fair dice, and the
mathematical reasons why other shapes are not fair.
• a complete list of all possible Fair Dice (http:// www.mathpuzzle. com/ Fairdice.htm) which has nice
• World's Largest Dice Collection (http:/ / www. dicecollector.com/ ) Links, Photos, Information about dice
• Computer Simulation of Irregular Dice (http:// www.physics. ox. ac. uk/ cm/ cmt/ cuboid)
• A Pair Of Dice Which Never Roll 7 (http:/ / www.chiark.greenend.org.uk/ ~sgtatham/ dice/ )
• The oldest backgammon set found in Iran (http:/ / www.gamblinggates. com/ news/ gaming/
oldest_backgammon_set31041. html)
• A Brief History of Dice (http:// www. wizards.com/ default.asp?x=dnd/ alumni/20070302a) (in Dungeons &
Dragons games)
• How do you make loaded dice? (http:/ / www. straightdope. com/ columns/ read/2878/
how-do-you-load-a-pair-of-dice), The Straight Dope, July 14, 2009
• a discussion linking dice and Tarot cards (http:/ / www.cs. utk. edu/ ~mclennan/ BA/ PT/Mintro. html)
• Popular Science July 1945 Why Dice Behave The Way They Do? (http:// books. google. com/
books?id=PiEDAAAAMBAJ& lpg=PA128&dq=popular science july 1945& pg=PA120#v=onepage&q&
• Dice size chart (http:// www. dicegamers. com/ dice-size-chart) shows common dice dimensions
 This article incorporates text from a publication now in the public domain: Chisholm, Hugh, ed (1911).
Encyclopædia Britannica (Eleventh ed.). Cambridge University Press.
Diehard tests
Diehard tests
The diehard tests are a battery of statistical tests for measuring the quality of a random number generator. They
were developed by George Marsaglia over several years and first published in 1995 on a CD-ROM of random
These are the tests:
• Birthday spacings: Choose random points on a large interval. The spacings between the points should be
asymptotically exponentially distributed. The name is based on the birthday paradox.
• Overlapping permutations: Analyze sequences of five consecutive random numbers. The 120 possible orderings
should occur with statistically equal probability.
• Ranks of matrices: Select some number of bits from some number of random numbers to form a matrix over
{0,1}, then determine the rank of the matrix. Count the ranks.
• Monkey tests: Treat sequences of some number of bits as "words". Count the overlapping words in a stream. The
number of "words" that don't appear should follow a known distribution. The name is based on the infinite
monkey theorem.
• Count the 1s: Count the 1 bits in each of either successive or chosen bytes. Convert the counts to "letters", and
count the occurrences of five-letter "words".
• Parking lot test: Randomly place unit circles in a 100 x 100 square. If the circle overlaps an existing one, try
again. After 12,000 tries, the number of successfully "parked" circles should follow a certain normal distribution.
• Minimum distance test: Randomly place 8,000 points in a 10,000 x 10,000 square, then find the minimum
distance between the pairs. The square of this distance should be exponentially distributed with a certain mean.
• Random spheres test: Randomly choose 4,000 points in a cube of edge 1,000. Center a sphere on each point,
whose radius is the minimum distance to another point. The smallest sphere's volume should be exponentially
distributed with a certain mean.
• The squeeze test: Multiply 2
by random floats on [0,1) until you reach 1. Repeat this 100,000 times. The
number of floats needed to reach 1 should follow a certain distribution.
• Overlapping sums test: Generate a long sequence of random floats on [0,1). Add sequences of 100 consecutive
floats. The sums should be normally distributed with characteristic mean and sigma.
• Runs test: Generate a long sequence of random floats on [0,1). Count ascending and descending runs. The counts
should follow a certain distribution.
• The craps test: Play 200,000 games of craps, counting the wins and the number of throws per game. Each count
should follow a certain distribution.
Diehard tests
External links
• The Marsaglia Random Number CDROM including the Diehard Battery of Tests of Randomness
• Mirror site
• DieHarder: a random number test suite including an alternative GPL implementation of Diehard tests in C
[1] http:/ / www. stat. fsu. edu/ pub/ diehard/
[2] http:/ / www. cs. hku. hk/ ~diehard/cdrom/
[3] http:/ / www. phy. duke. edu/ ~rgb/ General/dieharder.php
Differential entropy
Differential entropy (also referred to as continuous entropy) is a concept in information theory that extends the
idea of (Shannon) entropy, a measure of average surprisal of a random variable, to continuous probability
Let X be a random variable with a probability density function f whose support is a set . The differential entropy
or is defined as
As with its discrete analog, the units of differential entropy depend on the base of the logarithm, which is usually 2
(i.e., the units are bits). See logarithmic units for logarithms taken in different bases. Related concepts such as joint,
conditional differential entropy, and relative entropy are defined in a similar fashion.
One must take care in trying to apply properties of discrete entropy to differential entropy, since probability density
functions can be greater than 1. For example, Uniform(0,1/2) has negative differential entropy
Thus, differential entropy does not share all properties of discrete entropy.
Note that the continuous mutual information has the distinction of retaining its fundamental significance
as a measure of discrete information since it is actually the limit of the discrete mutual information of partitions of X
and Y as these partitions become finer and finer. Thus it is invariant under non-linear homeomorphisms (continuous
and uniquely invertible maps)
, including linear
transformations of X and Y, and still represents the amount of
discrete information that can be transmitted over a channel that admits a continuous space of values.
Properties of differential entropy
• For two densities f and g, with equality if almost everywhere. Similarly, for two random
variables X and Y, and with equality if and only if X and Y are independent.
• The chain rule for differential entropy holds as in the discrete case
• Differential entropy is translation invariant, ie, for a constant c.
• Differential entropy is in general not invariant under arbitrary invertible maps. In particular, for a constant a,
. For a vector valued random variable X and a matrix A,
Differential entropy
• In general, for a bijective transformation from a random vector X to a random vector with same dimension Y
, the corresponding entropies are related via where
is the Jacobian of the transformation m.
• If a random vector has mean zero and covariance matrix K, with
equality if and only if X is jointly gaussian.
However, differential entropy does not have other desirable properties:
• It is not invariant under change of variables.
• It can be negative.
A modification of differential entropy that addresses this is the relative information entropy, also known as the
Kullback–Leibler divergence, which includes an invariant measure factor (see limiting density of discrete points).
Normal Distribution Maximizes The Differential Entropy For a Given
The following is a proof that a Gaussian variable has the largest entropy amongst all random variables of equal
Let be a Gaussian PDF with mean and variance and an arbitrary PDF with the same variance.
Consider the Kullback-Leibler divergence between the two distributions
Now note that
because the result does not depend on other than through the variance. Combining the two results yields
with equality when following from the properties of Kullback-Leibler divergence.
Differential entropy
Example: Exponential distribution
Let X be an exponentially distributed random variable hain with parameter , that is, with probability density
Its differential entropy is then
Here, was used rather than to make it explicit that the logarithm was taken to base e, to simplify the
Differential entropies for various distributions
In the table below, (the gamma function), ,
, and is Euler's constant.
Table of differential entropies.
Probability density function (pdf) Entropy in nats

Differential entropy
Student's t

(Many of these are from
As described above, differential entropy does not share all properties of discrete entropy. A modification of
differential entropy adds an invariant measure factor to correct this, and the resulting notion is called relative
information entropy; see limiting density of discrete points:
The definition of differential entropy above can be obtained by partitioning the range of X into bins of length
with associated sample points within the bins, for X Riemann integrable. This gives a quantized version of X,
defined by if . Then the entropy of is
The first term approximates the differential entropy, while the second term is approximately . Note that
this procedure suggests that the entropy in the discrete sense of a continuous random variable should be .
Differential entropy
[1] Kraskov, Alexander; Stögbauer, Grassberger (2004). "Estimating mutual information". Phys. Rev. E 60: 066138. arXiv:cond-mat/0305641.
Bibcode 2004PhRvE..69f6138K. doi:10.1103/PhysRevE.69.066138.
[2] Fazlollah M. Reza (1961, 1994). An Introduction to Information Theory (http:/ / books. google. com/ books?id=RtzpRAiX6OgC& pg=PA8&
dq=intitle:"An+ Introduction+to+ Information+Theory"++ "entropy+of+a+ simple+ source"& as_brr=0&
ei=zP79Ro7UBovqoQK4g_nCCw& sig=j3lPgyYrC3-bvn1Td42TZgTzj0Q). Dover Publications, Inc., New York. ISBN 0-486-68210-2. .
[3] Lazo, A. and P. Rathie. On the entropy of continuous probability distributions Information Theory, IEEE Transactions on, 1978. 24(1): p.
• Thomas M. Cover, Joy A. Thomas. Elements of Information Theory New York: Wiley, 1991. ISBN
External links
• Differential entropy (http:// planetmath. org/ ?op=getobj&amp;from=objects&amp;id=1915) on PlanetMath
Entropy (information theory)
In information theory, entropy is a measure of the uncertainty associated with a random variable. In this context, the
term usually refers to the Shannon entropy, which quantifies the expected value of the information contained in a
message, usually in units such as bits. Equivalently, the Shannon entropy is a measure of the average information
content one is missing when one does not know the value of the random variable. The concept was introduced by
Claude E. Shannon in his 1948 paper "A Mathematical Theory of Communication".
Shannon's entropy represents an absolute limit on the best possible lossless compression of any communication,
under certain constraints: treating messages to be encoded as a sequence of independent and identically-distributed
random variables, Shannon's source coding theorem shows that, in the limit, the average length of the shortest
possible representation to encode the messages in a given alphabet is their entropy divided by the logarithm of the
number of symbols in the target alphabet.
A single toss of a fair coin has an entropy of one bit. Two tosses has an entropy of two bits. The entropy rate for the
coin is one bit per toss. However, if the coin is not fair, then the uncertainty is lower (if asked to bet on the next
outcome, we would bet preferentially on the most frequent result), and thus the Shannon entropy is lower.
Mathematically, a single coin flip (fair or not) is an example of a Bernoulli trial, and its entropy is given by the
binary entropy function. A series of tosses of a two-headed coin will have zero entropy, since the outcomes are
entirely predictable. The entropy rate of English text is between 1.0 and 1.5 bits per letter,
or as low as 0.6 to 1.3
bits per letter, according to estimates by Shannon based on human experiments.
Entropy is a measure of disorder, or more precisely unpredictability. For example, a series of coin tosses with a fair
coin has maximum entropy, since there is no way to predict what will come next. A string of coin tosses with a
two-headed coin has zero entropy, since the coin will always come up heads. Most collections of data in the real
world lie somewhere in between. It is important to realize the difference between the entropy of a set of possible
outcomes, and the entropy of a particular outcome. A single toss of a fair coin has an entropy of one bit, but a
particular result (e.g. "heads") has zero entropy, since it is entirely "predictable".
English text has fairly low entropy. In other words, it is fairly predictable. Even if we don't know exactly what is
going to come next, we can be fairly certain that, for example, there will be many more e's than z's, or that the
combination 'qu' will be much more common than any other combination with a 'q' in it and the combination 'th' will
be more common than any of them. Uncompressed, English text has about one bit of entropy for each byte (eight
Entropy (information theory)
bits) of message.
If a compression scheme is lossless—that is, you can always recover the entire original message by
uncompressing—then a compressed message has the same total entropy as the original, but in fewer bits. That is, it
has more entropy per bit. This means a compressed message is more unpredictable, which is why messages are often
compressed before being encrypted. Shannon's source coding theorem says (roughly) that a lossless compression
scheme cannot compress messages, on average, to have more than one bit of entropy per bit of message. The entropy
of a message is in a certain sense a measure of how much information it really contains.
Shannon's theorem also implies that no lossless compression scheme can compress all messages. If some messages
come out smaller, at least one must come out larger. In the real world, this is not a problem, because we are generally
only interested in compressing certain messages, for example English documents as opposed to random bytes, and
don't care if our compressor makes random messages larger.
Named after Boltzmann's H-theorem, Shannon denoted the entropy H of a discrete random variable X with possible
values {x
, ..., x
} as,
Here E is the expected value, and I is the information content of X.
I(X) is itself a random variable. If p denotes the probability mass function of X then the entropy can explicitly be
written as
where b is the base of the logarithm used. Common values of b are 2, Euler's number e, and 10, and the unit of
entropy is bit for b = 2, nat for b = e, and dit (or digit) for b = 10.
In the case of p
 = 0 for some i, the value of the corresponding summand 0 log
 0 is taken to be 0, which is consistent
with the limit:
Entropy (information theory)
Entropy H(X) (i.e. the expected surprisal) of a coin
flip, measured in bits, graphed versus the fairness of
the coin Pr(X=1), where X=1 represents a result of
Note that the maximum of the graph depends on the
distribution. Here, at most 1 bit is required to
communicate the outcome of a fair coin flip, but the
result of a fair die would require at most log
6 bits.
Consider tossing a coin with known, not necessarily fair,
probabilities of coming up heads or tails.
The entropy of the unknown result of the next toss of the coin is
maximized if the coin is fair (that is, if heads and tails both have
equal probability 1/2). This is the situation of maximum
uncertainty as it is most difficult to predict the outcome of the next
toss; the result of each toss of the coin delivers a full 1 bit of
However, if we know the coin is not fair, but comes up heads or
tails with probabilities p and q, then there is less uncertainty.
Every time it is tossed, one side is more likely to come up than the
other. The reduced uncertainty is quantified in a lower entropy: on
average each toss of the coin delivers less than a full 1 bit of
The extreme case is that of a double-headed coin that never comes
up tails, or a double-tailed coin that never results in a head. Then
there is no uncertainty. The entropy is zero: each toss of the coin
delivers no information.
For a random variable with outcomes , the Shannon entropy, a measure of
uncertainty (see further below) and denoted by , is defined as
where is the probability mass function of outcome .
To understand the meaning of Eq. (1), first consider a set of possible outcomes (events) ,
with equal probability . An example would be a fair die with values, from to . The
uncertainty for such a set of outcomes is defined by
The logarithm is used so as to provide the additivity characteristic for independent uncertainty. For example,
consider appending to each value of the first die the value of a second die, which has possible outcomes
. There are thus possible outcomes . The
uncertainty for such a set of outcomes is then
Thus the uncertainty of playing with two dice is obtained by adding the uncertainty of the second die to
the uncertainty of the first die .
Now return to the case of playing with one die only (the first one). Since the probability of each event is , we
can write
Entropy (information theory)
In the case of a non-uniform probability mass function (or density in the case of continuous random variables), we
which is also called a surprisal; the lower the probability , i.e. , the higher the uncertainty or the
surprise, i.e. , for the outcome .
The average uncertainty , with being the average operator, is obtained by
and is used as the definition of the entropy in Eq. (1). The above also explained why information entropy
and information uncertainty can be used interchangeably.
One may also define the conditional entropy of two events X and Y taking values x
and y
respectively, as
where p(x
) is the probability that X=x
and Y=y
. This quantity should be understood as the amount of randomness
in the random variable X given that you know the value of Y. For example, the entropy associated with a six-sided
die is H(die), but if you were told that it had in fact landed on 1, 2, or 3, then its entropy would be equal to H(die: the
die landed on 1, 2, or 3).
Relationship to thermodynamic entropy
The inspiration for adopting the word entropy in information theory came from the close resemblance between
Shannon's formula and very similar known formulae from thermodynamics.
In statistical thermodynamics the most general formula for the thermodynamic entropy S of a thermodynamic system
is the Gibbs entropy,
where k
is the Boltzmann constant, and p
is the probability of a microstate. The Gibbs entropy was defined by J.
Willard Gibbs in 1878 after earlier work by Boltzmann (1872).
The Gibbs entropy translates over almost unchanged into the world of quantum physics to give the von Neumann
entropy, introduced by John von Neumann in 1927,
where ρ is the density matrix of the quantum mechanical system and Tr is the trace.
At an everyday practical level the links between information entropy and thermodynamic entropy are not close.
Physicists and chemists are apt to be more interested in changes in entropy as a system spontaneously evolves away
from its initial conditions, in accordance with the second law of thermodynamics, rather than an unchanging
probability distribution. And, as the minuteness of Boltzmann's constant k
indicates, the changes in S / k
for even
tiny amounts of substances in chemical and physical processes represent amounts of entropy which are so large as to
be off the scale compared to anything seen in data compression or signal processing.
Entropy (information theory)
But, at a multidisciplinary level, connections can be made between thermodynamic and informational entropy,
although it took many years in the development of the theories of statistical mechanics and information theory to
make the relationship fully apparent. In fact, in the view of Jaynes (1957), thermodynamics should be seen as an
application of Shannon's information theory: the thermodynamic entropy is interpreted as being an estimate of the
amount of further Shannon information needed to define the detailed microscopic state of the system, that remains
uncommunicated by a description solely in terms of the macroscopic variables of classical thermodynamics. For
example, adding heat to a system increases its thermodynamic entropy because it increases the number of possible
microscopic states for the system, thus making any complete state description longer. (See article: maximum entropy
thermodynamics). Maxwell's demon can (hypothetically) reduce the thermodynamic entropy of a system by using
information about the states of individual molecules; but, as Landauer (from 1961) and co-workers have shown, to
function the demon himself must increase thermodynamic entropy in the process, by at least the amount of Shannon
information he proposes to first acquire and store; and so the total entropy does not decrease (which resolves the
Entropy as information content
Entropy is defined in the context of a probabilistic model. Independent fair coin flips have an entropy of 1 bit per
flip. A source that always generates a long string of B's has an entropy of 0, since the next character will always be a
The entropy rate of a data source means the average number of bits per symbol needed to encode it. Shannon's
experiments with human predictors show an information rate of between 0.6 and 1.3 bits per character,
on the experimental setup; the PPM compression algorithm can achieve a compression ratio of 1.5 bits per character
in English text.
From the preceding example, note the following points:
1. The amount of entropy is not always an integer number of bits.
2. Many data bits may not convey information. For example, data structures often store information redundantly, or
have identical sections regardless of the information in the data structure.
Shannon's definition of entropy, when applied to an information source, can determine the minimum channel
capacity required to reliably transmit the source as encoded binary digits (see caveat below in italics). The formula
can be derived by calculating the mathematical expectation of the amount of information contained in a digit from
the information source. See also Shannon-Hartley theorem.
Shannon's entropy measures the information contained in a message as opposed to the portion of the message that is
determined (or predictable). Examples of the latter include redundancy in language structure or statistical properties
relating to the occurrence frequencies of letter or word pairs, triplets etc. See Markov chain.
Data compression
Entropy effectively bounds the performance of the strongest lossless (or nearly lossless) compression possible, which
can be realized in theory by using the typical set or in practice using Huffman, Lempel-Ziv or arithmetic coding. The
performance of existing data compression algorithms is often used as a rough estimate of the entropy of a block of

See also Kolmogorov complexity.
Limitations of entropy as information content
There are a number of entropy-related concepts that mathematically quantify information content in some way:
• the self-information of an individual message or symbol taken from a given probability distribution,
• the entropy of a given probability distribution of messages or symbols, and
• the entropy rate of a stochastic process.
Entropy (information theory)
(The "rate of self-information" can also be defined for a particular sequence of messages or symbols generated by a
given stochastic process: this will always be equal to the entropy rate in the case of a stationary process.) Other
quantities of information are also used to compare or relate different sources of information.
It is important not to confuse the above concepts. Oftentimes it is only clear from context which one is meant. For
example, when someone says that the "entropy" of the English language is about 1.5 bits per character, they are
actually modeling the English language as a stochastic process and talking about its entropy rate.
Although entropy is often used as a characterization of the information content of a data source, this information
content is not absolute: it depends crucially on the probabilistic model. A source that always generates the same
symbol has an entropy rate of 0, but the definition of what a symbol is depends on the alphabet. Consider a source
that produces the string ABABABABAB... in which A is always followed by B and vice versa. If the probabilistic
model considers individual letters as independent, the entropy rate of the sequence is 1 bit per character. But if the
sequence is considered as "AB AB AB AB AB..." with symbols as two-character blocks, then the entropy rate is 0
bits per character.
However, if we use very large blocks, then the estimate of per-character entropy rate may become artificially low.
This is because in reality, the probability distribution of the sequence is not knowable exactly; it is only an estimate.
For example, suppose one considers the text of every book ever published as a sequence, with each symbol being the
text of a complete book. If there are N published books, and each book is only published once, the estimate of the
probability of each book is 1/N, and the entropy (in bits) is -log
1/N = log
N. As a practical code, this corresponds
to assigning each book a unique identifier and using it in place of the text of the book whenever one wants to refer to
the book. This is enormously useful for talking about books, but it is not so useful for characterizing the information
content of an individual book, or of language in general: it is not possible to reconstruct the book from its identifier
without knowing the probability distribution, that is, the complete text of all the books. The key idea is that the
complexity of the probabilistic model must be considered. Kolmogorov complexity is a theoretical generalization of
this idea that allows the consideration of the information content of a sequence independent of any particular
probability model; it considers the shortest program for a universal computer that outputs the sequence. A code that
achieves the entropy rate of a sequence for a given model, plus the codebook (i.e. the probabilistic model), is one
such program, but it may not be the shortest.
For example, the Fibonacci sequence is 1, 1, 2, 3, 5, 8, 13, ... . Treating the sequence as a message and each number
as a symbol, there are almost as many symbols as there are characters in the message, giving an entropy of
approximately log
(n). So the first 128 symbols of the Fibonacci sequence has an entropy of approximately 7
bits/symbol. However, the sequence can be expressed using a formula [F(n) = F(n-1) + F(n-2) for n={3,4,5,...},
F(1)=1, F(2)=1] and this formula has a much lower entropy and applies to any length of the Fibonacci sequence.
Limitations of entropy as a measure of unpredictability
In cryptanalysis, entropy is often roughly used as a measure of the unpredictability of a cryptographic key. For
example, a 128-bit key that is randomly generated has 128 bits of entropy. It takes (on average) guesses to
break by brute force. If the key's first digit is 0, and the others random, then the entropy is 127 bits, and it takes (on
average) guesses.
However, this measure fails if the possible keys are not of equal probability. If the key is half the time "password"
and half the time a true random 128-bit key, then the entropy is approximately 65 bits. Yet half the time the key may
be guessed on the first try, if your first guess is "password", and on average, it takes around guesses (not
) to break this password.
Similarly, consider a 1000000-digit binary one-time pad. If the pad has 1000000 bits of entropy, it is perfect. If the
pad has 999999 bits of entropy, evenly distributed (each individual bit of the pad having 0.999999 bits of entropy) it
may still be considered very good. But if the pad has 999999 bits of entropy, where the first digit is fixed and the
remaining 999999 digits are perfectly random, then the first digit of the ciphertext will not be encrypted at all.
Entropy (information theory)
Data as a Markov process
A common way to define entropy for text is based on the Markov model of text. For an order-0 source (each
character is selected independent of the last characters), the binary entropy is:
where p
is the probability of i. For a first-order Markov source (one in which the probability of selecting a character
is dependent only on the immediately preceding character), the entropy rate is:
where i is a state (certain preceding characters) and is the probability of given as the previous character.
For a second order Markov source, the entropy rate is
b-ary entropy
In general the b-ary entropy of a source = (S,P) with source alphabet S = {a
, ..., a
} and discrete probability
distribution P = {p
, ..., p
} where p
is the probability of a
(say p
= p(a
)) is defined by:
Note: the b in "b-ary entropy" is the number of different symbols of the "ideal alphabet" which is being used as the
standard yardstick to measure source alphabets. In information theory, two symbols are necessary and sufficient for
an alphabet to be able to encode information, therefore the default is to let b = 2 ("binary entropy"). Thus, the
entropy of the source alphabet, with its given empiric probability distribution, is a number equal to the number
(possibly fractional) of symbols of the "ideal alphabet", with an optimal probability distribution, necessary to encode
for each symbol of the source alphabet. Also note that "optimal probability distribution" here means a uniform
distribution: a source alphabet with n symbols has the highest possible entropy (for an alphabet with n symbols)
when the probability distribution of the alphabet is uniform. This optimal entropy turns out to be .
A source alphabet with non-uniform distribution will have less entropy than if those symbols had uniform
distribution (i.e. the "optimized alphabet"). This deficiency in entropy can be expressed as a ratio:
Efficiency has utility in quantifying the effective use of a communications channel.
Entropy (information theory)
Shannon entropy is characterized by a small number of criteria, listed below. Any definition of entropy satisfying
these assumptions has the form
where K is a constant corresponding to a choice of measurement units.
In the following, and .
The measure should be continuous, so that changing the values of the probabilities by a very small amount should
only change the entropy by a small amount.
The measure should be unchanged if the outcomes x
are re-ordered.
The measure should be maximal if all the outcomes are equally likely (uncertainty is highest when all possible
events are equiprobable).
For equiprobable events the entropy should increase with the number of outcomes.
The amount of entropy should be independent of how the process is regarded as being divided into parts.
This last functional relationship characterizes the entropy of a system with sub-systems. It demands that the entropy
of a system can be calculated from the entropies of its sub-systems if the interactions between the sub-systems are
Given an ensemble of n uniformly distributed elements that are divided into k boxes (sub-systems) with b
, b
, ... , b
elements each, the entropy of the whole ensemble should be equal to the sum of the entropy of the system of boxes
and the individual entropies of the boxes, each weighted with the probability of being in that particular box.
For positive integers b
where b
+ ... + b
= n,
Choosing k = n, b
= ... = b
= 1 this implies that the entropy of a certain outcome is zero:
This implies that the efficiency of a source alphabet with n symbols can be defined simply as being equal to its n-ary
entropy. See also Redundancy (information theory).
Entropy (information theory)
Further properties
The Shannon entropy satisfies the following properties, for some of which it is useful to interpret entropy as the
amount of information learned (or uncertainty eliminated) by revealing the value of a random variable X:
• Adding or removing an event with probability zero does not contribute to the entropy:
• It can be confirmed using the Jensen inequality that
This maximal entropy of is effectively attained by a source alphabet having a uniform probability
distribution: uncertainty is maximal when all possible events are equiprobable.
• The entropy or the amount of information revealed by evaluating (X,Y) (that is, evaluating X and Y
simultaneously) is equal to the information revealed by conducting two consecutive experiments: first evaluating
the value of Y, then revealing the value of X given that you know the value of Y. This may be written as
• If X and Y are two independent experiments, then knowing the value of Y doesn't influence our knowledge of the
value of X (since the two don't influence each other by independence):
• The entropy of two simultaneous events is no more than the sum of the entropies of each individual event, and are
equal if the two events are independent. More specifically, if X and Y are two random variables on the same
probability space, and (X,Y) denotes their Cartesian product, then
Proving this mathematically follows easily from the previous two properties of entropy.
Extending discrete entropy to the continuous case: differential entropy
The Shannon entropy is restricted to random variables taking discrete values. The formula
where f denotes a probability density function on the real line, is analogous to the Shannon entropy and could thus be
viewed as an extension of the Shannon entropy to the domain of real numbers.
A precursor of the continuous entropy given in (1) is the expression for the functional in the H-theorem of
Formula (1) is usually referred to as the continuous entropy, or differential entropy. Although the analogy between
both functions is suggestive, the following question must be set: is the differential entropy a valid extension of the
Shannon discrete entropy? Differential entropy lacks a number of properties that the Shannon discrete entropy has –
it can even be negative – and thus corrections have been suggested, notably limiting density of discrete points.
To answer this question, we must establish a connection between the two functions:
We wish to obtain a generally finite measure as the bin size goes to zero. In the discrete case, the bin size is the
(implicit) width of each of the n (finite or infinite) bins whose probabilities are denoted by p
. As we generalize to
the continuous domain, we must make this width explicit.
To do this, start with a continuous function f discretized as shown in the figure. As the figure indicates, by the
mean-value theorem there exists a value x
in each bin such that
Entropy (information theory)
and thus the integral of the function f can be approximated (in the Riemannian sense) by
where this limit and "bin size goes to zero" are equivalent.
We will denote
and expanding the logarithm, we have
As , we have
and also
But note that as , therefore we need a special definition of the differential or continuous
which is, as said before, referred to as the differential entropy. This means that the differential entropy is not a limit
of the Shannon entropy for . Rather, if differs from the limit of the Shannon entropy by an infinite offset.
It turns out as a result that, unlike the Shannon entropy, the differential entropy is not in general a good measure of
uncertainty or information. For example, the differential entropy can be negative; also it is not invariant under
continuous co-ordinate transformations.
Another useful measure of entropy for the continuous case is the relative entropy of a distribution, defined as the
Kullback-Leibler divergence from the distribution to a reference measure m(x),
The relative entropy carries over directly from discrete to continuous distributions, and is invariant under co-ordinate
Entropy (information theory)
Use in combinatorics
Entropy has become a useful quantity in combinatorics.
Loomis-Whitney inequality
A simple example of this is an alternate proof of the Loomis-Whitney inequality: for every subset , we
where , that is, is the orthogonal projection in
the ith coordinate.
The proof follows as a simple corollary of Shearer's inequality: if are random variables and
are subsets of such that every integer between 1 and d lie in exactly r of these subsets,
where is the Cartesian product of random variables with indexes j in (so the dimension of this
vector is equal to the size of ).
We sketch how Loomis-Whitney follows from this: Indeed, let X be a uniformly distributed random variable with
values in A and so that each point in A occurs with equal probability. Then (by the further properties of entropy
mentioned above) , where |A| denotes the cardinality of A. Let
. The range of is contained in and hence
. Now use this to bound the right side of Shearer's inequality and exponentiate the
opposite sides of the resulting inequality you obtain.
Approximation to binomial coefficient
For integers let . Then
Here is a sketch proof. Note that is one term of the expression
. Rearranging gives the upper bound. For the lower bound one
first shows, using some algebra, that it is the largest term in the summation. But then,
since there are terms in the summation. Rearranging gives the lower bound.
A nice interpretation of this is that the number of binary strings of length with exactly many 1's is
approximately .
Entropy (information theory)
[1] Schneier, B: Applied Cryptography, Second edition, page 234. John Wiley and Sons.
[2] Shannon, Claude E.: Prediction and entropy of printed English, The Bell System Technical Journal, 30:50-64, January 1951.
[3] Schneider, T.D, Information theory primer with an appendix on logarithms (http:// www.lecb.ncifcrf.gov/ ~toms/ paper/ primer/primer.
pdf), National Cancer Institute, 14 April 2007.
[4] Jaynes, E.T. (May 1957). "Information Theory and Statistical Mechanics" (http:// bayes. wustl. edu/ etj/ articles/ theory.1. pdf). Physical
Review 106 (4): 620–630. Bibcode 1957PhRv..106..620J. doi:10.1103/PhysRev.106.620. .
[5] Compare: Boltzmann, Ludwig (1896, 1898). Vorlesungen über Gastheorie : 2 Volumes - Leipzig 1895/98 UB: O 5262-6. English version:
Lectures on gas theory. Translated by Stephen G. Brush (1964) Berkeley: University of California Press; (1995) New York: Dover ISBN
[6] Mark Nelson (2006-08-24). "The Hutter Prize" (http:// marknelson.us/ 2006/ 08/ 24/ the-hutter-prize/). . Retrieved 2008-11-27.
[7] T. Schürmann and P. Grassberger, Entropy Estimation of Symbol Sequences (http:/ / arxiv.org/abs/ cond-mat/0203436), CHAOS,Vol. 6,
No. 3 (1996) 414-427
[8] T. Schürmann, Bias Analysis in Entropy Estimation (http:// arxiv.org/abs/ cond-mat/0403192) J. Phys. A: Math. Gen. 37 (2004)
[9] Probability and Computing, M. Mitzenmacher and E. Upfal, Cambridge University Press
This article incorporates material from Shannon's entropy on PlanetMath, which is licensed under the Creative
Commons Attribution/Share-Alike License.
External links
• Entropy (http:// www. mdpi.com/ journal/entropy) an interdisciplinary journal on all aspect of the entropy
concept. Open access.
• Information is not entropy, information is not uncertainty ! (http:// www.lecb. ncifcrf.gov/ ~toms/ information.
is. not. uncertainty.html) - a discussion of the use of the terms "information" and "entropy".
• I'm Confused: How Could Information Equal Entropy? (http:// www.ccrnp.ncifcrf.gov/ ~toms/ bionet.
info-theory. faq.html#Information.Equal.Entropy) - a similar discussion on the bionet.info-theory FAQ.
• Description of information entropy from "Tools for Thought" by Howard Rheingold (http:// www. rheingold.
com/ texts/ tft/6. html)
• A java applet representing Shannon's Experiment to Calculate the Entropy of English (http:// math. ucsd.edu/
~crypto/ java/ ENTROPY/ )
• Slides on information gain and entropy (http:// www.autonlab. org/tutorials/ infogain.html)
• An Intuitive Guide to the Concept of Entropy Arising in Various Sectors of Science (http:/ / en.wikibooks. org/
wiki/An_Intuitive_Guide_to_the_Concept_of_Entropy_Arising_in_Various_Sectors_of_Science) - a wikibook
on the interpretation of the concept of entropy.
Entropy estimation
Entropy estimation
Estimating the differential entropy of a system or process, given some observations, is useful in various
science/engineering applications, such as Independent Component Analysis,
image analysis,
genetic analysis,
speech recognition,
manifold learning,
and time delay estimation.
The simplest and most common approach
uses histogram-based estimation, but other approaches have been developed and used, each with their own benefits
and drawbacks.
The main factor in choosing a method is often a trade-off between the bias and the variance of the
although the nature of the (suspected) distribution of the data may also be a factor.
Histogram estimator
The histogram approach uses the idea that the differential entropy,
can be approximated by producing a histogram of the observations, and then finding the discrete entropy
of that histogram (which is itself a maximum-likelihood estimate of the discretized frequency distribution).
Histograms can be quick to calculate, and simple, so this approach has some attractions. However, the estimate
produced is biased, and although corrections can be made to the estimate, they may not always be satisfactory.
A method better suited for multidimensional pdf's is to first make a pdf estimate with some method, and then, from
the pdf estimate, compute the entropy. A useful pdf estimate method is e.g. Gaussian Mixture Modeling (GMM),
where the Expectation Maximization (EM) algorithm is used to find an ML estimate of a weighted sum of Gaussian
pdf's approximating the data pdf.
Estimates based on sample-spacings
If the data is one-dimensional, we can imagine taking all the observations and putting them in order of their value.
The spacing between one value and the next then gives us a rough idea of (the reciprocal of) the probability density
in that region: the closer together the values are, the higher the probability density. This is a very rough estimate with
high variance, but can be improved, for example by thinking about the space between a given value and the one m
away from it, where m is some fixed number.
The probability density estimated in this way can then be used to calculate the entropy estimate, in a similar way to
that given above for the histogram, but with some slight tweaks.
One of the main drawbacks with this approach is going beyond one dimension: the idea of lining the data points up
in order falls apart in more than one dimension. However, using analogous methods, some multidimensional entropy
estimators have been developed.

Entropy estimation
Estimates based on nearest-neighbours
For each point in our dataset, we can find the distance to its nearest neighbour. We can in fact estimate the entropy
from the distribution of the nearest-neighbour-distance of our datapoints.
(In a uniform distribution these distances
all tend to be fairly similar, whereas in a strongly nonuniform distribution they may vary a lot more.)
[1] Dinh-Tuan Pham (2004) Fast algorithms for mutual information based independent component analysis. In Signal Processing. Volume 52,
Issue 10, 2690 - 2700, doi:10.1109/TSP.2004.834398
[2] Chang, C.-I.; Du, Y.; Wang, J.; Guo, S.-M.; Thouin, P.D. (2006) Survey and comparative analysis of entropy and relative entropy
thresholding techniques. In Vision, Image and Signal Processing, Volume 153, Issue 6, 837 - 850, doi:10.1049/ip-vis:20050032
[3] Martins, D. C. et al. (2008) Intrinsically Multivariate Predictive Genes. In Selected Topics in Signal Processing. Volume 2, Issue 3, 424 - 439,
[4] Gue Jun Jung; Yung-Hwan Oh (2008) Information Distance-Based Subvector Clustering for ASR Parameter Quantization. In Signal
Processing Letters, Volume 15, 209 - 212, doi:10.1109/LSP.2007.913132
[5] Costa, J.A.; Hero, A.O. (2004), Geodesic entropic graphs for dimension and entropy estimation in manifold learning. In Signal Processing,
Volume 52, Issue 8, 2210 - 2221, doi:10.1109/TSP.2004.831130
[6] Benesty, J.; Yiteng Huang; Jingdong Chen (2007) Time Delay Estimation via Minimum Entropy. In Signal Processing Letters, Volume 14,
Issue 3, March 2007 157 - 160 doi:10.1109/LSP.2006.884038
[7] J. Beirlant, E. J. Dudewicz, L. Gyorfi, and E. C. van der Meulen (1997) Nonparametric entropy estimation: An overview (http:// www. its.
caltech. edu/ ~jimbeck/ summerlectures/ references/Entropy estimation.pdf). In International Journal of Mathematical and Statistical
Sciences, Volume 6, pp. 17– 39.
[8] T. Schürmann, Bias analysis in entropy estimation. In J. Phys. A: Math. Gen, 37 (2004), pp. L295–L301. doi:10.1088/0305-4470/37/27/L02
[9] G. Miller (1955) Note on the bias of information estimates. In Information Theory in Psychology: Problems and Methods, pp. 95–100.
[10] E. G. Learned-Miller (2003) A new class of entropy estimators for multi-dimensional densities, in Proceedings of the International
Conference on Acoustics, Speech, and Signal Processing (ICASSP’03), vol. 3, April 2003, pp. 297–300.
[11] I. Lee (2010) Sample-spacings based density and entropy estimators for spherically invariant multidimensional data, In Neural Computation,
vol. 22, issue 8, April 2010, pp. 2208–2227.
Fisher–Yates shuffle
Fisher–Yates shuffle
The Fisher–Yates shuffle (named after Ronald Fisher and Frank Yates), also known as the Knuth shuffle (after
Donald Knuth), is an algorithm for generating a random permutation of a finite set—in plain terms, for randomly
shuffling the set. A variant of the Fisher–Yates shuffle, known as Sattolo's algorithm, may be used to generate
random cycles of length n instead. Properly implemented, the Fisher–Yates shuffle is unbiased, so that every
permutation is equally likely. The modern version of the algorithm is also rather efficient, requiring only time
proportional to the number of items being shuffled and no additional storage space.
The basic process of Fisher–Yates shuffling is similar to randomly picking numbered tickets out of a hat, or cards
from a deck, one after another until there are no more left. What the specific algorithm provides is a way of doing
this numerically in an efficient and rigorous manner that, properly done, guarantees an unbiased result.
Fisher and Yates' original method
The Fisher–Yates shuffle, in its original form, was described in 1938 by Ronald A. Fisher and Frank Yates in their
book Statistical tables for biological, agricultural and medical research.
(Later editions describe a somewhat
different method attributed to C. R. Rao.) Their method was designed to be implemented using pencil and paper,
with a precomputed table of random numbers as the source of randomness. The basic method given for generating a
random permutation of the numbers 1–N goes as follows:
1. Write down the numbers from 1 to N.
2. Pick a random number k between one and the number of unstruck numbers remaining (inclusive).
3. Counting from the low end, strike out the kth number not yet struck out, and write it down elsewhere.
4. Repeat from step 2 until all the numbers have been struck out.
5. The sequence of numbers written down in step 3 is now a random permutation of the original numbers.
Provided that the random numbers picked in step 2 above are truly random and unbiased, so will the resulting
permutation be. Fisher and Yates took care to describe how to obtain such random numbers in any desired range
from the supplied tables in a manner which avoids any bias. They also suggested the possibility of using a simpler
method — picking random numbers from one to N and discarding any duplicates—to generate the first half of the
permutation, and only applying the more complex algorithm to the remaining half, where picking a duplicate number
would otherwise become frustratingly common.
The modern algorithm
The modern version of the Fisher–Yates shuffle, designed for computer use, was introduced by Richard Durstenfeld
in 1964 in Communications of the ACM volume 7, issue 7, as "Algorithm 235: Random permutation",
and was
popularized by Donald E. Knuth in volume 2 of his book The Art of Computer Programming as "Algorithm P".
Neither Durstenfeld nor Knuth, in the first edition of his book, acknowledged the earlier work of Fisher and Yates in
any way, and may not have been aware of it. Subsequent editions of The Art of Computer Programming do,
however, mention Fisher and Yates' contribution.
The algorithm described by Durstenfeld differs from that given by Fisher and Yates in a small but significant way.
Whereas a naive computer implementation of Fisher and Yates' method would spend needless time counting the
remaining numbers in step 3 above, Durstenfeld's solution is to move the "struck" numbers to the end of the list by
swapping them with the last unstruck number at each iteration. This reduces the algorithm's time complexity to O(n),
compared to O(n
) for the naive implementation.
This change gives the following algorithm (for a zero-based
Fisher–Yates shuffle
To shuffle an array a of n elements (indexes 0..n-1):
for i from n − 1 downto 1 do
j ← random integer with 0 ≤ j ≤ i
exchange a[j] and a[i]
The "inside-out" algorithm
The Fisher–Yates shuffle, as implemented by Durstenfeld, is an in-place shuffle. That is, given a preinitialized array,
it shuffles the elements of the array in place, rather than producing a shuffled copy of the array. This can be an
advantage if the array to be shuffled is large.
To simultaneously initialize and shuffle an array, a bit more efficiency can be attained by doing an "inside-out"
version of the shuffle. In this version, one successively places element number i into a random position among the
first i positions in the array, after moving the element previously occupying that position to position i. In case the
random position happens to be number i, this "move" (to the same place) involves an uninitialised value, but that
does not matter, as the value is then immediately overwritten. No separate initialization is needed, and no exchange
is performed. In the common case where source is defined by some simple function, such as the integers from 0 to n
- 1, source can simply be replaced with the function since source is never altered during execution.
To initialize an array a of n elements to a randomly shuffled copy of source, both 0-based:
a[0] ← source[0]
for i from 1 to n − 1 do
j ← random integer with 0 ≤ j ≤ i
a[i] ← a[j]
a[j] ← source[i]
The inside-out shuffle can be seen to be correct by induction; every one of the n! different sequences of random
numbers that could be obtained from the calls of random will produce a different permutation of the values, so all of
these are obtained exactly once.
Pencil-and-paper method
As an example, we'll permute the numbers from 1 to 8 using Fisher and Yates' original method. We'll start by writing
the numbers out on a piece of scratch paper:
Range Roll Scratch Result
1 2 3 4 5 6 7 8
Now we roll a random number k from 1 to 8—let's make it 3—and strike out the kth (i.e. third) number (3, of course)
on the scratch pad and write it down as the result:
Range Roll Scratch Result
1–8 3 1 2 3 4 5 6 7 8 3
Now we pick a second random number, this time from 1 to 7: it turns out to be 4. Now we strike out the fourth
number not yet struck off the scratch pad—that's number 5—and add it to the result:
Range Roll Scratch Result
1–7 4 1 2 3 4 5 6 7 8 3 5
Fisher–Yates shuffle
Now we pick the next random number from 1 to 6, and then from 1 to 5, and so on, always repeating the strike-out
process as above:
Range Roll Scratch Result
1–6 5 1 2 3 4 5 6 7 8 3 5 7
1–5 3 1 2 3 4 5 6 7 8 3 5 7 4
1–4 4 1 2 3 4 5 6 7 8 3 5 7 4 8
1–3 1 1 2 3 4 5 6 7 8 3 5 7 4 8 1
1–2 2 1 2 3 4 5 6 7 8 3 5 7 4 8 1 6
1 2 3 4 5 6 7 8 3 5 7 4 8 1 6 2
Modern method
We'll now do the same thing using Durstenfeld's version of the algorithm: this time, instead of striking out the
chosen numbers and copying them elsewhere, we'll swap them with the last number not yet chosen. We'll start by
writing out the numbers from 1 to 8 as before. For clarity, we'll use a vertical bar (|) to separate the part of the list
that has already been processed from the part that hasn't been permuted yet; of course, no such separator is actually
used in the real algorithm:
Range Roll Scratch | Result
1 2 3 4 5 6 7 8 |
For our first roll, we roll a random number from 1 to 8: this time it's 6, so we swap the 6th and 8th numbers in the
Range Roll Scratch | Result
1–8 6 1 2 3 4 5 8 7 | 6
The next random number we roll from 1 to 7, and turns out to be 2. Thus, we swap the 2nd and 7th numbers and
move on:
Range Roll Scratch | Result
1–7 2 1 7 3 4 5 8 | 2 6
The next random number we roll is from 1 to 6, and just happens to be 6, which means we leave the 6th number in
the list (which, after the swap above, is now number 8) in place and just move to the next step. Again, we proceed
the same way until the permutation is complete:
Fisher–Yates shuffle
Range Roll Scratch | Result
1–6 6 1 7 3 4 5 | 8 2 6
1–5 1 5 7 3 4 | 1 8 2 6
1–4 3 5 7 4 | 3 1 8 2 6
1–3 3 5 7 | 4 3 1 8 2 6
1–2 1 7 | 5 4 3 1 8 2 6
At this point there's nothing more that can be done, so the resulting permutation is 7 5 4 3 1 8 2 6.
Sattolo's algorithm
A very similar algorithm was published in 1986 by Sandra Sattolo for generating uniformly distributed cycles of
(maximal) length n.
The only difference between Durstenfeld's and Sattolo's algorithms is that in the latter, in step
2 above, the random number j is chosen from the range between 1 and i−1 (rather than between 1 and i) inclusive. To
turn the Java example above into an example of Sattolo's algorithm, simply replace rng.nextInt(i) with
rng.nextInt(i-1) in the code. This simple change modifies the algorithm so that the resulting permutation always
consists of a single cycle.
In fact, as described below, it's quite easy to accidentally implement Sattolo's algorithm when the ordinary
Fisher–Yates shuffle is intended. This will bias the results by causing the permutations to be picked from the smaller
set of (n−1)! cycles of length N, instead of from the full set of all n! possible permutations.
The fact that Sattolo's algorithm always produces a cycle of length n can be shown by induction. Assume by
induction that after the initial iteration of the loop, the remaining iterations permute the first n − 1 elements
according to a cycle of length n − 1 (those remaining iterations are just Sattolo's algorithm applied to those first n − 1
elements). This means that tracing the initial element to its new position p, then the element originally at position p
to its new position, and so forth, one only gets back to the initial position after having visited all other positions.
Suppose the initial iteration swapped the final element with the one at (non-final) position k, and that the subsequent
permutation of first n − 1 elements then moved it to position l; we compare the permutation π of all n elements with
that remaining permutation σ of the first n − 1 elements. Tracing successive positions as just mentioned, there is no
difference between π and σ until arriving at position k. But then, under π the element originally at position k is
moved to the final position rather than to position l, and the element originally at the final position is moved to
position l. From there on, the sequence of positions for π again follows the sequence for σ, and all positions will
have been visited before getting back to the initial position, as required.
As for the equal probability of the permutations, it suffices to observe that the modified algorithm involves (n−1)!
distinct possible sequences of random numbers produced, each of which clearly produces a different permutation,
and each of which occurs—assuming the random number source is unbiased—with equal probability. The (n−1)!
different permutations so produced precisely exhaust the set of cycles of length n: each such cycle has a unique cycle
notation with the value n in the final position, which allows for (n−1)! permutations of the remaining values to fill
the other positions of the cycle notation.
A sample implementation of Sattolo's algorithm in Python is:
from random import randrange
def sattoloCycle(items):
i = len(items)
Fisher–Yates shuffle
while i > 1:
i = i - 1
j = randrange(i) # 0 <= j <= i-1
items[j], items[i] = items[i], items[j]
Comparison with other shuffling algorithms
The Fisher–Yates shuffle is quite efficient; indeed, its asymptotic time and space complexity are optimal. Combined
with a high-quality unbiased random number source, it is also guaranteed to produce unbiased results. Compared to
some other solutions, it also has the advantage that, if only part of the resulting permutation is needed, it can be
stopped halfway through, or even stopped and restarted repeatedly, generating the permutation incrementally as
In high-level programming languages with a fast built-in sorting algorithm, an alternative method, where each
element of the set to be shuffled is assigned a random number and the set is then sorted according to these numbers,
may be faster in practice despite having worse asymptotic time complexity (O(n log n) vs. O(n)). Like the
Fisher–Yates shuffle, this method produces unbiased results, but may be more tolerant of certain kinds of bias in the
random numbers. However, care must be taken to ensure that the assigned random numbers are never duplicated,
since sorting algorithms typically don't order elements randomly in case of a tie.
A variant of the above method that has seen some use in languages that support sorting with user-specified
comparison functions is to shuffle a list by sorting it with a comparison function that returns random values.
However, this is an extremely bad method: it is very likely to produce highly non-uniform distributions, which in
addition depends heavily on the sorting algorithm used.

For instance suppose quicksort is used as sorting
algorithm, with a fixed element selected as first pivot element. The algorithm starts comparing the pivot with all
other elements to separate them into those less and those greater than it, and the relative sizes of those groups will
determine the final place of the pivot element. For a uniformly distributed random permutation, each possible final
position should be equally likely for the pivot element, but if each of the initial comparisons returns "less" or
"greater" with equal probability, then that position will have a binomial distribution for p = 1/2, which gives
positions near the middle of the sequence with a much higher probability for than positions near the ends. Other
sorting methods like merge sort may produce results that appear more uniform, but are not quite so either, since
merging two sequences by repeatedly choosing one of them with equal probability (until the choice is forced by the
exhaustion of one sequence) does not produce results with a uniform distribution; instead the probability to choose a
sequence should be proportional to the number of elements left in it. In fact no method that uses only two-way
random events with equal probability ("coin flipping"), repeated a bounded number of times, can produce
permutations of a sequence (of more than two elements) with a uniform distribution, because every execution path
will have as probability a rational number with as denominator a power of 2, while the required probability 1/n! for
each possible permutation is not of that form.
In principle this shuffling method can even result in program failures like endless loops or access violations, because
the correctness of a sorting algorithm may depend on properties of the order relation (like transitivity) that a
comparison producing random values will certainly not have.
While this kind of behaviour should not occur with
sorting routines that never perform a comparison whose outcome can be predicted with certainty (based on previous
comparisons), there can be valid reasons for deliberately making such comparisons. For instance the fact that any
element should compare equal to itself allows using them as sentinel value for efficiency reasons, and if this is the
case, a random comparison function would break the sorting algorithm.
Fisher–Yates shuffle
Potential sources of bias
Care must be taken when implementing the Fisher–Yates shuffle, both in the implementation of the algorithm itself
and in the generation of the random numbers it is built on, otherwise the results may show detectable bias. A number
of common sources of bias have been listed below.
Implementation errors
A common error when implementing the Fisher–Yates shuffle is to pick the random numbers from the wrong range.
The resulting algorithm may appear to work, but will produce biased results. For example, a common off-by-one
error would be choosing the index j of the entry to swap in the example above to be always strictly less than the
index i of the entry it will be swapped with. This turns the Fisher–Yates shuffle into Sattolo's algorithm, which
produces only permutations consisting of a single cycle involving all elements: in particular with this modification
no element of the array can ever end up in its original position.
Order bias from incorrect implementation
Order bias from incorrect implementation - n =
Similarly, always selecting j from the entire range of valid array
indexes on every iteration also produces a result which is biased, albeit
less obviously so. This can be seen from the fact that doing so yields n
distinct possible sequences of swaps, whereas there are only n!
possible permutations of an n-element array. Since n
can never be
evenly divisible by n! when n > 2 (as the latter is divisible by n−1,
which shares no prime factors with n), some permutations must be
produced by more of the n
sequences of swaps than others. As a
concrete example of this bias, observe the distribution of possible
outcomes of shuffling a three-element array [1, 2, 3]. There are 6
possible permutations of this array (3! = 6), but the algorithm produces
27 possible shuffles (3
= 27). In this case, [1, 2, 3], [3, 1, 2], and [3, 2,
1] each result from 4 of the 27 shuffles, while each of the remaining 3
permutations occurs in 5 of the 27 shuffles.
The matrix to the right shows the probability of each element in a list
of length 7 ending up in any other position. Observe that for most
elements, ending up in their original position (the matrix's main
diagonal) has lowest probability, and moving one slot backwards has
highest probability.
Modulo bias
Doing a Fisher–Yates shuffle involves picking uniformly distributed
random integers from various ranges. Most random number generators,
however—whether true or pseudorandom—will only directly provide
numbers in some fixed range, such as, say, from 0 to 2
−1. A simple and commonly used way to force such
numbers into a desired smaller range is to apply the modulo operator; that is, to divide them by the size of the range
and take the remainder. However, the need, in a Fisher–Yates shuffle, to generate random numbers in every range
from 0–1 to 0–n pretty much guarantees that some of these ranges will not evenly divide the natural range of the
random number generator. Thus, the remainders will not always be evenly distributed and, worse yet, the bias will be
systematically in favor of small remainders.
For example, assume that your random number source gives numbers from 0 to 99 (as was the case for Fisher and
Yates' original tables), and that you wish to obtain an unbiased random number from 0 to 15. If you simply divide
the numbers by 16 and take the remainder, you'll find that the numbers 0–3 occur about 17% more often than others.
Fisher–Yates shuffle
This is because 16 does not evenly divide 100: the largest multiple of 16 less than or equal to 100 is 6×16 = 96, and
it is the numbers in the incomplete range 96–99 that cause the bias. The simplest way to fix the problem is to discard
those numbers before taking the remainder and to keep trying again until a number in the suitable range comes up.
While in principle this could, in the worst case, take forever, in practice the expected number of retries will always
be less than one.
A related problem occurs with implementations that first generate a random floating-point number—usually in the
range [0,1)—and then multiply it by the size of the desired range and round down. The problem here is that random
floating-point numbers, however carefully generated, always have only finite precision. This means that there are
only a finite number of possible floating point values in any given range, and if the range is divided into a number of
segments that doesn't divide this number evenly, some segments will end up with more possible values than others.
While the resulting bias will not show the same systematic downward trend as in the previous case, it will still be
Pseudorandom generators: problems involving state space, seeding, and usage
An additional problem occurs when the Fisher–Yates shuffle is used with a pseudorandom number generator or
PRNG: as the sequence of numbers output by such a generator is entirely determined by its internal state at the start
of a sequence, a shuffle driven by such a generator cannot possibly produce more distinct permutations than the
generator has distinct possible states. Even when the number of possible states exceeds the number of permutations,
the irregular nature of the mapping from sequences of numbers to permutations means that some permutations will
occur more often than others. Thus, to minimize bias, the number of states of the PRNG should exceed the number
of permutations by at least several orders of magnitude.
For example, the built-in pseudorandom number generator provided by many programming languages and/or
libraries may often have only 32 bits of internal state, which means it can only produce 2
different sequences of
numbers. If such a generator is used to shuffle a deck of 52 playing cards, it can only ever produce a very small
fraction of the 52! ≈ 2
possible permutations. It's impossible for a generator with less than 226 bits of internal
state to produce all the possible permutations of a 52-card deck. It has been suggested that confidence that the shuffle
is unbiased can only be attained with a generator with more than about 250 bits of state.
Also, of course, no pseudorandom number generator can produce more distinct sequences, starting from the point of
initialization, than there are distinct seed values it may be initialized with. Thus, a generator that has 1024 bits of
internal state but which is initialized with a 32-bit seed can still only produce 2
different permutations right after
initialization. It can produce more permutations if one exercises the generator a great many times before starting to
use it for generating permutations, but this is a very inefficient way of increasing randomness: supposing one can
arrange to use the generator a random number of up to a billion, say 2
for simplicity, times between initialization
and generating permutations, then the number of possible permutations is still only 2
A further problem occurs when a simple linear congruential PRNG is used with the divide-and-take-remainder
method of range reduction described above. The problem here is that the low-order bits of a linear congruential
PRNG are less random than the high-order ones: the low n bits of the generator themselves have a period of at most
. When the divisor is a power of two, taking the remainder essentially means throwing away the high-order bits,
such that one ends up with a significantly less random value. This is an example of the general rule that a
poor-quality RNG or PRNG will produce poor-quality shuffles.
Finally, it is to be noted that even with perfect random number generation, flaws can be introduced into an
implementation by improper usage of the generator. For example, suppose a Java implementation creates a new
generator for each call to the shuffler, without passing constructor arguments. The generator will then be
default-seeded by the language's time-of-day (System.currentTimeMillis() in the case of Java). So if two callers call
the shuffler within a time-span less than the granularity of the clock (one millisecond in the case of Java), the
generators they create will be identical, and (for arrays of the same length) the same permutation will be generated.
Fisher–Yates shuffle
This is almost certain to happen if the shuffler is called many times in rapid succession, leading to an extremely
non-uniform distribution in such cases; it can also apply to independent calls from different threads. A more robust
Java implementation would use a single static instance of the generator defined outside the shuffler function.
[1] Fisher, R.A.; Yates, F. (1948) [1938]. Statistical tables for biological, agricultural and medical research (3rd ed.). London: Oliver & Boyd.
pp. 26–27. OCLC 14222135. (note: 6th edition, ISBN 0-02-844720-4, is available on the web (http:// digital. library.adelaide.edu. au/ coll/
special/ fisher/stat_tab. pdf), but gives a different shuffling algorithm by C. R. Rao)
[2] Durstenfeld, Richard (July 1964). "Algorithm 235: Random permutation" (http:/ / doi. acm.org/ 10. 1145/ 364520.364540). Communications
of the ACM 7 (7): 420. doi:10.1145/364520.364540. .
[3] Knuth, Donald E. (1969). The Art of Computer Programming volume 2: Seminumerical algorithms. Reading, MA: Addison–Wesley.
pp. 124–125. OCLC 85975465.
[4] Knuth (1998) [1969]. The Art of Computer Programming vol. 2 (3rd ed.). Boston: Addison–Wesley. pp. 145–146. ISBN 0-201-89684-2.
OCLC 38207978.
[5] Black, Paul E. (2005-12-19). "Fisher–Yates shuffle" (http:/ / www.nist. gov/ dads/ HTML/fisherYatesShuffle.html). Dictionary of
Algorithms and Data Structures. National Institute of Standards and Technology. . Retrieved 2007-08-09.
[6] Wilson, Mark C. (2004-06-21). "Overview of Sattolo's Algorithm" (http:/ / algo.inria. fr/seminars/ summary/ Wilson2004b. pdf). In F.
Chyzak (ed.). INRIA Research Report. 5542. Algorithms Seminar 2002–2004 (http:/ / algo. inria. fr/seminars/ allyears.html), summary by
Éric Fusy.. pp. 105–108. ISSN 0249-6399. .
[7] "A simple shuffle that proved not so simple after all" (http:/ / szeryf.wordpress. com/ 2007/ 06/ 19/
a-simple-shuffle-that-proved-not-so-simple-after-all/ ). require ‘brain’. 2007-06-19. . Retrieved 2007-08-09.
[8] "Doing the Microsoft Shuffle: Algorithm Fail in Browser Ballot" (http:// www.robweir.com/ blog/ 2010/ 02/
microsoft-random-browser-ballot. html). Rob Weir: An Antic Disposition. 2010-02-27. . Retrieved 2010-02-28.
[9] "Writing a sort comparison function, redux" (http:// blogs.msdn. com/ oldnewthing/archive/ 2009/ 05/ 08/ 9595334. aspx). require ‘brain’.
2009-05-08. . Retrieved 2009-05-08.
Global Consciousness Project
The Global Consciousness Project (GCP, also called the EGG Project) is a parapsychology experiment begun in
1998, described as an attempt to detect potential interactions of "global consciousness" with physical systems. The
project monitors a geographically distributed network of hardware random number generators to uncover potential
anomalies in their output which might correlate with world events that elicit widespread emotional response or
focused attention by large numbers of people.
According to the GCP, the experiment aims to test a conjecture they
feel would extend the range of anomalous phenomena currently encompassed by psi research. The GCP is privately
funded through the Institute of Noetic Sciences
and describes itself as an international collaboration of about 100
research scientists and engineers.
Skeptics such as Robert T. Carroll, Claus Larsen, and others have questioned the methodology of the Global
Consciousness Project, particularly how the data are selected and interpreted,

saying the data anomalies
reported by the project are the result of "pattern matching" and selection bias which ultimately fail to support a belief
in psi or global consciousness.
Global Consciousness Project
Roger D. Nelson developed the project as an extrapolation of two decades of experiments from the controversial
Princeton Engineering Anomalies Research Lab (PEAR),
which Nelson says appeared to show that electronic
noise-based, random number generators (RNG or REG, random event generators) seem to be influenced by human
consciousness to bring about a less-than-random sequence of data.
In an extension of the laboratory research called FieldREG, investigators examined the outputs of REGs in the field,
before, during and after highly focused or coherent group events. The group events studied included psychotherapy
sessions, theater presentations, religious rituals, sports competitions such as the Football World Cup, and television
broadcasts like the Academy Awards.
FieldREG was extended to global dimensions in studies looking at data from 12 independent REGs in the US and
Europe during a web-promoted "Gaiamind Meditation" in January 1997, and then again in September 1997 after the
death of Diana, Princess of Wales. The results suggested it would be worthwhile to build a permanent network of
continuously-running REGs.
This became the EGG project or Global Consciousness Project.
Comparing the GCP to PEAR, Nelson, referring to the "field" studies with REGs done by PEAR, said the GCP used
"exactly the same procedure... applied on a broader scale."
The GCP's methodology is based on the hypothesis that events which elicit widespread emotion or draw the
simultaneous attention of large numbers of people may affect the output of hardware random number generators in a
statistically significant way.
The GCP maintains a network of hardware random number generators which are
interfaced to computers at 65 locations around the world. Custom software reads the output of the random number
generators and records a trial (sum of 200 bits) once every second. The data are sent to a server in Princeton, creating
a database of synchronized parallel sequences of random numbers. The GCP is run as a replication experiment,
essentially combining the results of many distinct tests of the hypothesis. The hypothesis is tested by calculating the
extent of data fluctuations at the time of events. The procedure is specified by a three-step experimental protocol.
In the first step, the event duration and the calculation algorithm are pre-specified and entered into a formal
In the second step, the event data are extracted from the database and a Z score, which indicates the
degree of deviation from the null hypothesis, is calculated from the pre-specified algorithm. In the third step, the
event Z-score is combined with the Z-scores from previous events to yield an overall result for the experiment. The
GCP claims that, as of late 2009, the cumulative result of more than 300 registered events significantly supports their
The remote devices have been dubbed Princeton Eggs, a reference to the coinage electrogaiagram, a portmanteau of
electroencephalogram and Gaia.
Supporters and skeptics have referred to the aim of the GCP as being analogous
to detecting "a great disturbance in The Force."


Based on an exploratory analysis of 'highly statistically significant' experimental results, the GCP has suggested
changes in the level of randomness may have occurred during the September 11, 2001 attacks at the times of the
plane impacts and the building collapses, and over the two days following the disaster.
Moreover, the GCP has
identified similar 'anomalies' in the EGG data hours and even days before the attacks; while the GCP does not claim
a causal relationship,
such changes—if genuine—would seem to imply either subconscious mass precognition, or
backwards causality.
Independent scientists Edwin May and James Spottiswoode conducted an analysis of the data around the 11
September 2001 events and concluded there was no statistically significant change in the randomness of the GCP
data during the attacks and the apparent significant deviation reported by Nelson and Radin existed only in their
Global Consciousness Project
chosen time window.
Spikes and fluctuations are to be expected in any random distribution of data, and there is
no set time frame for how close a spike has to be to a given event for the GCP to say they have found a
Wolcotte Smith said "A couple of additional statistical adjustments would have to be made to
determine if there really was a spike in the numbers," referencing the data related to September 11, 2001.
Similarly, Jeffrey D. Scargle believes unless both Bayesian and classical p-value analysis agree and both show the
same anomalous effects, the kind of result GCP proposes will not be generally accepted.
In 2003, a New York Times article concluded "All things considered at this point, the stock market seems a more
reliable gauge of the national—if not the global—emotional resonance."
According to The Age, Nelson concedes "the data, so far, is not solid enough for global consciousness to be said to
exist at all. It is not possible, for example, to look at the data and predict with any accuracy what (if anything) the
eggs may be responding to."
Robert Matthews called it "the most sophisticated attempt yet" to prove psychokinesis existed, but cited the
unreliability of significant events to cause statistically significant spikes, concluding "the only conclusion to emerge
from the Global Consciousness Project so far is that data without a theory is as meaningless as words without a
[1] Bancel, P, & Nelson, R. "The GCP Event Experiment: Design, Analytical Methods, Results" (http:/ / noosphere. princeton.edu/ papers/ pdf/
GCP. Events.Mar08. prepress. pdf). Journal Scientific of Exploration (2008) Section 2 of this research article by GCP scientists provides a
concise presentation of the GCP hypothesis and methodology. .
[2] "''Global Consciousness Project: Contributions''" (http:/ / noosphere.princeton.edu/ contributions. html). Noosphere.princeton.edu. .
Retrieved 2010-01-05.
[3] ""Terry Schiavo and the Global Consciousness Project" (Skeptic News, April 27, 2005)" (http:/ / www. skepticnews. com/ 2005/ 04/
terry_schiavo_a. html). . Retrieved 2008-05-05.
[4] Larsen, Claus (1 January 2003). ""An Evening with Dean Radin"" (http:// skepticreport.com/ sr/ ?p=560). Skeptic Report. . Retrieved
[5] "The Skeptic's Dictionary" (http:// www.skepdic. com/ globalconsciousness. html). Skepdic.com. . Retrieved 2010-01-05.
[6] Carey, Benedict (2007-02-06). "A Princeton Lab on ESP Plans to Close Its Doors" (http:/ / www. nytimes. com/ 2007/ 02/ 10/ science/
10princeton. html?pagewanted=1&ei=5090& en=2f8f7bdba3ac59f1&ex=1328763600). New York Times. . Retrieved 2007-08-03.
[7] Bierman, 1996; Blasband, 2000; Nelson, 1995, 1997; Nelson et al., 1996, 1998a, 1998b; Radin, 1997; Radin et al., 1996.
[8] "ejap/diana/abstract" (http:/ / noosphere. princeton. edu/ ejap/ diana/ abstract.html). . Retrieved 2009-10-28.
[9] "The EGG Story" (http:// noosphere. princeton. edu/ story.html). Noosphere.princeton.edu. . Retrieved 2010-01-05.
[10] "Ibid. p.6" (http:// noosphere. princeton.edu/ papers/ pdf/GCP. Events.Mar08.prepress.pdf). .
[11] "GCP Event registry" (http:/ / noosphere.princeton. edu/ pred_formal.html). . Retrieved 2009-10-17.
[12] "GCP Event summaries" (http:// noosphere. princeton.edu/ results. html). . Retrieved 2009-10-17.
[13] "Gathering of a global mind" (http:/ / noosphere. princeton.edu/ story.html). . Retrieved 2008-03-23.
[14] Williams, Bryan J. (12 August 2002). "Exploratory Block Analysis of Field Consciousness Effects on Global RNGs on September 11, 2001"
(http:// noosphere. princeton. edu/ williams/ GCP911. html). Noosphere. . Retrieved 2009-10-07.
[15] "A disturbance in the Force...?" (http:/ / www.boundaryinstitute. org/bi/ randomness. htm). Boundary Institute. . Retrieved 2009-10-07.
[16] ""September 11, 2001: Exploratory and Contextual Analyses"" (http:/ / noosphere.princeton. edu/ terror.html). . Retrieved 2008-07-12.
[17] ""Extended Analysis: September 11, 2001 in Context"" (http:// noosphere.princeton.edu/ terror1.html). . Retrieved 2008-07-12.
[18] Nelson et al. (2002). "Correlations of Continuous Random Data with Major World Events" (http:// noosphere.princeton. edu/ papers/
nelson-pp.pdf). Foundations of Physics Letters. . Retrieved 2009-10-10.
[19] May, E.C e.a. Global Consciousness Project: An Independent Analysis of The 11 September 2001 Events (http:// www. lfr.org/LFR/ csl/
library/ Sep1101.pdf)
[20] "USA Today" (http:// www.usatoday. com/ tech/ 2001/ 12/ 06/ net-interest.htm). USA Today. 2001-12-06. . Retrieved 2010-01-05.
[21] Scargle, Jeffrey D. Was There Evidence of Global Consciousness on September 11, 2001? (http:// noosphere.princeton. edu/ papers/
[22] By J.D. Reed (2003-03-09). "New York Times" (http:// www. nytimes. com/ 2003/ 03/ 09/ nyregion/so-just-what-makes-the-earth-move.
html). Nytimes.com. . Retrieved 2010-01-05.
[23] "Theage.com.au" (http:// www. theage. com. au/ news/ in-depth/ mind-over-matter/2007/ 04/ 26/ 1177459869857. html). Melbourne:
Theage.com.au. 2007-04-28. . Retrieved 2010-01-05.
[24] "The National (Abu Dhabi)" (http:// www.thenational. ae/ article/ 20090209/ FRONTIERS/272091981/ 1036/ OPINION). Thenational.ae.
2009-02-09. . Retrieved 2010-01-05.
Global Consciousness Project
External links
• The Internet-based Global Consciousness Project (http:/ / noosphere. princeton.edu) Global Correlations in
Random Data
• Audio Interview with Global Consciousness Project researchers Dr. Roger Nelson and Dr. Dean Radin (http:/ /
www.skeptiko. com/ 74-radin-nelson-global-consciousness)
Hardware random number generator
This SSL Accelerator computer card uses a
hardware random number generator to generate
cryptographic keys to encrypt data sent over
computer networks.
In computing, a hardware random number generator is an apparatus
that generates random numbers from a physical process. Such devices
are often based on microscopic phenomena that generate a low-level,
statistically random "noise" signal, such as thermal noise or the
photoelectric effect or other quantum phenomena. These processes are,
in theory, completely unpredictable, and the theory's assertions of
unpredictability are subject to experimental test. A quantum-based
hardware random number generator typically consists of a transducer
to convert some aspect of the physical phenomena to an electrical
signal, an amplifier and other electronic circuitry to bring the output of
the transducer into the macroscopic realm, and some type of analog to
digital converter to convert the output into a digital number, often a simple binary digit 0 or 1. By repeatedly
sampling the randomly varying signal, a series of random numbers is obtained.
Hardware random number generators differ from pseudo-random number generators (PRNGs), which are commonly
used in software. These PRNGs use a deterministic algorithm to produce numerical sequences. Although these
pseudo-random sequences pass statistical pattern tests for randomness, by knowing the algorithm and the conditions
used to initialize it, called the "seed", the output can be predicted. While this can quickly generate large quantities of
pseudorandom data, it is vulnerable to cryptanalysis of the algorithm. Cryptographic PRNGs resist determining the
seed from their output, but still require a small amount of high-quality random data for the seed. Hardware RNGs
can generate the seed, or they may be used directly for the random data to protect against potential vulnerabilities in
a PRNG algorithm.
Random number generators can also be built from "random" macroscopic processes, using devices such as coin
flipping, dice, roulette wheels and lottery machines. The presence of unpredictability in these phenomena can be
justified by the theory of unstable dynamical systems and chaos theory. Even though macroscopic processes are
deterministic under Newtonian mechanics, the output of a well-designed device like a roulette wheel cannot be
predicted in practice, because it depends so sensitively on the micro-details of the initial conditions of each use.
Although dice have been mostly used in gambling, and in recent times as "randomizing" elements in games (e.g. role
playing games), the Victorian scientist Francis Galton described a way to use dice to explicitly generate random
numbers for scientific purposes in 1890.
Hardware random number generators are often relatively slow, and they may produce a biased sequence (i.e., some
values are more common than others) that requires debiasing.
Hardware random number generator
Early work
One early way of producing random numbers was by a variation of the same machines used to play keno or select
lottery numbers. Basically, these mixed numbered ping-pong balls with blown air, perhaps combined with
mechanical agitation, and use some method to withdraw balls from the mixing chamber (U.S. Patent 4786056
This method gives reasonable results in some senses, but the random numbers generated by this means are
expensive. The method is inherently slow, and is unusable in most automated situations (i.e., with computers).
On 29 April 1947 RAND Corporation began generating random digits with an "electronic roulette wheel", consisting
of a random frequency pulse source of about 100,000 pulses per second gated once per second with a constant
frequency pulse and fed into a 5-bit binary counter. Douglas Aircraft built the equipment, implementing Cecil
Hasting’s suggestion (RAND P-113)
for a noise source (most likely the well known behavior of the 6D4 miniature
gas thyratron tube, when placed in a magnetic field
). Twenty of the 32 possible counter values were mapped onto
the 10 decimal digits and the other 12 counter values were discarded.
The results of a long run from the RAND machine, carefully filtered and tested, were converted into a table, which
was published in 1955 in the book A Million Random Digits with 100,000 Normal Deviates. The RAND table was a
significant breakthrough in delivering random numbers because such a large and carefully prepared table had never
before been available. It has been a useful source for simulations, modeling, and even for deriving the arbitrary
constants in cryptographic algorithms to demonstrate that the constants had not been selected for (in B. Schneier’s
words) "nefarious purpose(es)." Khufu and Khafre do this, for example.
See: Nothing up my sleeve numbers.
The RAND book is still in print, and remains an important source of random numbers.
A Million Random Digits with 100,000 Normal Deviates
This 1955 book was a product of RAND’s computing power (and patience). The tables of random numbers in
the book have become a standard reference in engineering and econometrics textbooks and have been widely
used in gaming and simulations that employ Monte Carlo trials. Still the largest known source of random
digits and normal deviates, the work is routinely used by statisticians, physicists, polltakers, market analysts,
lottery administrators, and quality control engineers.
— Rand Corporation
Physical phenomena with quantum-random properties
There are two fundamental sources of practical quantum mechanical physical randomness: quantum mechanics at the
atomic or sub-atomic level and thermal noise (some of which is quantum mechanical in origin). Quantum mechanics
predicts that certain physical phenomena, such as the nuclear decay of atoms, are fundamentally random and cannot,
in principle, be predicted (for a discussion of empirical verification of quantum unpredictability, see Bell test
experiments.) And, because we live at a finite, non-zero temperature, every system has some random variation in its
state; for instance, molecules of air are constantly bouncing off each other in a random way (see statistical
mechanics.) This randomness is a quantum phenomenon as well (see phonon.)
Because the outcome of quantum-mechanical events cannot in principle be predicted, they are the ‘gold standard’ for
random number generation. Some quantum phenomena used for random number generation include:
• Shot noise, a quantum mechanical noise source in electronic circuits. The name ‘shot noise’ refers to the sound of
shotgun pellets, dropped, striking a taut membrane. A simple example is a lamp shining on a photodiode. Due to
the uncertainty principle, arriving photons create noise in the circuit. Collecting the noise for use poses some
problems, but this is an especially simple random noise source. However, shot noise energy is not always well
distributed throughout the bandwidth of interest. Gas diode and thyratron electron tubes in a crosswise magnetic
field can generate substantial noise energy (10 volts or more into high impedance loads) but have a very peaked
energy distribution and require careful filtering to achieve flatness across a broad spectrum
Hardware random number generator
• A nuclear decay radiation source (as, for instance, from some kinds of commercial smoke detectors), detected by
a Geiger counter attached to a PC.
• Photons travelling through a semi-transparent mirror, as in the commercial product, Quantis from id Quantique.
The mutually exclusive events (reflection — transmission) are detected and associated to ‘0’ or ‘1’ bit values
Physical phenomena without quantum-random properties
Thermal phenomena are easier to detect. They are (somewhat) vulnerable to attack by lowering the temperature of
the system, though most systems will stop operating at temperatures low enough to reduce noise by a factor of two
(e.g., ~150 K). Some of the thermal phenomena used include:
• Thermal noise from a resistor, amplified to provide a random voltage source.
• Avalanche noise generated from an avalanche diode, or Zener breakdown noise from a reverse-biased zener
• Atmospheric noise, detected by a radio receiver attached to a PC (though much of it, such as lightning noise, is
not properly thermal noise, but most likely a chaotic phenomenon).
Another variable physical phenomenon that is easy to measure is clock drift.
In the absence of quantum effects or thermal noise, other phenomena that tend to be random, although in ways not
easily characterized by laws of physics, can be used. When several such sources are combined carefully (as in, for
example, the Yarrow algorithm or Fortuna CSPRNGs), enough entropy can be collected for the creation of
cryptographic keys and nonces, though generally at restricted rates. The advantage is that this approach needs, in
principle, no special hardware. The disadvantage is that a sufficiently knowledgeable attacker can surreptitiously
modify the software or its inputs, thus reducing the randomness of the output, perhaps substantially. The primary
source of randomness typically used in such approaches is the precise timing of the interrupts caused by mechanical
input/output devices, such as keyboards and disk drives, various system information counters, etc.
This last approach must be implemented carefully and may be subject to attack if it is not. For instance, the
forward-security of the generator in Linux 2.6.10 kernel could be broken with 2
or 2
time complexity.
random number generator used for cryptographic purposes in an early version of the Netscape browser was certainly
vulnerable (and was promptly changed).
One approach in using physical randomness is to convert a noise source into a random bit sequence in a separate
device that is then connected to the computer through an I/O port. The acquired noise signal is amplified, filtered,
and then run through a high-speed voltage comparator to produce a logic signal that alternates states at random
intervals. At least in part, the randomness produced depends on the specific details of the 'separate device'. Care must
also always be taken when amplifying low-level noise to keep out spurious signals, such as power line hum and
unwanted broadcast transmissions, and to avoid adding bias during acquisition and amplification. In some simple
designs, the fluctuating logic value is converted to an RS-232 type signal and presented to a computer’s serial port.
Software then sees this series of logic values as bursts of "line noise" characters on an I/O port. More sophisticated
systems may format the bit values before passing them into a computer.
Another approach is to feed an analog noise signal to an analog to digital converter, such as the audio input port built
into most personal computers. The digitized signal may then be processed further in software to remove bias.
However, digitization is itself often a source of bias, sometimes subtle, so this approach requires considerable
caution and care.
Some have suggested using digital cameras, such as webcams, to photograph chaotic macroscopic phenomena. A
group at Silicon Graphics imaged Lava lamps to generate random numbers (U.S. Patent 5732138
). One problem
was determining whether the chaotic shapes generated were actually random — the team decided that they are in
properly operating Lava lamps. Other chaotic scenes could be employed, such as the motion of streamers in a fan air
Hardware random number generator
stream or, probably, bubbles in a fish tank (fish optional). The digitized image will generally contain additional
noise, perhaps not very random, resulting from the video to digital conversion process. A higher quality device might
use two sources and eliminate signals that are common to both — depending on the sources and their physical
locations, this reduces or eliminates interference from outside electric and magnetic fields. This is often
recommended for gambling devices, to reduce cheating by requiring attackers to exploit bias in several "random bit"
Clock drift
There are several ways to measure and use clock drift as a source of randomness.
The Intel 80802 Firmware Hub chip included a hardware RNG using two free running oscillators, one fast and one
slow. A thermal noise source (non-commonmode noise from two diodes) is used to modulate the frequency of the
slow oscillator, which then triggers a measurement of the fast oscillator. That output is then debiased using a von
Neumann type decorrelation step (see below). The output rate of this device is somewhat less than 100,000 bit/s.
This chip was an optional component of the 840 chipset family that supported an earlier Intel bus. It is not included
in modern PCs.
All VIA C3 microprocessors have included a hardware RNG on the processor chip since 2003. Instead of using
thermal noise, raw bits are generated by using four freerunning oscillators which are designed to run at different
rates. The output of two are XORed to control the bias on a third oscillator, whose output clocks the output of the
fourth oscillator to produce the raw bit. Minor variations in temperature, silicon characteristics, and local electrical
conditions cause continuing oscillator speed variations and thus produce the entropy of the raw bits. To further
ensure randomness, there are actually two such RNGs on each chip, each positioned in different environments and
rotated on the silicon. The final output is a mix of these two generators. The raw output rate is tens to hundreds of
megabits per second, and the whitened rate is a few megabits per second. User software can access the generated
random bit stream using new non-privileged machine language instructions.
A software implementation of a related idea on ordinary hardware is included in CryptoLib, a cryptographic routine
library (JB Lacy, DP Mitchell, WM Schell, CryptoLib: Cryptography in software, Proc 4th USENIX Security Symp,
pg 1-17, 1993). The algorithm is called truerand. Most modern computers have two crystal oscillators, one for the
real-time clock and one for the primary CPU clock; truerand exploits this fact. It uses an operating system service
that sets an alarm, running off the real-time clock. One subroutine sets that alarm to go off in one clock tick (usually
1/60th of a second). Another then enters a while loop waiting for the alarm to trigger. Since the alarm will not always
trigger in exactly one tick, the least significant bits of a count of loop iterations, between setting the alarm and its
trigger, will vary randomly, possibly enough for some uses. Truerand doesn't require additional hardware, but in a
multi-tasking system great care must be taken to avoid non-randomizing interference from other processes (e.g., in
the suspension of the counting loop process as the operating system scheduler starts and stops assorted processes).
Dealing with bias
The bit-stream from such systems is prone to be biased, with either 1s or 0s predominating. There are two
approaches to dealing with bias and other artifacts. The first is to design the RNG to minimize bias inherent in the
operation of the generator. One method to correct this feeds back the generated bit stream, filtered by a low-pass
filter, to adjust the bias of the generator. By the central limit theorem, the feedback loop will tend to be well-adjusted
'almost all the time'. Ultra-high speed random number generators often use this method. Even then, the numbers
generated are usually somewhat biased.
Limitation: This bias is only observed in case of uniform type random number generator. There are other types of
random number generation method, and the most common way is exponential distribution. This distribution was
proofed in the discussion of dice rollings. Once the number of dice rolling between the same dice number, can be
measured, it is the exponential distribution: P(x)= (1/6)*(5/6)^x In such case, the generated random number is free
Hardware random number generator
from the bias problem.
Software whitening
A second approach to coping with bias is to reduce it after generation (in software or hardware). Even if the above
hardware bias reduction steps have been taken, the bit-stream should still be assumed to contain bias and correlation.
There are several techniques for reducing bias and correlation, often known by the name "whitening" algorithms, by
analogy with the related problem of producing white noise from a correlated signal. There is another way,
dynamic-statics test that is to make statics randomness check in each random number block in dynamically. This can
be done usable in a short time, 1 gigabytes per second or more. In this method, one block shall be determined as
doubtful one, the block is disregarded and canceled. This is the request of draft of ANSI(X9F1).
John von Neumann invented a simple algorithm to fix simple bias, and reduce correlation. It considers bits two at a
time, taking one of three actions: when two successive bits are equal, they are not used as a random bit; a sequence
of 1,0 becomes a 1; and a sequence of 0,1 becomes a zero. This eliminates simple bias, and is easy to implement as a
computer program or in digital logic. This technique works no matter how the bits have been generated. It cannot
assure randomness in its output, however. What it can do (with significant numbers of discarded bits) is transform a
random bit stream with a frequency of 1’s different from 50% into a stream closer to that frequency.
Another technique for improving a near random bit stream is to exclusive-or the bit stream with the output of a
high-quality cryptographically secure pseudorandom number generator such as Blum Blum Shub or a strong stream
cipher. This can cheaply improve decorrelation and digit bias. This can be done by hardware, like as FPGA and in
case, this can be done faster than software.
A related method which reduces bias in a near random bit stream is to take two or more uncorrelated near random bit
streams, and exclusive or them together. Let the probability of a bit stream producing a 0 be 1/2 + e, where -1/2 ≤ e
≤ 1/2. Then e is the bias of the bitstream. If two uncorrelated bit streams with bias e are exclusive-or-ed together,
then the bias of the result will be 2e². This may be repeated with more bit streams (see also Piling-up lemma).
Some designs apply cryptographic hash functions such as MD5, SHA-1, or RIPEMD-160 or even a CRC function to
all or part of the bit stream, and then use the output as the random bit stream. This is attractive, partly because it is
relatively fast compared to some other methods, but depends entirely on qualities in the hash output for which there
may be little theoretical basis.
Many physical phenomena can be used to generate bits that are highly biased, but each bit is independent from the
others. A Geiger counter (with a sample time longer than the tube recovery time) or a semi-transparent mirror photon
detector both generate bit streams that are mostly "0" (silent or transmission) with the occasional "1" (click or
reflection). If each bit is independent from the others, the Von Neumann strategy generates one random, unbiased
output bit for each of the rare "1" bits in such a highly biased bit stream. Whitening techniques such as the Advanced
Multi-Level Strategy (AMLS)
can extract more output bits—output bits that are just as random and
unbiased—from such a highly biased bit stream.
Hardware random number generator
PRNG with periodically refreshed random key
Other designs use what are believed to be true random bits as the key for a high quality block cipher algorithm,
taking the encrypted output as the random bit stream. Care must be taken in these cases to select an appropriate block
mode, however. In some implementations, the PRNG is run for a limited number of digits, while the hardware
generating device produces a new seed.
Using observed events
Software engineers without true random number generators often try to develop them by measuring physical events
available to the software. An example is measuring the time between user keystrokes, and then taking the least
significant bit (or two or three) of the count as a random digit. A similar approach measures task-scheduling,
network hits, disk-head seek times and other internal events. One Microsoft design includes a very long list of such
internal values (see the CSPRNG article).
The method is risky when it uses computer-controlled events because a clever, malicious attacker might be able to
predict a cryptographic key by controlling the external events. It is also risky because the supposed user-generated
event (e.g., keystrokes) can be spoofed by a sufficiently ingenious attacker, allowing control of the "random values"
used by the cryptography.
However, with sufficient care, a system can be designed that produces cryptographically secure random numbers
from the sources of randomness available in a modern computer. The basic design is to maintain an "entropy pool"
of random bits that are assumed to be unknown to an attacker. New randomness is added whenever available (for
example, when the user hits a key) and an estimate of the number of bits in the pool that cannot be known to an
attacker is kept. Some of the strategies in use include:
• When random bits are requested, return that many bits derived from the entropy pool (by a cryptographic hash
function, say) and decrement the estimate of the number of random bits remaining in the pool. If not enough
unknown bits are available, wait until enough are available. This is the top-level design of the "/dev/random"
device in Linux, written by Theodore Ts'o and used in many other Unix-like operating systems. It provides
high-quality random numbers so long as the estimates of the input randomness are sufficiently cautious. The
Linux "/dev/urandom" device is a simple modification which disregards estimates of input randomness, and is
therefore rather less likely to have high entropy as a result.
• Maintain a stream cipher with a key and Initialization vector (IV) obtained from an entropy pool. When enough
bits of entropy have been collected, replace both key and IV with new random values and decrease the estimated
entropy remaining in the pool. This is the approach taken by the yarrow library. It provides resistance against
some attacks and conserves hard-to-obtain entropy.
It is very easy to misconstruct hardware or software devices which attempt to generate random numbers. Also, most
'break' silently, often producing decreasingly random numbers as they degrade. A physical example might be the
rapidly decreasing radioactivity of the smoke detectors mentioned earlier. Failure modes in such devices are plentiful
and are complicated, slow, and hard to detect.
Because many entropy sources are often quite fragile, and fail silently, statistical tests on their output should be
performed continuously. Many, but not all, such devices include some such tests into the software that reads the
Just as with other components of a cryptosystem, a software random number generator should be designed to resist
certain attacks. Defending against these attacks is difficult. See: random number generator attack.
Hardware random number generator
Estimating entropy
There are mathematical techniques for estimating the entropy of a sequence of symbols. None are so reliable that
their estimates can be fully relied upon; there are always assumptions which may be very difficult to confirm. These
are useful for determining if there is enough entropy in a seed pool, for example, but they cannot, in general,
distinguish between a true random source and a pseudo-random generator.
Performance test
Hardware random number generators should be constantly monitored for proper operation. RFC 4086 and FIPS Pub
140-2 include tests which can be used for this. Also see the documentation for the New Zealand cryptographic
software library cryptlib.
Since many practical designs rely on a hardware source as an input, it will be useful to at least check that the source
is still operating. Statistical tests can often detect failure of a noise source, such as a radio station transmitting on a
channel thought to be empty, for example. Noise generator output should be sampled for testing before being passed
through a "whitener." Some whitener designs can pass statistical tests with no random input. While detecting a large
deviation from perfection would be a sign that a true random noise source has become degraded, small deviations are
normal and can be an indication of proper operation. Correlation of bias in the inputs to a generator design with other
parameters (e.g., internal temperature, bus voltage) might be additionally useful as a further check. Unfortunately,
with currently available (and foreseen) tests, passing such tests is not enough to be sure the output sequences are
random. A carefully chosen design, verification that the manufactured device implements that design and continuous
physical security to insure against tampering may all be needed in addition to testing for high value uses.
[1] http:/ / www. google. com/ patents?vid=4786056
[2] P-113 (http:// www. rand.org/pubs/ papers/ P113), Papers, Rand Corporation, .
[3] Cobine, Curry (1947), "Electrical Noise Generators", Proceedings of the I.R.E. (September 1947): 875–9
[4] Monograph report (http:/ /www. rand.org/pubs/ monograph_reports/MR1418/ index2.html), Rand Corporation, .
[5] Schneier, Bruce, Applied Cryptography.
[6] "Tools" (http:// www.rand. org/about/ tools), About, Rand Corporation, May 2009,
[7] 6D4 electron tube reference, Sylvania.
[8] (PDF), IACR, http:// eprint. iacr.org/2006/ 086. pdf
[9] http:/ / www. google. com/ patents?vid=5732138
[10] Peres, Yuval (March 1992), "Iterating Von Neumann's Procedure for Extracting Random Bits", Annals of Statistics 20 (1): 590–97,
[11] Crowley, Paul, Generating random binary data from Geiger counters (http:// www.ciphergoth.org/crypto/ unbiasing/ ), Cipher Goth, .
• Brown, George W (June 1949), History of Rand’s Million Digits (http:/ / www.rand.org/pubs/ papers/ P113),
papers, RAND Corporation.
• Brown, Bernice (October 1948), Some Tests of the Randomness of a Million Digits (http:/ / www.rand.org/pubs/
papers/ P44), Papers, RAND Corporation.
• "Tube type 6D4", Electron Tube Data handbook, Sylvania, 1957.
• A Million Random Digits with 100,000 Normal Deviates (http:/ / www.rand.org/publications/ classics/
randomdigits/), RAND Corporation.
• Galton, Francis (1890), "Dice for statistical experiments" (http:// www.mugu.com/ galton/ statistician. html),
Nature (42): 13–4.
• (PDF) Randomness and Genuine Random Number Generator With Self-testing Functions (http:/ / www.
letech-rng.jp/ SNA+MC2010-Paper. pdf), Japan: LE Tech RNG.
Hardware random number generator
External links
• RFC 4086 on Randomness Recommendations for Security (replaces earlier RFC 1750 (http:/ / www.ietf. org/
rfc/rfc4086.txt), IETF.
• A Statistical Test Suite for the Validation of Random Number Generators and Pseudo Random Number
Generators for Cryptographic Applications (http:/ / csrc. nist. gov/ rng/ rng2.html), Special Publication, NIST.
• The history of generating random numbers (http:/ / www.americanscientist. org/template/ AssetDetail/ assetid/
20829/page/ 3), American Scientist, p. 3.
• (PDF) Intel hardware Random Number Generator built into Pentium family CPUs (after the PIII) (http:/ /
download.intel. com/ design/ chipsets/ rng/ CRIwp.pdf), Intel.
• Entropy Key (http:// www. entropykey.co. uk/ tech), Simtec, "uses P-N semiconductor junctions reverse biassed
with a high enough voltage to bring them near to, but not beyond, breakdown in order to generate noise".
• Theodore Ts'o (November 1995), random.c — A strong random number generator (http:// www. cs. berkeley.
edu/ ~daw/ rnd/linux-rand).
• Pars Mutaf (February 2006), True random numbers from Wi-Fi background noise (http:/ / www.freewebs. com/
pmutaf/ iwrandom. html), retrieved 2007-04-16.
• video_entropyd (randomness from video) (http:/ / www.vanheusden. com/ ved), Van Heusden.
• audio_entropyd (randomness from audio) (http:// www.vanheusden. com/ aed), Van Heusden.
• randaudio (randomness from audio) (http:/ / randaudio.sourceforge.net), SourceForge.
• Math::TrulyRandom (http:/ /search. cpan. org/~gary/Math-TrulyRandom-1.0/ TrulyRandom.pod), CPAN, a
Perl module that claims to generate actual random numbers from interrupt timing discrepancies.
• Denker, John S, Turbid — High-Entropy Randomness Generator (http:/ / www.av8n. com/ turbid/ paper/turbid.
History of randomness
History of randomness
Ancient fresco of dice players in Pompei
In ancient history, the concepts of chance and
randomness were intertwined with that of fate. Many
ancient peoples threw dice to determine fate, and this
later evolved into games of chance. At the same time,
most ancient cultures used various methods of
divination to attempt to circumvent randomness and

The Chinese were perhaps the earliest people to
formalize odds and chance 3,000 years ago. The Greek
philosophers discussed randomness at length, but only
in non-quantitative forms. It was only in the sixteenth
century that Italian mathematicians began to formalize
the odds associated with various games of chance. The
invention of modern calculus had a positive impact on
the formal study of randomness. In the 19th century the concept of entropy was introduced in physics.
The early part of the twentieth century saw a rapid growth in the formal analysis of randomness, and mathematical
foundations for probability were introduced, leading to its axiomatization in 1933. At the same time, the advent of
quantum mechanics changed the scientific perspective on determinacy. In the mid to late 20th-century, ideas of
algorithmic information theory introduced new dimensions to the field via the concept of algorithmic randomness.
Although randomness had often been viewed as an obstacle and a nuisance for many centuries, in the twentieth
century computer scientists began to realize that the deliberate introduction of randomness into computations can be
an effective tool for designing better algorithms. In some cases, such randomized algorithms are able to outperform
the best deterministic methods.
History of randomness
Antiquity to the Middle Ages
Depiction of Roman Goddess Fortuna who
determined fate, by Hans Beham, 1541
In ancient history, the concepts of chance and randomness were
intertwined with that of fate. Pre-Christian people along the
Mediterranean threw dice to determine fate, and this later evolved into
games of chance.
There is also evidence of games of chance played
by ancient Egyptians, Hindus and Chinese, dating back to 2100 BC.
The Chinese used dice before the Europeans, and have a long history
of playing games of chance.
Over 3,000 years ago, the problems concerned with the tossing of
several coins were considered in the I Ching, one of the oldest Chinese
mathematical texts, that probably dates to 1150 BC. The two principal
elements yin and yang were combined in the I Ching in various forms
to produce Heads and Tails permutations of the type HH, TH, HT, etc.
and the Chinese seem to have been aware of Pascal's triangle long
before the Europeans formalized it in the 17th century.
Western philosophy focused on the non-mathematical aspects of
chance and randomness until the 16th century.
The development of the concept of chance throughout history has been
very gradual. Historians have wondered why progress in the field of
randomness was so slow, given that humans have encountered chance
since antiquity. Deborah Bennett suggests that ordinary people face an inherent difficulty in understanding
randomness, although the concept is often taken as being obvious and self-evident. She cites studies by Kahneman
and Tversky; these concluded that statistical principles are not learned from everyday experience because people do
not attend to the detail necessary to gain such knowledge.
The Greek philosophers were the earliest Western thinkers to address chance and randomness. Around 400 BC,
Democritus presented a view of the world as governed by the unambiguous laws of order and considered
randomness as a subjective concept that only originated from the inability of humans to understand the nature of
events. He used the example of two men who would send their servants to bring water at the same time to cause
them to meet. The servants, unaware of the plan, would view the meeting as random.
Aristotle saw chance and necessity as opposite forces. He argued that nature had rich and constant patterns that could
not be the result of chance alone, but that these patterns never displayed the machine-like uniformity of necessary
determinism. He viewed randomness as a genuine and widespread part of the world, but as subordinate to necessity
and order.
Aristotle classified events into three types: certain events that happen necessarily; probable events that
happen in most cases; and unknowable events that happen by pure chance. He considered the outcome of games of
chance as unknowable.
Around 300 BC Epicurus proposed the concept that randomness exists by itself, independent of human knowledge.
He believed that in the atomic world, atoms would swerve at random along their paths, bringing about randomness at
higher levels.
History of randomness
Hotei, the deity of fortune observing a cock fight
in a 16th-century Japanese print
For several centuries thereafter, the idea of chance continued to be
intertwined with fate. Divination was practiced in many cultures, using
diverse methods. The Chinese analyzed the cracks in turtle shells,
while the Germans, who according to Tacitus had the highest regards
for lots and omens, utilized strips of bark.
In the Roman Empire,
chance was personified by the Goddess Fortuna. The Romans would
partake in games of chance to simulate what Fortuna would have
decided. In 49 BC, Julius Caesar allegedly decided on his fateful
decision to cross the Rubicon after throwing dice.
Aristotle's classification of events into the three classes: certain,
probable and unknowable was adopted by Roman philosophers, but
they had to reconcile it with deterministic Christian teachings in which
even events unknowable to man were considered to be predetermined
by God. About 960 Bishop Wibold of Cambrai correctly enumerated
the 56 different outcomes (without permutations) of playing with three dice. No reference to playing cards has been
found in Europe before 1350. The Church preached against card playing, and card games spread much more slowly
than games based on dice.
The Christian Church specifically forbade divination; and wherever Christianity went,
divination lost most of its old-time power.

Over the centuries, many Christian scholars wrestled with the conflict between the belief in free will and its implied
randomness, and the idea that God knows everything that happens. Saints Augustine and Aquinas tried to reach an
accommodation between foreknowledge and free will, but Martin Luther argued against randomness and took the
position that God's omniscience renders human actions unavoidable and determined.
In the 13th century, Thomas
Aquinas viewed randomness not as the result of a single cause, but of several causes coming together by chance.
While he believed in the existence of randomness, he rejected it as an explanation of the end-directedness of nature,
for he saw too many patterns in nature to have been obtained by chance.
The Greeks and Romans had not noticed the magnitudes of the relative frequencies of the games of chance. For
centuries, chance was discussed in Europe with no mathematical foundation and it was only in the 16th century that
Italian Mathematicians began to discuss the outcomes of games of chance as ratios.


In his 1565 Liber de
Lude Aleae (a gambler's manual published after his death) Gerolamo Cardano wrote one of the first formal tracts to
analyze the odds of winning at various games.
History of randomness
17th–19th centuries
Statue of Blaise Pascal, Louvre
Around 1620 Galileo wrote a paper called On a discovery concerning
dice that used an early probabilistic model to address specific
In 1654, prompted by Chevalier de Méré's interest in
gambling, Blaise Pascal corresponded with Pierre de Fermat, and much
of the groundwork for probability theory was laid. Pascal's Wager was
noted for its early use of the concept of infinity, and the first formal use
of decision theory. The work of Pascal and Fermat influenced Leibniz's
work on the infinitesimal calculus, which in turn provided further
momentum for the formal analysis of probability and randomness.
The first known suggestion for viewing randomness in terms of
complexity was made by Leibniz in an obscure 17th-century document
discovered after his death. Leibniz asked how one could know if a set
of points on a piece of paper were selected at random (e.g. by
splattering ink) or not. Given that for any set of finite points there is
always a mathematical equation that can describe the points, (e.g. by
Lagrangian interpolation) the question focuses on the way the points
are expressed mathematically. Leibniz viewed the points as random if
the function describing them had to be extremely complex. Three
centuries later, the same concept was formalized as algorithmic randomness by A. N. Kolmogorov and Gregory
Chaitin as the minimal length of a computer program needed to describe a finite string as random.
The Doctrine of Chances, the first textbook on probability theory was published in 1718 and the field continued to
grow thereafter.
The frequency theory approach to probability was first developed by Robert Ellis and John Venn
late in the 19th century.
The Fortune Teller by Vouet, 1617
While the mathematical elite was making progress in understanding
randomness from the 17th to the 19th century, the public at large
continued to rely on practices such as fortune telling in the hope of
taming chance. Fortunes were told in a multitude of ways both in the
Orient (where fortune telling was later termed an addiction) and in
Europe by gypsies and others.

English practices such as the
reading of eggs dropped in a glass were exported to Puritan
communities in North America.
The term entropy, which is now a key element in the study of
randomness, was coined by Rudolf Clausius in 1865 as he studied heat
engines in the context of the second law of thermodynamics. Clausius was the first to state "entropy always
From the time of Newton until about 1890, it was generally believed that if one knows the initial state of a system
with great accuracy, and if all the forces acting on the system can be formulated with equal accuracy, it would be
possible, in principle, to make predictions of the state of the universe for an infinitely long time. The limits to such
predictions in physical systems became clear as early as 1893 when Henri Poincaré showed that in the three-body
problem in astronomy, small changes to the initial state could result in large changes in trajectories during the
numerical integration of the equations.
During the 19th century, as probability theory was formalized and better understood, the attitude towards
"randomness as nuisance" began to be questioned. Goethe wrote:
History of randomness
The tissue of the world is built from necessities and randomness; the intellect of men places itself
between both and can control them; it considers the necessity and the reason of its existence; it knows
how randomness can be managed, controlled, and used.
The words of Goethe proved prophetic, when in the 20th century randomized algorithms were discovered as
powerful tools.
By the end of the 19th century, Newton's model of a mechanical universe was fading away as the
statistical view of the collision of molecules in gases was studied by Maxwell and Boltzmann.
equation S = k log
 W (inscribed on his tombstone) first related entropy with logarithms.
20th century
Antony Gormley's Quantum Cloud sculpture in
London was designed by a computer using a
random walk algorithm.
During the 20th century, the five main interpretations of probability
theory (e.g., classical, logical, frequency, propensity and subjective)
became better understood, were discussed, compared and
A significant number of application areas were
developed in this century, from finance to physics. In 1900 Louis
Bachelier applied Brownian motion to evaluate stock options,
effectively launching the fields of financial mathematics and stochastic
Émile Borel was one of the first mathematicians to formally address
randomness in 1909, and introduced normal numbers.
In 1919
Richard von Mises gave the first definition of algorithmic randomness
via the impossibility of a gambling system. He advanced the frequency
theory of randomness in terms of what he called the collective, i.e. a
random sequence. Von Mises regarded the randomness of a collective
as an empirical law, established by experience. He related the
"disorder" or randomness of a collective to the lack of success of
attempted gambling systems. This approach led him to suggest a
definition of randomness that was later refined and made
mathematically rigorous by Alonso Church by using computable functions in 1940.
Richard von Mises likened
the principle of the impossibility of a gambling system to the principle of the conservation of energy, a law that
cannot be proven, but has held true in repeated experiments.
Von Mises never totally formalized his rules for sub-sequence selection, but in his 1940 paper "On the concept of
random sequence", Alonzo Church suggested that the functions used for place settings in the formalism of von Mises
be recursive functions rather than arbitrary functions of the initial segments of the sequence, appealing to the
Church–Turing thesis on effectiveness.

The advent of quantum mechanics in the early 20th century and the formulation of the Heisenberg uncertainty
principle in 1927 saw the end to the Newtonian mindset among physicists regarding the determinacy of nature. In
quantum mechanics, there is not even a way to consider all observable elements in a system as random variables at
once, since many observables do not commute.
History of randomness
Café Central, one of the early meeting places of
the Vienna circle
By the early 1940s, the frequency theory approach to probability was
well accepted within the Vienna circle, but in the 1950s Karl Popper
proposed the propensity theory.

Given that the frequency
approach cannot deal with "a single toss" of a coin, and can only
address large ensembles or collectives, the single-case probabilities
were treated as propensities or chances. The concept of propensity was
also driven by the desire to handle single-case probability settings in
quantum mechanics, e.g. the probability of decay of a specific atom at
a specific moment. In more general terms, the frequency approach can
not deal with the probability of the death of a specific person given that
the death can not be repeated multiple times for that person. Karl
Popper echoed the same sentiment as Aristotle in viewing randomness
as subordinate to order when he wrote that "the concept of chance is not opposed to the concept of law" in nature,
provided one considers the laws of chance.

Claude Shannon's development of Information theory in 1948 gave rise to the entropy view of randomness. In this
view, randomness is the opposite of determinism in a stochastic process. Hence if a stochastic system has entropy
zero it has no randomness and any increase in entropy increases randomness. Shannon's formulation defaults to
Boltzmann's entropy in case all probabilities are equal.

Entropy is now widely used in diverse fields of
science from thermodynamics to quantum chemistry.
Martingales for the study of chance and betting strategies were introduced by Lévy in the 1930s and were formalized
by Doob in the 1950s.
The application of random walk hypothesis in financial theory was first proposed by
Maurice Kendall in 1953.
It was later promoted by Eugene Fama and Burton Malkiel.
A. N. Kolmogorov
Random strings were first studied by in the 1960s by A. N.
Kolmogorov (who had provided the first axiomatic definition of
probability theory in 1933),
Chaitin and Martin-Löf.
algorithmic randomness of a string was defined as the minimum size of
a program (e.g. in bits) executed on a universal computer that yields
the string. Chaitin's Omega number later related randomness and the
halting probability for programs.
In 1964, Benoît Mandelbrot suggested that most statistical models
approached only a first stage of dealing with indeterminism, and that
they ignored many aspects of real world turbulance.

In his 1997
he defined seven states of randomness ranging from "mild to wild",
with traditional randomness being at the mild end of the scale.
Despite mathematical advances, reliance on other methods of dealing with chance, such as fortune telling and
astrology continued in the 20th century. The government of Myanmar reportedly shaped 20th century economic
policy based on fortune telling and planned the move of the capital of the country based on the advice of


White House Chief of Staff Donald Regan criticized the involvement of astrologer Joan
Quigley in decisions made during Ronald Reagan's presidency in the 1980s.


Quigley claims to have been
the White House astrologer for seven years.
During the 20th century, limits in dealing with randomness were better understood. The best-known example of both
theoretical and operational limits on predictability is weather forecasting, simply because models have been used in
the field since the 1950s. Predictions of weather and climate are necessarily uncertain. Observations of weather and
climate are uncertain and incomplete, and the models into which the data are fed are uncertain.
In 1961, Edward
Lorenz noticed that a very small change to the initial data submitted to a computer program for weather simulation
History of randomness
could result in a completely different weather scenario. This later became known as the butterfly effect, often
paraphrased as the question: "Does the flap of a butterfly’s wings in Brazil set off a tornado in Texas?".
A key
example of serious practical limits on predictability is in geology, where the ability to predict earthquakes either on
an individual or on a statistical basis remains a remote prospect.
In the late 1970s and early 1980s, computer scientists began to realize that the deliberate introduction of randomness
into computations can be an effective tool for designing better algorithms. In some cases, such randomized
algorithms outperform the best deterministic methods.
[1] Handbook to Life in Ancient Rome, Lesley Adkins, 1998 ISBN 0195123328 p. 279
[2] Religions of the Ancient World, Sarah Iles Johnston, 2004 ISBN 0674015177 p. 370
[3] What is Random?: Chance and Order in Mathematics and Life, Edward J. Beltrami, 1999, Springer ISBN 0387987371 pp. 2-4
[4] Encyclopedia of Leisure and Outdoor Recreation, John Michael Jenkins, 2004 ISBN 0415252261 p. 194
[5] Audacious Angles of China, Elise Mccormick, 2007 ISBN 1406753327 p. 158
[6] The Nature and Growth of Modern Mathematics, Edna Ernestine Kramer, 1983 ISBN p. 313
[7] Randomness, Deborah J. Bennett, Harvard University Press, 1998. ISBN 0-674-10745-4 pp. 8-9 and 24
[8] Design and Analysis of Randomized Algorithms, Juraj Hromkovič, 2005 ISBN 3540239499 p. 1
[9] Aristotle's Physics: a Guided Study, Joe Sachs, 1995 ISBN 0813521920 p. 70
[10] A History of Probability and Statistics and Their Applications before 1750, Anders Hald, 2003 ISBN 0471471291 p. 30
[11] Epicurus: an Introduction, John M. Rist, 1972 ISBN 0521084261 p. 52
[12] The Age of Chance, Gerda Reith, 2000 ISBN 0415179971 p. 15; Tac. Germ. 10
[13] What is Random?: Chance and Order in Mathematics and Life, Edward J. Beltrami, 1999, Springer ISBN 0387987371 pp. 3-4
[14] A History of Probability and Statistics and Their Applications before 1750, Anders Hald, 2003 ISBN 0471471291 pp. 29-36
[15] A general history of the Christian church Volume 2 by Joseph Priestley 1804 ASIN B002KW4M6O page 11
[16] Catholic encyclopedia (http:/ / www.newadvent. org/cathen/ 05048b. htm)
[17] The Case for Humanism, Lewis Vaughn, Austin Dacey, 2003 ISBN 0742513939 p. 81
[18] The treatise on the divine nature: Summa theologiae I, 1-13, by Saint Thomas Aquinas, Brian J. Shanley, 2006 ISBN 0872208052 p. 198
[19] A History of Probability and Statistics and Their Applications before 1750, Anders Hald, 2003 ISBN 0471471291 pp. 30-4
[20] World of Scientific Discovery , Kimberley A. McGrath and Bridget Traverspage, 1999 ISBN 0787627607 p. 893
[21] Randomness, Deborah J. Bennett, Harvard University Press, 1998. ISBN 0-674-10745-4 p. 8
[22] A Dictionary of Scientists, John Daintith, Derek Gjertsen, 1999 ISBN 0192800868 p. 88
[23] A History of Probability and Statistics and Their Applications before 1750, Anders Hald, 2003 ISBN 0471471291 p. 41
[24] Thinking about Gödel and Turing, Gregory J. Chaitin, 2007 ISBN 9812708960 p. 242
[25] Schneider, Ivo (2005), "Abraham De Moivre, The Doctrine of Chances (1718, 1738, 1756)", in Grattan-Guinness, I., Landmark Writings in
Western Mathematics 1640-1940, Amsterdam: Elsevier, p. 105–120, ISBN 0444508716
[26] Asia in the Making of Europe, Volume 3, Donald Frederick Lach, Edwin J. Van Kley, 1998 ISBN 0226467694 p. 1660
[27] A History of the Gypsies of Eastern Europe and Russia, David M. Crowe, 1996 ISBN 0312129467 p. 36
[28] Events that Changed America through the Seventeenth Century, John E. Findling, Frank W. Thackeray, 2000 ISBN 0313290830 p. 168
[29] Great physicists by William H. Cropper 2004 ISBN page 93
[30] On Limited Predictability, A. Wiin-Nielsen, 1999 ISBN 8773041858 p. 3
[31] Design and Analysis of Randomized Algorithms, Juraj Hromkovič, 2005 ISBN 3540239499 p. 4
[32] Encyclopedia of science and technology by James S. Trefil 2001 ISBN 0415937248 Page cxxxiii
[33] Stanford Encyclopedia of Philosophy (http:/ / plato. stanford.edu/ entries/ probability-interpret/)
[34] E. Borel, Les probabilites denombrables et leurs applications arithmetique Rend. Circ. Mat. Palermo 27 (1909) 247-271
[35] Companion Encyclopedia of the History and Philosophy Volume 2, Ivor Grattan-Guinness 0801873975 p. 1412
[36] The Philosophy of Karl Popper, Herbert Keuth ISBN 0521548306 p. 171
[37] Alonzo Church, "On the concept of random sequence," Bull. Amer. Math. Soc., 46 (1940), 254–260
[38] J. Alberto Coffa, "Randomness and knowledge," in PSA 1972: proceedings of the 1972 Biennial Meeting Philosophy of Science Association,
Volume 20, Springer, 1974 ISBN 9027704082 p. 106
[39] Introduction to random time and quantum randomness by Kai Lai Chung, Jean-Claude Zambrini 2003 ISBN 9812384154 page
[40] Karl Popper, 1957, "The propensity interpretation of the calculus of probability and the quantum theory”, in S. Körner (ed.), The Colston
Papers, 9: 65–70.
[41] Karl Popper, 1959, "The propensity interpretation of probability", British Journal of the Philosophy of Science, 10: 25–42.
[42] Karl Popper, The Logic of Scientific Discovery p. 206
[43] The Philosophy of Karl Popper, Herbert Keuth ISBN 0521548306 p. 170
[44] Single Orbit Dynamics, Benjamin Weiss 1999 ISBN 0821804146 p. 83
History of randomness
[45] The mathematical theory of information by Jan Kåhre 2002 ISBN 1402070640 page 218
[46] Reviews in Computational Chemistry, Volume 23 by Kenneth B. Lipkowitz ISBN 0470082011 page 279
[47] Martingale approximation by Yu. V. Borovskikh, 1997 ISBN 9067642711 page 287
[48] Kendall, M. G. (1953). "The analysis of economic time-series-part I: prices", Journal of the Royal Statistical Society. A (General) 116 (1):
11–34. (http:// www. jstor. org/ stable/ 2980947)
[49] Probability theory: the logic of science by Edwin T. Jaynes, G. Larry Bretthorst 2003 ISBN 0521592712 page 49
[50] Information and Randomness: an Algorithmic Perspective, Cristian Calude, 2002 ISBN 3540434666 p. 145
[51] Thinking about Gödel and Turing, Gregory J. Chaitin, 2007 ISBN 9812708960 p. 185
[52] Gaussian Self-Affinity and Fractals by Benoit Mandelbrot, F.J. Damerau, M. Frame, and K. McCamy 2001 ISBN 0387989935 page 20
[53] The effortless economy of science? by Philip Mirowski 2004 ISBN 0822333228 page 255
[54] Fractals and scaling in finance by Benoît Mandelbrot 1997 ISBN 0387983635 pages 136-142
[55] Myanmar (Burma) since 1962: the failure of development by Peter John Perry 2007 ISBN 0754645347 page 10
[56] Asia Times, June 18, 2009 Instant karma in Myanmar (http:// www. atimes. com/ atimes/ Southeast_Asia/ KF18Ae02.html)
[57] NY Times, November 11, 2005 A government on the move to a half-built capital, (http:// www.nytimes. com/ 2005/ 11/ 10/ world/asia/
[58] Time Magazine, May 16, 1988, Good Heavens! (http:// www.cnn. com/ ALLPOLITICS/1997/ 05/ 19/ back.time/ )
[59] Encyclopedia of the Reagan-Bush years 1996 by Peter B. Levy ISBN 0313290180 page 25
[60] Exit with honor: the life and presidency of Ronald Reagan by William E. Pemberton 1997 ISBN 0765600951 page 123
[61] Quigley, Joan. What Does Joan Say?: My Seven Years as White House Astrologer to Nancy and Ronald Reagan. Carol Publishing Group.
New York, NY; 1990
[62] Predictability of Weather and Climate, Tim Palmer, Renate Hagedorn, 2006 ISBN 0521848822 p. 1
[63] Storm Warning: The Story of a Killer Tornado, Nancy Mathis, 2007 ISBN 0-7432-8053-2 p. x
[64] L. Knopoff, "Earthquake prediction: the scientific challenge", Proceedings of the National Academy of Sciences, 1999 ISBN 0309058376 p.
[65] Design and Analysis of Randomized Algorithms, Juraj Hromkovič 2005 ISBN 3540239499 p. 4
Ignorance space
The ignorance space is the first component of the Bernoulli space which constitutes the stochastic model within
Bernoulli stochastics
. Ignorance means lack of knowledge about facts. Since Bernoulli Stochastics is a
mathematical science, the facts refer to characteristics that are quantified by variables, and each fact is therefore
represented by a unique real number. Because the considered variable has a fixed, i.e., determinate value, the
variable is called deterministic variable denoted D. For example, consider a given object and let the characteristic of
interest be the mass of the object. Then the deterministic variable D stands for mass which is quantified by the unit
kilogram and for the object at hand D is fixed by a real number.
Ignorance, knowlede and, truth
In the above example the mass of the object is fixed and the corresponding real number, say specifies the value of
D kilogram. The real number constitutes the truth about the mass of the object. However, human beings are not
able to determine the truth given by , since whatever measurement device is used, the measurement process is
subject to randomness and the determination of the true value of the deterministic variable D is in principle
It follows that human beings in general cannot know "what is", for example, the true value . But, they can, of
course know "what is not" since any weighung device yields a more or less small set which contains the true but
unknown value . Thus human knowledge generally refers to "what is not" namely all numbers which are not
contained in , but it remains unknown which of the elements of is the true one.
Thus, human ignorance refers to truth, i.e., "what is", while human knowledge refers "what is not". The set
describes therefore the state of ignorance about the true value of the deterministic variable D. Therefore the set
is called ignorance space of D denoted .
Ignorance space
Learning and ignorance space
Any reduction of a ignorance space represents a the result of a learning process. In order to obtain a quantified
desription of a learning process, the ignorance space must be specified before the learning prcess is performed.
Moreover, for judging the achieved result of a learning process, the hopefully reduced ignorance space must be
Whenever the truth is given by a real number, say then a learning process is generally called measurement
procedure or in case that it is based on a Bernoulli Space it is called stochastic measurement procedure.
Measurement and learning are probably the most important activities of mankind and it is therefor of utmost
imporatnce that measurement and learning are performed in a rational and effective way in order to avoid or at least
reduce waste of resources.
[1] Elart von Collani (ed.), Defining the Science Stochastics, Heldermann Verlag, Lemgo, 2004.
[2] Elart von Collani, Defining and medelling uncertainty, Journal of Uncertain Systems, Vol. 2, 202–211, 2008, (http:/ / www.
worldacademicunion. com/ journal/ jus/ jusVol02No3paper05. pdf).
External links
• Stochastikon Ecyclopedia, (http:// www. encyclopedia. stochastikon. com)
• E-Learning Programme Stochastikon Magister, (http:// www.magister. stochastikon. com)
• Homepage of Stochastikon GmbH, (http:/ / www. stochastikon. com/ )
• Economic Quality Control, (http:// www. heldermann-verlag.de/ eqc/ eqc23/ eqc23003. pdf)
Indeterminacy (philosophy)
Indeterminacy, in philosophy, can refer both to common scientific and mathematical concepts of uncertainty and
their implications and to another kind of indeterminacy deriving from the nature of definition or meaning. It is
related to deconstructionism and to Nietzsche's criticism of the Kantian noumenon.
Indeterminacy in philosophy
Indeterminacy was discussed in one of Jacques Derrida's early works Plato's Pharmacy (1969)
, a reading of
Plato's Phaedrus and Phaedo. Plato writes of a fictionalized conversation between Socrates and a student, in which
Socrates tries to convince the student that writing is inferior to speech. Socrates uses the Egyptian myth of Thoth's
creation of writing to illustrate his point. As the story goes, Thoth presents his invention to the god-king of Upper
Egypt for judgment. Upon its presentation, Thoth offers script as a pharmakon for the Egyptian people. The Greek
word pharmakon poses a quandary for translators- it is both a remedy and a poison. In the proffering of a
pharmakon, Thoth presents it as its true meaning- a harm and benefit. The god-king, however, refuses the invention.
Through various reasonings, he determines the pharmakon of writing to be a bad thing for the Egyptian people. The
pharmakon, the undecidable, has been returned decided. The problem, as Derrida reasons, is this: since the word
pharmakon, in the original Greek, means both a remedy and a poison, it cannot be determined as fully remedy or
fully poison. Amon rejected writing as fully poison in Socrates' retelling of the tale, thus shutting out the other
The problem of indeterminacy arises when one observes the eventual circularity of virtually every possible
definition. It is easy to find loops of definition in any dictionary, because this seems to be the only way that certain
Indeterminacy (philosophy)
concepts, and generally very important ones such as that of existence, can be defined in the English language. A
definition is a collection of other words, and in any finite dictionary if one continues to follow the trail of words in
search of the precise meaning of any given term, one will inevitably encounter this linguistic indeterminacy.
Philosophers and scientists generally try to eliminate indeterminate terms from their arguments, since any
indeterminate thing is unquantifiable and untestable; similarly, any hypothesis which consists of a statement of the
properties of something unquantifiable or indefinable cannot be falsified and thus cannot be said to be supported by
evidence that does not falsify it. This is related to Popper's discussions of falsifiability in his works on the scientific
method. The quantifiability of data collected during an experiment is central to the scientific method, since reliable
conclusions can only be drawn from replicable experiments, and since in order to establish observer agreement
scientists must be able to quantify experimental evidence.
Immanuel Kant unwittingly proposed one answer to this question in his Critique of Pure Reason by stating that there
must "exist" a "thing in itself" – a thing which is the cause of phenomena, but not a phenomenon itself. But, so to
speak, "approximations" of "things in themselves" crop up in many models of empirical phenomena: singularities in
physics, such as gravitational singularities, certain aspects of which (e.g., their unquantifiability) can seem almost to
mirror various "aspects" of the proposed "thing in itself", are generally eliminated (or attempts are made at
eliminating them) in newer, more precise models of the universe; and definitions of various psychiatric disorders
stem, according to philosophers who draw on the work of Michel Foucault, from a belief that something
unobservable and indescribable is fundamentally "wrong" with the mind of whoever suffers from such a disorder:
proponents of Foucault's treatment of the concept of insanity would assert that one need only try to quantify various
characteristics of such disorders as presented in today's Diagnostic and Statistical Manual – delusion, one of the
diagnostic criteria which must be exhibited by a patient if he or she is to be considered schizophrenic, for example –
in order to discover that the field of study known as abnormal psychology relies upon indeterminate concepts in
defining virtually each "mental disorder" it describes. The quality that makes a belief a delusion is indeterminate to
the extent to which it is unquantifiable; arguments that delusion is determined by popular sentiment (i.e., "almost
no-one believes that he or she is made of cheese, and thus that belief is a delusion") would lead to the conclusion
that, for example, Alfred Wegener's assertion of continental drift was a delusion since it was dismissed for decades
after it was made.
Nietzsche and the indeterminacy of the "thing in itself"
Relevant criticism of Kant's original formulation of the "thing in itself" can be found in the works of Friedrich
Wilhelm Nietzsche, who argued against what he held to be the indeterminacy of such concepts as the Platonic idea,
the subject, the Kantian noumenon, the opposition of "appearance" to "reality", etc. Nietzsche concisely argued
against Kant's noumenon in his On Truth and Lies in a Nonmoral Sense as follows:
"The 'thing in itself' (which is precisely what the pure truth, apart from any of its consequences, would be) is likewise
something quite incomprehensible to the creator of language and something not in the least worth striving for."
In his Beyond Good and Evil, Nietzsche argues against the "misleading significance of words" and its production of
a "thing in itself":
"I would repeat it, however, a hundred times, that 'immediate certainty,' as well as 'absolute knowledge' and the
'thing in itself,' involve a CONTRADICTIO IN ADJECTO; we really ought to free ourselves from the misleading
significance of words!"
Furthermore, Nietzsche argued against such singularities as the atom in the scientific models of his day in The Will
to Power:
"For all its detachment and freedom from emotion, our science is still the dupe of linguistic habits; it has never got
rid of those changelings called 'subjects.' The atom is one such changeling, another is the Kantian 'thing-in-itself.'"
Indeterminacy (philosophy)
Approximation versus equality
The concept of something that is unapproachable but always further-approximable has led to a rejection by
philosophers like Nietzsche of the concept of exact equality in general in favor of that of approximate similarity:
"Every word instantly becomes a concept precisely insofar as it is not supposed to serve as a reminder of the unique
and entirely individual original experience to which it owes its origin; but rather, a word becomes a concept insofar
as it simultaneously has to fit countless more or less similar cases – which means, purely and simply, cases which
are never equal and thus altogether unequal."
"What then is truth? A movable host of metaphors, metonymies, and; anthropomorphisms: in short, a sum of human
relations which have been poetically and rhetorically intensified, transferred, and embellished, and which, after long
usage, seem to a people to be fixed, canonical, and binding. Truths are illusions which we have forgotten are
illusions- they are metaphors that have become worn out and have been drained of sensuous force, coins which have
lost their embossing and are now considered as metal and no longer as coins."
If one states an equation between two things, one states, in effect, that they are the same thing. It can be argued that
this cannot possibly be true, since one will then consider the properties which the two sides of the equation share –
that which makes them "equal" – but one also can, and does, consider them as two separate concepts. Even in a
mathematical statement as simple as "x=x", one encounters fundamental differences between the two "x"es under
consideration: firstly, that there are two distinct "x"es, in that they neither occupy the same space on this page nor in
one's own mind. There would otherwise be only one "x". Secondly, that if two things were absolutely equal in every
possible respect, then there would necessarily be no reason to consider their equality. Nothing could lead anyone to
consider the possibility or impossibility of their equality if there were no properties not shared between "them", since
there would necessarily be no relationship between them whatsoever. Thirdly, and most importantly, if two things
were equal in every possible respect they would necessarily not be two things, but the very same thing, since there
would be no difference to separate them.
In examples as odd as this, the differences between two approximately-equal things may be very small indeed, and it
is certainly true that they are quite irrelevant to most discussions. Acceptance of the reflexive property illustrated
above has led to useful mathematical discoveries which have influenced the life of anyone reading this article on a
computer. But in an examination of the possibility of the determinacy of any possible concept, differences like this
are supremely relevant since that quality which could possibly make two separate things "equal" seems to be
Indeterminacy of meaning and translation
• Willard Van Orman Quine, Donald Davidson etc.
Foucault and the indeterminacy of insanity
The philosopher Michel Foucault wrote about the existence of such problems of precise definition in the very
concept of insanity itself – a very rough approximation of his argument can be found in the late social commentator
and journalist Hunter S. Thompson's book, Kingdom of Fear:
"The only difference between the Sane and the Insane, is IN and yet within this world, the Sane have the power to
have the Insane locked up."
Another summary of Foucault's original argument against the indeterminacy of the concept of insanity in his
Madness and Civilization can be found in the following excerpt from the Literature, Arts, and Medicine Database:
"Central to this is the notion of confinement as a meaningful exercise. Foucault's history explains how the mad came
first to be confined; how they became identified as confined due to moral and economic factors that determined
those who ought to be confined; how they became perceived as dangerous through their confinement, partly by way
of atavistic identification with the lepers whose place they had come to occupy; how they were 'liberated' by Pinel
Indeterminacy (philosophy)
and Tuke, but in their liberation remained confined, both physically in asylums and in the designation of being mad;
and how this confinement subsequently became enacted in the figure of the psychiatrist, whose practice is 'a certain
moral tactic contemporary with the end of the eighteenth century, preserved in the rites of the asylum life, and
overlaid by the myths of positivism.' Science and medicine, notably, come in at the later stages, as practices
'elaborated once this division' between the mad and the sane has been made (ix)."
In The Archaeology of Knowledge, Foucault addresses indeterminacy directly by discussing the origin of the
meaning of concepts:
"Foucault directs his analysis toward the 'statement', the basic unit of discourse that he believes has been ignored up
to this point. 'Statement' is the English translation from French énoncé (that which is enunciated or expressed), which
has a peculiar meaning for Foucault. 'Énoncé' for Foucault means that which makes propositions, utterances, or
speech acts meaningful. In this understanding, statements themselves are not propositions, utterances, or speech acts.
Rather, statements create a network of rules establishing what is meaningful, and it is these rules that are the
preconditions for propositions, utterances, or speech acts to have meaning. Statements are also 'events'. Depending
on whether or not they comply with the rules of meaning, a grammatically correct sentence may still lack meaning
and inversely, an incorrect sentence may still be meaningful. Statements depend on the conditions in which they
emerge and exist within a field of discourse. It is huge collections of statements, called discursive formations, toward
which Foucault aims his analysis. [...] Rather than looking for a deeper meaning underneath discourse or looking for
the source of meaning in some transcendental subject, Foucault analyzes the conditions of existence for meaning. In
order to show the principles of meaning production in various discursive formations he details how truth claims
emerge during various epochs on the basis of what was actually said and written during these periods of time."
The difference described by Foucault between the sane and the insane does have observable and very real effects on
millions of people daily and can be characterized in terms of those effects, but it can also serve to illustrate a
particular effect of the indeterminacy of definition: i.e., that insofar as the general public tends not to characterize or
define insanity in very precise terms, it tends, according to Foucault, unnecessarily and arbitrarily to confine some of
its members on an irrational basis. The less-precisely such states as "insanity" and "criminality" are defined in a
society, the more likely that society is to fail to continue over time to describe the same behaviors as characteristic of
those states (or, alternately, to characterize such states in terms of the same behaviors).
Indeterminacy in Discourse Analysis
Steve Hoenisch asserts in his article Interpretation and Indeterminacy in Discourse Analysis that "[T]he exact
meaning of a speaker's utterance in a contextualized exchange is often indeterminate. Within the context of the
analysis of the teacher-pupil exchange, I will argue for the superiority of interactional linguistics over speech act
theory because it reduces the indeterminacy and yields a more principled interpretation[...]"
Current work
Richard Dawkins, the man who coined the term meme in the 1970s, described the concept of faith in his
documentary, Root of All Evil?, as "the process of non-thinking". In the documentary, he used Bertrand Russell's
analogy between a teapot orbiting the sun (something that cannot be observed because the brightness of the sun
would obscure it even from the best telescope's view) and the object of one's faith (in this particular case, God) to
explain that a highly indeterminate idea can self-replicate freely: "Everybody in the society had faith in the teapot.
Stories of the teapot had been handed down for generations as part of the tradition of society. There are holy books
about the teapot."
In Darwin's Dangerous Idea, Dennett argues against the existence of determinate meaning (in this case, of the
subjective experience of vision for frogs) via an explanation of their indeterminacy in the chapter entitled The
Evolution of Meanings, in the section The Quest for Real Meanings:
Indeterminacy (philosophy)
"Unless there were 'meaningless' or 'indeterminate' variation in the triggering conditions of the various frogs' eyes,
there could be no raw material [...] for selection for a new purpose to act upon. The indeterminacy that Fodor (and
others) see as a flaw [...] is actually a prediction for such evolution [of "purpose"]. The idea that there must be
something determinate that the frog's eye really means – some possibly unknowable proposition in froggish that
expresses exactly what the frog's eye is telling the frog's brain – is just essentialism applied to meaning (or function).
Meaning, like function on which it so directly depends, is not something determinate at its birth. [...]"
Dennet argues, controversially

, against qualia in Consciousness Explained. Qualia are attacked from several
directions at once: he maintains they do not exist (or that they are too ill-defined to play any role in science, or that
they are really something else, i.e. behavioral dispositions). They cannot simultaneously have all the properties
attributed to them by philosophers—incorrigible, ineffable, private, directly accessible and so on. The multiple drafts
theory is leveraged to show that facts about qualia are not definite. Critics object that one's own qualia are
subjectively quite clear and distinct to oneself.
The self-replicating nature of memes is a partial explanation of the recurrence of indeterminacies in language and
thought. The wide influences of Platonism and Kantianism in Western philosophy can arguably be partially
attributed to the indeterminacies of some of their most fundamental concepts (namely, the Idea and the Noumenon,
For a given meme to exhibit replication and heritability – that is, for it to be able to make an imperfect copy of itself
which is more likely to share any given trait with its "parent" meme than with some random member of the general
"population" of memes – it must in some way be mutable, since memetic replication occurs by means of human
conceptual imitation rather than via the discrete molecular processes that govern genetic replication. (If a statement
were to generate copies of itself that didn't meaningfully differ from it, that process of copying would more
accurately be described as "duplication" than as "replication", and it would be incorrect to term these statements
"memes"; the same would be true if the "child" statements did not noticeably inherit a substantial proportion of their
traits from their "parent" statements.) In other words, if a meme is defined roughly (and somewhat arbitrarily) as a
statement (or as a collection of statements, like Foucault's "discursive formations") that inherits some, but not all, of
its properties (or elements of its definition) from its "parent" memes and which self-replicates, then indeterminacy of
definition could be seen as advantageous to memetic replication, since an absolute rigidity of definition would
preclude memetic adaptation.
It is important to note that indeterminacy in linguistics can arguably partially be defeated by the fact that languages
are always changing. However, what the entire language and its collected changes continue to reflect is sometimes
still considered to be indeterminate.
Persons of faith argue that faith "is the basis of all knowledge". The Wikipedia article on faith states that "one must
assume, believe, or have faith in the credibility of a person, place, thing, or idea in order to have a basis for
knowledge." In this way the object of one's faith is similar to Kant's noumenon.
This would seem to attempt to make direct use of the indeterminacy of the object of one's faith as evidential support
of its existence: if the object of one's faith were to be proven to exist (i.e., if it were no longer of indeterminate
definition, or if it were no longer unquantifiable, etc.), then faith in that object would no longer be necessary;
arguments from authority such as those mentioned above wouldn't either; all that would be needed to prove its
existence would be scientific evidence. Thus, if faith is to be considered as a reliable basis for knowledge, persons of
faith would seem, in effect, to assert that indeterminacy is not only necessary, but good (see Nassim Taleb).
Indeterminacy (philosophy)
Indeterminacy in new physical theories
Science generally attempts to eliminate vague definitions, causally inert entities, and indeterminate properties, via
further observation, experimentation, characterization, and explanation. Occam's razor tends to eliminates causally
inert entities from functioning models of quantifiable phenomena, but some quantitative models, such as quantum
mechanics, actually imply certain indeterminacies, such as the relative indeterminacy of quantum particles' positions
to the precision with which their momenta can be measured (and vice versa). (See Heisenberg's indeterminacy
One ardent supporter of the possibility of a final unifying theory (and thus, arguably, of the possibility of the end of
some current indeterminacies) in physics, Steven Weinberg, stated in an interview with PBS
"Sometimes [...] people say that surely there's no final theory because, after all, every time we've made a step toward
unification or toward simplification we always find more and more complexity there. That just means we haven't
found it yet. Physicists never thought they had the final theory."
The wikipedia article on the possibility of such a "theory of everything" notes that
"Other possibilities which may frustrate the explanatory capacity of a TOE may include sensitivity to the boundary
conditions of the universe, or the existence of mathematical chaos in its solutions, making its predictions precise, but
Chaos theory argues that precise prediction of the behavior of complex systems becomes impossible because of the
observer's inability to gather all necessary data.
As yet, it seems entirely possible that there shall never be any "final theory" of all phenomena, and that, rather,
explanations may instead breed more and more complex and exact explanations of the new phenomena uncovered by
current experimentation. In this argument, the "indeterminacy" or "thing in itself" is the "final explanation" that will
never be reached; this can be compared to the concept of the limit in calculus, in that quantities may approach, but
never reach, a given limit in certain situations.
Proponents of a deterministic universe have criticised various applications of the concept of indeterminacy in the
sciences; for instance, Einstein once stated that "God does not play dice" in a succinct (but now unpopular) argument
against the theory of quantum indeterminacy, which states that the actions of particles of extremely low mass or
energy are unpredictable because an observer's interaction with them changes either their positions or momenta. (The
"dice" in Einstein's metaphor refer to the probabilities that these particles will behave in particular ways, which is
how quantum mechanics addressed the problem.)
At first it might seem that a criticism could be made from a biological standpoint in that an indeterminate idea would
seem not to be beneficial to the species that holds it. A strong counterargument, however, is that not all traits
exhibited by living organisms will be seen in the long term as evolutionarily advantageous, given that extinctions
occur regularly and that phenotypic traits have often died out altogether – in other words, an indeterminate meme
may in the long term demonstrate its evolutionary value to the species that produced it in either direction; humans
are, as yet, the only species known to make use of such concepts. It might also be argued that conceptual vagueness
is an inevitability, given the limited capacity of the human nervous systems. We just do not have enough neurons to
maintain separate concepts for "dog with 1,000,000 hairs", "dog with 1,000,001 hairs" and so on. But conceptual
vagueness is not metaphysical indeterminacy.
Indeterminacy (philosophy)
Indeterminacy and consciousness
It has been speculated that there is a connection between consciousness and the quantum uncertainty underlying all
observable phenomena, since the brain's activity can be correlated to a great degree with the phenomenon of
consciousness and all physical activity is to some extent unpredictable. According to some philosophers, such as Dr.
William Plank, this would tend to agree with a Nietzschean view of causality.

Furthermore, qualia, a way of talking about the way things appear to subjective consciousess, were argued to be
indeterminate by Dennett in the works noted above.
If communication and memetic replication are taken as necessary to human consciousness, then the indeterminacy of
definition can arguably be seen as necessary to human consciousness as well inasmuch as it facilitates (or, possibly,
enables) memetic replication; however, such a proposition is currently untestable and cannot predict any real events
except, perhaps, for the continuation of indeterminacy. The indeterminacy of definition is itself determined by
physical events, according to a biological psychology, and does not demonstrably cause them: like qualia,
indeterminacy might only appear to accompany observable, quantifiable processes. The proposition that
indeterminacy has a definite effect on observable phenomena (such as on the wide influences of Platonism and
Kantianism) is based on historical evidence rather than on scientific experiment; however, it is nevertheless not an
untenable position in modern philosophy if it does not treat indeterminacy as a "transcendental cause" but as a
phenomenon or process which can be precisely characterized and which is evidenced by other observable
Synonymous concepts in philosophy
Uncertainty and indeterminacy are words for essentially the same concept in both quantum mechanics.
Unquantifiability, and undefinability (or indefinability), can also sometimes be synonymous with indeterminacy. In
science, indeterminacy can sometimes be interchangeable with unprovability or unpredictability. Also, anything
entirely inobservable can be said to be indeterminate in that it cannot be precisely characterized.
[1] Derrida, Plato's Pharmacy in Dissemination, 1972, Athlone Press, London, 1981 (http:/ / social. chass. ncsu. edu/ wyrick/debclass/ pharma.
[2] Nietzsche, F. On Truth and Lies (http:// www.publicappeal. org/library/nietzsche/ Nietzsche_various/ on_truth_and_lies.htm)
[3] Nietzsche, F. Beyond Good and Evil (http:// www.marxists. org/ reference/archive/nietzsche/ 1886/ beyond-good-evil/ch01. htm)
[4] Nietzsche quotes (http:/ / www.wutsamada. com/ alma/ modern/ nietzquo.htm)
[5] Nietzsche quote
[6] Thompson, Hunter.S.
[7] Foucault, M. Madness and Civilisation (http:/ / mchip00. nyu.edu/ lit-med/lit-med-db/webdocs/ webdescrips/ foucault12432-des-.html)
[8] Foucault, M. The Archaeology of Knowledge
[9] Hoenisch, S. Interpretation and Indeterminacy in Discourse Analysis (http:/ / www. criticism.com/ da/ da_indet.htm)
[10] Dawkins World of Dawkins (http:/ / www. simonyi. ox. ac.uk/ dawkins/ WorldOfDawkins-archive/Dawkins/ Work/ Articles/
1999-10-04snakeoil. shtml)
[11] Lormand, E. Qualia! Now Showing ata Theatre near you (http:// www-personal.umich.edu/ ~lormand/phil/ cons/ qualia.htm)
[12] De Leon, D. The Qualities of Qualia (http:// www.lucs. lu. se/ ftp/pub/ LUCS_Studies/ LUCS58. pdf)
[13] Weinberg, S. PBS interview (http:/ / www. pbs. org/ wgbh/ nova/ elegant/ view-weinberg.html)
expert opinion on the relationship between Nietzsche's critique of Kant's "thing in itself" and quantum indeterminacy. (http:// www.
msubillings.edu/ CASFaculty/ Plank/ THE IMPLICATIONS OF QUANTUM NON.htm)
[15] The Quantum Nietzsche-- a site explaining the same ideas, also run by William Plank. (http:// www.quantumnietzsche. com/ )
Infinite monkey theorem
Infinite monkey theorem
Given enough time, a hypothetical monkey typing at random would,
as part of its output, almost surely produce all of Shakespeare's plays.
In this image a chimpanzee is giving it a try.
The infinite monkey theorem states that a monkey
hitting keys at random on a typewriter keyboard for an
infinite amount of time will almost surely type a given
text, such as the complete works of William
In this context, "almost surely" is a mathematical term
with a precise meaning, and the "monkey" is not an
actual monkey, but a metaphor for an abstract device
that produces a random sequence of letters ad
infinitum. The probability of a monkey exactly typing a
complete work such as Shakespeare's Hamlet is so tiny
that the chance of it occurring during a period of time
of the order of the age of the universe is extremely low, but not zero.
Variants of the theorem include multiple and even infinitely many typists, and the target text varies between an entire
library and a single sentence. The history of these statements can be traced back to Aristotle's On Generation and
Corruption and Cicero's De natura deorum, through Blaise Pascal and Jonathan Swift, and finally to modern
statements with their iconic typewriters. In the early 20th century, Émile Borel and Arthur Eddington used the
theorem to illustrate the timescales implicit in the foundations of statistical mechanics.
Direct proof
There is a straightforward proof of this theorem. If two events are statistically independent, then the probability of
both happening equals the product of the probabilities of each one happening independently. For example, if the
chance of rain in Montreal on a particular day is 0.3 and the chance of an earthquake in San Francisco on that day is
0.008, then the chance of both happening on that same day is 0.3 × 0.008 = 0.0024.
Suppose the typewriter has 50 keys, and the word to be typed is banana. If we assume that the keys are pressed
randomly (i.e., with equal probability) and independently, then the chance that the first letter typed is 'b' is 1/50, and
the chance that the second letter typed is a is also 1/50, and so on, because events are independent. Therefore, the
chance of the first six letters matching banana is
(1/50) × (1/50) × (1/50) × (1/50) × (1/50) × (1/50) = (1/50)
= 1/15 625 000 000 ,
less than one in 15 billion. For the same reason, the chance that the next 6 letters match banana is also (1/50)
, and
so on.
From the above, the chance of not typing banana in a given block of 6 letters is 1 − (1/50)
. Because each block is
typed independently, the chance X
of not typing banana in any of the first n blocks of 6 letters is
As n grows, X
gets smaller. For an n of a million, X
is roughly 0.9999, but for an n of 10 billion X
is roughly 0.53
and for an n of 100 billion it is roughly 0.0017. As n approaches infinity, the probability X
approaches zero; that is,
by making n large enough, X
can be made as small as is desired,

and the chance of typing banana approaches
Infinite monkey theorem
The same argument shows why at least one of infinitely many monkeys will produce a text as quickly as it would be
produced by a perfectly accurate human typist copying it from the original. In this case X
= (1 − (1/50)
where X
represents the probability that none of the first n monkeys types banana correctly on their first try. When we
consider 100 billion monkeys, the probability falls to 0.17%, and as the number of monkeys n increases, the value of
– the probability of the monkeys failing to reproduce the given text – approaches zero arbitrarily closely. The
limit, for n going to infinity, is zero.
However, for physically meaningful numbers of monkeys typing for physically meaningful lengths of time the
results are reversed. If there are as many monkeys as there are particles in the observable universe (10
), and each
types 1,000 keystrokes per second for 100 times the life of the universe (10
seconds), the probability of the
monkeys replicating even a short book is nearly zero. See Probabilities, below.
Infinite strings
The two statements above can be stated more generally and compactly in terms of strings, which are sequences of
characters chosen from some finite alphabet:
• Given an infinite string where each character is chosen uniformly at random, any given finite string almost surely
occurs as a substring at some position.
• Given an infinite sequence of infinite strings, where each character of each string is chosen uniformly at random,
any given finite string almost surely occurs as a prefix of one of these strings.
Both follow easily from the second Borel–Cantelli lemma. For the second theorem, let E
be the event that the kth
string begins with the given text. Because this has some fixed nonzero probability p of occurring, the E
independent, and the below sum diverges,
the probability that infinitely many of the E
occur is 1. The first theorem is shown similarly; one can divide the
random string into nonoverlapping blocks matching the size of the desired text, and make E
the event where the kth
block equals the desired string.
Ignoring punctuation, spacing, and capitalization, a monkey typing letters uniformly at random has a chance of one
in 26 of correctly typing the first letter of Hamlet. It has a chance of one in 676 (26 × 26) of typing the first two
letters. Because the probability shrinks exponentially, at 20 letters it already has only a chance of one in 26
19,928,148,895,209,409,152,340,197,376 (almost 2 × 10
). In the case of the entire text of Hamlet, the probabilities
are so vanishingly small they can barely be conceived in human terms. The text of Hamlet contains approximately
130,000 letters.
Thus there is a probability of one in 3.4 × 10
to get the text right at the first trial. The
average number of letters that needs to be typed until the text appears is also 3.4 × 10
or including
punctuation, 4.4 × 10
Even if the observable universe were filled with monkeys the size of atoms typing from now until the heat death of
the universe, their total probability to produce a single instance of Hamlet would still be many orders of magnitude
less than one in 10
. As Kittel and Kroemer put it, "The probability of Hamlet is therefore zero in any
operational sense of an event…", and the statement that the monkeys must eventually succeed "gives a misleading
conclusion about very, very large numbers." This is from their textbook on thermodynamics, the field whose
statistical foundations motivated the first known expositions of typing monkeys.
Infinite monkey theorem
Statistical mechanics
In one of the forms in which probabilists now know this theorem, with its "dactylographic" [i.e., typewriting]
monkeys (French: singes dactylographes; the French word singe covers both the monkeys and the apes), appeared in
Émile Borel's 1913 article "Mécanique Statistique et Irréversibilité" (Statistical mechanics and irreversibility),
in his book "Le Hasard" in 1914. His "monkeys" are not actual monkeys; rather, they are a metaphor for an
imaginary way to produce a large, random sequence of letters. Borel said that if a million monkeys typed ten hours a
day, it was extremely unlikely that their output would exactly equal all the books of the richest libraries of the world;
and yet, in comparison, it was even more unlikely that the laws of statistical mechanics would ever be violated, even
The physicist Arthur Eddington drew on Borel's image further in The Nature of the Physical World (1928), writing:
If I let my fingers wander idly over the keys of a typewriter it might happen that my screed made an
intelligible sentence. If an army of monkeys were strumming on typewriters they might write all the books in
the British Museum. The chance of their doing so is decidedly more favourable than the chance of the
molecules returning to one half of the vessel.
These images invite the reader to consider the incredible improbability of a large but finite number of monkeys
working for a large but finite amount of time producing a significant work, and compare this with the even greater
improbability of certain physical events. Any physical process that is even less likely than such monkeys' success is
effectively impossible, and it may safely be said that such a process will never happen.
Origins and "The Total Library"
In a 1939 essay entitled "The Total Library", Argentine writer Jorge Luis Borges traced the infinite-monkey concept
back to Aristotle's Metaphysics. Explaining the views of Leucippus, who held that the world arose through the
random combination of atoms, Aristotle notes that the atoms themselves are homogeneous and their possible
arrangements only differ in shape, position and ordering. In De Generatione et Corruptione (On Generation and
Corruption), the Greek philosopher compares this to the way that a tragedy and a comedy consist of the same
"atoms", i.e., alphabetic characters.
Three centuries later, Cicero's De natura deorum (On the Nature of the Gods)
argued against the atomist worldview:
He who believes this may as well believe that if a great quantity of the one-and-twenty letters, composed either
of gold or any other matter, were thrown upon the ground, they would fall into such order as legibly to form
the Annals of Ennius. I doubt whether fortune could make a single verse of them.
Borges follows the history of this argument through Blaise Pascal and Jonathan Swift,
then observes that in his
own time, the vocabulary had changed. By 1939, the idiom was "that a half-dozen monkeys provided with
typewriters would, in a few eternities, produce all the books in the British Museum." (To which Borges adds,
"Strictly speaking, one immortal monkey would suffice.") Borges then imagines the contents of the Total Library
which this enterprise would produce if carried to its fullest extreme:
Everything would be in its blind volumes. Everything: the detailed history of the future, Aeschylus' The
Egyptians, the exact number of times that the waters of the Ganges have reflected the flight of a falcon, the
secret and true nature of Rome, the encyclopedia Novalis would have constructed, my dreams and half-dreams
at dawn on August 14, 1934, the proof of Pierre Fermat's theorem, the unwritten chapters of Edwin Drood,
those same chapters translated into the language spoken by the Garamantes, the paradoxes Berkeley invented
concerning Time but didn't publish, Urizen's books of iron, the premature epiphanies of Stephen Dedalus,
which would be meaningless before a cycle of a thousand years, the Gnostic Gospel of Basilides, the song the
sirens sang, the complete catalog of the Library, the proof of the inaccuracy of that catalog. Everything: but for
Infinite monkey theorem
every sensible line or accurate fact there would be millions of meaningless cacophonies, verbal farragoes, and
babblings. Everything: but all the generations of mankind could pass before the dizzying shelves—shelves that
obliterate the day and on which chaos lies—ever reward them with a tolerable page.
Borges's total library concept was the main theme of his widely read 1941 short story "The Library of Babel", which
describes an unimaginably vast library consisting of interlocking hexagonal chambers, together containing every
possible volume that could be composed from the letters of the alphabet and some punctuation characters.
Applications and criticisms
Thomas Huxley is sometimes misattributed with
proposing a variant of the theory in his debates with
Samuel Wilberforce.
In his 1931 book The Mysterious Universe, Eddington's rival
James Jeans attributed the monkey parable to a "Huxley",
presumably meaning Thomas Henry Huxley. This attribution is
Today, it is sometimes further reported that Huxley
applied the example in a now-legendary debate over Charles
Darwin's On the Origin of Species with the Anglican Bishop of
Oxford, Samuel Wilberforce, held at a meeting of the British
Association for the Advancement of Science at Oxford on June 30,
1860. This story suffers not only from a lack of evidence, but the
fact that in 1860 the typewriter itself had yet to emerge.
Despite the original mix-up, monkey-and-typewriter arguments are
now common in arguments over evolution. For example, Doug
Powell argues as a Christian apologist that even if a monkey
accidentally types the letters of Hamlet, it has failed to produce
Hamlet because it lacked the intention to communicate. His
parallel implication is that natural laws could not produce the
information content in DNA.
A more common argument is
represented by Reverend John F. MacArthur, who claims that the
genetic mutations necessary to produce a tapeworm from an
amoeba are as unlikely as a monkey typing Hamlet's soliloquy,
and hence the odds against the evolution of all life are impossible to overcome.
Evolutionary biologist Richard Dawkins employs the typing monkey concept in his book The Blind Watchmaker to
demonstrate the ability of natural selection to produce biological complexity out of random mutations. In a
simulation experiment Dawkins has his weasel program produce the Hamlet phrase METHINKS IT IS LIKE A
WEASEL, starting from a randomly typed parent, by "breeding" subsequent generations and always choosing the
closest match from progeny that are copies of the parent, with random mutations. The chance of the target phrase
appearing in a single step is extremely small, yet Dawkins showed that it could be produced rapidly (in about 40
generations) using cumulative selection of phrases. The random choices furnish raw material, while cumulative
selection imparts information. As Dawkins acknowledges, however, the weasel program is an imperfect analogy for
evolution, as "offspring" phrases were selected "according to the criterion of resemblance to a distant ideal target."
In contrast, Dawkins affirms, evolution has no long-term plans and does not progress toward some distant goal (such
as humans). The weasel program is instead meant to illustrate the difference between nonrandom cumulative
selection, and random single-step selection.
In terms of the typing monkey analogy, this means that Romeo and
Juliet could be produced relatively quickly if placed under the constraints of a nonrandom, Darwinian-type selection,
Infinite monkey theorem
by freezing in place any letters that happened to match the target text, and making that the template for the next
generation of typing monkeys.
A different avenue for exploring the analogy between evolution and an unconstrained monkey lies in the problem
that the monkey types only one letter at a time, independently of the other letters. Hugh Petrie argues that a more
sophisticated setup is required, in his case not for biological evolution but the evolution of ideas:
In order to get the proper analogy, we would have to equip the monkey with a more complex typewriter. It
would have to include whole Elizabethan sentences and thoughts. It would have to include Elizabethan beliefs
about human action patterns and the causes, Elizabethan morality and science, and linguistic patterns for
expressing these. It would probably even have to include an account of the sorts of experiences which shaped
Shakespeare's belief structure as a particular example of an Elizabethan. Then, perhaps, we might allow the
monkey to play with such a typewriter and produce variants, but the impossibility of obtaining a
Shakespearean play is no longer obvious. What is varied really does encapsulate a great deal of
already-achieved knowledge.
Petrie's argument however fails because if one were to provide all the knowledge and culture of human being to the
monkey then it would essentially be the work of an intelligent human being not of some random, non-intelligent
happenstance that the infinite monkey theorem claims.
James W. Valentine, while admitting that the classic monkey's task is impossible, finds that there is a worthwhile
analogy between written English and the metazoan genome in this other sense: both have "combinatorial,
hierarchical structures" that greatly constrain the immense number of combinations at the alphabet level.
Literary theory
R. G. Collingwood argued in 1938 that art cannot be produced by accident, and wrote as a sarcastic aside to his
…some … have denied this proposition, pointing out that if a monkey played with a typewriter … he would
produce … the complete text of Shakespeare. Any reader who has nothing to do can amuse himself by
calculating how long it would take for the probability to be worth betting on. But the interest of the suggestion
lies in the revelation of the mental state of a person who can identify the 'works' of Shakespeare with the series
of letters printed on the pages of a book…
Nelson Goodman took the contrary position, illustrating his point along with Catherine Elgin by the example of
Borges' “Pierre Menard, Author of the Quixote”,
What Menard wrote is simply another inscription of the text. Any of us can do the same, as can printing
presses and photocopiers. Indeed, we are told, if infinitely many monkeys … one would eventually produce a
replica of the text. That replica, we maintain, would be as much an instance of the work, Don Quixote, as
Cervantes' manuscript, Menard's manuscript, and each copy of the book that ever has been or will be
In another writing, Goodman elaborates, "That the monkey may be supposed to have produced his copy randomly
makes no difference. It is the same text, and it is open to all the same interpretations…." Gérard Genette dismisses
Goodman's argument as begging the question.
For Jorge J. E. Gracia, the question of the identity of texts leads to a different question, that of author. If a monkey is
capable of typing Hamlet, despite having no intention of meaning and therefore disqualifying itself as an author, then
it appears that texts do not require authors. Possible solutions include saying that whoever finds the text and
identifies it as Hamlet is the author; or that Shakespeare is the author, the monkey his agent, and the finder merely a
user of the text. These solutions have their own difficulties, in that the text appears to have a meaning separate from
the other agents: what if the monkey operates before Shakespeare is born, or if Shakespeare is never born, or if no
one ever finds the monkey's typescript?
Infinite monkey theorem
Random document generation
The theorem concerns a thought experiment which cannot be fully carried out in practice, since it is predicted to
require prohibitive amounts of time and resources. Nonetheless, it has inspired efforts in finite random text
One computer program run by Dan Oliver of Scottsdale, Arizona, according to an article in The New Yorker, came
up with a result on August 4, 2004: After the group had worked for 42,162,500,000 billion billion monkey-years, one
of the "monkeys" typed, “VALENTINE. Cease toIdor:eFLP0FRjWK78aXzVOwm)-‘;8.t" The first 19
letters of this sequence can be found in "The Two Gentlemen of Verona". Other teams have reproduced 18
characters from "Timon of Athens", 17 from "Troilus and Cressida", and 16 from "Richard II".
A website entitled The Monkey Shakespeare Simulator, launched on July 1, 2003, contained a Java applet that
simulates a large population of monkeys typing randomly, with the stated intention of seeing how long it takes the
virtual monkeys to produce a complete Shakespearean play from beginning to end. For example, it produced this
partial line from Henry IV, Part 2, reporting that it took "2,737,850 million billion billion billion monkey-years" to
reach 24 matching characters:
RUMOUR. Open your ears; 9r"5j5&?OWTY Z0d...
Due to processing power limitations, the program uses a probabilistic model (by using a random number generator or
RNG) instead of actually generating random text and comparing it to Shakespeare. When the simulator "detects a
match" (that is, the RNG generates a certain value or a value within a certain range), the simulator simulates the
match by generating matched text.
More sophisticated methods are used in practice for natural language generation. If instead of simply generating
random characters one restricts the generator to a meaningful vocabulary and conservatively following grammar
rules, like using a context-free grammar, then a random document generated this way can even fool some humans (at
least on a cursory reading) as shown in the experiments with SCIgen, snarXiv, and the Postmodernism Generator.
Testing of random number generators
Questions about the statistics describing how often an ideal monkey is expected to type certain strings translate into
practical tests for random number generators; these range from the simple to the "quite sophisticated". Computer
science professors George Marsaglia and Arif Zaman report that they used to call one such category of tests
"overlapping m-tuple tests" in lecture, since they concern overlapping m-tuples of successive elements in a random
sequence. But they found that calling them "monkey tests" helped to motivate the idea with students. They published
a report on the class of tests and their results for various RNGs in 1993.
Real monkeys
Primate behaviorists Cheney and Seyfarth remark that real monkeys would indeed have to rely on chance to have
any hope of producing Romeo and Juliet. Monkeys lack a theory of mind and are unable to differentiate between
their own and others' knowledge, emotions, and beliefs. Even if a monkey could learn to write a play and describe
the characters' behavior, it could not reveal the characters' minds and so build an ironic tragedy.
In 2003, lecturers and students from the University of Plymouth MediaLab Arts course used a £2,000 grant from the
Arts Council to study the literary output of real monkeys. They left a computer keyboard in the enclosure of six
Celebes Crested Macaques in Paignton Zoo in Devon in England for a month, with a radio link to broadcast the
results on a website.
Not only did the monkeys produce nothing but five pages
consisting largely of the letter S, the lead male began
by bashing the keyboard with a stone, and the monkeys continued by urinating and defecating on it. Phillips said that
the artist-funded project was primarily performance art, and they had learned "an awful lot" from it. He concluded
that monkeys "are not random generators. They're more complex than that. … They were quite interested in the
Infinite monkey theorem
screen, and they saw that when they typed a letter, something happened. There was a level of intention there."

Popular culture
The infinite monkey theorem and its associated imagery is considered a popular and proverbial illustration of the
mathematics of probability, widely known to the general public because of its transmission through popular culture
rather than because of its transmission via the classroom.
This theorem was mentioned in part and used as a joke in the novel The Hitchhiker's Guide to the Galaxy by Douglas
Adams: “Ford! There’s an infinite number of monkeys outside who want to talk to us about this script for Hamlet
they’ve worked out” (chapter 9).
In the American cartoon The Simpsons episode "Last Exit to Springfield", Mr Burns states: "This is a thousand
monkeys working at a thousand typewriters. Soon they'll have written the greatest novel known to man. Let's see.
(reading) 'It was the best of times, it was the "blurst" of times'? You stupid monkey!"
The enduring, widespread popularity of the theorem was noted in the introduction to a 2001 paper, "Monkeys,
Typewriters and Networks: The Internet in the Light of the Theory of Accidental Excellence" (Hoffmann and
In 2002, an article in the Washington Post said: "Plenty of people have had fun with the famous
notion that an infinite number of monkeys with an infinite number of typewriters and an infinite amount of time
could eventually write the works of Shakespeare."
In 2003, the previously mentioned Arts Council funded
experiment involving real monkeys and a computer keyboard received widespread press coverage.
In 2007, the
theorem was listed by Wired magazine in a list of eight classic thought experiments.
[1] Isaac, Richard E. (1995). The Pleasures of Probability. Springer. pp. 48–50. ISBN 0-387-94415-X. Isaac generalizes this argument
immediately to variable text and alphabet size; the common main conclusion is on p.50.
[2] This shows that the probability of typing "banana" in one of the predefined non-overlapping blocks of six letters tends to 1. In addition the
word may appear across two blocks, so the estimate given is conservative.
[3] The first theorem is proven by a similar if more indirect route in Gut, Allan (2005). Probability: A Graduate Course. Springer. pp. 97–100.
ISBN 0-387-22833-0.
[4] Using the Hamlet text from gutenberg (http:// www.gutenberg. org/dirs/ etext99/ 1ws2611. txt), there are 132680 alphabetical letters and
199749 characters overall
[5] For any required string of 130,000 letters from the set a-z, the average number of letters that needs to be typed until the string appears is
(rounded) 3.4 × 10
, except in the case that all letters of the required string are equal, in which case the value is about 4% more,
3.6 × 10
. In that case failure to have the correct string starting from a particular position reduces with about 4% the probability of a
correct string starting from the next position (i.e., for overlapping positions the events of having the correct string are not independent; in this
case there is a positive correlation between the two successes, so the chance of success after a failure is smaller than the chance of success in
general). The figure 3.4 × 10
is derived from n = 26
by taking the logarithm of both sides: log
(n) = 1300000×log
= 183946.5352, therefore n = 10
 × 10
= 3.429 × 10
[6] 26 letters ×2 for capitalisation, 12 for punctuation characters = 64, 199749×log
(64) = 4.4 × 10
[7] Kittel, Charles and Herbert Kroemer (1980). Thermal Physics (2nd ed.). W. H. Freeman Company. pp. 53. ISBN 0-7167-1088-9.
[8] Émile Borel (1913). "Mécanique Statistique et Irréversibilité". J. Phys. 5e série 3: 189–196.
[9] Arthur Eddington (1928). The Nature of the Physical World: The Gifford Lectures. New York: Macmillan. pp. 72. ISBN 0-8414-3885-4.
[10] Aristotle, De Generatione et Corruptione, 315b14.
[11] Marcus Tullius Cicero, De natura deorum, 2.37. Translation from Cicero's Tusculan Disputations; Also, Treatises On The Nature Of The
Gods, And On The Commonwealth, C. D. Yonge, principal translator, New York, Harper & Brothers Publishers, Franklin Square. (1877).
Downloadable text (http:// www. gutenberg. org/etext/ 14988).
[12] The English translation of "The Total Library" lists the title of Swift's essay as "Trivial Essay on the Faculties of the Soul." The appropriate
reference is, instead: Swift, Jonathan, Temple Scott et.al. "A Tritical Essay upon the Faculties of the Mind." The Prose Works of Jonathan
Swift, Volume 1. London: G. Bell, 1897, pp. 291-296. Google Books (http:// books. google. com/ books?id=FctEAAAAYAAJ&
printsec=frontcover&dq=The+Prose+Works+ of+Jonathan+ Swift& hl=en&ei=JdyDTb-yM8u3tweNmcy8BA& sa=X& oi=book_result&
ct=result&resnum=1&ved=0CC4Q6AEwAA#v=onepage& q& f=false)
[13] Borges, Jorge Luis. "La biblioteca total" (The Total Library), Sur No. 59, August 1939. Trans. by Eliot Weinberger. In Selected
Non-Fictions (Penguin: 1999), ISBN 0-670-84947-2.
Infinite monkey theorem
[14] Padmanabhan, Thanu (2005). "The dark side of astronomy". Nature 435 (7038): 20–21. doi:10.1038/435020a. Platt, Suzy; Library of
Congress Congressional Research Service (1993). Respectfully quoted: a dictionary of quotations. Barnes & Noble. pp. 388–389.
ISBN 0-88029-768-9.
[15] Rescher, Nicholas (2006). Studies in the Philosophy of Science. ontos verlag. pp. 103. ISBN 3-938793-20-1.
[16] Powell, Doug (2006). Holman Quicksource Guide to Christian Apologetics. Broadman & Holman. pp. 60, 63. ISBN 0-8054-9460-X.
[17] MacArthur, John (2003). Think Biblically!: Recovering a Christian Worldview. Crossway Books. pp. 78–79. ISBN 1-58134-412-0.
[18] Dawkins, Richard (1996). The Blind Watchmaker. W.W. Norton & Co.. pp. 46–50. ISBN 0-393-31570-3.
[19] As quoted in Blachowicz, James (1998). Of Two Minds: Nature of Inquiry. SUNY Press. pp. 109. ISBN 0-7914-3641-1.
[20] Valentine, James (2004). On the Origin of Phyla. University of Chicago Press. pp. 77–80. ISBN 0-226-84548-6.
[21] p.126 of The Principles of Art, as summarized and quoted by Sclafani, Richard J. (1975). "The logical primitiveness of the concept of a work
of art". British Journal of Aesthetics 15 (1): 14. doi:10.1093/bjaesthetics/15.1.14.
[22] John, Eileen and Dominic Lopes, editors (2004). The Philosophy of Literature: Contemporary and Classic Readings: An Anthology.
Blackwell. pp. 96. ISBN 1-4051-1208-5.
[23] Genette, Gérard (1997). The Work of Art: Immanence and Transcendence. Cornell UP. ISBN 0-8014-8272-0.
[24] Gracia, Jorge (1996). Texts: Ontological Status, Identity, Author, Audience. SUNY Press. pp. 1–2, 122–125. ISBN 0-7914-2901-6.
[25] (http:// www. newyorker.com/ arts/ critics/ books/ 2007/ 04/ 09/ 070409crbo_books_acocella?currentPage=all) Acocella, Joan, "The
Typing Life: How writers used to write", The New Yorker, April 9, 2007, a review of The Iron Whim: A Fragmented History of Typewriting
(Cornell) 2007, by Darren Wershler-Henry
[26] Marsaglia G. and Zaman A. (1993). "Monkey tests for random number generators". Computers & mathematics with applications (Elsevier,
Oxford) 26: 1–10. doi:10.1016/0898-1221(93)90001-C. ISSN 0898-1221 PostScript version (http:/ / stat. fsu.edu/ pub/ diehard/ cdrom/
pscript/ monkey. ps)
[27] Cheney, Dorothy L. and Robert M. Seyfarth (1992). How Monkeys See the World: Inside the Mind of Another Species. University of
Chicago Press. pp. 253–255. ISBN 0-226-10246-7.
[28] "No words to describe monkeys' play" (http:/ / news. bbc. co. uk/ 2/ hi/ 3013959. stm). BBC News. 2003-05-09. . Retrieved 2009-07-25.
[29] "Notes Towards the Complete Works of Shakespeare" (http:// www.vivaria.net/ experiments/ notes/ publication/ NOTES_EN.pdf) (PDF).
vivaria.net. 2002. . Retrieved 2006-06-13.
[30] Associated Press (2003-05-09). "Monkeys Don't Write Shakespeare" (http:// www. wired.com/ news/ culture/0,1284,58790,00. html).
Wired News. . Retrieved 2007-03-02.
[31] Examples of the theorem being referred to as proverbial include: Why Creativity Is Not like the Proverbial Typing Monkey. Jonathan W.
Schooler, Sonya Dougal, Psychological Inquiry, Vol. 10, No. 4 (1999); and The Case of the Midwife Toad (Arthur Koestler, New York, 1972,
page 30): "Neo-Darwinism does indeed carry the nineteenth-century brand of materialism to its extreme limits—to the proverbial monkey at
the typewriter, hitting by pure chance on the proper keys to produce a Shakespeare sonnet." The latter is sourced from Parable of the Monkeys
(http:/ / www. angelfire.com/ in/ hypnosonic/ Parable_of_the_Monkeys. html), a collection of historical references to the theorem in various
[32] Monkeys, Typewriters and Networks (http:/ / skylla. wz-berlin.de/ pdf/ 2002/ii02-101. pdf), Ute Hoffmann & Jeanette Hofmann,
Wissenschaftszentrum Berlin für Sozialforschung gGmbH (WZB), 2001.
[33] "Hello? This is Bob" (http:/ / www.washingtonpost. com/ ac2/ wp-dyn/A28521-2002Oct27?language=printer), Ken Ringle, Washington
Post, 28 October 2002, page C01.
[34] Notes Towards the Complete Works of Shakespeare (http:/ / www.vivaria.net/ experiments/ notes/ documentation/ press/ ) – some press
[35] The Best Thought Experiments: Schrödinger's Cat, Borel's Monkeys (http:// www. wired. com/ science/ discoveries/ magazine/ 15-06/
st_best), Greta Lorge, Wired Magazine: Issue 15.06, May 2007.
External links
• Ask Dr. Math article (http:// mathforum.org/library/drmath/view/ 55871. html), August 1998, Adam Bridge
• The Parable of the Monkeys (http:/ / www. angelfire.com/ in/ hypnosonic/ Parable_of_the_Monkeys. html), a
bibliography with quotations
• Planck Monkeys (http:/ / azureworld.blogspot. com/ 2007/ 04/ planck-monkeys. html), on populating the cosmos
with monkey particles
• PixelMonkeys.org (http:/ / www. pixelmonkeys. org) - Artist, Matt Kane's application of the Infinite Monkey
Theorem on pixels to create images.
Edward Kofler
Edward Kofler
Edward Kofler
Edward Kofler in 1940
2007-04-22Zürich,  Switzerland
Fields Mathematics
Institutions University of Warsaw
University of Zürich
Swiss National Science Foundation
Alma mater Lwów University
University of Cracow
Known for Developing the linear partial information theory (LPI)
Edward Kofler (November 16, 1911 – April 22, 2007) was a mathematician who made important contributions to
game theory and fuzzy logic by working out the theory of linear partial information.
He was born in Brzeżany, Imperial Russia (now Ukraine) and graduated as a disciple of among others Hugo
Steinhaus and Stefan Banach from the University of Lwów Poland (now Ukraine) and the University of Cracow,
having studied game theory. After the graduation in 1939 Kofler returned to his family in Kolomyia (today Kolomea
in Ukraine), where he taught mathematics in a Polish high school. After German attack on the town 1 July 1941 he
succeeded to flight to Kazakhstan together with his wife. Here at Alma-Ata he managed a Polish school with
orphanage in exile and worked there as mathematics teacher. After the World War II was ended he returned home to
Polen with the orphanage. He was accompanied by the wife and his baby son. The family settled in Poland. From
1959 he accepted the position of lecturer at the University of Warsaw in the faculty of economics. In 1962 he gained
a Ph.D. with his thesis Economic Decisions, Applying Game Theory. Then in 1962 he became assistant professor at
the faculty of social science in the same university, specializing in econometrics.
In 1969 he migrated to Zürich, Switzerland, where he was employed at the Institute for Empirical Research in
Economics at the University of Zürich and scientific advisor at the Swiss National Science Foundation
(Schweizerische Nationalfonds zur Förderung der wissenschaftlichen Forschung). In Zürich in 1970 Kofler
developed his linear partial information (LPI) theory allowing qualified decisions to be made on the basis of fuzzy
logic: incomplete or fuzzy a priori information.
Kofler was visiting professor at the University of St Petersburg (former Leningrad, Russia), University of Heidelberg
(Germany), McMaster University (Hamilton, Ontario, Canada) and University of Leeds (England). He collaborated
with many well known specialists in information theory, such as Oskar R. Lange in Poland, Nicolai Vorobiev in the
Soviet Union, Günter Menges in Germany, and Heidi Schelbert and Peter Zweifel in Zürich. He was the author of
many books and articles. He died in Zürich.
Edward Kofler
• E. Kofler (2000). Linear partial information with applications.
• E. Kofler & P. Zweifel (2005). One-shot decisions under Linear Partial Information. Springer Netherlands.
ISSN 0040-5833.(ISSN= 0040-5833)/[2]
• A. Zimmermann, P. Zweifel, E. Kofler (2006). Application of the linear partial information model to forecasting
the Swiss timber market. Journal of Forecasting, Volume 4 Issue 4, Pages 387 - 398./[3]
• M. Behara, E. Kofler and G. Menges (2008). Entropy and informativity in decision situations under partial
information. Springer Netherlands.(ISSN=0932-5026, Online:1613-9798)/[4]
• E. Kofler (2008). Taking into account confidence measures in the valuation of statistical inferences. Springer
• "Set theory Considerations on the Chess Game and the Theory of Corresponding Elements"- Mathematics
Seminar at the University of Lvov, 1936
• On the history of mathematics (Fejezetek a matematika történetéből) – book, 339 pages, Warsaw 1962 and
Budapest 1965
• From the digit to infinity – book, 312 pages, Warsaw 1960
• Economic decisions and the theory of games – Dissertation, University of Warsaw 1961
• Introduction to game theory – book, 230 pages, Warsaw 1962
• Optimization of multiple goals, Przeglad Statystyczny, Warsaw 1965
• The value of information – book, 104 pages, Warsaw 1967
• (With H. Greniewski and N. Vorobiev) Strategy of games, book, 80 pages, Warsaw 1968
• "Das Modell des Spiels in der wissenschaftlichen Planung" Mathematik und Wirtschaft No.7, East Berlin 1969
• Entscheidungen bei teilweise bekannter Verteilung der Zustände, Zeitschrift für OR, Vol. 18/3, 1974
• Konfidenzintervalle in Entscheidungen bei Ungewissheit, Stattliche Hefte, 1976/1
• "Entscheidungen bei teilweise bekannter Verteilung der Zustande", Zeitschrift für OR, Bd. 18/3, 1974, S 141-157
• "Konfidenzintervalle in Entscheidungen bei Ungewissheit", Statistische Hefte, 1976/1, S. 1-21
• (With G. Menges) Entscheidungen bei unvollständiger Information, Springer Verlag, 1976
• (With G. Menges) "Cognitive Decisions under Partial Information", in R.J. Bogdan (ed.), Local Induction, Reidel,
Dodrecht-Holland, 1976
• (With G. Menges) "Entscheidungen bei unvollständiger Information", volume 136 of Lecture Notes in Economics
and Mathematical Systems. Springer, Berlin, 1976.
• (With G. Menges) "Stochastic Linearisation of Indeterminateness" in Mathematical Economics and Game
Theory, (Springer) Berlin-Heidelberg-New York 1977, S. 20-63
• (With G. Menges) "Die Strukturierung von Unbestimmtheiten und eine Verallgemeinegung des Axiomensystems
von Kolmogoroff", Statistische Hefte 1977/4, S. 297-302
• (With G. Menges) "Lineare partielle Information, fuzziness und Vielziele-Optimierung", Proceedings in
Operations Research 8, Physica-Verlag 1979
• (With Fahrion, R., Huschens, S., Kuß, U., and Menges, G.) "Stochastische partielle Information (SPI)",
Statistische Hefte, Bd. 21, Jg. 1980, S. 160-167
• "Fuzzy sets- oder LPI-Theorie?" in G. Menges, H. Schelbert, P. Zweifel (eds.), Stochastische Unschärfe in
Wirtschaftswissenschaften, Haag & Herchen, Frankfurt-am-Main, 1981
• (With P. Zweifel)"Decisions under Fuzzy State Distribution with Application to the dealt Risks of Nuclear
Power", in Haag, W. (ed.), Large Scale Energy Systems, (Pergamon), Oxford 1981, S: 437-444
Edward Kofler
• "Extensive Spiele bei unvollständiger Information", in Information in der Wirtschaft, Gesellschaft für
Wirtschafts- und Sozialwissenschaften, Band 126, Berlin 1982
• "Equilibrium Points, Stability and Regulation in Fuzzy Optimisation Systems under Linear Partial Stochastic
Information (LPI)", Proceedings of the International Congress of Cybernetics and Systems, AFCET, Paris 1984,
pp. 233-240
• "Fuzzy Weighing in Multiple Objective Decision Making, G. Menges Contribution and Some New
Developments", Beitrag zum Gedenkband G. Menges, Hrgb. Schneeweiss, H., Strecker H., Springer Verlag 1984
• (With Z. W. Kmietowicz, and A. D. Pearman) "Decision making with Linear Partial Information (L.P.I.)". The
Journal of the Operational Research Society, 35(12):1079-1090, 1984
• (With P. Zweifel, A. Zimmermann) "Application of the Linear Partial Information (LPI) to forecasting the Swiss
timber market" Journal of Forecasting 1985, v4(4),387-398
• (With Peter Zweifel) "Exploiting linear partial information for optimal use of forecasts with an application to U.S.
economic policy, International Journal of Forecasting, 1988
E. Kofler's work from 1989 "Forecasting and Stability
under Fuzzy Information"
• "Prognosen und Stabilität bei unvollständiger Information",
Campus 1989
• (With P. Zweifel) "Convolution of Fuzzy Distributions in
Decision Making", Statistical Papers 32, Springer 1991, p.
• (With P. Zweifel) "One-Shot Decisions under Linear Partial
Information" Theory and Decision 34, 1993, p. 1-20
• "Decision Making under Linear Partial Information".
Proceedings of the European Congress EUFIT, Aachen, 1994,
p. 891-896
• (With P. Zweifel) "Linear Partial Information in One-Shot
Decisions", Selecta Statistica Vol. IX, 1996
• Mehrfache Zielsetzung in wirtschaftlichen Entscheidungen bei
unscharfen Daten, Institut für Empirische
Wirtschaftsforschung, 9602, 1996
• "Linear Partial Information with Applications". Proceedings of
ISFL 1997 (International Symposium on Fuzzy Logic), Zürich,
1997, p.235-239
• (With Thomas Kofler) "Forecasting Analysis of the Economic
Growth", Selecta Statistica Canadiana, 1998
• "Linear Partial Information with Applications in Fuzzy Sets and Systems", 1998. North-Holland
• (With Thomas Kofler) Fuzzy Logic and Economic Decisions, 1998
• (With L. Götte) "Fuzzy Systems and their Game Theoretical Solution", International Conference on Operations
Research, ETH, Zürich, August 1998
• "Forecasting and Optimal Strategies in Fuzzy Chess Situations ("Prognosen und Optimale Strategien in
unscharfen Schachsituationen"), Idee & Form No. 70, 2001 Zürich, pp. 2065 & 2067
• (With P. Zweifel) "One-shot decisions under Linear Partial Information" - Springer Netherlands, 2005
Edward Kofler
External links
• How to apply the Linear Partial Information (LPI)
• Linear Partial Information (LPI) theory and its applications
• Applying the Linear Partial Information (LPI) for USA's economy policy
• Practical decision making with the Linear Partial Information (LPI)
• Stochastic programming applying fuzzy Linear Partial Information (LPI)
• One-shot decisions applying the Linear Partial Information (LPI)
[1] http:/ / www. sciencedirect. com/ science?_ob=ArticleURL& _udi=B6V05-41N5GTV-F&_user=10& _rdoc=1&_fmt=&_orig=search&
_sort=d&_docanchor=&view=c&_acct=C000050221& _version=1&_urlVersion=0&_userid=10&
[2] http:// springerlink.com/ content/ l625g27848x67v0j/
[3] http:// www3. interscience. wiley. com/ journal/113337798/ abstract
[4] http:/ / www. springerlink.com/ content/ g137w7r0w503x431/
[5] http:// www. springerlink.com/ content/ u125751263481687/
[6] http:// direct.bl. uk/ bld/ PlaceOrder.do?UIN=148552859&ETOC=RN& from=searchengine
[7] http:// www. ingentaconnect. com/ content/ els/ 01650114/ 2001/ 00000118/ 00000001/ art00088
[8] http:/ / ideas. repec.org/ a/ eee/ intfor/v4y1988i1p15-32. html
[9] http:// econpapers.repec. org/article/eeeintfor/v_3A4_3Ay_3A1988_3Ai_3A1_3Ap_3A15-32. htm
[10] http:/ / www. sciencedirect. com/ science?_ob=ArticleURL& _udi=B6VCT-4BX7BW4-4& _user=10& _coverDate=05%2F01%2F2005&
_rdoc=1&_fmt=&_orig=search&_sort=d& view=c&_acct=C000050221& _version=1&_urlVersion=0&_userid=10&
[11] http:/ / www. springerlink.com/ content/ l625g27848x67v0j
Lavarand was Silicon Graphics' name for its hardware random number generator that worked by taking pictures of
the patterns made by the floating material in lava lamps, extracting random data from the pictures, and using the
result to seed a pseudo-random number generator.
Although the second part of the number generation uses a
pseudo-random number generator, it is a "true" random number generator due to the random seed that is used.
It is covered under U.S. Patent 5732138
, titled "Method for seeding a pseudo-random number generator with a
cryptographic hash of a digitization of a chaotic system."
From 1997 through 2001,
there was a web site at http:/ / lavarand. sgi. com/
demonstrating the technique.
Landon Curt Noll, one of the originators, went on to help develop LavaRnd, which does not use lava lamps. Despite
the short life of lavarand.sgi.com, it is often cited as an example of an online random number source.

[1] "Totally Random" (http:/ / www.wired.com/ wired/ archive/11. 08/ random. html). Wired Magazine 11 (08). August 2003. .
[2] "Internet Archive Wayback Machine" (http:// web. archive. org/web/ */ http:/ / lavarand.sgi. com/ ). . Retrieved 2010-01-05.
[3] http:/ / lavarand.sgi. com/
[4] U.S. Patent 6889236 (http:// www.google. com/ patents?vid=6889236)
[5] U.S. Patent 7,031,991 (http:// www.google. com/ patents?vid=7,031,991)
External links
• Archived version of Lavarand.com from Archive.org (http:// web.archive. org/web/ 20010926221159/ http:/ /
lavarand. sgi. com/ ) (note that pictures do not work)
LavaRnd is a random number generator that works by measuring noise from a CCD (typically, an inexpensive
webcam). The CCD is enclosed in a light-proof container, and operated at a high gain. The resulting images are not
perfectly black—they contain noise. The LavaRnd system takes noisy data from the CCD and runs it through an
algorithm called the Digital Blender
to produce data that is more uniformly random. The resulting data is much
more random than typical pseudorandom numbers.
The source code for LavaRnd has been released as free software (with an LGPL license). The authors of LavaRnd
deliberately did not patent the method.
Landon Curt Noll, one of the developers of LavaRnd, was also a developer of Lavarand.
External links
• LavaRnd official web site
• How LavaRnd works
[1] http:/ / www. lavarnd.org/ what/ digital-blender.html
[2] http:/ / www. lavarnd.org
[3] http:/ / www. lavarnd.org/ what/ how-it-works.html
Linear partial information
Linear partial information
Linear partial information (LPI) is a method of making decisions based on insufficient or fuzzy information. LPI
was introduced in 1970 by Polish - Swiss mathematician Edward Kofler (1911–2007) to simplify decision processes.
Comparing to other methods the LPI-fuzziness is algorithmically simple and particularly in decision making, more
practically oriented. Instead of an indicator function the decision maker linearizes any fuzziness by establishing of
linear restrictions for fuzzy probability distributions or normalized weights. In the LPI-procedure the decision maker
linearizes any fuzziness instead of applying a membership function. This can be done by establishing stochastic and
non-stochastic LPI-relations. A mixed stochastic and non-stochastic fuzzification is often a basis for the
LPI-procedure. By using the LPI-methods any fuzziness in any decision situation can be considered on the base of
the linear fuzzy logic.
Any Stochastic Partial Information SPI(p), which can be considered as a solution of a linear inequality
system, is called Linear Partial Information LPI(p) about probability p. It can be considered as an
LPI-fuzzification of the probability p corresponding to the concepts of linear fuzzy logic.
a) The MaxEmin Principle
To obtain the maximally warranted expected value, the decision maker has to choose the strategy which maximizes
the minimal expected value. This procedure leads to the MaxEmin - Principle and is an extension of the Bernoulli's
b) The MaxWmin Principle
This principle leads to the maximal guaranteed weight function, regarding the extreme weights.
c) The Prognostic Decision Principle (PDP)
This principle is based on the prognosis interpretation of strategies under fuzziness.
Fuzzy equilibrium and stability
Despite the fuzziness of information, it is often necessary to choose the optimal, most cautious strategy, for example
in economic planning, in conflict situations or in daily decisions. This is impossible without the concept of fuzzy
equilibrium. The concept of fuzzy stability is considered as an extension into a time interval, taking into account the
corresponding stability area of the decision maker. The more complex is the model, the softer a choice has to be
considered. The idea of fuzzy equilibrium is based on the optimization principles. Therefore the MaxEmin-,
MaxGmin- and PDP-stability have to be analyzed. The violation of these principles leads often to wrong predictions
and decisions.
Linear partial information
LPI equilibrium point
Considering a given LPI-decision model, as a convolution of the corresponding fuzzy states or a disturbance set, the
fuzzy equilibrium strategy remains the most cautious one, despite of the presence of the fuzziness. Any deviation
from this strategy can cause a loss for the decision maker.
External links
• Tools for establishing dominance with linear partial information and attribute hierarchy
• Linear Partial Information with applications
• Linear Partial Information (LPI) with applications to the U.S. economic policy
• Practical decision making with Linear Partial Information (LPI)
• Stochastic programming with fuzzy linear partial information on probability distribution
• One-shot decisions under Linear Partial Information
Selected references
• Edward Kofler - Equilibrium Points, Stability and Regulation in Fuzzy Optimisation Systems under Linear Partial
Stochastic Information (LPI), Proceedings of the International Congress of Cybernetics and Systems, AFCET,
Paris 1984, pp. 233–240
• Edward Kofler - Decision Making under Linear Partial Information. Proceedings of the European Congress
EUFIT, Aachen, 1994, p. 891-896.
• Edward Kofler - Linear Partial Information with Applications. Proceedings of ISFL 1997 (International
Symposium on Fuzzy Logic), Zurich, 1997, p. 235-239.
• Edward Kofler – Entscheidungen bei teilweise bekannter Verteilung der Zustände, Zeitschrift für OR, Vol. 18/3,
• Edward Kofler - Extensive Spiele bei unvollständiger Information, in Information in der Wirtschaft, Gesellschaft
für Wirtschafts- und Sozialwissenschaften, Band 126, Berlin 1982
[1] http:/ / direct.bl. uk/ bld/ PlaceOrder.do?UIN=148552859&ETOC=RN& from=searchengine/
[2] http:// www. ingentaconnect. com/ content/ els/ 01650114/ 2001/ 00000118/ 00000001/ art00088/
[3] http:/ / ideas. repec.org/ a/ eee/ intfor/v4y1988i1p15-32. html/
[4] http:// econpapers.repec. org/article/eeeintfor/v_3A4_3Ay_3A1988_3Ai_3A1_3Ap_3A15-32. htm/
[5] http:/ / www. sciencedirect. com/ science?_ob=ArticleURL& _udi=B6VCT-4BX7BW4-4& _user=10& _coverDate=05%2F01%2F2005&
_rdoc=1&_fmt=&_orig=search&_sort=d& view=c&_acct=C000050221& _version=1&_urlVersion=0&_userid=10&
[6] http:/ / www. springerlink.com/ content/ l625g27848x67v0j/
Mendelian randomization
Mendelian randomization
In epidemiology, Mendelian randomization is a method of using measured variation in genes of known function to
examine the causal effect of a modifiable exposure on disease in non-experimental studies. The design was first
described by Gray and Wheatley (1991) as a method for obtaining unbiased estimates of the effects of a putative
casual variable without conducting a traditional randomised trial
. These authors also coined the term Mendelian
Background - spurious findings from observational epidemiology
An important focus of observational epidemiology is the identification of modifiable causes of common diseases that
are of public health interest. In order to have firm evidence that a recommended public health intervention will have
the desired beneficial effect, the observed association between the particular risk factor and disease must imply that
the risk factor actually causes the disease.
Well-known successes include the identified causal links between smoking and lung cancer, and between blood
pressure and stroke. However, there have also been notable failures when identified exposures were later shown by
randomised controlled trials (RCTs) to be non-causal. For instance, it has now been shown that hormone
replacement therapy will not prevent cardiovascular disease, as was previously thought, and may have other adverse
health effects (Rossouw et al. 2002). The reason for such spurious findings in observational epidemiology is most
likely to be confounding by social, behavioural or physiological factors which are difficult to control for and
particularly difficult to measure accurately. Moreover, many findings cannot be replicated by RCTs for ethical
Implementing Mendelian randomization
Mendelian randomization is a method that allows one to test for, or in certain cases to estimate, a causal effect from
observational data in the presence of confounding factors. It uses common genetic polymorphisms with
well-understood effects on exposure patterns (e.g., propensity to drink alcohol) or effects that mimic those produced
by modifiable exposures (e.g., raised blood cholesterol (Katan 1986)). Importantly, the genotype must only affect the
disease status indirectly via its effect on the exposure of interest. Because genotypes are assigned randomly when
passed from parents to offspring during meiosis, if we assume that choice of mate is not associated with genotype
(panmixia), then the population genotype distribution should be unrelated to the confounders that typically plague
observational epidemiology studies. In this regard, Mendelian randomization can be thought of as a “natural” RCT.
From a statistical perspective, it is an application of the technique of instrumental variables (Thomas & Conti 2004,
Didelez & Sheehan 2007), with genotype acting as an instrument for the exposure of interest.
Mendelian randomization relies on getting good estimates from genetic association studies. Misleading conclusions
can also be drawn in the presence of linkage disequilibrium, genetic heterogeneity, pleiotropy, or population
stratification (Davey Smith & Ebrahim 2003).
Mendelian randomization
[1] R. Gray and K. Wheatley. (1991). How to avoid bias when comparing bone marrow transplantation with chemotherapy. Bone Marrow
Transplant, 7 Suppl 3, (http:/ / dx. doi. org/ 9-12)
• G. Davey Smith and S. Ebrahim (2003) Mendelian randomization: can genetic epidemiology contribute to
understanding environmental determinants of disease? International Journal of Epidemiology 32:
• G. Davey Smith, S. Ebrahim, S. Lewis, A.L.Hansell, L.J. Palmer and P.R. Burton (2005) Genetic epidemiology
and public health: hope, hype, and future prospects. Lancet 366: 1484-1498.doi:10.1016/S0140-6736(05)67601-5
• G. Davey Smith and S. Ebrahim (2005) What can Mendelian randomisation tell us about modifiable behavioural
and environmental exposures? BMJ 330: 1076-1079.doi:10.1136/bmj.330.7499.1076
• V. Didelez and N. Sheehan (2007) Mendelian randomization as an instrumental variable approach to causal
inference. Statistical Methods in Medical Research 16:309-330 doi:10.1177/0962280206077743
• M.B. Katan (1986) Apolipoprotein E isoforms, serum cholesterol and cancer. Lancet, 327:507-508.
• J.E. Rossouw et al. (2002) Risks and benefits of estrogen plus progestin in healthy post-menopausal women:
principal results from the Women’s Health Initiative randomized controlled trial. JAMA 288: 321-333.
• D.C. Thomas and D.V. Conti (2004) Commentary: The concept of Mendelian randomization. International
Journal of Epidemiology 32: 21-25 doi:10.1093/ije/dyh048
External links
• Mendelian Randomization: A Perfect Causal Epidemiologic Approach to Simulate a Randomized Trial? (http://
epidemiologic. blogspot. com/ 2006/ 03/ mendelian-randomization-perfect-causal.html) Epidemiologic Inquiry
2006, 1: 16
• G. Davey Smith (2006). Capitalising on Mendelian randomization to assess the effects of treatments. (http:/ /
www.jameslindlibrary.org/ trial_records/20th_Century/ 1990s/ gray/gray-commentary.html) James Lind
Library (http:/ / www. jameslindlibrary.org).
• PHOEBE Biostatistics Group (2007) Mendelian randomisation: Inferring causality in observational epidemiology
(http:/ / www. genestat. org/index. php?n=GeneStat. MendelianRandomisation)
A Million Random Digits with 100,000 Normal Deviates
A Million Random Digits with 100,000 Normal
A random sampling of 300 random digits from A
Million Random Digits with 100,000 Normal
A Million Random Digits with 100,000 Normal Deviates is a 1955
book by the RAND Corporation. The book, comprising primarily of a
random number table, was an important 20th century work in the field
of statistics and random numbers. It was produced starting in 1947 by
an electronic simulation of a roulette wheel attached to a computer, the
results of which were then carefully filtered and tested before being
used to generate the table. The RAND table was an important
breakthrough in delivering random numbers, because such a large and
carefully prepared table had never before been available. In addition to
being available in book form, one could also order the digits on a series
of punched cards. The main use of the tables was in statistics and the
experimental design of scientific experiments, especially those which
employed the Monte Carlo method; in cryptography, they have also
been used as "nothing up my sleeve numbers", for example in the
design of the Khafre cipher. The book was one of the last of a series of
random number tables produced from the mid-1920s through the
1950s, after which the development of high speed computers allowed
faster operation through the generation of pseudorandom numbers rather than reading them from tables.
The book was reissued in 2001 (ISBN 0-8330-3047-7) with a new foreword by RAND Executive Vice President
Michael D. Rich. It has generated many humorous user reviews on Amazon.com.


The digits and the deviates are available for free online, at: Datafile: A Million Random Digits
and 100,000
Normal Deviates
. The text of the book is also available at [6].
[1] Amazon.com Customer Reviews: A Million Random Digits with 100,000 Normal Deviates (http:/ / www. amazon.com/
Million-Random-Digits-Normal-Deviates/product-reviews/0833030477/ ) (Paperback version)]
[2] Heffernan, Virginia (January 15, 2010). "The Reviewing Stand" (http:/ / www. nytimes.com/ 2010/ 01/ 17/ magazine/ 17FOB-Medium-t.
html). The New York Times Magazine. . Retrieved 2011-03-09.
[3] Swaine, Michael (2005). "Is the Future Random and Faceless?" (http:// drdobbs. com/ open-source/184406045). Dr Dobbs Journal 30: 95. .
[4] http:// www. rand.org/ content/ dam/ rand/pubs/ monograph_reports/2005/ digits. txt. zip
[5] http:// www. rand.org/ content/ dam/ rand/pubs/ monograph_reports/2005/ deviates.txt.zip
[6] http:// www. rand.org/ pubs/ monograph_reports/MR1418/
• George W. Brown, "History of RAND's random digits—Summary," in A.S. Householder, G.E. Forsythe, and
H.H. Germond, eds., Monte Carlo Method, National Bureau of Standards Applied Mathematics Series, 12
(Washington, D.C.: U.S. Government Printing Office, 1951): 31-32. (Available here (http:// www.rand.org/
pubs/ papers/ 2008/ P113. pdf) for download from the RAND Corporation.)
A Million Random Digits with 100,000 Normal Deviates
External links
• Information at rand.org (http:// www. rand.org/publications/ MR/ MR1418/ )
Datafile: A Million Random Digits (http:// www. rand. org/pubs/ monograph_reports/2005/ digits. txt. zip),
100,000 Normal Deviates (http:/ / www. rand. org/pubs/ monograph_reports/2005/ deviates. txt. zip)
• Tom Jennings' page about the book (http:/ / www.wps. com/ projects/ million/ index. html)
• Another million random digits and 100,000 normal deviates (http:// hcoop. net/ ~ntk/ random/)
Monte Carlo method
Monte Carlo methods (or Monte Carlo experiments) are a class of computational algorithms that rely on repeated
random sampling to compute their results. Monte Carlo methods are often used in simulating physical and
mathematical systems. These methods are most suited to calculation by a computer and tend to be used when it is
infeasible to compute an exact result with a deterministic algorithm.
This method is also used to complement the
theoretical derivations.
Monte Carlo methods are especially useful for simulating systems with many coupled degrees of freedom, such as
fluids, disordered materials, strongly coupled solids, and cellular structures (see cellular Potts model). They are used
to model phenomena with significant uncertainty in inputs, such as the calculation of risk in business. They are
widely used in mathematics, for example to evaluate multidimensional definite integrals with complicated boundary
conditions. When Monte Carlo simulations have been applied in space exploration and oil exploration, their
predictions of failures, cost overruns and schedule overruns are routinely better than human intuition or alternative
"soft" methods.
The Monte Carlo method was coined in the 1940s by John von Neumann, Stanislaw Ulam and Nicholas Metropolis,
while they were working on nuclear weapon projects in the Los Alamos National Laboratory. It was named in
homage to Monte Carlo casino, a famous casino, where Ulam's uncle would often gamble away his money.
Monte Carlo method applied to approximating
the value of π
Monte Carlo methods vary, but tend to follow a particular pattern:
1. Define a domain of possible inputs.
2. Generate inputs randomly from a probability distribution over the
3. Perform a deterministic computation on the inputs.
4. Aggregate the results.
For example, given that a circle inscribed in a square and the square
itself have a ratio of areas that is π/4, the value of π can be
approximated using a Monte Carlo method:
1. Draw a square on the ground, then inscribe a circle within it.
2. Uniformly scatter some objects of uniform size (grains of rice or
sand) over the square.
3. Count the number of objects inside the circle and the total number
of objects.
4. The ratio of the two counts is an estimate of the ratio of the two areas, which is π/4. Multiply the result by 4 to
estimate π.
Monte Carlo method
In this procedure the domain of inputs is the square that circumscribes our circle. We generate random inputs by
scattering grains over the square then perform a computation on each input (test whether it falls within the circle).
Finally, we aggregate the results to obtain our final result, the approximation of π.
To get an accurate approximation for π this procedure should have two other common properties of Monte Carlo
methods. First, the inputs should truly be random. If grains are purposefully dropped into only the center of the
circle, they will not be uniformly distributed, and so our approximation will be poor. Second, there should be a large
number of inputs. The approximation will generally be poor if only a few grains are randomly dropped into the
whole square. On average, the approximation improves as more grains are dropped.
Before the Monte Carlo method was developed, simulations tested a previously understood deterministic problem
and statistical sampling was used to estimate uncertainties in the simulations. Monte Carlo simulations invert this
approach, solving deterministic problems using a probabilistic analog (see Simulated annealing).
An early variant of the Monte Carlo method can be seen in the Buffon's needle experiment, in which π can be
estimated by dropping needles on a floor made of parallel strips of wood. In the 1930s, Enrico Fermi first
experimented with the Monte Carlo method while studying neutron diffusion, but did not publish anything on it.
In 1946, physicists at Los Alamos Scientific Laboratory were investigating radiation shielding and the distance that
neutrons would likely travel through various materials. Despite having most of the necessary data, such as the
average distance a neutron would travel in a substance before it collided with an atomic nucleus or how much energy
the neutron was likely to give off following a collision, the problem could not be solved with analytical calculations.
Stanisław Ulam had the idea of using random experiments. He recounts his inspiration as follows:
The first thoughts and attempts I made to practice [the Monte Carlo Method] were suggested by a question
which occurred to me in 1946 as I was convalescing from an illness and playing solitaires. The question was
what are the chances that a Canfield solitaire laid out with 52 cards will come out successfully? After spending
a lot of time trying to estimate them by pure combinatorial calculations, I wondered whether a more practical
method than "abstract thinking" might not be to lay it out say one hundred times and simply observe and count
the number of successful plays. This was already possible to envisage with the beginning of the new era of fast
computers, and I immediately thought of problems of neutron diffusion and other questions of mathematical
physics, and more generally how to change processes described by certain differential equations into an
equivalent form interpretable as a succession of random operations. Later [in 1946], I described the idea to
John von Neumann, and we began to plan actual calculations.
–Stanisław Ulam
Being secret, the work of von Neumann and Ulam required a code name. Von Neumann chose the name "Monte
Carlo". The name is a reference to the Monte Carlo Casino in Monaco where Ulam's uncle would borrow money to


Using lists of "truly" random numbers was extremely slow, von Neumann developed a form of
making pseudorandom numbers, using the middle-square method. Though this method has been criticized as crude,
von Neumann was aware of this: he justified it as being faster than any other method at his disposal, and also noted
that when it went awry it did so obviously, unlike methods which could be subtly incorrect.
Monte Carlo methods were central to the simulations required for the Manhattan Project, though severely limited by
the computational tools at the time. In the 1950s they were used at Los Alamos for early work relating to the
development of the hydrogen bomb, and became popularized in the fields of physics, physical chemistry, and
operations research. The Rand Corporation and the U.S. Air Force were two of the major organizations responsible
for funding and disseminating information on Monte Carlo methods during this time, and they began to find a wide
application in many different fields.
Monte Carlo method
Uses of Monte Carlo methods require large amounts of random numbers, and it was their use that spurred the
development of pseudorandom number generators, which were far quicker to use than the tables of random numbers
that had been previously used for statistical sampling.
There is no consensus on how Monte Carlo should be defined. For example, Ripley
defines most probabilistic
modeling as stochastic simulation, with Monte Carlo being reserved for Monte Carlo integration and Monte Carlo
statistical tests. Sawilowsky
distinguishes between a simulation, Monte Carlo method, and a Monte Carlo
simulation. A simulation is a fictitious representation of reality. A Monte Carlo method is a technique that can be
used to solve a mathematical or statistical problem. A Monte Carlo simulation uses repeated sampling to determine
the properties of some phenomenon. Examples:
• Drawing a pseudo-random uniform variable from the interval [0,1] can be used to simulate the tossing of a coin:
If the value is less than or equal to 0.50 designate the outcome as heads, but if the value is greater than 0.50
designate the outcome as tails. This is a simulation, but not a Monte Carlo simulation.
• The area of an irregular figure inscribed in a unit square can be determined by throwing darts at the square and
computing the ratio of hits within the irregular figure to the total number of darts thrown. This is a Monte Carlo
method of determining area, but not a simulation.
• Drawing a large number of pseudo-random uniform variables from the interval [0,1], and assigning values less
than or equal to 0.50 as heads and greater than 0.50 as tails, is a Monte Carlo simulation of the behavior of
repeatedly tossing a coin.
Kalos and Whitlock
point out that such distinctions are not always easy to maintain. For example, the emission of
radiation from atoms is a natural stochastic process. It can be simulated directly, or its average behavior can be
described by stochastic equations that can themselves be solved using Monte Carlo methods. "Indeed, the same
computer code can be viewed simultaneously as a 'natural simulation' or as a solution of the equations by natural
Monte Carlo and random numbers
Monte Carlo simulation methods do not always require truly random numbers to be useful — while for some
applications, such as primality testing, unpredictability is vital.
Many of the most useful techniques use
deterministic, pseudorandom sequences, making it easy to test and re-run simulations. The only quality usually
necessary to make good simulations is for the pseudo-random sequence to appear "random enough" in a certain
What this means depends on the application, but typically they should pass a series of statistical tests. Testing that
the numbers are uniformly distributed or follow another desired distribution when a large enough number of
elements of the sequence are considered is one of the simplest, and most common ones.
Sawilowsky lists the characteristics of a high quality Monte Carlo simulation:
• the (pseudo-random) number generator has certain characteristics (e. g., a long “period” before the sequence
• the (pseudo-random) number generator produces values that pass tests for randomness
• there are enough samples to ensure accurate results
• the proper sampling technique is used
• the algorithm used is valid for what is being modeled
• it simulates the phenomenon in question.
Pseudo-random number sampling algorithms are used to transform uniformly distributed pseudo-random numbers
into numbers that are distributed according to a given probability distribution.
Monte Carlo method
Monte Carlo simulation versus “what if” scenarios
There are ways of using probabilities that are definitely not Monte Carlo simulations—for example, deterministic
modeling using single-point estimates. Each uncertain variable within a model is assigned a “best guess” estimate.
Scenarios (such as best, worst, or most likely case) for each input variable are chosen and the results recorded.
By contrast, Monte Carlo simulations sample probability distribution for each variable to produce hundreds or
thousands of possible outcomes. The results are analyzed to get probabilities of different outcomes occurring.
example, a comparison of a spreadsheet cost construction model run using traditional “what if” scenarios, and then
run again with Monte Carlo simulation and Triangular probability distributions shows that the Monte Carlo analysis
has a narrower range than the “what if” analysis. This is because the “what if” analysis gives equal weight to all
scenarios (see quantifying uncertainty in corporate finance).
Monte Carlo methods are especially useful for simulating phenomena with significant uncertainty in inputs and
systems with a large number of coupled degrees of freedom. Areas of application include:
Physical sciences
Monte Carlo methods are very important in computational physics, physical chemistry, and related applied fields,
and have diverse applications from complicated quantum chromodynamics calculations to designing heat shields and
aerodynamic forms. In statistical physics Monte Carlo molecular modeling is an alternative to computational
molecular dynamics, and Monte Carlo methods are used to compute statistical field theories of simple particle and
polymer systems.
Quantum Monte Carlo methods solve the many-body problem for quantum systems. In
experimental particle physics, Monte Carlo methods are used for designing detectors, understanding their behavior
and comparing experimental data to theory. In astrophysics, they are used to model the evolution of galaxies.
Monte Carlo methods are also used in the ensemble models that form the basis of modern weather forecasting.
Monte Carlo methods are widely used in engineering for sensitivity analysis and quantitative probabilistic analysis in
process design. The need arises from the interactive, co-linear and non-linear behavior of typical process
simulations. For example,
• in microelectronics engineering, Monte Carlo methods are applied to analyze correlated and uncorrelated
variations in analog and digital integrated circuits. This enables designers to estimate realistic 3–sigma corners
and effectively optimize circuit yields.
• in geostatistics and geometallurgy, Monte Carlo methods underpin the design of mineral processing flowsheets
and contribute to quantitative risk analysis.
• impacts of pollution are simulated
and diesel compared with petrol.
• In autonomous robotics, Monte Carlo localization can be used to determine the position of a robot, it is often
applied to stochastic filters such as the Kalman filter or Particle filter which form the heart of the SLAM (
simultaneous Localisation and Mapping ) algorithm.
Monte Carlo method
Computational Biology
Monte Carlo methods are used in computational biology, such for as Bayesian inference in phylogeny.
Biological systems such as proteins
images of cancer,
are being studied by means of computer
The systems can be studied in the coarse-grained or ab initio frameworks depending on the desired accuracy.
Computer simulations allow us to monitor the local environment of a particular molecule to see if some chemical
reaction is happening for instance. We can also conduct thought experiments when the physical experiments are not
feasible, for instance breaking bonds, introducing impurities at specific sites, changing the local/global structure, or
introducing external fields.
Applied statistics
In applied statistics, Monte Carlo methods are generally used for two purposes:
1. To compare competing statistics for small samples under realistic data conditions. Although Type I error and
power properties of statistics can be calculated for data drawn from classical theoretical distributions (e.g., normal
curve, Cauchy distribution) for asymptotic conditions (i. e, infinite sample size and infinitesimally small treatment
effect), real data often do not have such distributions.
2. To provide implementations of hypothesis tests that are more efficient than exact tests such as permutation tests
(which are often impossible to compute) while being more accurate than critical values for asymptotic
Monte Carlo methods are also a compromise between approximate randomization and permutation tests. An
approximate randomization test is based on a specified subset of all permutations (which entails potentially
enormous housekeeping of which permutations have been considered). The Monte Carlo approach is based on a
specified number of randomly drawn permutations (exchanging a minor loss in precision if a permutation is drawn
twice – or more frequently – for the efficiency of not having to track which permutations have already been
Monte Carlo method
Monte Carlo tree search applied to a game of
Battleship. Initially the algorithm takes random
shots, but as possible states are eliminated, the
shots can be more selective. As a crude example,
if a ship is hit (figure A), then adjacent squares
become much higher priorities (figures B and C).
Monte Carlo methods have recently been incorporated in algorithms
for playing games that have outperformed previous algorithms in
games like Go and Battleship. These algorithms employ Monte Carlo
tree search. Possible algorithms are organized in a tree and a large
number of random simulations are used to estimate the long-term
potential of each move. A black box simulator represents the
opponent's moves. In games like Battleship, where there is only limited
knowledge of the state of the system (i.e., the positions of the ships), a
belief state is constructed consisting of probabilities for each state and
then initial states are sampled for running simulations. The belief state
is updated as the game proceeds, as in the figure. On a 10 x 10 grid, in
which the total possible number of moves is 100, one algorithm sank
all the ships 50 moves faster, on average, than random play.
One of the main problems that this approach has in game playing is
that it sometimes misses an isolated, very good move. These
approaches are often strong strategically but weak tactically, as tactical
decisions tend to rely on a small number of crucial moves which are
easily missed by the randomly searching Monte Carlo algorithm.
Design and visuals
Monte Carlo methods have also proven efficient in solving coupled
integral differential equations of radiation fields and energy transport,
and thus these methods have been used in global illumination
computations which produce photo-realistic images of virtual 3D models, with applications in video games,
architecture, design, computer generated films, and cinematic special effects.
Finance and business
Monte Carlo methods in finance are often used to calculate the value of companies, to evaluate investments in
projects at a business unit or corporate level, or to evaluate financial derivatives. They can be used to model project
schedules, where simulations aggregate estimates for worst-case, best-case, and most likely durations for each task to
determine outcomes for the overall project.
When planning a wireless network, design must be proved to work for a wide variety of scenarios that depend
mainly on the number of users, their locations and the services they want to use. Monte Carlo methods are typically
used to generate these users and their states. The network performance is then evaluated and, if results are not
satisfactory, the network design goes through an optimization process.
Monte Carlo method
Use in mathematics
In general, Monte Carlo methods are used in mathematics to solve various problems by generating suitable random
numbers and observing that fraction of the numbers which obeys some property or properties. The method is useful
for obtaining numerical solutions to problems which are too complicated to solve analytically. The most common
application of the Monte Carlo method is Monte Carlo integration.
Monte-Carlo integration works by comparing
random points with the value of the function
Errors reduce by a factor of
Deterministic numerical integration algorithms work well in a small
number of dimensions, but encounter two problems when the functions
have many variables. First, the number of function evaluations needed
increase rapidly with the number of dimensions. For example, if 10
evaluations provide adequate accuracy in one dimension, then 10
points are needed for 100 dimensions—far too many to be computed.
This is called the curse of dimensionality. Second, the boundary of a
multidimensional region may be very complicated, so it may not be
feasible to reduce the problem to a series of nested one-dimensional
100 dimensions is by no means unusual, since in many
physical problems, a "dimension" is equivalent to a degree of freedom.
Monte Carlo methods provide a way out of this exponential increase in computation time. As long as the function in
question is reasonably well-behaved, it can be estimated by randomly selecting points in 100-dimensional space, and
taking some kind of average of the function values at these points. By the law of large numbers, this method will
display convergence—i.e., quadrupling the number of sampled points will halve the error, regardless of the
number of dimensions.
A refinement of this method, known as importance sampling in statistics, involves sampling the points randomly, but
more frequently where the integrand is large. To do this precisely one would have to already know the integral, but
one can approximate the integral by an integral of a similar function or use adaptive routines such as Stratified
sampling, recursive stratified sampling, adaptive umbrella sampling

or the VEGAS algorithm.
A similar approach, the quasi-Monte Carlo method, uses low-discrepancy sequences. These sequences "fill" the area
better and sample the most important points more frequently, so quasi-Monte Carlo methods can often converge on
the integral more quickly.
Monte Carlo method
Another class of methods for sampling points in a volume is to simulate random walks over it (Markov chain Monte
Carlo). Such methods include the Metropolis-Hastings algorithm, Gibbs sampling and the Wang and Landau
Another powerful and very popular application for random numbers in numerical simulation is in numerical
optimization. The problem is to minimize (or maximize) functions of some vector that often has a large number of
dimensions. Many problems can be phrased in this way: for example, a computer chess program could be seen as
trying to find the set of, say, 10 moves that produces the best evaluation function at the end. In the traveling
salesman problem the goal is to minimize distance traveled. There are also applications to engineering design, such
as multidisciplinary design optimization.
Most Monte Carlo optimization methods are based on random walks. Essentially, the program moves randomly on a
multi-dimensional surface, preferring moves that reduce the function, but sometimes moving "uphill".
Inverse problems
Probabilistic formulation of inverse problems leads to the definition of a probability distribution in the model space.
This probability distribution combines a priori information with new information obtained by measuring some
observable parameters (data). As, in the general case, the theory linking data with model parameters is nonlinear, the
a posteriori probability in the model space may not be easy to describe (it may be multimodal, some moments may
not be defined, etc.).
When analyzing an inverse problem, obtaining a maximum likelihood model is usually not sufficient, as we
normally also wish to have information on the resolution power of the data. In the general case we may have a large
number of model parameters, and an inspection of the marginal probability densities of interest may be impractical,
or even useless. But it is possible to pseudorandomly generate a large collection of models according to the posterior
probability distribution and to analyze and display the models in such a way that information on the relative
likelihoods of model properties is conveyed to the spectator. This can be accomplished by means of an efficient
Monte Carlo method, even in cases where no explicit formula for the a priori distribution is available.
The best-known importance sampling method, the Metropolis algorithm, can be generalized, and this gives a method
that allows analysis of (possibly highly nonlinear) inverse problems with complex a priori information and data with
an arbitrary noise distribution.

Computational mathematics
Monte Carlo methods are useful in many areas of computational mathematics, where a "lucky choice" can find the
correct result. A classic example is Rabin's algorithm for primality testing: for any n which is not prime, a random x
has at least a 75% chance of proving that n is not prime. Hence, if n is not prime, but x says that it might be, we have
observed at most a 1-in-4 event. If 10 different random x say that "n is probably prime" when it is not, we have
observed a one-in-a-million event. In general a Monte Carlo algorithm of this kind produces one correct answer with
a guarantee n is composite, and x proves it so, but another one without, but with a guarantee of not getting this
answer when it is wrong too often—in this case at most 25% of the time. See also Las Vegas algorithm for a related,
but different, idea.
Monte Carlo method
[1] Hubbart 2007
[2] Hubbard 2009
[3] Metropolis 1987
[4] Kalos & Whitlock 2008
[5] Eckardt 1987
[6] Grinstead & Snell 1997
[7] Anderson 1986
[8] Ripley 1987
[9] Sawilowsky 2003
[10] Davenport 1992
[11] Vose 2000, p. 13
[12] Vose 2000, p. 16
[13] Baeurle 2009
[14] MacGillivray & Dodd 1982
[15] Int Panis et al. 2001
[16] Int Panis et al. 2002
[17] Ojeda & et al. 2009,
[18] Milik & Skolnick 1993
[19] Forastero et al. 2010
[20] Sawilowsky & Fahoome 2003
[21] Silver & Veness 2010
[22] Szirmay-Kalos 2008
[23] Press et al. 1996
[24] MEZEI, M (31 December 1986). "Adaptive umbrella sampling: Self-consistent determination of the non-Boltzmann bias". Journal of
Computational Physics 68 (1): 237–248. Bibcode 1987JCoPh..68..237M. doi:10.1016/0021-9991(87)90054-4.
[25] Bartels, Christian; Karplus, Martin (31 December 1997). "Probability Distributions for Complex Systems:  Adaptive Umbrella Sampling of
the Potential Energy". The Journal of Physical Chemistry B 102 (5): 865–880. doi:10.1021/jp972280j.
[26] Mosegaard & Tarantola 1995
[27] Tarantola 2005
• Anderson, H.L. (1986). "Metropolis, Monte Carlo and the MANIAC" (http:/ / library.lanl. gov/ cgi-bin/
getfile?00326886. pdf). Los Alamos Science 14: 96–108.
• Baeurle, Stephan A. (2009). "Multiscale modeling of polymer materials using field-theoretic methodologies: a
survey about recent developments" (http:/ / www.springerlink.com/ content/ xl057580272w8703/ ). Journal of
Mathematical Chemistry 46 (2): 363–426. doi:10.1007/s10910-008-9467-3.
• Berg, Bernd A. (2004). Markov Chain Monte Carlo Simulations and Their Statistical Analysis (With Web-Based
Fortran Code). Hackensack, NJ: World Scientific. ISBN 9812389350.
• Binder, Kurt (1995). The Monte Carlo Method in Condensed Matter Physics. New York: Springer.
ISBN 0387543694.
• Caflisch, R. E. (1998). Monte Carlo and quasi-Monte Carlo methods. Acta Numerica. 7. Cambridge University
Press. pp. 1–49.
• Davenport, J. H.. "Primality testing revisited". Proceeding ISSAC '92 Papers from the international symposium on
Symbolic and algebraic computation: 123 129. doi:10.1145/143242.143290.
• Doucet, Arnaud; Freitas, Nando de; Gordon, Neil (2001). Sequential Monte Carlo methods in practice. New
York: Springer. ISBN 0387951466.
• Eckhardt, Roger (1987). "Stan Ulam, John von Neumann, and the Monte Carlo method". Los Alamos Science,
Special Issue (15): 131–137.
• Fishman, G. S. (1995). Monte Carlo: Concepts, Algorithms, and Applications. New York: Springer.
ISBN 038794527X.
Monte Carlo method
• C. Forastero and L. Zamora and D. Guirado and A. Lallena (2010). "A Monte Carlo tool to simulate breast cancer
screening programmes". Phys. In Med. And Biol. 55 (17): 5213. Bibcode 2010PMB....55.5213F.
• Gould, Harvey; Tobochnik, Jan (1988). An Introduction to Computer Simulation Methods, Part 2, Applications to
Physical Systems. Reading: Addison-Wesley. ISBN 020116504X.
• Grinstead, Charles; Snell, J. Laurie (1997). Introduction to Probability. American Mathematical Society.
pp. 10–11.
• Hammersley, J. M.; Handscomb, D. C. (1975). Monte Carlo Methods. London: Methuen. ISBN 0416523404.
• Hubbard, Douglas (2007). How to Measure Anything: Finding the Value of Intangibles in Business. John Wiley &
Sons. p. 46.
• Hubbard, Douglas (2009). The Failure of Risk Management: Why It's Broken and How to Fix It. John Wiley &
• Kahneman, D.; Tversky, A. (1982). Judgement under Uncertainty: Heuristics and Biases. Cambridge University
• Kalos, Malvin H.; Whitlock, Paula A. (2008). Monte Carlo Methods. Wiley-VCH. ISBN 978-3527407606.
• Kroese, D. P.; Taimre, T.; Botev, Z.I. (2011). Handbook of Monte Carlo Methods (http:// www.
montecarlohandbook. org). New York: John Wiley & Sons. pp. 772. ISBN 0470177934.
• MacGillivray, H. T.; Dodd, R. J. (1982). "Monte-Carlo simulations of galaxy systems" (http:/ / www.
springerlink.com/ content/ rp3g1q05j176r108/fulltext.pdf). Astrophysics and Space Science (Springer
Netherlands) 86 (2).
• MacKeown, P. Kevin (1997). Stochastic Simulation in Physics. New York: Springer. ISBN 9813083263.
• Metropolis, N. (1987). "The beginning of the Monte Carlo method" (http:/ / library.lanl. gov/ la-pubs/ 00326866.
pdf). Los Alamos Science (1987 Special Issue dedicated to Stanisław Ulam): 125–130.
• Metropolis, Nicholas; Rosenbluth, Arianna W.; Rosenbluth, Marshall N.; Teller, Augusta H.; Teller, Edward
(1953). "Equation of State Calculations by Fast Computing Machines". Journal of Chemical Physics 21 (6): 1087.
Bibcode 1953JChPh..21.1087M. doi:10.1063/1.1699114.
• Metropolis, N.; Ulam, S. (1949). "The Monte Carlo Method". Journal of the American Statistical Association
(American Statistical Association) 44 (247): 335–341. doi:10.2307/2280232. JSTOR 2280232. PMID 18139350.
• M. Milik and J. Skolnick (Jan 1993). "Insertion of peptide chains into lipid membranes: an off-lattice Monte
Carlo dynamics model". Proteins 15 (1): 10–25. doi:10.1002/prot.340150104. PMID 8451235.
• Mosegaard, Klaus; Tarantola, Albert (1995). "Monte Carlo sampling of solutions to inverse problems". J.
Geophys. Res. 100 (B7): 12431–12447. Bibcode 1995JGR...10012431M. doi:10.1029/94JB03097.
• P. Ojeda and M. Garcia and A. Londono and N.Y. Chen (Feb 2009). "Monte Carlo Simulations of Proteins in
Cages: Influence of Confinement on the Stability of Intermediate States". Biophys. Jour. (Biophysical Society) 96
(3): 1076–1082. Bibcode 2009BpJ....96.1076O. doi:10.1529/biophysj.107.125369.
• Int Panis L; De Nocker L, De Vlieger I, Torfs R (2001). "Trends and uncertainty in air pollution impacts and
external costs of Belgian passenger car traffic International". Journal of Vehicle Design 27 (1–4): 183–194.
• Int Panis L, Rabl A, De Nocker L, Torfs R (2002). P. Sturm. ed. "Diesel or Petrol ? An environmental comparison
hampered by uncertainty". Mitteilungen Institut für Verbrennungskraftmaschinen und Thermodynamik
(Technische Universität Graz Austria) Heft 81 Vol 1: 48–54.
• Press, William H.; Teukolsky, Saul A.; Vetterling, William T.; Flannery, Brian P. (1996) [1986]. Numerical
Recipes in Fortran 77: The Art of Scientific Computing. Fortran Numerical Recipes. 1 (Second ed.). Cambridge
University Press. ISBN 0-521-43064-X.
• Ripley, B. D. (1987). Stochastic Simulation. Wiley & Sons.
• Robert, C. P.; Casella, G. (2004). Monte Carlo Statistical Methods (2nd ed.). New York: Springer.
ISBN 0387212396.
Monte Carlo method
• Rubinstein, R. Y.; Kroese, D. P. (2007). Simulation and the Mont Carlo Method (2nd ed.). New York: John Wiley
& Sons. ISBN 9780470177938.
• Savvides, Savvakis C. (1994). "Risk Analysis in Investment Appraisal". Project Appraisal Journal 9 (1).
doi:10.2139/ssrn.265905*Sawilowsky, Shlomo S.; Fahoome, Gail C. (2003). Statistics via Monte Carlo
Simulation with Fortran. Rochester Hills, MI: JMASM. ISBN 0-9740236-0-4.
• Sawilowsky, Shlomo S. (2003). "You think you've got trivials?". Journal of Modern Applied Statistical Methods
2 (1): 218–225.
• Silver, David; Veness, Joel (2010). "Monte-Carlo Planning in Large POMDPs" (http:/ / books. nips. cc/ papers/
files/nips23/ NIPS2010_0740.pdf). In Lafferty, J.; Williams, C. K. I.; Shawe-Taylor, J. et al.. Advances in
Neural Information Processing Systems 23. Neural Information Processing Systems Foundation
• Szirmay-Kalos, László (2008). Monte Carlo Methods in Global Illumination - Photo-realistic Rendering with
Randomization. VDM Verlag Dr. Mueller e.K.. ISBN 978-3836479196.
• Tarantola, Albert (2005). Inverse Problem Theory (http:// www.ipgp.jussieu. fr/~tarantola/Files/ Professional/
SIAM/ index. html). Philadelphia: Society for Industrial and Applied Mathematics. ISBN 0898715725.
• Vose, David (2000). Risk Analysis, A Quantitative Guide (Second ed.). John Wiley & Sons.
External links
• Overview of Top Excel Monte-Carlo Tools (http:// www.crystalballservices. com/ Resources/
ConsultantsCornerBlog/ EntryId/71/
Excel-Simulation-Show-Down-Comparing-the-top-Monte-Carlo-Simulation-Tools.aspx), Eric Torkia
• Overview and reference list (http:/ / mathworld.wolfram.com/ MonteCarloMethod. html), Mathworld
• Introduction to Monte Carlo Methods (http:/ / www.phy. ornl.gov/ csep/ CSEP/ MC/ MC. html), Computational
Science Education Project
• The Basics of Monte Carlo Simulations (http:// www.chem. unl.edu/ zeng/ joy/ mclab/ mcintro.html),
University of Nebraska-Lincoln
• Introduction to Monte Carlo simulation (http:// office.microsoft.com/ en-us/ excel-help/
introduction-to-monte-carlo-simulation-HA010282777.aspx) (for Microsoft Excel), Wayne L. Winston
• Monte Carlo Methods – Overview and Concept (http:/ / www.brighton-webs.co. uk/ montecarlo/concept. asp),
• Molecular Monte Carlo Intro (http:// www. cooper.edu/ engineering/chemechem/ monte. html), Cooper Union
• Monte Carlo techniques applied in physics (http:/ / www.princeton.edu/ ~achremos/ Applet1-page.htm)
• Monte Carlo Method Example (http:// waqqasfarooq.com/ waqqasfarooq/ index. php?option=com_content&
view=article& id=47:monte-carlo&catid=34:statistics& Itemid=53), A step-by-step guide to creating a monte
carlo excel spreadsheet
• Pricing using Monte Carlo simulation (http:/ / knol. google. com/ k/ giancarlo-vercellino/
pricing-using-monte-carlo-simulation/11d5i2rgd9gn5/3#), a practical example, Prof. Giancarlo Vercellino
• Approximate And Double Check Probability Problems Using Monte Carlo method (http:/ / orcik.net/
programming/ approximate-and-double-check-probability-problems-using-monte-carlo-method/) at Orcik Dot
Nothing up my sleeve number
Nothing up my sleeve number
In cryptography, nothing up my sleeve numbers are any numbers which, by their construction, are above suspicion
of hidden properties. They are used in creating cryptographic functions such as hashes and ciphers. These algorithms
often need randomized constants for mixing or initialization purposes. The cryptographer may wish to pick these
values in a way that demonstrates the constants were not selected for (in Bruce Schneier's words) a "nefarious
purpose", for example, to create a "backdoor" to the algorithm.
These fears can be allayed by using numbers
created in a way that leaves little room for adjustment. An example would be the use of initial digits from the
number π as the constants.
Using digits of π millions of places into its definition would not be considered as
trustworthy. The algorithm designer might have selected that starting point because it created a secret weakness the
designer could later exploit.
Digits in the positional representation of real numbers such as π, e and irrational roots are believed to appear
random. See normal number. Such numbers can be viewed as the opposite extreme of Chaitin–Kolmogorov random
numbers in that they appear random but have very low information entropy. Their use is motivated by early
controversy over the U.S. Government's 1975 Data Encryption Standard, which came under criticism because no
explanation was supplied for the constants used in its S-box (though they were later found to have good justification,
see Differential cryptanalysis).

Thus a need was felt for a more transparent way to generate constants used in
"Nothing up my sleeve" is a phrase associated with magicians, who sometimes preface a magic trick by holding open
their sleeves to show they have no objects hidden inside.
• The cipher Khafre, designed in 1989, includes constants from the book A Million Random Digits with 100,000
Normal Deviates, published by the RAND Corporation in 1951.
• Ron Rivest used the trigonometric sine function to generate constants for the widely-used MD5 hash.
• The U.S. National Security Agency used the square roots of small integers to produce the constants used in its
"Secure Hash Algorithm" SHA-1. The SHA-2 functions use the square roots and cube roots of small primes.
• The Blowfish encryption algorithm uses the binary representation of π to initialize its key schedule.
• RFC 3526 describes prime numbers for internet key exchange that are also generated from π.
• The S-box of the NewDES cipher is derived from the United States Declaration of Independence.
• The AES candidate DFC derives all of its arbitrary constants, including all entries of the S-box, from the binary
expansion of e.
• The ARIA key schedule uses the binary expansion of 1/π.
• The key schedule of the RC5 cipher uses binary digits from both e and the golden ratio.
• Dual EC DRBG, a NIST-recommended cryptographic random bit generator, came under criticism in 2007
because constants recommended for use in the algorithm could have been selected in a way that would permit
their author to predict future outputs given a sample of past generated values.
Nothing up my sleeve number
[1] Bruce Schneier (2007-11-15). "Did NSA Put a Secret Backdoor in New Encryption Standard?" (http:// www. wired.com/ politics/ security/
commentary/securitymatters/2007/ 11/ securitymatters_1115). Wired News. .
[2] http:/ / www. schneier. com/ paper-blowfish-fse.html
[3] Bruce Schneier. Applied Cryptography, second edition, John Wiley and Sons, 1996.
[4] http:// tvtropes.org/pmwiki/ pmwiki. php/ Main/ NothingUpMySleeve TV Tropes entry for "nothing up my sleeve"
[5] RFC 1321 Sec. 3.4
[6] FIPS 180-2: Secure Hash Standard (SHS) (http:// csrc. nist.gov/ publications/ fips/ fips180-2/fips180-2withchangenotice.pdf) (PDF, 236
kB) – Current version of the Secure Hash Standard (SHA-1, SHA-224, SHA-256, SHA-384, and SHA-512), 1 August 2002, amended 25
February 2004
[7] Revision of NEWDES, Robert Scott, 1996 (http:// groups. google.com/ group/sci. crypt/msg/ 7fb986b231fa9dc5)
[8] Henri Gilbert, M. Girault, P. Hoogvorst, F. Noilhan, T. Pornin, G. Poupard, J. Stern, S. Vaudenay (May 19, 1998) (PDF/PostScript).
Decorrelated Fast Cipher: an AES candidate (http:// citeseer. ist. psu.edu/ gilbert98decorrelated.html). .
[9] A. Biryukov, C. De Cannière, J. Lano, B. Preneel, S. B. Örs (January 7, 2004) (PostScript). Security and Performance Analysis of ARIA
(http:/ / www. cosic. esat. kuleuven. be/ publications/ article-500.ps). Version 1.2—Final Report. Katholieke Universiteit Leuven. .
[10] Rivest, R. L. (1994). "The RC5 Encryption Algorithm" (http:// theory.lcs.mit. edu/ ~rivest/ Rivest-rc5rev.pdf) (pdf). Proceedings of the
Second International Workshop on Fast Software Encryption (FSE) 1994e. pp. 86–96. .
• Bruce Schneier. Applied Cryptography, second edition. John Wiley and Sons, 1996.
• Eli Biham, Adi Shamir, (1990). Differential Cryptanalysis of DES-like Cryptosystems. Advances in Cryptology
— CRYPTO '90. Springer-Verlag. 2–21.
Philosophical interpretation of classical physics
Classical Newtonian physics has, formally, been replaced by quantum mechanics on the small scale and relativity on
the large scale. Because most humans continue to think in terms of the kind of events we perceive in the human scale
of daily life, it became necessary to provide a new philosophical interpretation of classical physics. Classical
mechanics worked extremely well within its domain of observation but made inaccurate predictions at very small
scale - atomic scale systems - and when objects moved very fast or were very massive. Viewed through the lens of
quantum mechanics or relativity, we can now see that classical physics, imported from the world of our everyday
experience, includes notions for which there is no actual evidence. For example, one commonly held idea is that
there exists one absolute time shared by all observers. Another is the idea that electrons are discrete entities like
miniature planets that circle the nucleus in definite orbits.
The correspondence principle says that classical accounts are approximations to quantum mechanics that are for all
practical purposes equivalent to quantum mechanics when dealing with macro-scale events.
Various problems occur if classical mechanics is used to describe quantum systems, such as the ultraviolet
catastrophe in black body radiation, the Gibbs paradox, and the lack of a zero point for entropy.
Since classical physics corresponds more closely to ordinary language than modern physics does, this subject is also
a part of the philosophical interpretation of ordinary language, which has other aspects, as well.
Philosophical interpretation of classical physics
The measurement process
In classical mechanics it is assumed that given properties - speed or mass of a particle; temperature of a gas, etc. -
can in principle be measured to any degree of accuracy desired.
Study of the problem of measurement in quantum mechanics has shown that measurement of any object involves
interactions between the measuring apparatus and that object that inevitably affect it in some way; at the scale of
particles this effect is necessarily large. On the everyday macroscopic scale the effect can be made small.
Furthermore, the classical idealization of a property simply being "measured" ignores the fact that measurement of a
property - temperature of a gas by thermometer, say - involves a pre-existing account of the behavior of the
measuring device. When effort was devoted to working out the operational definitions involved in precisely
determining position and momentum of micro-scale entities, physicists were required perforce to provide such an
account for measuring devices to be used at that scale. The key thought experiment in this regard is known as
Heisenberg's microscope.
The problem for the individual is how to properly characterize a part of reality of which one has no direct sense
experience. Our inquiries into the quantum domain find most pertinent whatever it is that happens in between the
events by means of which we obtain our only information. Our accounts of the quantum domain are based on
interactions of macro domain instruments and sense organs with physical events, and those interactions give us some
but not all of the information we seek. We then seek to derive further information from series of those experiments in
an indirect way.
One interpretation of this conundrum is given by Werner Heisenberg in his 1958 book, Physics and Philosophy,p.
We can say that physics is a part of science and as such aims at a description and understanding of
nature. Any kind of understanding, scientific or not, depends on our language, on the communication of
ideas. Every description of phenomena, of experiments and their results, rests upon language as the only
means of communication. The words of this language represent the concepts of daily life, which in the
scientific language of physics may be refined to the concepts of classical physics. These concepts are the
only tools for an unambiguous communication about events, about the setting up of experiments, and
about their results. If therefore the atomic physicist is asked to give a description of what really happens
in his experiments, the words "description" and "really" and "happens" can only refer to the concepts of
daily life or of classical physics. As soon as the physicist gave up this basis he would lose the means of
unambiguous communication and could not continue in his science. Therefore, any statement about
what has "actually happened" is a statement in terms of the classical concepts and -- because of
thermodynamics and of the uncertainty relations -- by its very nature incomplete with respect to the
details of the atomic events involved. The demand to "describe what happens" in the
quantum-theoretical process between two successive observations is a contradiction in adjecto, since the
word "describe" refers to the use of the classical concepts, while these concepts cannot be applied in the
space between the observations; they can only be applied at the points of observation.
Primacy of observation in quantum mechanics and special relativity
Both quantum mechanics and special relativity begin their divergence from classical mechanics by insisting on the
primacy of observations and a refusal to admit unobservable entities. Thus special relativity rejects the absolute
simultaneity assumed by classical mechanics; and quantum mechanics does not permit one to speak of properties of
the system (exact position, say) other than those that can be connected to macro scale observations. Position and
momentum are not things waiting for us to discover; rather, they are the results that are obtained by performing
certain procedures.
Philosophical interpretation of classical physics
1. Messiah, Albert, Quantum Mechanics, volume I, pp. 45-50.
• Albert Messiah, Quantum Mechanics, English translation by G. M. Temmer of Mécanique Quantique, 1966, John
Wiley and Sons
• A lecture to his statistical mechanics class at the University of California at Santa Barbara by Dr. Herbert P.
Broida [1] (1920-1978)
• "Physics and the Real World" by George F. R. Ellis, Physics Today, July, 2005
External links
• Bohmian Mechanics website
[1] http:/ / sunsite. berkeley. edu/ uchistory/ archives_exhibits/ in_memoriam/ catalog/ broida_herbert.html
[2] http:// www. bohmian-mechanics. net
Physical Unclonable Function
In practical cryptography, a Physical Unclonable Function or PUF is a function that is embodied in a physical
structure and is easy to evaluate but hard to predict. Further, an individual PUF device must be easy to make but
practically impossible to duplicate, even given the exact manufacturing process that produced it. In this respect it is
the hardware analog of a one-way function. Early references that exploit the physical properties of disordered
systems for authentication purposes date back to Bauder in 1983
and Simmons in 1984

. Naccache and
Frémanteau provided an authentication scheme in 1992 for memory cards
. The terms POWF (Physical One-Way
Function) and PUF (Physical Unclonable Function) were coined in 2001
and 2002
, the latter publication
describing the first integrated PUF where unlike PUFs based on optics, the measurement circuitry and the PUF are
integrated onto the same electrical circuit (and fabricated on silicon).
Rather than embodying a single cryptographic key, PUFs implement challenge-response authentication. When a
physical stimulus is applied to the structure, it reacts in an unpredictable way due to the complex interaction of the
stimulus with the physical microstructure of the device. This exact microstructure depends on physical factors
introduced during manufacture which are unpredictable (like a Fair coin). The applied stimulus is called the
challenge, and the reaction of the PUF is called the response. A specific challenge and its corresponding response
together form a challenge-response pair or CRP. The device's identity is established by the properties of the
microstructure itself. As this structure is not directly revealed by the challenge-response mechanism such a device is
resistant to spoofing attacks.
PUFs can be implemented with a very small hardware investment. Unlike a ROM containing a table of responses to
all possible challenges, which would require hardware exponential in the number of challenge bits, a PUF can be
constructed in hardware proportional to the number of challenge and response bits.
Unclonability means that each PUF device has a unique and unpredictable way of mapping challenges to responses,
even if it was manufactured with the same process as a similar device, and it is infeasible to construct a PUF with the
same challenge-response behavior as another given PUF because exact control over the manufacturing process is
infeasible. Mathematical unclonability means that it should be very hard to compute an unknown response given the
other CRPs or some of the properties of the random components from a PUF. This is because a response is created
Physical Unclonable Function
by a complex interaction of the challenge with many or all of the random components. In other words, given the
design of the PUF system, without knowing all of the physical properties of the random components, the CRPs are
highly unpredictable. The combination of physical and mathematical unclonability renders a PUF truly unclonable.
Different sources of physical randomness can be used in PUFs. A distinction is made between PUFs in which
physical randomness is explicitly introduced and PUFs that use randomness that is intrinsically present in a physical
Types of PUFs
All PUFs are subject to environmental variations such as temperature, supply voltage and Electromagnetic
interference, which can affect their performance. Therefore, rather than just being random, the real power of a PUF is
its ability to be different between devices, but simultaneously to be the same under different environmental
PUFs using explicitly-introduced randomness
This type of PUF can have a much greater ability to distinguish devices from one another and have minimal
environmental variations compared to PUFs that utilize intrinsic randomness. This is due to the use of different
underlying principles and the ability for parameters to be directly controlled and optimized.
Optical PUF
An optical PUF which was termed POWF

consists of a transparent material that is doped with light scattering
particles. When a laser beam shines on the material, a random and unique speckle pattern will arise. The placement
of the light scattering particles is an uncontrolled process and the interaction between the laser and the particles is
very complex. Therefore, it is very hard to duplicate the optical PUF such that the same speckle pattern will arise.
We say the optical PUF is practically unclonable.
Coating PUF
A coating PUF


can be built in the top layer of an IC. Above a normal IC, a network of metal wires is laid
out in a comb shape. The space between and above the comb structure is filled with an opaque material and
randomly doped with dielectric particles. Because of the random placement, size and dielectric strength of the
particles, the capacitance between each couple of metal wires will be random up to a certain extent. This unique
randomness can be used to obtain a unique identifier for the device carrying the Coating PUF. Moreover, the
placement of this opaque PUF in the top layer of an IC protects the underlying circuits from being inspected by an
attacker, e.g. for reverse-engineering. When an attacker tries to remove (a part of) the coating, the capacitance
between the wires is bound to change and the original unique identifier will be destroyed. In
it was shown how an
unclonable RFID Tag is built with Coating PUFs.
PUFs using intrinsic randomness
Unlike PUFs that utilize explicitly-introduced randomness, PUFs using intrinsic randomness are highly attractive
because they can be included in a design without modifications to the manufacturing process.
Silicon PUF
A silicon PUF exploits the random variations in delays of wires and gates. Given an input challenge, a race condition
is set up in the circuit, and two transitions that propagate along different paths are compared to see which comes
first. An arbiter, typically implemented as a latch, produces a 1 or a 0, depending on which transition comes first.
Many circuits realizations are possible and at least two have been fabricated. When a circuit with the same layout
mask is fabricated on different chips, the logic function implemented by the circuit is different for each chip due to
Physical Unclonable Function
the random variations of delays.
A PUF based on a delay loop, i.e., a ring oscillator with logic, is described in
. This was the publication that
introduced the PUF acronym and the first integrated PUF of any type. A multiplexor-based PUF is described in
A secure processor design using a PUF is described in
. A multiplexor-based PUF with an RF interface for use in
RFID anti-counterfeiting applications is described in
These PUFs are present in all ICs having SRAM memory on board. Their behavior and application for
anti-counterfeiting purposes were investigated in detail in
and in
On top of this they permit the implementation of secure secret key storage without storing the key in digital form.
An example would be an RFID tag, which can easily be cloned. When equipped with a PUF however, creating a
clone in a reasonable timeframe can be next to impossible.
Butterfly PUF
Recently a new PUF was introduced
: the Butterfly PUF. The Butterfly PUF is based on cross-coupling of two
latches or flip-flops. The mechanism being this PUF is similar to the one behind the SRAM PUF but has the
advantage that it can be implemented on any SRAM FPGA.
Magnetic PUF
A magnetic PUF exists on a magnetic stripe card. The physical structure of the magnetic media applied to a card is
fabricated by blending billions of particles of barium ferrite together in a slurry during the manufacturing process.
The particles have many different shapes and sizes. The slurry is applied to a receptor layer. The particles land in a
random fashion, much like pouring a handful of wet magnetic sand onto a carrier. To pour the sand to land in exactly
the same pattern a second time is physically impossible due to the inexactness of the process, the sheer number of
particles, and the random geometry of their shape and size. The randomness introduced during the manufacturing
process cannot be controlled. This is a classic example of a PUF using intrinsic randomness.
When the slurry dries, the receptor layer is sliced into strips and applied to plastic cards, but the random pattern on
the magnetic stripe remains and cannot be changed. Because of their physically unclonable functions, it is highly
improbable that two magnetic stripe cards will ever be identical. In fact, using a standard size card, the odds of any
two cards having an exact matching magnetic PUF are calculated to be 1 in 900 million. Further, because the PUF is
magnetic, we know that each card will carry a distinctive, repeatable and readable magnetic signal.
Personalizing the PUF
The personal data encoded on the magnetic stripe contributes another layer of randomness. When the card is encoded
with personal identifying information, the odds of two encoded magstripe cards having an identical magnetic
signature are approximately 1 in 10 Billion. The encoded data can be used as a marker to locate significant elements
of the PUF. This signature can be digitized and is generally called a magnetic fingerprint. An example of its use is in
the Magneprint brand system.


Stimulating the PUF
The magnetic head acts as a stimulus on the PUF and amplifies the random magnetic signal. Because of the complex
interaction of the magnetic head, influenced by speed, pressure, direction and acceleration, with the random
components of the PUF, each swipe of the head over the magnetic PUF will yield a stochastic, but very distinctive
signal. Think of it as a song with thousands of notes. The odds of the same notes recurring in an exact pattern from a
single card swiped many times are 1 in 100 million, but overall the melody remains very recognizable.
Physical Unclonable Function
Uses for a Magnetic PUF
The stochastic behavior of the PUF in concert with the stimulus of the head makes the magnetic stripe card an
excellent tool for Dynamic Token Authentication, Forensic Identification, Key generation, One-Time Passwords,
and Digital Signatures.
[1] D.W. Bauder, "An anti-counterfeiting concept for currency systems", Research report PTK-11990. Sandia National Labs. Albuquerque, NM,
[2] G. Simmons, “A system for verifying user identity and authorization at the point-of sale or access,” Cryptologia, vol. 8, no. 1, pp. 1–21, 1984.
[3] G. Simmons, “Identification of data, devices, documents and individuals,” in IEEE International Carnahan Conference on Security
Technology, 1991, pp. 197–218.
[4] David Naccache and Patrice Frémanteau, Unforgeable identification device, identification device reader and method of identification, August
1992. (http:// v3. espacenet. com/ publicationDetails/ biblio?DB=EPODOC& adjacent=true&locale=en_gb& FT=D&date=19940223&
CC=EP&NR=0583709A1& KC=A1)
[5] R. Pappu, B. Recht, J. Taylor, and N. Gershenfeld. Physical One-Way functions. Science, 297(5589):2026–2030, Sep 2002. http:// dx. doi.
org/ 10. 1126/ science. 1074376.
[6] B. Gassend, D. Clarke, M. van Dijk and S. Devadas. Silicon Physical Random Functions. Proceedings of the Computer and Communications
Security Conference, November 2002
[7] R. Pappu, "Physical One-Way Functions", PhD Thesis, MIT, 2001. Physical One-Way Functions (http:/ / alumni. media.mit. edu/ ~pappu/
pdfs/ Pappu-PhD-POWF-2001.pdf).
[8] R. Pappu, B. Recht, J. Taylor, and N. Gershenfeld. Physical One-Way functions. Science, 297(5589):2026–2030, Sep 2002. http:// dx. doi.
org/ 10. 1126/ science. 1074376.
[9] B. Skoric, S. Maubach, T. Kevenaar, and P. Tuyls. Information-theoretic analysis of capacitive physical unclonable functions. J. Appl. Phys.,
100(2):024902, Jul 2006. http:// dx. doi.org/ 10. 1063/ 1. 2209532
[10] B. Skoric, G.-J. Schrijen, W. Ophey, R. Wolters, N. Verhaegh, and J. van Geloven. Experimental hardware for coating PUFs and optical
PUFs. In P. Tuyls, B. Skoric, and T. Kevenaar, editors, Security with Noisy Data - On Private Biometrics, Secure Key Storage and
Anti-Counterfeiting, pages 255-268. Springer London, 2008. http:// dx.doi. org/10.1007/ 978-1-84628-984-2_15
[11] Pim Tuyls, Geert-Jan Schrijen, Boris Skoric, Jan van Geloven, Nynke Verhaegh and Rob Wolters: "Read-proof hardware from protective
coatings", CHES 2006, p 369- 383.
[12] Pim Tuyls, Lejla Batina RFID-Tags for Anti-counterfeiting. CT-RSA, 2006, pp:115-131
[13] B. Gassend, D. Clarke, M. van Dijk and S. Devadas. Silicon Physical Random Functions. Proceedings of the Computer and
Communications Security Conference, November 2002
[14] D. Lim, J-W. Lee, B. Gassend, M. van Dijk, E. Suh, and S. Devadas. Extracting Secret Keys from Integrated Circuits, IEEE Transactions on
VLSI Systems, volume 13, Number 10, pages 1200–1205, October 2005
[15] G. E. Suh, C. W. O'Donnell, and S. Devadas. Aegis: A Single-Chip secure processor. IEEE Design and Test of Computers, 24(6):570-580,
Nov 2007. http:/ / dx. doi. org/10. 1109/ MDT. 2007. 179
[16] S. Devadas, V. Khandelwal, S. Paral, R. Sowell, E. Suh, T. Ziola, Design and Implementation of `Unclonable' RFID ICs for
Anti-Counterfeiting and Security Applications, RFID World 2008, March 2008
[17] Jorge Guajardo, Sandeep S. Kumar, Geert-Jan Schrijen, Pim Tuyls, “FPGA Intrinsic PUFs and Their Use for IP Protection”, Workshop on
Cryptographic Hardware and Embedded Systems (CHES), Sep 10-13, 2007, Vienne, Austria
[18] Jorge Guajardo, Sandeep S. Kumar, Geert-Jan Schrijen, Pim Tuyls, “Physical Unclonable Functions and Public-Key Crypto for FPGA IP
Protection”, International Conference on Field Programmable Logic and Applications (FPL), Aug 27-29, 2007, Amsterdam, The Netherlands.
[19] Christoph Böhm, Maximilian Hofer, “Using SRAMs as Physical Unclonable Functions”, Austrochip - Workshop on Microelectronics, Oct 7,
2009, Graz, Austria.
[20] L. Bolotnyy and G. Robins. Physically unclonable Function-Based security and privacy in RFID systems. In 5th IEEE Int. Conf. on
Pervasive Computing and Communications (PERCOM), pages 211-220, Washington, DC, USA, 2007. IEEE Computer Society. http:// dx.
doi. org/10.1109/ PERCOM. 2007. 26
[21] S. Kumar, J. Guajardo, R. Maes, G.J. Schrijen qnd P. Tuyls, The Buttefly PUF: Protecting IP on every FPGA, In IEEE International
Workshop on Hardware Oriented Security and Trust, Anaheim 2008. (http:/ / www.cosic.esat. kuleuven. be/ publications/ article-1154.pdf)
[22] http:// www. aip. org/dbis/ stories/ 2005/14405. html
[23] Tony Fitzpatrick, Nov. 11, 2004, "Magneprint technology licensed to TRAX Systems, Inc." http:// news-info.wustl. edu/ tips/ page/
normal/ 4159. html
[24] Patrick L. Thimangu,January 7, 2005, "Washington U. cashing in with MagnePrint licensing," St. Louis Business Journal http:// www.
bizjournals. com/ stlouis/ stories/ 2005/ 01/ 10/ story7. html?jst=s_cn_hl
• http:/ / people. csail. mit. edu/ rudolph/Teaching/Lectures/ Security/Lecture-Security-PUFs-2.pdf
• "Ultra-low-cost true randomness AND physical fingerprinting" (http:/ / tshb. livejournal.com/ 2989. html)
Random binary tree
Random binary tree
In computer science and probability theory, a random binary tree refers to a binary tree selected at random from
some probability distribution on binary trees. Two different distributions are commonly used: binary trees formed by
inserting nodes one at a time according to a random permutation, and binary trees chosen from a uniform discrete
distribution in which all distinct trees are equally likely. It is also possible to form other distributions, for instance by
repeated splitting. Adding and removing nodes directly in a random binary tree will in general disrupt its random
structure, but the treap and related randomized binary search tree data structures use the principle of binary trees
formed from a random permutation in order to maintain a balanced binary search tree dynamically as nodes are
inserted and deleted.
For random trees that are not necessarily binary, see random tree.
Binary trees from random permutations
For any set of numbers (or, more generally, values from some total order), one may form a binary search tree in
which each number is inserted in sequence as a leaf of the tree, without changing the structure of the previously
inserted numbers. The position into which each number should be inserted is uniquely determined by a binary search
in the tree formed by the previous numbers. For instance, if the three numbers (1,3,2) are inserted into a tree in that
sequence, the number 1 will sit at the root of the tree, the number 3 will be placed as its right child, and the number 2
as the left child of the number 3. There are six different permutations of the numbers (1,2,3), but only five trees may
be constructed from them. That is because the permutations (2,1,3) and (2,3,1) form the same tree.
Expected depth of a node
For any fixed choice of a value x in the given set of numbers, if one randomly permutes the numbers and forms a
binary tree from them as described above, the expected value of the length of the path from the root of the tree to x is
at most 2 log x + O(1), where "log" denotes the natural logarithm function and the O introduces big O notation. For,
the expected number of ancestors of x is by linearity of expectation equal to the sum, over all other values y in the
set, of the probability that y is an ancestor of x. And a value y is an ancestor of x exactly when y is the first element
to be inserted from the elements in the interval [x,y]. Thus, the values that are adjacent to x in the sorted sequence of
values have probability 1/2 of being an ancestor of x, the values one step away have probability 1/3, etc. Adding
these probabilities for all positions in the sorted sequence gives twice a Harmonic number, leading to the bound
above. A bound of this form holds also for the expected search length of a path to a fixed value x that is not part of
the given set.
The longest path
Although not as easy to analyze as the average path length, there has also been much research on determining the
expectation (or high probability bounds) of the length of the longest path in a binary search tree generated from a
random insertion order. It is now known that this length, for a tree with n nodes, is almost surely
where β is the unique number in the range 0 < β < 1 satisfying the equation
Random binary tree
Expected number of leaves
In the random permutation model, each of the numbers from the set of numbers used to form the tree, except for the
smallest and largest of the numbers, has probability 1/3 of being a leaf in the tree, for it is a leaf when it inserted after
its two neighbors, and any of the six permutations of these two neighbors and it are equally likely. By similar
reasoning, the smallest and largest of the numbers have probability 1/2 of being a leaf. Therefore, the expected
number of leaves is the sum of these probabilities, which for n ≥ 2 is exactly (n + 1)/3.
Treaps and randomized binary search trees
In applications of binary search tree data structures, it is rare for the values in the tree to be inserted without deletion
in a random order, limiting the direct applications of random binary trees. However, algorithm designers have
devised data structures that allow insertions and deletions to be performed in a binary search tree, at each step
maintaining as an invariant the property that the shape of the tree is a random variable with the same distribution as a
random binary search tree.
If a given set of ordered numbers is assigned numeric priorities (distinct numbers unrelated to their values), these
priorities may be used to construct a Cartesian tree for the numbers, a binary tree that has as its inorder traversal
sequence the sorted sequence of the numbers and that is heap-ordered by priorities. Although more efficient
construction algorithms are known, it is helpful to think of a Cartesian tree as being constructed by inserting the
given numbers into a binary search tree in priority order. Thus, by choosing the priorities either to be a set of
independent random real numbers in the unit interval, or by choosing them to be a random permutation of the
numbers from 1 to n (where n is the number of nodes in the tree), and by maintaining the heap ordering property
using tree rotations after any insertion or deletion of a node, it is possible to maintain a data structure that behaves
like a random binary search tree. Such a data structure is known as a treap or a randomized binary search tree.
Uniformly random binary trees
The number of binary trees with n nodes is a Catalan number: for n = 1, 2, 3, ... these numbers of trees are
1, 2, 5, 14, 42, 132, 429, 1430, 4862, 16796, … (sequence A000108
in OEIS).
Thus, if one of these trees is selected uniformly at random, its probability is the reciprocal of a Catalan number.
Trees in this model have expected depth proportional to the square root of n, rather than to the logarithm;
the Strahler number of a uniformly random binary tree, a more sensitive measure of the distance from a leaf in which
a node has Strahler number i whenever it has either a child with that number or two children with number i − 1, is
with high probability logarithmic.
Due to their large heights, this model of equiprobable random trees is not generally used for binary search trees, but
it has been applied to problems of modeling the parse trees of algebraic expressions in compiler design
(where the
above-mentioned bound on Strahler number translates into the number of registers needed to evaluate an
) and for modeling evolutionary trees.
In some cases the analysis of random binary trees under the
random permutation model can be automatically transferred to the uniform model.
Random split trees
Devroye & Kruszewski (1996) generate random binary trees with n nodes by generating a real-valued random
variable x in the unit interval (0,1), assigning the first xn nodes (rounded down to an integer number of nodes) to the
left subtree, the next node to the root, and the remaining nodes to the right subtree, and continuing recursively in
each subtree. If x is chosen uniformly at random in the interval, the result is the same as the random binary search
tree generated by a random permutation of the nodes, as any node is equally likely to be chosen as root; however,
this formulation allows other distributions to be used instead. For instance, in the uniformly random binary tree
model, once a root is fixed each of its two subtrees must also be uniformly random, so the uniformly random model
Random binary tree
may also be generated by a different choice of distribution for x. As Devroye and Kruszewski show, by choosing a
beta distribution on x and by using an appropriate choice of shape to draw each of the branches, the mathematical
trees generated by this process can be used to create realistic-looking botanical trees.
[1] Hibbard (1962); Knuth (1973); Mahmoud (1992), p. 75.
[2] Robson (1979); Pittel (1985); Devroye (1986); Mahmoud (1992), pp. 91–99; Reed (2003).
[3] Martinez & Roura (1992); Seidel & Aragon (1996).
[4] http:/ / en.wikipedia. org/wiki/ Oeis%3Aa000108
[5] Knuth (2005), p. 15.
[6] Devroye & Kruszewski (1995). That it is at most logarithmic is trivial, because the Strahler number of every tree is bounded by the logarithm
of the number of its nodes.
[7] Mahmoud (1992), p. 63.
[8] Flajolet, Raoult & Vuillemin (1979).
[9] Aldous (1996).
[10] Mahmoud (1992), p. 70.
• Aldous, David (1996), "Probability distributions on cladograms", in Aldous, David; Pemantle, Robin, Random
Discrete Structures, The IMA Volumes in Mathematics and its Applications, 76, Springer-Verlag, pp. 1–18.
• Devroye, Luc (1986), "A note on the height of binary search trees", Journal of the ACM 33 (3): 489–498,
• Devroye, Luc; Kruszewski, Paul (1995), "A note on the Horton-Strahler number for random trees", Information
Processing Letters 56 (2): 95–99, doi:10.1016/0020-0190(95)00114-R.
• Devroye, Luc; Kruszewski, Paul (1996), "The botanical beauty of random binary trees", in Brandenburg, Franz J.,
Graph Drawing: 3rd Int. Symp., GD'95, Passau, Germany, September 20-22, 1995, Lecture Notes in Computer
Science, 1027, Springer-Verlag, pp. 166–177, doi:10.1007/BFb0021801.
• Drmota, Michael (2009), Random Trees : An Interplay between Combinatorics and Probability, Springer-Verlag,
ISBN 9783211753552.
• Flajolet, P.; Raoult, J. C.; Vuillemin, J. (1979), "The number of registers required for evaluating arithmetic
expressions", Theoretical Computer Science 9 (1): 99–125, doi:10.1016/0304-3975(79)90009-4.
• Hibbard, T. (1962), "Some combinatorial properties of certain trees with applications to searching and sorting",
Journal of the ACM 9 (1): 13–28, doi:10.1145/321105.321108.
• Knuth, Donald M. (1973), "6.2.2 Binary Tree Searching", The Art of Computer Programming, III,
Addison-Wesley, pp. 422–451.
• Knuth, Donald M. (2005), "Draft of Section Generating All Trees" (http:// www-cs-faculty.stanford.
edu/ ~knuth/ fasc4a. ps. gz), The Art of Computer Programming, IV.
• Mahmoud, Hosam M. (1992), Evolution of Random Search Trees, John Wiley & Sons.
• Martinez, Conrado; Roura, Salvador (1998), "Randomized binary search trees" (http:// citeseer. ist. psu. edu/
article/martinez97randomized.html), Journal of the ACM (ACM Press) 45 (2): 288–323,
• Pittel, B. (1985), "Asymptotical growth of a class of random trees", Annals of Probability 13 (2): 414–427,
• Reed, Bruce (2003), "The height of a random binary search tree", Journal of the ACM 50 (3): 306–332,
• Robson, J. M. (1979), "The height of binary search trees", Australian Computer Journal 11: 151–153.
• Seidel, Raimund; Aragon, Cecilia R. (1996), "Randomized Search Trees" (http:/ / citeseer. ist. psu. edu/
seidel96randomized.html), Algorithmica 16 (4/5): 464–497, doi:10.1007/s004539900061.
Random compact set
Random compact set
In mathematics, a random compact set is essentially a compact set-valued random variable. Random compact sets
are useful in the study of attractors for random dynamical systems.
Let be a complete separable metric space. Let denote the set of all compact subsets of . The
Hausdorff metric on is defined by
is also а complete separable metric space. The corresponding open subsets generate a σ-algebra on , the
Borel sigma algebra of .
A random compact set is а measurable function from а probability space into .
Put another way, a random compact set is a measurable function such that is almost surely
compact and
is a measurable function for every .
Random compact sets in this sense are also random closed sets as in Matheron (1975). Consequently their
distribution is given by the probabilities
(The distribution of а random compact convex set is also given by the system of all inclusion probabilities
For , the probability is obtained, which satisfies
Thus the covering function is given by
Of course, can also be interpreted as the mean of the indicator function
The covering function takes values between and . The set of all with is called the
support of . The set , of all with is called the kernel, the set of fixed points, or
essential minimum . If , is а sequence of i.i.d. random compact sets, then almost surely
and converges almost surely to
Random compact set
• Matheron, G. (1975) Random Sets and Integral Geometry. J.Wiley & Sons, New York.
• Molchanov, I. (2005) The Theory of Random Sets. Springer, New York.
• Stoyan D., and H.Stoyan (1994) Fractals, Random Shapes and Point Fields. John Wiley & Sons, Chichester,
New York.
Random number generation
A random number generator (often abbreviated as RNG) is a computational or physical device designed to
generate a sequence of numbers or symbols that lack any pattern, i.e. appear random.
The many applications of randomness have led to the development of several different methods for generating
random data. Many of these have existed since ancient times, including dice, coin flipping, the shuffling of playing
cards, the use of yarrow stalks (by divination) in the I Ching, and many other techniques. Because of the mechanical
nature of these techniques, generating large amounts of sufficiently random numbers (important in statistics)
required a lot of work and/or time. Thus, results would sometimes be collected and distributed as random number
tables. Nowadays, after the advent of computational random number generators, a growing number of
government-run lotteries, and lottery games, are using RNGs instead of more traditional drawing methods. RNGs are
also used today to determine the odds of modern slot machines.
Several computational methods for random number generation exist, but often fall short of the goal of true
randomness — though they may meet, with varying success, some of the statistical tests for randomness intended to
measure how unpredictable their results are (that is, to what degree their patterns are discernible).
Practical applications and uses
Random number generators have applications in gambling, statistical sampling, computer simulation, cryptography,
completely randomized design, and other areas where producing an unpredictable result is desirable.
Note that, in general, wher