114 views

Uploaded by ltsn

- NetSuite Data Sheet
- Jason Brownlee - Clever Algorithms
- Learning C# by Developing Games with Unity 5.x - Second Edition - Sample Chapter
- Cpp Tutorial
- roboti v
- Programming Concepts in C++ _Version 3
- 40443498 Programming in C
- Bca
- Wrapper Class Example
- Working With Classes and Properties for Game Development in VB
- CS101x S443 Constructor and Destructor Functions IIT Bombay
- C++_Coding_Style
- 5-OOP2
- lklklklkl
- Chapter 3 Lesson 2 Functions.docx
- defensesm pseudocode
- Operation Manual T-EM 1-2 en 3
- Functions S10
- chapter 3- functions sir shahzad
- UNIT II

You are on page 1of 880

Contents I

1 Introduction

2 Foundations

5 Orderings

6 Combining concepts

Contents II

8 Permutations and rearrangements

9 Rotations

11 Coordinate structures

12 Composite objects

14 Mathematical notation

Contents III

15 C++ machinery

16 Acknowledgments

17 Index

Approach

Mathematics provides tools for

reasoning

deriving

composing

optimizing

Mathematics applied to programming addresses

security

reliability

performance

productivity

Blending programming and mathematics

2 Define mathematical requirements on which it depends

3 Implement it abstractly and efficiently

4 Place it in the general taxonomy of requirements and algorithms

Intended audience

programming and are willing to invest substantial effort working

through the material

The book is not encyclopedic and is designed to be read

sequentially, in its entirety1

While this book does not have formal prerequisites, programming

maturity and a readiness to deal with serious mathematics are

assumed

We present a unified approach to programming spanning from the

highest level of abstraction down to the machine

1

Perhaps more than once

Stepanov, McJones Elements of Programming March 14, 2008 7 / 880

The choice of C++

teaching language

Programs in this book deal with both memory addresses and

abstract mathematical theories

C defines an abstract machine that serves as a foundation for

modern processor architectures2

C++ extends C with mechanisms for writing abstract versions of

algorithms and data structures without sacrificing efficiency

2

C does not address the issues of memory hierarchies, instruction-level parallelism,

and thread-level parallelism; research is needed to incorporate these features into the

system programming language of the future

Stepanov, McJones Elements of Programming March 14, 2008 8 / 880

The relationship to C++ STL

The book and C++ STL are results of the same research

The book departs from it significantly by providing improved

interfaces in a number of places

The book does not attempt to provide a full set of library

interfaces or deal with all the issues a real library must address

The machine model

Sequential (single-threaded) stacked-based execution

A linear address space, not necessarily all accessible

Memory allocation and deallocation under programmer control

Our programs and techniques can be used as building blocks for

systems with concurrent execution and automatic storage

deallocation

C++ usage conventions

with respect to concepts 3

We do not use postincrement ++ nor do we depend on the return

value of preincrement ++

We omit inline since its use depends on compiler, target

platform, and the code

3

Therefore all the constructors defined in the book should be considered to be

explicit

Stepanov, McJones Elements of Programming March 14, 2008 11 / 880

Our use of mathematics

settings for the programs in the book

We strive for the level of rigor typical in modern mathematical

textbooks, while attempting to preserve mathematical and

algorithmic intuitions

All the lemmas in this book can be proved via straightforward

derivations from the provided definitions

We urge the reader to prove every lemma to ensure true

comprehension

Outline of the book

Chapter 1 - Foundations

Chapter 2 - Transformations and their orbits

Chapter 3 - Algorithms on algebraic structures

Chapter 4 - Orderings

Chapter 5 - Combining concepts

Part II: Algorithms on abstractions of memory

Chapter 6 - Refining concepts of iterators

Chapter 7 - Permutations and rearrangements

Chapter 8 - Rotations

Chapter 9 - Algorithms on increasing ranges

Chapter 10 - Coordinate structures

Chapter 11 - Composite objects

Chapter 12 - Iterative algorithms for divide-and-conquer

Appendix 1 - Mathematical notation

Appendix 2 - C++ machinery

Contents I

1 Introduction

2 Foundations

Meaning

Objects

Relations on objects

Procedures

Assignment, destruction, and construction

Relationships between types

Concept Regular

Procedures as objects

Generalized procedures

Parameter passing

Regular functions

Contents II

Complexity

5 Orderings

6 Combining concepts

Contents III

9 Rotations

11 Coordinate structures

12 Composite objects

14 Mathematical notation

15 C++ machinery

Contents IV

16 Acknowledgments

17 Index

Things, ideas, and descriptions

We observe things: our minds form ideas (mental images) of

them; we act: our ideas affect things or create new things

We create descriptions of our ideas to preserve them,

communicate them, or allow them to participate in manual and

automatic processes

Note that a description is itself a thing

Universe of ideas

or 2 or a geometrical point

An entity is an idea of a thing such as a car or an email message

An entity has attributes with values, such as job level or salary of an

employee

An entity can be composed of other entities called parts, such as the

engine of a car or the body of an email message

An entity can be in relationships with other entities, such as the

supervisor of an employee

An activity is an idea of changes performed on entities that

changes their attributes or creates new entities, such as promoting

a given employee

An abstraction is what is common to a class of entities

An abstraction of values, such as Integer

An abstraction of entities, such as Employee

An abstraction of activities, such as Promotion

Stepanov, McJones Elements of Programming March 14, 2008 19 / 880

Entities

Its existence

Its nature

Activities affecting it

Its particulars: attributes, parts, and relationships

In computer programs

Existence corresponds to object identity

Nature corresponds to object type

Activities correspond to procedures

Particulars correspond to object state, object components, and

object relationships

Values

belong to different sorts

The universality of mathematics

which investigates the means of measuring quantity” 4

Mathematical reasoning, including discrete arithmetic reasoning

and continuous spatial reasoning, is a fundamental, innate ability

of the human mind

4

Leonard Euler. Elements of Algebra. Third Edition, Longman, London, 1822.

Stepanov, McJones Elements of Programming March 14, 2008 22 / 880

Continuity and discreteness

space can’t be represented using discrete quantities

The arithmetization of geometry required the introduction of the

real numbers

The continuity of physical processes allows us to effectively

approximate real numbers with discrete, finite-precision quantities

Scientists and engineers have encountered no problem where

careful approximation has failed to work

This is the key to the applicability of digital computers

Computer descriptions of entities

Computer programmers map these entities into objects containing

discrete, bounded quantities approximating the values of the

attributes

The meaning of a program is in the mind, not in the computer

Objects

Definition

An object is a set of memory locations and other resources together

with primitive procedures defined on it that provide its

computational basis

The state of an object is the addresses of its memory locations

together with the content of those locations at a particular point in

time

The interpretation of an object is a mapping from its state to an

entity

An interpretation maps states in such a way that procedures on the

object are consistent with the corresponding activities on entities

Well-formed object states

Definition

An object is in a well-formed state if there is an entity corresponding to it

Examples

Every state of a computer word is well-formed when interpreted

as an integer

A pointer containing an illegal address is not well-formed

An IEEE 754 floating point NaN (Not a Number) is not

well-formed when interpreted as a real number

Partial and total representations

Definition

A representation for an entity of a given nature is an object state

corresponding to it; the entity is representable if such a state exists

Definition

A representation is partial if not every entity is representable by an

object state; a representation is total if it is not partial

Example

The type int is a partial representation of integers

Unique representations

Definition

A representation is unique if there is at most one object state

corresponding to every set of particulars of the entity

Examples

A representation of a truth value as a byte that interprets zero as

false and nonzero as true is not unique

Signed-magnitude representation does not provide a unique

representation of zero

Identity

often slippery to deal with in specific situations

It was a major astronomical discovery that the Morning Star and

Evening Star are identical

The question of whether the author of Hamlet and William

Shakespeare identical is still debated

When does replacing parts change the identity?

A computer programmer must determine how to map the identity

of real things to objects

Multiple techniques are possible for representing object identity

A memory address

An index into a data structure

A unique value of an attribute (e.g., Social Security Number)

Equality

Leibniz’s law

x = y if, and only if, everything that may be said about any one of the

things x or y may also be said about the other a

a

Alfred Tarski, Introduction to Logic and to the Methodology of Science, Oxford, 1965

Properties of equality

x = y ⇔ ∀P(P(x) ⇔ P(y))

reflexivity x = x

symmetry (x = y) ⇔ (y = x)

transitivity ((x = y) ∧ (y = z)) ⇒ (x = z)

Equality in mathematics

2

Arithmetic: 4 = 21

Geometry: 4ABC = 4ACB

Algebra: (a + b)2 = a2 + 2ab + b2

Equality in the real world

Not all attributes are necessarily essential

Essentialness of attributes often depends on context

All US quarters are equal to a fare collector, but not to a coin

collector

Equality is not the same as identity

A thing’s attributes can change without affecting its identity

Distinct things can have equal attributes and, therefore, be equal

themselves

Equality in programming

mutable software objects

Leibniz’s law is the fundamental property of equality, but is not a

constructive definition

It establishes the consequences of the equality of two integers

It does not describe how to compare them for equality

Developing constructive implementations of equality is one of the

tasks of this book

Genus

Definition

Two objects are of the same genus if they represent entities with the

same nature under their corresponding interpretations

Examples

Polar and Cartesian representations of a complex number are of

the same genus

Two’s-complement and signed-magnitude representations of

integers are of the same genus

16-bit and 32-bit unsigned representations of natural numbers are

of the same genus

Type

Definition

Two objects have the same type if they share the same procedures and

the same interpretation

Intuition for type

The English word type derives from the Greek word túpoc (tupos):

“the impress made by the blow, what is formed, what leaves its

impress, the form-giving form, hence form generally as outline”5

The technical sense of tupos was “. . . the pattern in conformity to

which a thing must be made” 6

In programming, type is a fundamental idea preceding any notion

of type in programming languages

A type is a design for objects, as an engineering drawing is a

design for manufactured objects

5

Kittel and Friedrich. Theological Dictionary of the New Testament, Volume 8,

Eerdmans, 1972, page 246

6

Thayer and Smith. “Greek Lexicon entry for Tupos”. “The New Testament Greek

Lexicon”

Stepanov, McJones Elements of Programming March 14, 2008 37 / 880

Equality of objects

Definition

Two objects of the same genus are equal if their states represent entities

with equal essential particulars

however, allows us to apply the mathematical understanding of

equality on abstract values to the world of programming

Nonessential objects

Definition

A nonessential object is one whose value does not affect the

computation

Example

An object used only to instrument or tune the performance is

nonessential

How procedures interact with objects

four kinds of objects:

Arguments are objects given to a procedure at its point of invocation

Local state is objects created and accessed during a single invocation

of the procedure

Global state is objects accessible to this and other procedures across

multiple invocations

Own state is objects accessible only to this procedure but shared

across multiple invocations

Domain, argument types, and definition space

operation such as addition takes two arguments

In programming, it is common for a procedure to take many

arguments

Rather than defining the domain of a procedure as the direct

product of the types of its arguments, we refer to the argument

types of a procedure

In the next chapter we extend the definition of domain to

homogeneous functions (all of whose arguments are of the same

types)

Definition

The definition space for a procedure is that subset of the direct product

of its argument types to which the procedure is intended to be applied

Codomain and result space

Definition

The codomain for a procedure is the type it returns

Definition

The result space for a procedure is the set of values from its codomain

returned by the procedure for some inputs from its definition space

Total and partial procedures

Definition

A procedure is partial if its definition space is a subset of the direct

product of the types of its arguments; it is total if they are equala

a

In mathematical usage, “partial” includes “total”; for example, total ordering

implies partial ordering

when applied to a procedure and set of arguments lying within

the definition space of that procedure

Preconditions are used to specify the definition spaces for

procedures

Before a partial procedure is called, either its precondition must be

satisfied, or the call must be guarded by a call on is_defined7

7

For example of the former, see weak_remainder in Chapter 5, and for an example

of the latter, see collision_point in Chapter 2

Stepanov, McJones Elements of Programming March 14, 2008 43 / 880

Terminating and non-terminating procedures

Definition

A procedure is terminating, semi-terminating, or non-terminating

depending on whether it always terminates, sometimes terminates, or

never terminates on its definition space

definition space

All the procedures in this book are terminating unless explicitly

stated otherwise

A terminating procedure is known as an algorithm

Stateful and indexed procedures

Definition

A procedure is stateful, indexed, or stateless depending on whether its

own state is mutable, immutable, or empty

Proper and improper procedures

Definition

A procedure is proper if it does not modify any essential objects in its

global state; otherwise it is improper

Functions and mutators

Definition

A function is a procedure that does not modify its arguments and

returns a newly constructed object called its result; a mutator is a

procedure whose main purpose is to modify its arguments

Procedure examples

Division is a partial function on integers since its definition space

omits zero as a value for its second argument

A daemon is a non-terminating procedure

The Knuth-Bendix procedure in automatic theorem proving is

semi-terminating

Sorting is a terminating proper mutator

A pseudo-random number generator that keeps its state in an own

variable is a stateful proper function

Multiplication mod k for a fixed k is an indexed proper function

with k being its index

Setting a global graphics state variable is an improper mutator

Assignment

Definition

An assignment is a mutator taking two objects of the same genus whose

effect is to make the first object equal to the second without modifying

the second

interpretation of the second object

Destruction

Definition

A destructor is a mutator causing the cessation of an object’s existence

After a destructor has been invoked on an object, no procedure

can be applied to it and its former memory locations and

resources may be reused for other purposes

Partially-formed state

Definition

An object is in a partially-formed state if it can be assigned to or

destroyed

effect of any procedure other than assignment (only on the left

side) and destruction is not defined

Construction

Definition

A constructor is a mutator transforming memory locations into an

object

complex object state

Type constructors

Definition

A type constructor is a mechanism for creating a new type from one or

more existing types

Examples

In C++, T* is the type “pointer to T”, for any type T

In C++, struct{T0 , ..., Tn−1 } is an n-ary type constructor

In Chapter 10 we show how to construct types corresponding to

data structures such as lists and arrays

Type attributes

Definition

A type attribute is a mapping from a type to a value describing some

characteristic of the type

Examples

The size of an object in bytes

The alignment of an object

The number of members in a struct

Definition

If F is a procedure type, Arity(F) returns its number of arguments

Type functions

Definition

A type function is a mapping from a type to an affiliated type

Examples

Given “pointer to T ”, the value type T

The result type of the difference of two pointers of a given type

The type of the ith member of a struct type (counting from 0)

Definition

If F is a procedure type and i < Arity(F), ArgumentT ype(F, i)

returns the type of the ith argument (counting from 0)

If F is a procedure type, Codomain(F) returns the type of the

result

Stepanov, McJones Elements of Programming March 14, 2008 55 / 880

Concepts

Definition

A concept is a predicate on types stated in terms of properties of

procedures, type attributes, and type functions defined on the types

Definition

A concept is said to be modeled by specific types, or the types model

the concept, if the properties of the concept are satisfied for those

types

To assert that a concept C is modeled by types t0 , . . . , tn−1 , we

write C(t0 , . . . , tn−1 )

Refinement and weakening of concepts

Definition

Concept C 0 refines concept C if whenever C 0 is satisfied for a set of

types, C is also satisfied for those types a

We say C weakens C 0 if C 0 refines C

a

Technically, it is sufficient for C to hold for a subset of the types, possibly reordered

Type concepts

Definition

A type concept is a concept defined on one type

Examples of type concepts in C++

Examples

Integral type

Unsigned integral type

Modeled by: unsigned, unsigned long, unsigned char

Signed integral type

Modeled by: int, long, char

Pointer type

Modeled by: int*, char*

Sequence

Modeled by: std::vector<int>, std::list<int>

Bidirectional iterator

Modeled by: std::list<int>::iterator, int*

Primitive type concepts

Procedure

ProperProcedure

Function

Concept HomogeneousFunction

Definition

HomogeneousFunction(F) ⇒

Function(F)

Arity(F) > 0

For all i, j such that 0 6 i, j < Arity(F),

ArgumentT ype(F, i) = ArgumentT ype(F, j)

Definition

The type function Domain(F) is defined on any function type F

satisfying HomogeneousFunction(F):

Requirements for interoperability of components

We want a sort procedure that works on an array of elements

The requirements on the element type allowing objects to be

sorted should be satisfied by arrays themselves, so we can sort an

array of arrays

All built-in types (e.g., int, float) should satisfy these common

requirements

The requirements are based on equational reasoning

Since the requirements assure regularity of behavior and

interoperability, we call the type concept corresponding to them

regular and the types modeling it regular types 8

8

Regular types were first introduced in:

James C. Dehnert and Alexander A. Stepanov.

Fundamentals of Generic Programming.

Report of the Dagstuhl Seminar on Generic Programming, Schloss Dagstuhl,

Germany, April 1998; also appears in Lecture Notes in Computer Science (LNCS)

volume 1766, pages 1-11.

Concept Regular

Equality

Assignment

Destructor

Default constructor

Copy constructor

Default total ordering (defined in Chapter 4)

9

In Chapter 12 we extend Regular with the notion of underlying type and its

affiliated types and functions

Stepanov, McJones Elements of Programming March 14, 2008 63 / 880

Equality for Regular

Equality should be defined on well-formed object states; two

objects are equal if and only if they represent the same abstract

value

Equality is written as a = b and inequality is written as a , b

Assignment for Regular

Assignment is written as a ← b;

Destructor for Regular

Default constructor for Regular

This is a constructor that takes no arguments and leaves the object

in a partially-formed state

The default construction for a local object a of type T is written T a;

Copy constructor for Regular

This is a constructor that takes an additional argument of the same

type and constructs a new object equal to it

The effect of the copy constructor is equivalent to default

construction followed by assignment

The copy constructor for a local object a of type T with initial

value b is written T a ← b;

Additional functions and affiliated types for Regular

types required for a Regular type

Procedures as objects

destructed, assigned, copied, and tested for equality

These operations on a procedure are all operations on its own state

Two procedures of different type that perform the same

computation belong to the same genus

For example, heapsort and quicksort

Generalized procedures

Definition

A generalized procedure is a procedure defined in terms of type

requirements

Each type requirement has a formal parameter name, introduced

via a typename clause following the template keyword

Requirements on the types are specified via the requires clause,

whose argument is an expression built up from actual types, formal

types, applications of type functions, type equality, concepts, and

their logical connectivesa

a

In this book we use only conjunction

Syntax of type expressions and requires clauses

<type function> ( <type expression> )

<type predicate> ::= <type expression> == <type expression> |

<type concept> ( <type expression )

<requires expression> ::= <type predicate> |

<requires expression> && <type predicate>

<requires clause> ::= requires( <requires expression> )

arguments and local variables

Our requires clause is implemented in Appendix 2 with a simple

macro10

10

In the future C++ is expected to include support for concepts with different syntax

Stepanov, McJones Elements of Programming March 14, 2008 72 / 880

Example of a generalized procedure

requires(SemigroupOperation(Op))

Domain(Op) square(const Domain(Op)& x)

{

return op(x, x);

}

Passing parameters to procedures

passed to it as a parameter:

For reading The procedure depends only on the initial value of

the object, not on its identity

For writing The procedure mutates the object

For selecting The procedure selects one of several objects

(without mutating any of them) and returns the

selected object (not a copy)

Passing parameters for reading

To pass a parameter for reading in C++, we use one of two

techniques

If the size of the parameter is small, we pass it by value (which

makes a local copy)

Otherwise, we pass it by const reference

If the procedure needs a local copy it can mutate, we pass it by

value even when it is large

requires(Readable(I) && Iterator(I))

I find(I f, I l, const ValueType(I)& x)

{

while (f != l && source(f) != x)

++f;

return f;

}

constructor, and it creates a copy equal to the original

Stepanov, McJones Elements of Programming March 14, 2008 75 / 880

Passing parameters for writing

requires(Regular(T))

void swap(T& x, T& y)

{

T tmp = x;

x = y;

y = tmp;

}

Passing parameters for selecting

by reference and by const reference versions of the procedure:

template <typename T>

requires(StrictTotallyOrdered(T))

T& min(T& a, T& b)

{

if (b < a) return b;

else return a;

}

requires(StrictTotallyOrdered(T))

const T& min(const T& a, const T& b)

{

if (b < a) return b;

else return a;

}

More on selecting

assignment when all its parameters are writable

Increase the smaller of two variables by 100: min(a, b) += 100;

Note the return type of the two versions of the procedure must

agree with the selecting parameters with respect to constness

The reasons why C++ requires both versions lie outside the scope

of this book

(In this book we often show only the non-const version)

Concept RegularFunction

Definition

RegularFunction(F) ⇒

Function(F)

Regular(F)

For all f, g ∈ F such that f = g and for all 0 6 i < n:

Regular(ArgumentT ype(F, i))

For all xi , yi ∈ ArgumentT ype(F, i) such that xi = yi ,

where n = Arity(F)

instantiations are regular

Reasons for non-regular functions

Leibniz’s Law

In programming, there are situations when it’s not so

When a function returns the address of a memory location

The “address of” operator (& in C++)

When the function returns a value determined by the state of the

real world

Input from a device or another process

When a stateful function returns a value depending on its own state

A pseudo-random number generator

When a function returns a nonessential attribute of an object

Amount of preallocated memory for a data structure

Program optimization

transformations, such as:

Common subexpression elimination

Loop hoisting

Copy propagation

Example

If f and g are regular, we can optimize f(g(x), g(x)) by evaluating g(x)

only once

Area of an object

A type is of constant area if every object of that type has the same

area

A type is of fixed area if every object of that type has an an area that

does not change during its lifetime

A type is of dynamic area if the area of an object of the type can

vary during the lifetime of the object

For every type, there is a (potentially implicit) function, area, that

returns the area of an object of that type

11

In a binary computer we should in principle measure areas in bits, but unless we

used packed representations we deal in bytes

Stepanov, McJones Elements of Programming March 14, 2008 82 / 880

Storage efficiency

Definition

dlog2 56s(x)e

The byte storage efficiency of a fixed-area object x is area(x) , where

s(x) is the number of distinct states of x

Example

The byte storage efficiency of struct{bool} is 1

The byte storage efficiency of struct{bool,bool} is 1/2

The byte storage efficiency of struct{bool,bool,bool} is 1/3

Cost of a procedure call

The cost of a procedure call is the time required for its execution

The worst-case cost of a procedure call for arguments of given areas

is the maximum cost across all procedure calls with arguments of

these areas

The average cost of a procedure call for arguments of given areas is

the average cost across all procedure calls with arguments of these

areas

A procedure has uniform cost if the cost of a procedure call

depends only on the areas of its arguments

A procedure has fixed cost if the cost of a procedure call does not

depend on its arguments

There is a (possibly implicit) function, cost, that returns the cost

when applied to a procedure and set of arguments lying within

the definition space of that procedure

Contents I

1 Introduction

2 Foundations

Transformations

Orbits

Applications

Examples

Conclusions

Project

Contents II

5 Orderings

6 Combining concepts

9 Rotations

11 Coordinate structures

Contents III

12 Composite objects

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

17 Index

Concept Operation

Definition

Operation(Op) ⇒

RegularFunction(Op)

HomogeneousFunction(Op)

Codomain(Op) = Domain(Op)

Example

Unary abs : double → double

Binary + : double × double → double

Ternary multiply_add : double × double × double → double

Concept Transformation

Definition

Transformation(F) ⇒

Operation(F)

Arity(F) = 1

The predicate is_defined : F × Domain(F) → bool is defined

Example

Numbers Square root

Geometry Plane rotations; space translations

Combinatorics Reversal of a sequence

Composition and reachability

f(x), f(f(x)), . . .

Self-composability allows us to define iterative algorithms

Definition

f0 (x) = x

fn+1 (x) = f(fn (x))

Definition

y is reachable from x under a transformation f if y = x or there is a z

reachable from x and f(z) = y

Cyclic and terminal elements

Definition

x is cyclic under f if f(x) is defined and x is reachable from f(x)

Definition

x is terminal under f if and only if f is not defined at x

Orbits

Definitions

An orbit of x under a transformation f is the set of all elements

reachable from x under f

Lemma

An orbit does not contain both a cyclic and a terminal element

Lemma

An orbit contains at most one terminal element

Classification of orbits

Definitions

An orbit of x under f is:

infinite if it has no cyclic or terminal elements

terminating if it has a terminal element

circular if x is cyclic

ρ-shaped if x is not cyclic and its orbit contains a cyclic element

An orbit of x is finite if it is not infinite

Finite orbits

terminating

circular

ρ-shaped

Structure of orbits

Definition

The orbit cycle is the set of cyclic elements in the orbit

Definition

The orbit handle is the complement of the orbit cycle with respect to the

orbit

Definition

The connection point is the first cyclic element

Sizes

Definitions

The orbit size o of an orbit is the number of distinct elements in it

The handle size h of an orbit is the number of elements in the orbit

handle

The cycle size c of an orbit is the number of elements in the orbit

cycle

Lemma

o=h+c

Finite orbit assumption

Thus no algorithm can determine whether an orbit is finite or

infinite for an arbitrary transformation

For many transformations, finiteness is easily provable

There is an implicit precondition of orbit finiteness for all the

algorithms in this chapter

Algorithmic intuition for cycle detection

If two cars, a fast car and a slow car, start along a path, the fast car

will catch up with the slow car if and only if there is a cycle

If there is no cycle, the fast car will reach the end of the path before

the slow car

If there is a cycle, by the time the slow car enters the cycle, the fast

car will already be there, and will catch up eventually

Collision point

Definition

The collision point of a transformation f and a starting point x is the

unique y such that

y = fn (x) = f2n+1 (x)

and n > 0 is the smallest integer satisfying this condition

collision_point

requires(Transformation(F))

Domain(F) collision_point(const Domain(F)& x, F f)

{

Domain(F) fast = x;

Domain(F) slow = x; // n ← 0 (completed iterations)

while (true) { // slow = fn (x) ∧ fast = f2n (x)

if (!is_defined(f, fast)) break;

fast = f(fast); // slow = fn (x) ∧ fast = f2n+1 (x)

A: if (fast == slow) break;

if (!is_defined(f, fast)) break;

fast = f(fast); // slow = fn (x) ∧ fast = f2n+2 (x)

slow = f(slow); // slow = fn+1 (x) ∧ fast = f2n+2 (x)

// n ← n + 1

}

return fast;

// Postcondition: fast is either the terminal point or the collision point of f and x

}

Proof of termination of collision_point

The movement of slow is unguarded, because by the regularity of

f, slow will be traversing the same orbit as fast

If there is no cycle, is_defined will eventually return false because

of finiteness

If there is a cycle, slow will eventually reach the connection point

(the first element in the cycle):

Consider the distance d from fast to slow at the point labeled A

once slow enters the cycle

06d<c

If d = 0 the procedure terminates

Otherwise the distance decreases by one on each iteration

By induction, the procedure always terminates

When it terminates, slow has moved a total of h + d steps

Position of collision_point when a cycle exists

The annotations show that when there is a cycle and n > 0 is the

number of completed iterations, then fn (x) = f2n+1 (x)

n is the smallest such integer since we checked the condition for

every i < n

We express n = h + d where d is the distance slow moves after

reaching the connection point

Therefore h + d + qc = 2h + 2d + 1 for some q > 0 that counts the

number of cycles completed by fast when slow enters the cycle

Simplifying gives qc = h + d + 1

Represent h in terms of its quotient and remainder when divided

by c: h = mc + r for 0 6 r < c

Substitution gives qc = mc + r + d + 1 or d = (q − m)c − r − 1

0 6 d < c implies q − m = 1, so d = c − r − 1

The distance from the collision point to the connection point is

e=c−d=r+1

Stepanov, McJones Elements of Programming March 14, 2008 102 / 880

Distinguishing circular from ρ-shaped case

h = 0, which implies r = 0 and so e = 1

So the distance from the collision point to the beginning of the

orbit is equal to 1

Circularity therefore can be checked with the following simple

procedure (we leave implementation of similar checks for

terminating and ρ-shaped to the reader)

is_circular

requires(Transformation(F))

bool is_circular(const Domain(F)& x, F f)

{

Domain(F) y = collision_point(x, f);

return is_defined(f, y) && x == f(y);

}

Finding connection point

collision point

The element h steps beyond xw is xw+h = xh+c−r+h

Substituting h = mc + r gives xh+c−r+mc+r = xh+(m+1)c = xh

since it is m + 1 times around the cycle from xh

This suggests an algorithm for determining xh , the connection

point

connection_point

requires(Transformation(F))

Domain(F) convergent_point(Domain(F) x, Domain(F) y, F f)

{

// Precondition: the orbits of x and y converge

while (x != y) {

x = f(x);

y = f(y);

}

return x;

}

requires(Transformation(F))

Domain(F) connection_point(const Domain(F)& x, F f)

{

Domain(F) y = collision_point(x, f);

if (!is_defined(f, y)) return y;

return convergent_point(x, f(y), f);

}

Complexity

the termination proofs and is left as an exercise for the reader

Applications

See for example the Common Lisp function list-length

Determining the length and the period of a random number

generator – see for example:

Donald Knuth.

Exercise 3.1.6.

The Art of Computer Programming, Volume 2, 3rd edition, 1998,

page 7.

Credits two-pointer orbit detection to R.W. Floyd.

Our code works for both cases

Problem: representing sizes

We need to define a type function to obtain an integer type big

enough to encode the orbit size for a given type T

Definition of distance type

Definition

The distance type for a type T is an integer type a that allows us to

encode the maximum number of transformations from one

element of T into another

If a type occupies k bits, its distance type can be represented with

an unsigned integer type occupying k bits

This avoids an infinite tower of types

(It is difficult to have a count type that could count the number of

elements in any collection of type T , because that would require

an extra value)

The type function DistanceT ype(T ) returns the distance type of

T ; for the implementation in C++, see Appendix 2

a

We will discuss integer types briefly in the next chapter and in depth in Chapter 4

distance

requires(Transformation(F))

DistanceType(Domain(F)) distance(const Domain(F)& x, const Domain(F)& y, F f)

{

// Precondition: y is reachable from x under f

typedef DistanceType(Domain(F)) D;

Domain(F) z = x;

D n = D(0);

while (z != y) {

z = f(z);

n = n + D(1);

}

return n;

}

Representing sizes

o−1

But each of these does fit: h−1

c−1

For non-terminating cases, h fits also

orbit_structure

requires(Transformation(F))

triple<DistanceType(Domain(F)), DistanceType(Domain(F)), Domain(F)>

orbit_structure(const Domain(F)& x, F f)

{

typedef DistanceType(Domain(F)) D;

Domain(F) y = connection_point(x, f);

D m = distance(x, y, f);

D n(0);

if (is_defined(f, y))

n = distance(f(y), y, f);

// Terminating: m = h − 1 ∧ n = 0

// Otherwise: m = h ∧ n = c − 1

return triple<D, D, Domain(F)>(m, n, y);

}

Postcondition of orbit_structure

terminating h−1 0 terminal

circular 0 c−1 x

ρ-shaped h c−1 connection

Testing random number generators

Exercise

Given the C++ function object

struct random_function

{

int operator()(int x)

{

srand(x);

return rand();

}

};

whether the cycle length generated by std::rand on your platform is

satisfactory

Conclusions

theories

Nomenclature helps

e.g., orbit kinds and sizes

Equational reasoning on types and functions is essential

Abstract code facilitates reasoning and complexity analysis

Project

R.T. Sedgwick, T.G. Szymanski and A.C. Yao.

The complexity of finding cycles in periodic functions.

Proc. 11th SIGACT Meeting, 1979, pages 376-390.

Richard P. Brent.

An improved Monte Carlo factorization algorithm.

BIT, Volume 20, 1980, pages 176-184.

Leon S. Levy.

An improved list-searching algorithm.

Information Processing Letters, Volume 15, Issue 1, August 1982,

pages 43-45.

Write generic versions of these algorithms and compare their

efficiencies

Contents I

1 Introduction

2 Foundations

Algebra and abstraction

Power on semigroups, monoids, and groups

Slow power algorithm

Stepwise derivation of fast power algorithm

Concept for the exponent

Complexity

Applications

Contents II

Overloading

Conclusions

Reference

Projects

5 Orderings

6 Combining concepts

9 Rotations

Contents III

10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

Contents IV

17 Index

Elementary algebra

Since its inception by the Arabs in the ninth century, algebra has

been viewed as the theory of operations on numbers irrespective

of their particular kinds (such as integers or rationals) or values

The common algebraic operations are addition, subtraction,

multiplication, and division

These operations possess fundamental properties such as

associativity of addition or distributivity of multiplication over

addition

Higher algebra

extended to include aggregates of numbers (such as matrices and

polynomials) together with the common algebraic operations on

them

The operations on aggregates turned out to satisfy the same

properties as the operations on numbers

Precursors of abstract algebra

etc.) observed that a cluster of properties called a group (namely

an associative, invertible binary operation) appears in several

different domains (such as theory of equations and geometry)

At a slightly later time, mathematicians (Dedekind, Weber and

Hilbert) observed that a cluster of properties called a ring

connecting addition and multiplication also appears in multiple

contexts

By the early twentieth century, people learned to recognize

groups, rings, vector spaces, and other similar clusters of

properties

Abstract algebra

developed by several German mathematicians (Noether, Artin,

etc.)

The new approach was to take a well-understood cluster of

properties (such as a group) as axioms, and explore their

consequences independently of any underlying setting

As a result, the same algebraic techniques can be used in any

setting where these axioms are satisfied, leading to effective

applications in domains ranging from quantum mechanics to

crystallography

Computer science has found specialized use of these techniques in

areas such as automata theory and coding theory

Abstract mathematics

portion of mathematics was reinterpreted putting classical results

in their most abstract settings

This process led to

an increase in soundness

reuse of techniques across different domains

Many of the known theories from abstract mathematics

(especially algebra) provide a natural setting for practical

algorithms, with a corresponding increase in soundness and reuse

Abstraction

that every formal discipline rested on what might be called

“the principle of voluntary denial of complete knowledge”:

abstraction or generalization literally indicates a systematic

discarding of certain aspects of the studied objects. 12

12

Jean Dieudonné. Linear Algebra and Geometry. Herman, Paris, 1969, page 17

Stepanov, McJones Elements of Programming March 14, 2008 127 / 880

Grounding abstraction

underlying realities and the problems to be solved

Abstraction in programming should always start with the best

existing and practically useful algorithms and data structures

Concept BinaryOperation

Definition

BinaryOperation(Op) ⇒

Operation(Op)

Arity(Op) = 2

Concept SemigroupOperation

Definition

SemigroupOperation(Op) ⇒

BinaryOperation(Op)

For all op ∈ Op and for all a, b, c ∈ Domain(Op):

A common convention is to represent a semigroup operation as

the infix operator ◦; for example: a ◦ (b ◦ c) = (a ◦ b) ◦ c

Powers

a2 = a ◦ a

a3 = a ◦ a ◦ a

a1 = a

an = a

| ◦ a ◦{z· · · ◦ a}

n

an ◦ am = an+m

Properties of powers

Lemma

an ◦ am = am ◦ an = an+m (powers of the same element commute)

Lemma

(an )m = anm

Frobenius’s theorem

Definition

An element x has finite order under a semigroup operation if there exist

integers 0 < n < m such that xn = xm

Definition

An element x is an idempotent element under a semigroup operation if

x = x2

Theorem

An element of finite order has an idempotent power a

a

Georg Ferdinand Frobenius. Über endliche Gruppen. Berlin 1895. In:

Sitzungesberichte der Königlich Preussischen Akademie der Wissenschaften zu Berlin.

Phys.-math. Classe 1895, pages 163-194

Proof of Frobenius’s theorem

requires(SemigroupOperation(Op))

struct multiply_transformation

{

Domain(Op) x;

Op op;

multiply_transformation(Domain(Op) x, Op op) : x(x), op(op) {}

Domain(Op) operator()(const Domain(Op)& y)

{

return op(y, x);

}

};

requires(SemigroupOperation(Op))

Domain(Op) idempotent_power(Domain(Op) x, Op op)

{

return collision_point(x, multiply_transformation<Op>(x, op));

}

Explanation of proof of Frobenius’s theorem

operation op

Let

Since x is an element of finite order, its orbit under g has a cycle

By the postcondition of collision_point, y = gn (x) for the

smallest n > 0 such that gn (x) = g2n+1 (x), where

g1 (x) = g(x) = x ◦ x = x2

Thus gn (x) = xn+1 and g2n+1 (x) = x2n+2 = x2(n+1) = (xn+1 )2

Therefore y = gn (x) = xn+1 is the idempotent power of x

Concept MonoidOperation

Definition

MonoidOperation(Op) ⇒

SemigroupOperation(Op)

identity_element : Op → Domain(Op) is defined

For all op ∈ Op, for all a ∈ Domain(Op), and

for e = identity_element(op):

op(a, e) = a = op(e, a)

Concept GroupOperation

Definition

GroupOperation(Op) ⇒

MonoidOperation(Op)

inverse_operation : Op → (Domain(Op) → Domain(Op)) is

defined

For all op ∈ Op, for all a ∈ Domain(Op), and for

g = inverse_operation(op):

Extending powers to non-positive exponents

a0 = e

an ◦ a0 = an+0 = an

a−n = (an )−1

an ◦ a−n = an−n = a0

Lemma

(a−1 )n = a−n

Concept Integer

In the meantime we will rely on the intuitive understanding of

what integers are

Models include:

all C++ integral types, signed and unsigned

bignums (type allowing arbitrary-precision integers)

Operations are + - * / % with their standard semantics

All integer types contain constants 0 and 1

Initially we use the constant 2, but later in the chapter we introduce

special-case functions that allow us to avoid its use

In C++ the constants 0, 1, and 2 of an integer type I are represented

as I(0), I(1), and I(2)

slow_power_positive

requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) slow_power_positive(Domain(Op) a, I n, Op op)

{

// Precondition: n > I(0)

if (n == I(1))

return a;

else

return op(a, slow_power_positive(a, n - I(1), op));

}

slow_power_nonnegative

requires(Integer(I) && MonoidOperation(Op))

Domain(Op) slow_power_nonnegative(Domain(Op) a, I n, Op op)

{

// Precondition: n > I(0)

if (n == I(0))

return identity_element(op);

else

return slow_power_positive(a, n, op);

}

Observations

We never perform the operation with the identity element

Except when a is the identity element

It is not worth adding an extra check for the case of a being the

identity element

This would slow down the average case

The caller can do this if it is important

slow_power

requires(Integer(I) && GroupOperation(Op))

Domain(Op) slow_power(Domain(Op) a, I n, Op op)

{

if (n < I(0))

{

a = inverse_operation(op)(a);

n = -n;

}

return slow_power_nonnegative(a, n, op);

}

Ancient Egyptian discovery

power algorithm that gives an efficient way to multiply 13

It is sometimes called the Russian peasant algorithm 14

13

Gay Robins and Charles Shute. The Rhind Mathematical Papyrus, British Museum

Press, 1987

14

The oldest reference to Russian origin we have found appears in: Sir Thomas

Heath. A History of Greek Mathematics, Volume I, Clarendon Press, 1921, page 53.

Reprint: Dover, 1981. Heath writes: “I have been told that there is a method in use

to-day (some say in Russia, but I have not been able to verify this) . . . ” We have not

been able to verify this either.

Stepanov, McJones Elements of Programming March 14, 2008 144 / 880

Exploiting associativity

In general: an = (a2 )n/2 an mod 2

This translates into the following procedure

Initial (recursive) version

requires(Integer(I) && MonoidOperation(Op))

Domain(Op) power_nonnegative_0(Domain(Op) a, I n, Op op)

{

// Precondition: n > I(0)

if (n == I(0))

return identity_element(op);

else if (n == I(1))

return a;

else

return op(power_nonnegative_0(op(a, a), n / I(2), op),

power_nonnegative_0(a, n % I(2), op));

}

Transforming a program

improve the performance of this algorithm without changing its

asymptotic complexity

For the rest of the book we will typically only show final or

almost-final versions

Eliminating unnecessary multiplication by identity

requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) power_positive_0(Domain(Op) a, I n, Op op)

{

// Precondition: n > I(0)

if (n == I(1))

return a;

else if (n % I(2) == I(0))

return power_positive_0(op(a, a), n / I(2), op);

else

return op(power_positive_0(op(a, a), n / I(2), op), a);

}

Eliminating common subexpression

requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) power_positive_1(Domain(Op) a, I n, Op op)

{

// Precondition: n > I(0)

if (n == I(1))

return a;

Domain(Op) result = power_positive_1(op(a, a), n / I(2), op);

if (n % I(2) != I(0))

result = op(result, a);

return result;

}

Introducing accumulation variable

requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) power_accumulate_nonnegative_0(Domain(Op) r, Domain(Op) a, I n, Op op)

{

// Precondition: n > I(0)

if (n == I(0))

return r;

if (n % I(2) != I(0)) r = op(r, a);

return power_accumulate_nonnegative_0(r, op(a, a), n / I(2), op);

}

Almost iterative

requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) power_accumulate_nonnegative_1(Domain(Op) r, Domain(Op) a, I n, Op op)

{

// Precondition: n > I(0)

if (n == I(0))

return r;

if (n % I(2) != I(0)) r = op(r, a);

a = op(a, a);

n = n / I(2);

return power_accumulate_nonnegative_1(r, a, n, op);

}

Iterative

requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) power_accumulate_nonnegative_2(Domain(Op) r, Domain(Op) a, I n, Op op)

{

// Precondition: n > I(0)

while (true) {

if (n == I(0))

return r;

if (n % I(2) != I(0)) r = op(r, a);

a = op(a, a); // wasted on last iteration

n = n / I(2);

}

}

Rotating the loop: reordering

requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) power_accumulate_nonnegative_3(Domain(Op) r, Domain(Op) a, I n, Op op)

{

// Precondition: n > I(0)

while (true) {

if (n == I(0))

return r;

if (n % I(2) != I(0)) r = op(r, a);

n = n / I(2); // reorder

a = op(a, a); // independent statements

}

}

Rotating the loop: duplicating exit

requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) power_accumulate_nonnegative_4(Domain(Op) r, Domain(Op) a, I n, Op op)

{

// Precondition: n > I(0)

while (true) {

if (n == I(0))

return r;

if (n % I(2) != I(0)) r = op(r, a);

n = n / I(2);

if (n == I(0))

return r; // early exit

a = op(a, a);

}

}

Rotating the loop: hoisting exit

requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) power_accumulate_nonnegative_5(Domain(Op) r, Domain(Op) a, I n, Op op)

{

// Precondition: n > I(0)

if (n == I(0))

return r; // moved first test out of loop

while (true) {

if (n % I(2) != I(0)) r = op(r, a);

n = n / I(2);

if (n == I(0))

return r;

a = op(a, a);

}

}

Dependency between conditions

1 is odd

The second condition will only be true when the first condition is

true:

if (n % I(2) != I(0)) r = op(r, a);

n = n / I(2);

if (n == I(0))

return r;

Utilizing dependency

requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) power_accumulate_nonnegative_6(Domain(Op) r, Domain(Op) a, I n, Op op)

{

// Precondition: n > I(0)

if (n == I(0))

return r;

while (true) {

bool odd = n % I(2) != I(0);

n = n / I(2);

if (odd) {

r = op(r, a);

if (n == I(0))

return r;

}

a = op(a, a);

}

}

Specializing for positive n

requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) power_accumulate_positive_0(Domain(Op) r, Domain(Op) a, I n, Op op)

{

// Precondition: n > I(0)

while (true) {

bool odd = n % I(2) != I(0);

n = n / I(2);

if (odd) {

r = op(r, a);

if (n == I(0))

return r;

}

a = op(a, a);

}

}

(Nearly final) power_accumulate_nonnegative

requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) power_accumulate_nonnegative_7(Domain(Op) r, Domain(Op) a, I n, Op op)

{

// Precondition: n > I(0)

if (n == I(0))

return r;

else

return power_accumulate_positive_0(r, a, n, op);

}

Eliminating accumulation variable

requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) power_positive_2(Domain(Op) a, I n, Op op)

{

// Precondition: n > I(0)

n = n - I(1);

return power_accumulate_nonnegative_7(a, a, n, op);

}

Observation

When n is odd, this code is fine

Factoring out powers of 2

requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) power_positive_3(Domain(Op) a, I n, Op op)

{

// Precondition: n > I(0)

while (n % I(2) == I(0)) {

a = op(a, a);

n = n / I(2);

}

n = n - I(1);

if (n == I(0))

return a;

else

return power_accumulate_positive_0(a, a, n, op);

}

Observation

We know n is even at that point

One extra operation!

Unrolling one iteration of

power_accumulate_positive_0

requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) power_positive_4(Domain(Op) a, I n, Op op)

{

// Precondition: n > I(0)

while (n % I(2) == I(0)) {

a = op(a, a);

n = n / I(2);

}

n = n / I(2);

if (n == I(0))

return a;

else

return power_accumulate_positive_0(a, op(a, a), n, op);

}

Operations on exponent

n - I(1)

n = n / T(2);

n % T(2) == T(0)

n == T(0)

General / and % are very expensive

For all integral types, if we know n is non-negative, we can use

shifts and masks

Special cases of procedures

involving procedures and constants of a type by defining

special-case procedures

Often these special cases can be implemented more efficiently

than the general case

For built-in types there may exist machine instructions for the

special cases (e.g., shifting instead of multiplication by a power of

two)

For complex user-defined types there are often even more

significant opportunities for optimizing special cases

Division of two arbitrary polynomials is more difficult than

division of a polynomial by x

Division of two Gaussian integers (numbers

√ of the form a + bi

where a and b are integers and i = −1) is more difficult than

division of a Gaussian integer by 1 + i

Special cases of Integer procedures

Integer(I) ∧ i ∈ I ⇒

successor(i) ≡ i + 1

predecessor(i) ≡ i − 1

half_nonnegative(i) ≡ bi/2c

binary_scale_down_nonnegative(i, k) ≡ i/2k

binary_scale_up_nonnegative(i, k) ≡ 2k i

is_positive(i) ≡ i > 0

is_negative(i) ≡ i < 0

is_zero(i) ≡ i = 0

is_even(i) ≡ (i mod 2) = 0

is_odd(i) ≡ (i mod 2) , 0

Stepanov, McJones Elements of Programming March 14, 2008 167 / 880

power_accumulate_positive

requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) power_accumulate_positive(Domain(Op) r, Domain(Op) a, I n, Op op)

{

// Precondition: is_positive(n)

while (true) {

bool odd = is_odd(n);

halve_nonnegative(n);

if (odd) {

r = op(r, a);

if (is_zero(n))

return r;

}

a = op(a, a);

}

}

power_accumulate_nonnegative

requires(Integer(I) && MonoidOperation(Op))

Domain(Op) power_accumulate_nonnegative(Domain(Op) r, Domain(Op) a, I n, Op op)

{

// Precondition: ¬is_negative(n)

if (is_zero(n))

return r;

else

return power_accumulate_positive(r, a, n, op);

}

power_positive

requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) power_positive(Domain(Op) a, I n, Op op)

{

// Precondition: is_positive(n)

while (is_even(n)) {

a = op(a, a);

halve_nonnegative(n);

}

halve_nonnegative(n);

if (is_zero(n))

return a;

else

return power_accumulate_positive(a, op(a, a), n, op);

}

power_nonnegative

requires(Integer(I) && MonoidOperation(Op))

Domain(Op) power_nonnegative(Domain(Op) a, I n, Op op)

{

// Precondition: ¬is_negative(n)

if (is_zero(n))

return identity_element(op);

else

return power_positive(a, n, op);

}

power

requires(Integer(I) && GroupOperation(Op))

Domain(Op) power(Domain(Op) a, I n, Op op)

{

if (is_negative(n)) {

a = inverse_operation(op)(a);

n = -n;

}

return power_nonnegative(a, n, op);

}

The number of operations performed by power

Let v = number of one-bits in n

Let p = number of operations performed

Then p = (u − 1) + (v − 1) = u + v − 2 6 2blog2 nc

Exercise

Prove that power achieves this complexity

The operation count of power is not optimal

But 15 = 3 × 5, so a15 = (a3 )5

a3 takes 2 operations

a5 takes 3 operations

(a3 )5 takes 5 total

There are faster ways for 23, 27, 39, 43, . . .

See Project 3

Choosing algorithms based on complexity

For example, a stack is not a stack if the cost of push grows linearly

with the size of the stack

When only one algorithm for solving a problem is available, the

main issue is whether the algorithm is feasible for a particular

situation (set of arguments determining problem size, functional

parameters, etc.)

When there are more than one algorithm for solving a problem, we

need to analyze which algorithm to use for a particular situation

The analysis depends on the complexity properties of the

alternative algorithms combined with the complexity properties

of the types to which these algorithms are applied

Specifying complexity

machine instructions performed by a MIX program

Since we write generalized procedures defined in terms of type

requirements and sometimes taking function objects as

arguments, we need to count applications of the function objects

and the operations on the types

Complexity of slow_power versus power

The total cost is the sum of the cost of the operations on the

exponent, the cost of the semigroup operation, and the cost of the

control structure

For all the models of Integer we are familiar with, the logarithmic

number of exponent operations performed by power will be faster

than the linear number of operations performed by slow_power,

except for very small n

For most models of the semigroup operation, power will be faster,

but there are exceptions

When is power faster than slow_power?

If α = 0, we have a fixed-cost operation and power is always

much faster than slow_power

Examples: fixed-precision multiplication; matrix multiplication

with fixed-precision elements

If α 6 1, power is always faster15

Examples: multiplication of univariate polynomials with

fixed-precision coefficients; multiplication of bignums

If α > 2, slow_power is always faster16

Examples: multiplication of bivariate polynomials with

fixed-precision coefficients; multiplication of univariate

polynomials with bignum coefficients

15

R.L. Graham, A.C.-C. Yao, and F.-F. Yao. Addition chains with multiplicative

costs. Discrete Mathematics, Volume 23, Number 2, 1978, pages 115-119.

16

D.P. McCarthy. Effect of Improved Multiplication Efficiency on Exponentation

Algorithms Derived from Addition Chains. Mathematics of Computation, Volume 46,

Number 174, April 1986, pages 603-608.

Stepanov, McJones Elements of Programming March 14, 2008 178 / 880

Applications

Donald Knuth.

Secret factors.

The Art of Computer Programming, Vol. 2, pages 403-407.

RSA.

Donald Knuth.

Improved primality tests.

The Art of Computer Programming, Vol. 2, pages 394-396.

Primality testing.

Manindra Agrawal, Neeraj Kayal, and Nitin Saxena.

PRIMES is in P.

Annals of Mathematics, 160 (2004), pages 781-793.

Primality testing.

Reflections on deriving useful interfaces

to compute axn

Even power_accumulate_positive is useful in a situation when it

is known that n > 0

Reflection

While developing a component, it is often possible to discover

additional useful interfaces

Reflection on code transformations

Reflection

Compilers can perform code transformations when the semantics

of the operations are known

Currently this is only for built-in types

Someday we will be able to tell the compiler the semantics of our

operations

Fibonacci numbers

Definition

f0 = 0

f1 = 1

fn+2 = fn+1 + fn

Example

0, 1, 1, 2, 3, 5, 8, 13, . . .

Recursive way to calculate fn

requires(Integer(I))

I fibonacci_recursive(I n)

{

if (n == I(0))

return I(0);

else if (n == I(1))

return I(1);

else

return fibonacci_recursive(n - I(1))

+ fibonacci_recursive(n - I(2));

}

Iterative way to calculate fn

requires(Integer(I))

I fibonacci_iterative(I n)

{

if (n == I(0))

return I(0);

I fib_i = I(0);

I fib_j = I(1);

while (n != I(1)) {

I next = fib_i + fib_j;

fib_i = fib_j;

fib_j = next;

n = n - I(1);

}

return fib_j;

}

Fibonacci matrices

1 1

F1 =

1 0

fn+1 fn

Fn =

fn fn−1

fn+1 + fn fn + fn−1 fn+2 fn+1

F1 Fn = = = Fn+1

fn+1 fn fn+1 fn

Fn = F1 F1 . . . F1 = Fn

| {z } 1

n

m n

Fm Fn = F1 F1 = Fm+n

1 = Fm+n

Fibonacci matrices

fm+1 fm

Fm =

fm fm−1

fn+1 fn

Fn =

fn fn−1

fm+1 fn+1 + fm fn fm+1 fn + fm fn−1

Fm Fn =

fm fn+1 + fm−1 fn fm fn + fm−1 fn−1

Observations

bottom row

pair::first is fn and pair::second is fn−1

The identity fn+1 = fn−1 + fn allows us to compute the missing

term from the top row of Fn needed to compute the bottom row of

Fm Fn

fibonacci_multiplies

requires(Integer(I))

struct fibonacci_multiplies

{

typedef pair<I, I> D;

D operator()(const D& x, const D& y) const

{

return D(

x.first * (y.first + y.second) + x.second * y.first,

x.first * y.first + x.second * y.second);

}

};

fibonacci

requires(Integer(I))

I fibonacci(I n)

{

// Precondition: n > I(0)

if (n == I(0))

return I(0);

fibonacci_multiplies<I> op;

return power_positive(pair<I, I>(I(1), I(0)), n, op).first;

}

Overloading

Definition

Overloading an operator symbol or function name means using it for

operations or functions on several types

Reflection

Progress in mathematics often involves extending an operator to a new

domain

Example

+ on

natural numbers

integers

rationals

polynomials

matrices

Stepanov, McJones Elements of Programming March 14, 2008 190 / 880

Overloading should preserve semantics

+ is associative

+ is commutative

+ obeys the cancellation law

a+c=b+c⇒a=b

× distributes over +

Type parameterization results in overloading

power, we are overloading the function name

If the actual type parameters satisfy the requirements the

semantics is preserved since the same algorithm is used

Two ways operations are provided to power

This allows power to be used with different operations on the same

type, and the operation can have a state such as a modulus for use

in encryption

The integer operations are provided through overloading of -, /,

%, and ==

Using explicit operators is convenient whenever there are several

related operations on the type and no other correct interpretation of

the operations on the type

Concept CommutativeOperation

Definition

CommutativeOperation(Op) ⇒

BinaryOperation(Op)

For all op ∈ Op and for all a, b ∈ Domain(Op),

op(a, b) = op(b, a)

Concept CancellableOperation

Definition

CancellableOperation(Op) ⇒

BinaryOperation(Op)

For all op ∈ Op and for all a, b, c ∈ Domain(Op),

op(a, c) = op(b, c) ⇒ a = b

op(c, a) = op(c, b) ⇒ a = b

Concept AdditiveSemigroup

Definition

AdditiveSemigroup(T ) ⇒

The binary operation infix + is defined on T

SemigroupOperation(+)

CommutativeOperation(+) a

a

It would have been nice to require the cancellation law, but digital logic designers

use + for an operation that is not cancellable

Concept MultiplicativeSemigroup

Definition

MultiplicativeSemigroup(T ) ⇒

The binary operation infix × is defined on T

SemigroupOperation(×)

Concept AdditiveMonoid

Definition

AdditiveMonoid(T ) ⇒

AdditiveSemigroup(T )

MonoidOperation(+)

The constant T (0) is defined

identity_element(+) = T (0)

Concept MultiplicativeMonoid

Definition

MultiplicativeMonoid(T ) ⇒

MultiplicativeSemigroup(T )

MonoidOperation(×)

The constant T (1) is defined

identity_element(×) = T (1)

Concept AdditiveGroup

Definition

AdditiveGroup(T ) ⇒

AdditiveMonoid(T )

GroupOperation(+)

The unary operation negation (prefix −) is defined on T

inverse_operation(+) = negation

Definition

For an additive group, subtraction is defined as a − b = a + (−b)

Concept MultiplicativeGroup

Definition

MultiplicativeGroup(T ) ⇒

MultiplicativeMonoid(T )

GroupOperation(×)

The unary operation reciprocal is defined on T

inverse_operation(×) = reciprocal

Definition

For a multiplicative group, division is defined as

a/b = a × reciprocal(b)

Power for multiplicative types

template <typename G, typename I>

requires(MultiplicativeSemigroup(G) && Integer(I))

G power_positive(G a, I n)

{

return power_positive(a, n, multiplies<G>());

}

requires(MultiplicativeMonoid(G) && Integer(I))

G power_nonnegative(G a, I n)

{

return power_nonnegative(a, n, multiplies<G>());

}

requires(MultiplicativeGroup(G) && Integer(I))

G power(G a, I n)

{

return power(a, n, multiplies<G>());

}

Stepanov, McJones Elements of Programming March 14, 2008 202 / 880

Conclusions

They can be used with different models satisfying the same

requirements

Algorithms are affiliated with algebraic structures: semigroup,

monoid, . . .

Stepwise refinement leads from mathematical definitions to

efficient code

Special-case procedures can make code more efficient and even

more generic

Mathematics leads to surprising algorithms: fibonacci

Reference

Operators and Algebraic Structures.

Proceedings of the 1981 conference on Functional programming

languages and computer architecture, pages 59-63.

One of the first presentations of algorithms on algebraic structures.

Project 1

Project

Our definition of the Fibonacci sequence starts from zero and goes up;

extend the definition and our code to work for negative indices a

a

Suggested by Oleg Zabluda

Project 2

Charles M. Fiduccia.

An efficient formula for linear recurrences.

SIAM Journal of Computing, Volume 14, Number 1, February 1985,

page 106-112.

Generalizes the fibonacci algorithm to arbitrary linear recurrences.

Project

Create a library implementing Fiduccia’s algorithm

Project 3

Donald E. Knuth.

Addition chains.

The Art of Computer Programming, Volume 2: Seminumerical

Algorithms, 3rd edition, Addison-Wesley, San Francisco, 1998,

pages 465-481.

Describes a way to do minimal-operation exponentiation using

addition chains.

Project

Implement a useful library doing power optimally for exponents

known at compile time

Project 4

Project

Floating-point multiplication and addition are not associative, so

may give different results when they are used as the operation for

slow_power and power; establish whether slow_power or power

gives a more accurate result for:

1 Raising a floating-point number to an integral power

2 Multiplying a floating-point number by an integral factor

Contents I

1 Introduction

2 Foundations

5 Orderings

Motivation

Relation; transitivity; ordering

Strict versus reflexive

Symmetric versus asymmetric

Contents II

Equivalence; equality

Symmetric complement; total ordering; weak ordering; partial

ordering

Multiple orderings; natural total ordering; default ordering;

overloading

Intervals

Order selection algorithms

Conclusions

Reference

Project

6 Combining concepts

Contents III

8 Permutations and rearrangements

9 Rotations

11 Coordinate structures

12 Composite objects

14 Mathematical notation

Contents IV

15 C++ machinery

16 Acknowledgments

17 Index

The importance of linear ordering

Searching a collection using only equality requires linear time

Intersecting two collections using only equality requires time

proportional to the product of the sizes of the collections

A linear ordering allows organizing elements so that their

collections can be

matched (intersection, subset, union) in linear time

searched in logarithmic time

Concepts Predicate and Relation

Definition

Predicate(Op) ⇒

RegularFunction(Op)

Codomain(Op) = bool

Definition

Relation(Op) ⇒

Predicate(Op)

HomogeneousFunction(Op)

Arity(Op) = 2

(Other books use this term for more general non-unary predicates)

Concept Ordering

Definition

Ordering(Op) ⇒

Relation(Op)

For all op ∈ Op and for all a, b, c ∈ Domain(Op)

Examples of ordering

Examples

Equality

Equality of the first character

Reachability in an orbit

Divisibility

Strict versus reflexive

Definition

An ordering r is strict if ¬r(a, a)

An ordering r is reflexive if r(a, a)

An ordering r is weakly reflexive if r(a, b) ⇒ r(a, a) ∧ r(b, b)

Examples

Reflexive: all examples on the previous slide

Strict: proper factor

Symmetric versus asymmetric

Definition

A relation r is symmetric if r(a, b) ⇒ r(b, a)

A relation r is asymmetric if r(a, b) ⇒ ¬r(b, a)

Lemma

A strict ordering is asymmetric

Lemma

A symmetric ordering is weakly reflexive

Examples

Symmetric: sibling

Asymmetric: parent

Concept EquivalenceRelation

Definition

EquivalenceRelation(R) ⇒

Ordering(R)

Reflexive(R)

Symmetric(R)

Examples

Equality

Geometric congruence

a ≡ b (mod n)

Equivalence and equality

Lemma

If r is an equivalence relation, a = b ⇒ r(a, b)

Key function

Definition

If T is a regular type and r ∈ R is an equivalence relation on T , a

function f ∈ F : T → T 0 is a key function for r if

r(x, x 0 ) ⇔ f(x) = f(x 0 )

Given the choice of a particular key function f, we say f(x) is a

canonical representation of the elements equivalent to x

function for it and then apply equality to the results

symmetric_complement

requires(Relation(R))

struct symmetric_complement

{

R r;

symmetric_complement(R r) : r(r) {}

{

return !r(a, b) && !r(b, a);

}

};

Concept StrictTotalOrdering

Definition

StrictTotalOrdering(R) ⇒

Ordering(R)

For all r ∈ R and for all a, b ∈ Domain(R)

symmetric_complementr (a, b) ⇔ a = b

Properties of total ordering

Lemma

A total ordering is strict

Lemma

A total ordering is asymmetric

Lemma

The trichotomy law holds: r(a, b) ∨ r(b, a) ∨ (a = b)

Lemma

Exactly one clause of the trichotomy law holds

Extending strict total orderings to a direct product

strict total ordering on the direct product

Domain(R0 ) × . . . × Domain(Rn ) via the lexicographic ordering r̄

defined by:

r̄((x0 , . . . , xn ), (y0 , . . . , yn )) ≡

(∃k ∈ [0, n])(x0 = y0 ∧ . . . ∧ xk−1 = yk−1 ∧ rk (xk , yk ))

Lemma

r̄ is a strict total ordering

Concept StrictWeakOrdering

Definition

StrictWeakOrdering(R) ⇒

Ordering(R)

EquivalenceRelation(symmetric_complementR )

Weak ordering is a weakening of total ordering

Lemma

StrictTotalOrdering(R) ⇒ StrictWeakOrdering(R)

Properties of weak ordering

Lemma

A weak ordering is strict

Lemma

A weak ordering is asymmetric

Lemma

The trichotomy law holds:

r(a, b) ∨ r(b, a) ∨ symmetric_complementr (a, b)

Lemma

Exactly one clause of the trichotomy law holds

Extending strict weak orderings to a direct product

lexicographic ordering, just as with strict total orderings

Defining a strict weak ordering using a key function

codomain of f define a weak ordering r̃(x, y) ⇔ r(f(x), f(y))

Example

Weak ordering on ith component of a direct product

requires(Function(F) && Arity(F) == 1 &&

StrictTotalOrdering(R) && Codomain(F) == Domain(R))

struct key_ordering models(StrictWeakOrdering)

{

F f;

R r;

key_ordering(F f, R r) : f(f), r(r) { }

bool operator()(const Domain(F)& x, const Domain(F)& y) const

{

return r(f(x), f(y));

}

};

Not every strict ordering is weak

b c

a d

e

If were an equivalence relation it would have to include b c

and c b by transitivity

Mathematical conventions for usage of weak and semi

A strict weak ordering replaces equality with equivalence

Semi refers to dropping an operation

A semigroup lacks the inverse operation

Partial ordering

There are algorithms that deal with orderings that are not weak:

algorithms on partially-ordered sets

The most important is topological sort:

Donald Knuth.

Topological sorting.

The Art of Computer Programming, Volume 1, Addison-Wesley,

1997, pages 261-268.

Operations as simple as max and min do not make sense without

a weak order

Multiple orderings on a type

There can be many equivalence relations

Similarly there can be many orderings

Natural total ordering

Definition

The total ordering on a type that is consistent with algebraic operations

on it is called the natural total ordering

Example

Examples of consistent axioms for natural ordering:

Incrementable a < successor(a)

Incrementable a < b ⇒ successor(a) < successor(b)

Ordered group a < b ⇒ a + c < b + c

Ordered ring (a < b) ∧ (0 < c) ⇒ ca < cb

Default ordering

Complex numbers

Iterators on linked lists

We still want a total ordering to enable logarithmic searching, so

we always define the default ordering for regular types

Lexicographic ordering for complex numbers

Address-based ordering for iterators on linked lists

When the natural order exists, it coincides with the default

ordering

Definition

less<T> defines the default ordering for T

Concept StrictTotallyOrdered

Definition

StrictTotallyOrdered(T ) ⇒

Regular(T )

The relation < is defined on T

StrictTotalOrdering(<)

Clusters of derived procedures

If some of the cluster are defined, the definitions of the others

naturally follow

Example

In an additive group, negation and subtraction constitute such a cluster

Derived relations

complement ¬r(a, b)

converse r(b, a)

complement of converse ¬r(b, a)

Given a symmetric relation, say r(a, b), since the converse is equal

to the original relation, the only derivable relation is the

complement, ¬r(a, b)

Properties of derived relations

Lemma

Given an ordering, its complement, its converse, and the

complement of its converse are orderings

If an ordering is strict, its converse is also strict and its

complement and complement of converse are reflexive

If an ordering is reflexive, its converse is also reflexive and its

complement and complement of converse are strict

There exist C++ template function object classes complement,

converse, and complement_of_converse which, given a relation,

construct the corresponding derived relations

Equality and inequality

We assure consistency by always defining equality and relying on

the following template for inequality

requires(Regular(T))

bool operator!=(const T& a, const T& b)

{

return !(a == b);

}

<, >, 6, and >

For every totally ordered type, we have <, >, 6, and >

We assure consistency by always defining < and relying on the

following templates for the others

requires(StrictTotallyOrdered(T))

bool operator>(const T& a, const T& b) { return b < a; }

requires(StrictTotallyOrdered(T))

bool operator<=(const T& a, const T& b) { return !(b < a); }

requires(StrictTotallyOrdered(T))

bool operator>=(const T& a, const T& b) { return !(a < b); }

Intervals

Definition

A closed interval [a, b] is the set of all elements x such that a 6 x 6 b

An open interval (a, b) is the set of all elements x such that

a<x<b

A half-open on right interval [a, b) is the set of all elements x such

that a 6 x < b

A half-open on left interval (a, b] is the set of all elements x such that

a<x6b

A half-open interval is our short-hand for half-open on right

These definitions generalize to weak orderings

min and max

requires(StrictWeakOrdering(R))

Domain(R)& min(Domain(R)& a, Domain(R)& b, R r)

{

if (r(b, a)) return b;

else return a;

}

requires(StrictWeakOrdering(R))

Domain(R)& max(Domain(R)& a, Domain(R)& b, R r)

{

if (r(b, a)) return a;

else return b;

}

Other min and max

requires(StrictWeakOrdering(R))

Domain(R)& other_min(Domain(R)& a, Domain(R)& b, R r)

{

if (r(a, b)) return a;

else return b;

}

requires(StrictWeakOrdering(R))

Domain(R)& other_max(Domain(R)& a, Domain(R)& b, R r)

{

if (r(a, b)) return b;

else return a;

}

other_min and other_max 17

17

The C++ standard library uses min and other_max

Stepanov, McJones Elements of Programming March 14, 2008 245 / 880

A convention for order selection functions

For the order selection procedures in this section, there are four

useful versions

Since the parameters are being passed for selecting, versions taking

both references and const references are needed

For convenience, we want versions for totally ordered types (with

<) as well as for an explicitly-supplied ordering

From now on, we show only the non-const reference version with

an explict ordering; see Appendix 2 for the other versions

The code below shows how to obtain a version for a totally

ordered type given a version for an explicitly supplied ordering

Stability

Definition

An algorithm is stable if it respects the original order of equivalent

elements

elements are equal and, therefore, indistinguishable

Example

Stable sort

Multiple passes compose naturally

Sorting by first name and then by last name results in expected order

Stable partition

Even/odd partition of {1, 2, 3, 4, 5} results in {1, 3, 5, 2, 4}, not

{1, 5, 3, 4, 2} as produced by a fast partitioning algorithm

Stability sometimes increases the complexity

Stepanov, McJones Elements of Programming March 14, 2008 247 / 880

Increasing order

Definition

increasing

A sequence . . . , x, . . . , x 0 , . . . is in order with

strictly increasing

¬r(x 0 , x) a

respect to a strict weak ordering r if

r(x, x 0 )

a

While some would use the terms non-decreasing and increasing, we follow Nicolas

Bourbaki, Theory of Sets, III.1.5, page 138.

sequences

sort_2

requires(StrictWeakOrdering(R))

void sort_2(Domain(R)& a, Domain(R)& b, R r)

{

if (r(b, a)) swap(a, b);

}

sort_2 is stable

It does the minimal amount of work since it does not swap

equivalent elements

Stability of min and max

a0 = a; b0 = b;

sort_2(a, b, r);

assert(a == min(a0, b0, r) && b == max(a0, b0, r));

min and max satisfy it, while other_min and other_max do not

Properties of min and max for StrictTotalOrdering r

Lemma

associativity minr (minr (a, b), c) = minr (a, minr (b, c))

associativity maxr (maxr (a, b), c) = maxr (a, maxr (b, c))

commutativity minr (a, b) = minr (b, a)

commutativity maxr (a, b) = maxr (b, a)

absorption minr (a, maxr (a, b)) = a

absorption maxr (minr (a, b), b) = b

idempotency minr (a, a) = a

idempotency maxr (a, a) = a

Properties of min and max for StrictWeakOrdering r

Lemma

associativity minr (minr (a, b), c) = minr (a, minr (b, c))

associativity maxr (maxr (a, b), c) = maxr (a, maxr (b, c))

commutativity symmetric_complementr (minr (a, b), minr (b, a))

commutativity symmetric_complementr (maxr (a, b), maxr (b, a))

Exercise

What are the absorption and idempotency laws for minr and maxr ?

min_3 and max_3

requires(StrictWeakOrdering(R))

Domain(R)& min_3(Domain(R)& a, Domain(R)& b, Domain(R)& c, R r)

{

return min(min(a, b, r), c, r);

}

requires(StrictWeakOrdering(R))

Domain(R)& max_3(Domain(R)& a, Domain(R)& b, Domain(R)& c, R r)

{

return max(max(a, b, r), c, r);

}

Implementing order selection

There are more complicated cases than min and max such as

median of 3 and select 2nd of 4

Writing order selections is somewhat complicated and can be

helped by decomposition into simpler subproblems

median_3

requires(StrictWeakOrdering(R))

Domain(R)& median_3(Domain(R)& a, Domain(R)& b, Domain(R)& c, R r)

{

if (r(b, a)) return median_3_stage1(b, a, c, r);

else return median_3_stage1(a, b, c, r);

}

requires(StrictWeakOrdering(R))

Domain(R)& median_3_stage1(Domain(R)& a, Domain(R)& b, Domain(R)& c, R r)

{

// Precondition: a, b are sorted

if (!r(c, b)) return b; // a, b, c are sorted

else return max(a, c, r); // b is not the median

}

Stability of median_3

Exercise

Define the appropriate stability property for median_3

Lemma

median_3 is stable

Complexity of median_3

The function does 2 comparisons only when c is max_3(a, b, c)

and that happens in one-third of the cases

The average number of comparison is 2 32

Selecting second smallest

If after finding second smallest, two candidates for smallest remain,

then the second smallest is not second smallest!

Finding second smallest from n elements requires at least

n + log n − 2 comparisons

Finding second out of four requires 4 comparisons

select_2nd_4

template <typename R> requires(StrictWeakOrdering(R))

Domain(R)&

select_2nd_4(Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, R r) {

if (r(b, a)) return select_2nd_4_stage1(b, a, c, d, r);

else return select_2nd_4_stage1(a, b, c, d, r);

}

Domain(R)&

select_2nd_4_stage1(Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, R r) {

// Precondition: a 6 b

if (r(d, c)) return select_2nd_4_stage2(a, b, d, c, r);

else return select_2nd_4_stage2(a, b, c, d, r);

}

Domain(R)&

select_2nd_4_stage2(Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, R r) {

// Precondition: a 6 b ∧ c 6 d

if (r(c, a)) return min(a, d, r);

else return min(b, c, r);

}

Exercises

Exercise

Is select_2nd_4 stable?

Exercise

Implement select_3rd_4

median_5

template <typename R> requires(StrictWeakOrdering(R))

Domain(R)& median_5(

Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, Domain(R)& e, R r) {

if (r(b, a)) return median_5_stage1(b, a, c, d, e, r);

else return median_5_stage1(a, b, c, d, e, r);

}

Domain(R)& median_5_stage1(

Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, Domain(R)& e, R r) {

// Precondition: a 6 b

if (r(d, c)) return median_5_stage2(a, b, d, c, e, r);

else return median_5_stage2(a, b, c, d, e, r);

}

Domain(R)& median_5_stage2(

Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, Domain(R)& e, R r) {

// Precondition: a 6 b ∧ c 6 d

if (r(c, a)) return select_2nd_4_stage1(a, b, d, e, r);

else return select_2nd_4_stage1(c, d, b, e, r);

}

Exercises

Exercise

Show that median_5 is not stable

Exercise

Design a stable version of median_5

Exercise

Find an algorithm for median of 5 that does slightly fewer

comparisons on average

Conclusions

elements

The axioms of ordering provide the interface to connect specific

orderings with general purpose algorithms

Overloaded operators should preserve their semantics in

mathematics

Reference

N. Bourbaki.

Chapter 3, Section 1: Order relations. Ordered Sets.

Theory of Sets, Springer, 2004, pages 131-148.

Project

Donald E. Knuth.

Section 5.3: Optimum Sorting.

The Art of Computer Programming, Volume 3: Sorting and

Searching, 2nd edition, Addison-Wesley, 1998.

create a library for generic minimum-comparison networks for

sorting, merging, and selection

For sorting and merging networks, minimize not only the number

of comparisons, but the number of data movements

Assure stability of these networks

Contents I

1 Introduction

2 Foundations

5 Orderings

6 Combining concepts

Combining algebraic structures and ordering

Remainder

Contents II

Greatest common divisor

Extending the domain of greatest common divisor

Quotient

Quotient and remainder for negative quantities

Integers

Conclusions

References

Project

9 Rotations

Contents III

10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

Contents IV

17 Index

Axioms combine concepts

Combining different types

Ordered structures

Definition

OrderedAdditiveSemigroup(T ) ⇒

AdditiveSemigroup(T )

StrictTotallyOrdered(T )

For all x, y, z ∈ T ,

x<y⇒x+z<y+z

Definition

OrderedAdditiveMonoid(T ) ⇒ Monoid(T ) ∧ OrderedAdditiveSemigroup(T )

Definition

OrderedAdditiveGroup(T ) ⇒ Group(T ) ∧ OrderedAdditiveMonoid(T )

Absolute value

requires(OrderedAdditiveGroup(T))

T abs(const T& x)

{

if (x < T(0)) return -x;

else return x;

}

Concept OrderedCancellableMonoid

Definition

OrderedCancellableMonoid(T ) ⇒

OrderedAdditiveMonoid(T )

The partial binary infix operator − is defined on T

For all a, b ∈ T

b 6 a ⇒ is_defined(−, a, b) ∧ (a − b) + b = a

Division with remainder

repeated subtraction induces division with remainder

Interestingly, it is easier to introduce the remainder computation,

leaving the quotient for later

weak_remainder

requires(OrderedCancellableMonoid(T))

T weak_remainder(T a, T b)

{

// Precondition: a > T (0) ∧ b > T (0)

while (b <= a) a = a - b;

return a;

}

prove termination of weak_remainder; we need another

property to ensure there are no unreachable (infinite) elements

Concept ArchimedeanMonoid

Definition

ArchimedeanMonoid(T) ⇒

OrderedCancellableMonoid(T )

For all a and all b > 0 ∈ T , weak_remainder(a, b) terminates

Reflection

A concept can be defined with an axiom stating that a particular

procedure terminates

The Axiom of Archimedes

integer m such that a < m · b”

Archimedes originally wrote: “... that the excess by which the

greater of (two) unequal areas exceeds the less can, by being

added to itself, be made to exceed any given finite area.” 18

Archimedes attributes this lemma to Eudoxus 19

18

T.L. Heath, translator. The Works of Archimedes, Preface to Quadrature of the

Parabola. Dover, 2002, page 234

19

Sir Thomas Heath. A History of Greek Mathematics, Volume I. Dover, 1981, page 327

Stepanov, McJones Elements of Programming March 14, 2008 277 / 880

Examples of Archimedean monoids

Rational numbers

Binary fractions { 2nk }

Ternary fractions { 3nk }

Intuition for a fast algorithm for remainder

A related algorithm is possible for remainder

As with power, the Egyptians used this algorithm to do division

with remainder

Deriving a fast algorithm for remainder

stands for c| + ·{z

· · + c}

m times

Let a = n · b + r, where r = weak_remainder(a, b)

Let n = 2q + p, where q = bn/2c and p = n mod 2

a = q · (2 · b) + (p · b + r) where (p · b + r) < 2 · b

remainder(a, b) = r =

a if a < b

a − b if a − b < b

remainder(a, 2 · b))

if remainder(a, 2 · b) < b

remainder(a, 2 · b)) − b otherwise

remainder_nonnegative

requires(ArchimedeanMonoid(T))

T remainder_nonnegative(T a, T b)

{

// Precondition: a > T (0) ∧ b > T (0)

if (a < b) return a;

if (a - b < b) return a - b;

a = remainder_nonnegative(a, b + b);

if (a < b) return a;

else return a - b;

}

The first comparison is not needed in the recursive calls and could

be eliminated by the introduction of a helper function

a − b < b serves as a guard to ensure b + b will not overflow

Complexity of remainder_nonnegative

Exercise

Determine the exact counts of different operations

Concept WeaklyHalvableMonoid

division by an integer, it has a division by 2

While general k-section of an angle by ruler and compass can not

be done, bisection is trivial

Divisibility by 2 gives an iterative version of remainder

Definition

WeaklyHalvableMonoid(T ) ⇒

ArchimedeanMonoid(T )

The partial unary operation half_nonnegative is defined on T +

where T + = {x ∈ T |x > T (0)}

For all a, b ∈ T +

a = b + b ⇒ half_nonnegative(a) = b

remainder_nonnegative_iterative 20

requires(WeaklyHalvableMonoid(T))

T remainder_nonnegative_iterative(T a, const T& b)

{

// Precondition: a > T (0) ∧ b > T (0)

if (a < b) return a;

T c = b;

while (a - c >= c) c = c + c;

a = a - c;

while (c != b) {

c = half_nonnegative(c);

if (c <= a) a = a - c;

}

return a;

}

20

Edsger W. Dijkstra attributes this algorithm to N.G. de Bruijn on page 13 of Notes

on Structured Programming, in: O.-J. Dahl, E. W. Dijkstra and C.A.R. Hoare.

Structured Programming. Academic Press, London and New York, 1972. The algorithm

also appears in: Niklaus Wirth. Systematic Programming: An Introduction.

Prentice-Hall, 1973, program 7.22, page 38.

Stepanov, McJones Elements of Programming March 14, 2008 284 / 880

Divisibility on an Archimedean monoid T

Definition

For a > T (0) and b > T (0), b divides a ⇔ remainder(a, b) = T (0)

Properties of divisibility on an Archimedean monoid

Lemma

In an Archimedean monoid T with positive x, a, b

b divides a ⇒ b 6 a

b > a ∧ x divides a ∧ x divides b ⇒ x divides (b − a)

x divides a ∧ x divides b ⇒ x divides remainder(a, b)

Concept DiscreteArchimedeanMonoid

Definition

DiscreteArchimedeanMonoid(T) ⇒

ArchimedeanMonoid(T )

(∃u ∈ T )(∀x ∈ T )x < u ⇒ ¬(T (0) < x)

Lemma

Every element of a discrete Archimedean monoid is divisible by the

unit

Greatest common divisor

Definition

The greatest common divisor of a and b, denoted by gcd(a, b), is a

divisor of a and b that is divisible by any other divisor of a and b

see later, for other structures

Note the definition does not depend on ordering but is expressed

strictly in terms of divisibility

Properties of greatest common divisor

Lemma

In an Archimedean monoid with positive x, a, b, the following hold

x divides a ∧ x divides b ⇒ x 6 gcd(a, b)

gcd(a, b) is unique

gcd(a, a) = a

subtractive_gcd

template <typename T>

requires(ArchimedeanMonoid(T))

T subtractive_gcd(T a, T b)

{

// Precondition: a > T (0) ∧ b > T (0)

while (true) {

if (b < a)

a = a - b;

else if (a < b)

b = b - a;

else

return a;

}

}

It is fairly certain that the algorithm had been known at least a

century before Euclid

21

Sir Thomas Heath, translator. Euclid’s Elements. Dover, 1956: Volume 2, Book VII,

Propositions 1 and 2, pages 296-300; Volume 3, Book X, Propositions 1-3, pages 14-22

Stepanov, McJones Elements of Programming March 14, 2008 290 / 880

Correctness of subtractive_gcd

Proof.

1 Each subtraction preserves the property of being divisible by any

divisor of the original a and b

2 If the variables become equal, they divide the original a and b

3 Therefore it returns a divisor of a and b that is divisible by any

other divisor

4 In other words, it returns the greatest common divisor

Termination of subtractive_gcd

Lemma

It always terminates for integers and rationals

Lemma

It does not always terminate for reals

Concept EuclideanMonoid

Definition

EuclideanMonoid(T ) ⇒

ArchimedeanMonoid(T )

For all a > T (0), b > T (0) ∈ T , subtractive_gcd(a, b) terminates

Lemma

Every discrete Archimedean monoid is Euclidean

Extending subtractive_gcd to zero

one of its arguments is zero

requires(EuclideanMonoid(T))

T subtractive_gcd_with_zero(T a, T b)

{

// Precondition: a > T (0) ∧ b > T (0) ∧ ¬(a = T (0) ∧ b = T (0))

while (true) {

if (b == T(0)) return a;

while (b < a) a = a - b;

if (a == T(0)) return b;

while (a < b) b = b - a;

}

}

Speeding up subtractive gcd

is equivalent to a call of slow_remainder

By using our (logarithmic) remainder algorithm, we can speed up

the case where a and b are very different in magnitude

fast_subtractive_gcd

requires(EuclideanMonoid(T))

T fast_subtractive_gcd(T a, T b)

{

// Precondition: a > T (0) ∧ b > T (0) ∧ ¬(a = T (0) ∧ b = T (0))

while (true) {

if (b == T(0)) return a;

a = remainder_nonnegative(a, b);

if (a == T(0)) return b;

b = remainder_nonnegative(b, a);

}

}

Generalizing the greatest common divisor algorithm

divisor is based on a particular remainder algorithm

If a structure has a properly-defined remainder, it does not have to

be a Euclidean monoid

Polynomials

Gaussian integers

Mathematicians refer to these other types as Euclidean domains

Our goal is to define a concept unifying Euclidean monoids and

Euclidean domains, so we can use the same gcd algorithm for both

Concept CommutativeSemiring

Definition

CommutativeSemiring(T ) ⇒

AdditiveMonoid(T )

MultiplicativeSemigroup(T ) ∧ CommutativeOperation(×)

For any a, T (0) × a = T (0)

The constant T (1) is defined

For any a , T (0), T (1) × a = a

For all a, b, c ∈ T , a × (b + c) = a × b + a × c

Examples of CommutativeSemiring

Example

Nonnegative integers constitute a commutative semiring

Concept Semimodule

Definition

Semimodule(T ) ⇒

AdditiveMonoid(T )

The type function Scalar(T ) is defined

CommutativeSemiring(Scalar(T ))

The binary infix function · : Scalar(T ) × T → T is defined

For all α, β ∈ Scalar(T ) and for all x, y ∈ T ,

α · (β · x) = (α × β) · x

(α + β) · x = α · x + β · x

α · (x + y) = α · x + α · y

Scalar(T )(1) · x = x

Examples of Semimodule

Example

The set {(a, b)} of two-dimensional vectors with nonnegative real

coefficients constitutes a semimodule over nonnegative integers

Polynomials with nonnegative integer coefficients constitute a

semimodule over nonnegative integers

Concept QRSemimodule

Definition

QRSemimodule(T) ⇒

Semimodule(T)

The partial binary operation remainder is defined on T

The partial binary infix function

quotient : T × T → Scalar(T ) is defined

For all a ∈ T , b , T (0) ∈ T ,

a = quotient(a, b) · b + remainder(a, b)

gcd for QRSemimodule

requires(QRSemimodule(T))

T gcd(T a, T b)

{

// Precondition: ¬(a = T (0) ∧ b = T (0))

while (true) {

if (b == T(0)) return a;

a = remainder(a, b);

if (a == T(0)) return b;

b = remainder(b, a);

}

}

Concept EuclideanSemimodule

Definition

EuclideanSemimodule(T) ⇒

QRSemimodule(T)

For all a, b ∈ T such that ¬(a = T (0) ∧ b = T (0)), gcd(a, b)

terminates

gcd

requires(EuclideanSemimodule(T))

T gcd(T a, T b)

{

// Precondition: ¬(a = T (0) ∧ b = T (0))

while (true) {

if (b == T(0)) return a;

a = remainder(a, b);

if (a == T(0)) return b;

b = remainder(b, a);

}

}

Concept EuclideanSemiring

Definition

EuclideanSemiring(T ) ⇒

CommutativeSemiring(T )

For all a, b ∈ T , a × b = T (0) ⇒ a = T (0) ∨ b = T (0)

A Euclidean function w : T → N is defined

For all a, b ∈ T , w(a × b) > w(a)

Partial binary operations remainder and quotient

are defined on T

For all a ∈ T , b , T (0) ∈ T ,

that gcd terminates

Stepanov, McJones Elements of Programming March 14, 2008 306 / 880

Euclidean semiring is Euclidean semimodule

This implies every Euclidean semiring is a Euclidean semimodule

This implies gcd can be used both for Euclidean monoids and the

traditional concept of Euclidean semirings 22 , such as polynomials

over reals

22

Technically, the traditional concept introduced by Noether and van der Waerden

is the Euclidean ring, but semirings suffice

Stepanov, McJones Elements of Programming March 14, 2008 307 / 880

Deriving a fast algorithm for quotient and remainder

Let n = 2q + p, where q = bn/2c and p = n mod 2

a = q · (2 · b) + (p · b + r) where (p · b + r) < 2 · b

remainder(a, b) = r =

if a < b

a

a − b if a − b < b

remainder(a, 2 · b)) if remainder(a, 2 · b) < b

remainder(a, 2 · b)) − b otherwise

quotient(a,

b) =

0 if a < b

1 if a − b < b

2 · quotient(a, 2 · b)) if remainder(a, 2 · b) < b

2 · quotient(a, 2 · b)) + 1 otherwise

quotient_remainder_nonnegative

requires(Integer(I) && ArchimedeanMonoid(T))

pair<I, T> quotient_remainder_nonnegative(T a, T b)

{

// Precondition: a > T (0) ∧ b > T (0)

if (a < b)

return pair<I, T>(I(0), a);

if (a - b < b)

return pair<I, T>(I(1), a - b);

pair<I, T> q = quotient_remainder_nonnegative(a, b + b);

I n = q.first + q.first;

a = q.second;

if (a < b)

return pair<I, T>(n, a);

else

return pair<I, T>(n + I(1), a - b);

}

quotient_nonnegative

know how to compute quotient without remainder

Most computer instruction sets include an instruction that

computes both quotient and remainder

It would be nice if programming languages provided a binding

for this instruction

template <typename I, typename T>

requires(Integer(I) && ArchimedeanMonoid(T))

I quotient_nonnegative(T a, T b)

{

return quotient_remainder_nonnegative<I>(a, b).first;

}

quotient_remainder_nonnegative_iterative

requires(Integer(I) && WeaklyHalvableMonoid(T))

pair<I, T> quotient_remainder_nonnegative_iterative(T a, const T& b)

{

// Precondition: a > T (0) ∧ b > T (0)

if (a < b) return pair<I, T>(I(0), a);

T c = b;

while (a - c >= c) c = c + c;

a = a - c;

I n = I(1);

while (c != b) {

n = n + n;

c = half_nonnegative(c);

if (c <= a) {

a = a - c;

n = n + I(1);

}

}

return pair<I, T>(n, a);

}

Requirements for extending quotient and remainder

to negative quantities

properties:

quotient(a, b) is an integer

a = b · quotient(a, b) + remainder(a, b)

|remainder(a, b)| < |b|

For any integer n, remainder(a, b) = remainder(a + n · b, b)

Problems with current implementations of quotient

and remainder

languages, in which quotient truncates toward zero 23

Truncation violates our fourth requirement 24

In addition, truncation is an inferior way of rounding because it

sends twice as many values to zero as to any other integer, thus

leading to a nonuniform distribution

23

For an excellent discussion of this and other problems, see: Raymond T. Boute.

The Euclidean Definition of the Functions div and mod. ACM Transactions on

Programming Languages and Systems, Volume 14, Number 2, April 1992, pages 127-144

24

This requirement is equivalent to the classical mathematical definition of

congruence: “If two numbers a and b have the same remainder r relative to the same

modulus k they will be called congruent relative to the modulus k (following Gauss).”

—P. G. L. Dirichlet, Lectures on Number Theory, American Mathematical Society, 1999

Stepanov, McJones Elements of Programming March 14, 2008 313 / 880

Adapting nonnegative remainder and quotient

The next two functions are adaptors that convert remainder and

quotient_remainder correctly defined for nonnegative inputs to

produce “correct” results for positive or negative inputs

remainder

requires(OrderedAdditiveGroup(T) && BinaryOperation(Op) && Domain(Op) == T)

T remainder(T a, T b, Op rem)

{

// Precondition: b , 0

T r;

if (a < T(0))

if (b < T(0)) {

r = -rem(-a, -b);

} else {

r = rem(-a, b); if (r != T(0)) r = b - r;

}

else

if (b < T(0)) {

r = rem(a, -b); if (r != T(0)) r = b + r;

} else {

r = rem(a, b);

}

return r;

}

quotient_remainder

requires(Integer(I) && OrderedAdditiveGroup(T) &&

HomogeneousFunction(Op) && Arity(Op) == 2 && Domain(Op) == T && Codomain(Op) == pair<I, T>)

pair<I, T> quotient_remainder(T a, T b, Op quo_rem)

{

// Precondition: b , 0

pair<I, T> q_r;

if (a < 0) {

if (b < 0) { q_r = quo_rem(-a, -b); q_r.second = -q_r.second; }

else {

q_r = quo_rem(-a, b);

if (q_r.second != 0) { q_r.second = b - q_r.second; q_r.first = q_r.first + 1; }

q_r.first = -q_r.first;

}

} else {

if (b < 0) {

q_r = quo_rem( a, -b);

if (q_r.second != 0) { q_r.second = b + q_r.second; q_r.first = q_r.first + 1; }

q_r.first = -q_r.first;

}

else

q_r = quo_rem( a, b);

}

return q_r;

}

Concept DiscreteArchimedeanSemiring

Definition

DiscreteArchimedeanSemiring(T ) ⇒

CommutativeSemiring(T )

ArchimedeanMonoid(T )

For all a, b, c ∈ T , a < b ∧ 0 < c ⇒ a × c < b × c

For all a ∈ T , a < T (1) ⇒ ¬(T (0) < a)

Concept NonnegativeDiscreteArchimedeanSemiring

Definition

NonnegativeDiscreteArchimedeanSemiring(T ) ⇒

NonnegativeDiscreteArchimedeanSemiring(T )

For all a ∈ T , T (0) 6 a

Concept DiscreteArchimedeanRing

Definition

DiscreteArchimedeanRing(T ) ⇒

DiscreteArchimedeanSemiring(T )

AdditiveGroup(T )

Univalent concepts

Definition

Two types T1 and T2 are isomorphic if it is possible to write conversion

functions from T1 to T2 and from T2 to T1 that preserve the procedures

and their semantics

Definition

A concept is univalent if any types satisfying it are isomorphic

satisfying it are isomorphic to N, the natural numbers 25

DiscreteArchimedeanRing is univalent; types satisfying it are

isomorphic to Z, the integers

25

We follow Peano and include 0 in the natural numbers: Giuseppe Peano.

Formulario Mathematico, Edizioni Cremonese, Roma, 1960, page 27

Stepanov, McJones Elements of Programming March 14, 2008 320 / 880

Computer integer types

partial implementations of natural numbers and integers

While providing a detailed formal specification for computer

integers lies outside the scope of this book, there are some

important points we must make

Few if any programming languages provide access to the full

power of hardware operations for addition, subtraction,

multiplication, quotient, and remainder

We sketch appropriate interfaces

Bounded unsigned and signed integer types

Definition

A bounded unsigned integer type, Un , where n = 8, 16, 32, 64, . . ., is an

unsigned integer type capable of representing a value in the interval

[0, 2n )

Definition

A bounded signed integer type, Sn , where n = 8, 16, 32, 64, . . ., is a

signed integer type capable of representing a value in the interval

[−2n−1 , 2n−1 )

Operations for bounded unsigned and signed types

within a single type

Arithmetic operations such as multiplication and addition return

results that are larger than the type of their operands

While computer instructions return full results, a programmer in a

higher-level language has no access to them

Extended operations such as the ones below should be provided

sum_extended : Un × Un × U1 → U1 × Un

difference_extended : Un × Un × U1 → U1 × Un

product_extended : Un × Un → U2n

quotient_remainder_extended : U2n × Un → Un × Un

Arithmetic in a programming language

shows the functionality that should be encompassed

A good abstraction of these instruction sets is provided by

Knuth’s MMIX 26

26

Donald E. Knuth. The Art of Computer Programming, Volume 1, Fascicle 1, MMIX : A

RISC Computer for the New Millenium, Addison-Wesley, 2005, pages 1-28

Stepanov, McJones Elements of Programming March 14, 2008 324 / 880

Conclusions

into a seamless whole by describing algorithms in abstract terms

and adjusting theories to fit algorithmic requirements

If this chapter seems too mathematical, it should be noted that the

algorithms and mathematics in it are modern restatements of

results that are thousands of years old

References

Donald Knuth.

The Greatest Common Divisor, and Division of Polynomials.

The Art of Computer Programming, Volume 2, 3rd edition, 1998,

Sections 4.5.2 and 4.6.1, pages 333-356 and 420-439.

Exhaustive coverage of both Euclidean and Stein (binary) gcd.

Pierre Samuel.

About Euclidean Rings.

Journal of Algebra, Volume 19, 1971, pages 282-301.

Exhaustive and relatively elementary treatment of Euclidean rings.

binary_gcd_nonnegative

In 1961, Josef Stein discovered the following gcd algorithm 27 :

template <typename T>

requires(Integer(T))

T binary_gcd_nonnegative(T a, T b)

{

if (is_zero(a)) return b;

if (is_zero(b)) return a;

int d = 0;

while (is_even(a) && is_even(b)) {

halve_nonnegative(a); halve_nonnegative(b); d = d + 1;

}

while (is_even(a)) halve_nonnegative(a);

while (is_even(b)) halve_nonnegative(b);

while (true)

if (a < b) {

b = b - a; do { halve_nonnegative(b); } while (is_even(b));

} else if (b < a) {

a = a - b; do { halve_nonnegative(a); } while (is_even(a));

} else return binary_scale_up_nonnegative(a, d);

}

27

Josef Stein. Computational problems associated with Racah algebra. J. Comput.

Phys., Volume 1, 1967, pages 397-405

Stepanov, McJones Elements of Programming March 14, 2008 327 / 880

Generalizations of binary_gcd_nonnegative

dependent on binary integers, it is actually much deeper

The key observation is that 2 is the smallest integer prime, and the

only nonzero remainder mod2 is 1, a unit or invertible ring

element

It is possible to generalize it to other domains by using smallest

primes in those domains, such as the monomial x for polynomials

over reals or 1 + i for Gaussian integers

Polynomials See Knuth (Exercise 4.6.1.6, page 435 and

Solution, page 673)

Gaussian integers See Weilert

Other algebraic integer rings See Damgård and Frandsen,

and Agarwal and Frandsen

Project

Project

Find the correct abstract setting for binary binary_gcd_nonnegative

(Stein domain)

Additional references for binary_gcd_nonnegative

Andre Weilert.

(1+ i)-ary GCD Computation in Z[i] as an Analogue of the Binary

GCD Algorithm.

J. Symbolic Computation (2000) 30, pages 605-617.

Ivan Bjerre Damgård and Gudmund Skovbjerg Frandsen.

Efficient algorithms for GCD and cubic residuosity in the ring of

Eisenstein integers.

Proceedings of the 14th International Symposium on Fundamentals of

Computation Theory, Lecture Notes in Computer Science 2751,

Springer-Verlag (2003), pages 109-117.

Saurabh Agarwal and Gudmund Skovbjerg Frandsen.

Binary GCD Like Algorithms for Some Complex Quadratic Rings.

ANTS 2004, pages 57-71.

Contents I

1 Introduction

2 Foundations

5 Orderings

6 Combining concepts

Contents II

Memory and dereferencing

Actions

Iterators

Ranges

Forward iterators

Bidirectional iterators

Indexed iterators

Random access iterators

Conclusions

Reference

Project

9 Rotations

Contents III

10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

Contents IV

17 Index

Memory

Intuition

Memory is a set of locations, each with an address and a value

Getting or setting the value given an address is “fast”

The association of a value with a location is changeable

Examples

The physical and virtual address spaces of a computer

The locations visited by an algorithm

The locations owned by a data structure

Concept Readable

Definition

Readable(T ) ⇒

The type function ValueT ype(T ) is defined

Regular(ValueT ype(T ))

source : T → ValueT ype(T ) is defined a

a

source returns its result by value, by reference, or by constant reference

Concept Writable

Definition

Writable(T ) ⇒

The type function ValueT ype(T ) is defined

Regular(ValueT ype(T ))

If x ∈ T and v ∈ ValueT ype(T ), then sink(x) ← v is defined

No other use of sink(x) can be justified by the concept Writable,

although other uses may be supported by a specific type modeling

Writable

For a particular state of an object x, only a single assignment to

sink(x) can be justified by the concept Writable; a specific type

might provide a protocol allowing subsequent assignments to

sink(x)

is_aliased

requires(Writable(T) && Readable(U) && ValueType(T) == ValueType(U))

bool is_aliased(const T& x, const U& y)

{

// return true ⇔ immediately after executing sink(x) ← v, source(y) = v

// For many types, this works:

typedef const ValueType(T)* P;

return P(&sink(x)) == P(&source(y));

}

Concept Mutable

Definition

Mutable(T ) ⇒

Readable(T ) ∧ Writable(T )

For all x ∈ T , is_aliased(x, x)

Concept Dereferenceable

Definition

Dereferenceable(T ) ⇒

Readable(T ) ∨ Writable(T ) ∨ Mutable(T )

though it is not used in programs (since it’s not clear which of

source or sink could be used)

Extending Dereferenceable functions to all regular types

Reflection

It is useful if dereferencing an object x that doesn’t refer to another

object returns x itself

Therefore we assume that unless otherwise defined,

ValueT ype(T ) = T and source and sink return the object to

which they are applied

Concept Action

Definition

Action(T ) ⇒

ProperProcedure(T )

Arity(T ) = 1

T takes its argument by reference

Duality of actions and transformations

versa

The following two functions demonstrate this equivalence

The complexity of an action when implemented independently of

the corresponding transformation could be smaller

Example: interchange the first and last elements of a sequence

transformation_from_action

requires(Action(A))

struct transformation_from_action

{

A a;

transformation_from_action() {}

transformation_from_action(const A& a) : a(a) {}

Domain(A) operator()(Domain(A) x) { a(x); return x; }

};

action_from_transformation

requires(Transformation(F))

struct action_from_transformation

{

F f;

action_from_transformation() {}

action_from_transformation(const F& f) : f(f) {}

void operator()(Domain(F)& x) { x = f(x); }

};

Concept RegularAction

Definition

RegularAction(T ) ⇒

Action(T )

RegularFunction(transformation_from_actionT )

Concept AmortizedConstantTime

Definition

The amortized complexity of an operation is the complexity averaged

over a worst-case sequence of operations a

a

For an extensive treatment of amortized complexity, see: Robert Endre Tarjan.

Amortized Computational Complexity. SIAM Journal on Algebraic and Discrete

Methods, Volume 6, Number 2, April 1985, pages 306-318

Definition

AmortizedConstantTime(T ) ⇒

ProperProcedure(T )

The amortized time complexity of it is constant and “small”

subject for future research

Stepanov, McJones Elements of Programming March 14, 2008 347 / 880

Concept Iterator 28

Definition

Iterator(T ) ⇒

Regular(T )

The type function DistanceT ype(T ) is defined

Integer(DistanceT ype(T ))

The prefix operator ++ is defined on T

Action(++)

AmortizedConstantTime(++)

28

Our treatment of iterators departs significantly from that in the STL

Stepanov, McJones Elements of Programming March 14, 2008 348 / 880

Iterator protocols

If there are two copies of an iterator and one is incremented, the

other may become invalid

Thus an iterator is too weak to be used in a multipass algorithm

Assignment to a sink is not idempotent: a call to ++ must separate

two assignments to an iterator

The asymmetry between readable and writable iterators is

intentional

The ability to read from source more than once allows us to write

simple, useful functions like find_if

No corresponding benefits seem to accrue in the case of sink

Examples of Iterator

An iterator where ++ advances an output stream

An iterator on a singly-linked list

An iterator on a doubly-linked list

An iterator on a one-dimensional array

int∗

int

successor and - for Iterator

requires(Iterator(I))

I successor(I f)

{

++f;

return f;

}

requires(Iterator(I))

DistanceType(I) operator-(I l, I f)

{

return distance(f, l, successor<I>);

}

+= and + for Iterator

requires(Iterator(I))

void operator+=(I& f, DistanceType(I) n)

{

typedef DistanceType(I) N;

while (n != N(0)) {

++f;

n = n - N(1);

}

}

requires(Iterator(I))

I operator+(I f, DistanceType(I) n)

{

f += n;

return f;

}

Orbit of successor

any iterator

In this book, we only deal with terminating iterator orbits

Therefore, if, for i > 0, successori (x) is defined, it is not equal to x

The natural ordering on an orbit under successor

of x under successor defined by

such as preconditions and postconditions of algorithms

For many pairs of values of an iterator type, ≺ is not defined, so

there is often no effective way to write code implementing ≺

There is no efficient way to determine if one node precedes another

in a linked structure (the nodes might not even be linked together)

Sequences of iterators

sequences and subsequences of iterators

To describe the input and/or output of an algorithm

To describe ranges in a data structure

It is useful to describe a (possibly empty) sequence of iterators

starting at a particular iterator

Example

Binary search looks for the sequence of iterators whose values are

equal to a given value

This sequence is empty if there are no such values

The location of this empty sequence carries information: the values

in the sequence approximating the given value

Ranges

iterator and a nonnegative integer

It is often convenient to specify a sequence by specifying a starting

iterator together with another one that bounds the sequence

A specification of a such a located sequence is called a range

Ranges can be either counted or bounded, and either semi-open or

closed

Different kinds of ranges

Definition

A semi-open counted range Ji, nM, where n > 0 is an integer, denotes the

sequence of iterators {j|i j ≺ successorn (i)}

Definition

A closed counted range Ji, nK, where n > 0 is an integer, denotes the

sequence of iterators {k|i k successorn−1 (i)}

Definition

A semi-open bounded range [i, j) denotes the sequence of iterators

{k|i k ≺ j}

Definition

A closed bounded range [i, j] denotes the sequence of iterators

{k|i k j}

Stepanov, McJones Elements of Programming March 14, 2008 357 / 880

Terminology for iterators in a range

Definition

An iterator f in a range [i, j) is called the first if f = i ∧ f , j

An iterator l in a range [i, j) is called the limit if l = j

An iterator k in a range [i, j) is called the lasta if successor(k) = j

Otherwise an iterator in the range is called not last

a

The terminology distinguishing limit and last was suggested by John Banning

Size of a range

Definition

The size of a counted range Ji, nM or Ji, nK is n

The size of a semi-open bounded range [i, j) is

distance(i, j, successor)

The size of a closed bounded range [i, j] is

distance(i, j, successor) + 1

algorithms even when it cannot be effectively computed because

of an iterator too weak for a multipass algorithm

Empty ranges

Definition

An empty semi-open range is specified by Ji, 0M or [i, i) for some

iterator i

There are no empty closed ranges

Definition space of ++ on ranges

Lemma

++ and successor are defined for every iterator in a semi-open range,

and for every iterator except the last in a closed range

Readable, writable, and mutable ranges

Definition

A range r is readable, writable, or mutable if, correspondingly, source,

sink, or both of them are defined on all the iterators in the range

Type requirements of copy

requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&

ValueType(I) == ValueType(O))

O copy(I f, I l, O o);

Iterator(I) ensures ++ is defined

Writable(O) ensures sink is defined

Iterator(O) ensures ++ is defined

ValueT ype(I) = ValueT ype(O) ensures assignment is defined

Precondition and postcondition of copy

requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&

ValueType(I) == ValueType(O))

O copy(I f, I l, O o);

The output range must be writable and of at least

the size of the input range

If the ith iterator in the input range is aliased to the

jth iterator of the output, then i 6 j (informally,

every value is used before it is overwritten)

Postcondition The sequence of values in the output range is equal

to the sequence of original values in the input range

Implementation of copy

requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&

ValueType(I) == ValueType(O))

O copy(I f, I l, O o)

{

while (f != l) {

sink(o) = source(f);

++f;

++o;

}

return o;

}

known to the caller, who might find it useful

It is worth the small constant time to return it

Example using copy

requires(Writable(O) && Iterator(O) && Integer(ValueType(O)))

O iota(ValueType(O) n, O o) // like APL ι

{

return copy(ValueType(O)(0), n, o);

}

copy_bounded

requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&

ValueType(I) == ValueType(O))

pair<I, O> copy_bounded(I f, I l, O o_f, O o_l)

{

while (f != l && o_f != o_l) {

sink(o_f) = source(f);

++f;

++o_f;

}

return pair<I, O>(f, o_f);

}

While the ends of both ranges are known to the caller, returning

the pair allows determining which range is smaller and where in

the larger range copying stopped

copy_n

requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&

ValueType(I) == ValueType(O))

pair<I, O> copy_n(I f, DistanceType(I) n, O o)

{

typedef DistanceType(I) N;

while (n != N(0)) {

sink(o) = source(f);

++f;

++o;

n = n - N(1);

}

return pair<I, O>(f, o);

}

copy_k

requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&

ValueType(I) == ValueType(O))

struct copy_k

{

void operator()(I& f, O& r)

{

copy_k<k - 1, I, O>(f, r);

sink(r) = source(f);

++f; ++r;

}

};

struct copy_k<0, I, O> {

void operator()(I&, O&) { }

};

copy_n_unrolled

requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&

ValueType(I) == ValueType(O))

pair<I, O> copy_n_unrolled(I f, DistanceType(I) n, I r)

{

typedef DistanceType(I) N;

const int k = 4; // unroll factor

while (n >= N(k)) {

copy_k<k, I, O>()(f, r);

n = n - N(k);

}

return copy_n(f, n, r);

}

Concept ForwardIterator

Definition

ForwardIterator(T ) ⇒

Iterator(T )

RegularAction(++)

new operations

Examples of ForwardIterator

An iterator on a doubly-linked list

An iterator on a one-dimensional array

int∗

int

Advantages of regularity of ++

Reflection

ForwardIterator allows

algorithms that maintain more than one iterator into a range

multipass algorithms

Concept BidirectionalIterator

Definition

BidirectionalIterator(T ) ⇒

ForwardIterator(T )

The prefix operator -- and the corresponding transformation

predecessor are defined

RegularAction(--)

AmortizedConstantTime(--)

For any iterator k in a semi-open range [i, j):

k , j ⇒ predecessor(successor(k)) = k

k , i ⇒ successor(predecessor(k)) = k

Examples of BidirectionalIterator

An iterator on a one-dimensional array

int∗

int

-= and - for BidirectionalIterator

requires(BidirectionalIterator(I))

void operator-=(I& f, DistanceType(I) n)

{

typedef DistanceType(I) N;

while (n != N(0)) {

--f;

n = n - N(1);

}

}

requires(BidirectionalIterator(I))

I operator-(I f, DistanceType(I) n)

{

f -= n;

return f;

}

Using copy with overlapping ranges

source range, copy cannot be used because its aliasing

precondition is violated

copy_backward

ith iterator in the input range is aliased to the jth iterator of the

output, then j 6 i

requires(Readable(I) && BidirectionalIterator(I) &&

Writable(O) && BidirectionalIterator(O) &&

ValueType(I) == ValueType(O))

O copy_backward(I f, I l, O o)

{

while (f != l) {

--l;

--o;

sink(o) = source(l);

}

return o;

}

copy_backward_n

requires(Readable(I) && BidirectionalIterator(I) &&

Writable(O) && BidirectionalIterator(O) &&

ValueType(I) == ValueType(O))

pair<I, O> copy_backward_n(I l, DistanceType(I) n, O o)

{

typedef DistanceType(I) N;

while (n != N(0)) {

--l;

--o;

sink(o) = source(l);

n = n - N(1);

}

return pair<I, O>(l, o);

}

Concept IndexedIterator

Definition

IndexedIterator(T ) ⇒

ForwardIterator(T )

The mutating assignment += is defined on T × DistanceT ype(T )

- : T × T → DistanceT ype(T ) is defined

AmortizedConstantTime(+=)

AmortizedConstantTime(-)

primitive

+ for IndexedIterator

terms of += for Iterator becomes constant-time for IndexedIterator

Examples of IndexedIterator

int∗

int

copy_k_indexed

template <int k, typename I, typename O>

requires(Readable(I) && IndexedIterator(I) &&

Writable(O) && IndexedIterator(O) &&

ValueType(I) == ValueType(O))

struct basic_copy_k_indexed {

void operator()(I f, O o)

{

basic_copy_k_indexed<k - 1, I, O>()(f, o);

sink(o + (k - 1)) = source(f + (k - 1));

}

};

struct basic_copy_k_indexed<0, I, O>

{ void operator()(I, O) { } };

requires(Readable(I) && IndexedIterator(I) &&

Writable(O) && IndexedIterator(O) &&

ValueType(I) == ValueType(O))

void copy_k_indexed(I& f, O& o) {

basic_copy_k_indexed<k, I, O>()(f, o);

f += k; o += k;

}

Stepanov, McJones Elements of Programming March 14, 2008 383 / 880

copy_n_unrolled_indexed

requires(Readable(I) && IndexedIterator(I) &&

Writable(O) && IndexedIterator(O) &&

ValueType(I) == ValueType(O))

pair<I, O> copy_n_unrolled_indexed(I f, DistanceType(I) n, O o)

{

typedef DistanceType(I) N;

const int k = 4; // unroll factor

while (n >= N(k)) {

copy_k_indexed<k, I, O>(f, o);

n = n - N(k);

}

return copy_n(f, n, o);

}

Out-of-order execution

execution of copy

A compiler cannot achieve this because it cannot determine that

the source and destination ranges do not alias

Programmers know when this is the case, and should be able to

specify when such interleaved execution is legitimate

copy_parallel for disjoint ranges

input and output ranges do not alias

A special forall executes its body for all values of its iteration

variable in arbitrary order, possibly concurrently

template <typename I, typename O>

requires(Readable(I) && IndexedIterator(I) &&

Writable(O) && IndexedIterator(O) &&

ValueType(I) == ValueType(O))

void copy_parallel(I f, DistanceType(I) n, O o)

{

typedef DistanceType(I) N;

forall(N i, N(0), n) {

sink(o + i) = source(f + i);

}

}

Concept RandomAccessIterator (1 of 2)

RandomAccessIterator(T ) ⇒

BidirectionalIterator(T ) ∧ IndexedIterator(T )

StrictTotallyOrdered(T )

The type function DifferenceT ype(T ) is defined

Integer(DifferenceT ype(T ))

DifferenceT ype(T ) is large enough to contain distances and their

negations

The mutating assignments += and -= are defined on

T × DifferenceT ype(T )

+ : T × DifferenceT ype(T ) → T is defined

- : T × DifferenceT ype(T ) → T is defined

- : T × T → DifferenceT ype(T ) is defined

Concept RandomAccessIterator (2 of 2)

RandomAccessIterator(T ) ⇒

AmortizedConstantTime(<)

AmortizedConstantTime(-=)

AmortizedConstantTime(- : T × DifferenceT ype(T ) → T )

+=, -=, +, and - : T × DifferenceT ype(T ) → T accept negative

values for their second argument

AmortizedConstantTime(- : T × T → DifferenceT ype(T ))

- : T × T → DifferenceT ype(T )) accepts iterators in either order

Axioms for RandomAccessIterator

Exercise

Write an appropriate set of axioms relating the operations to each other

Examples of RandomAccessIterator

int∗

int

Equivalence of

RandomAccessIterator and IndexedIterator

Theorem

For any function defined on a range of random access iterators, there is

another function defined on indexed iterators with the same

asymptotic complexity

Proof of the equivalence

U ≡ DistanceT ype(I)

W ≡ sign_extendedU

appropriate integer operations, we rewrite the function with these

substitutions:

Replace With

DifferenceType(I) W

i < j i - f < j - f

i += n when n < 0 i = f + ((i - f) - U(-n))

i -= n i += -n

i - j W(i - f) - W(j - f)

Reflection on RandomAccessIterator and IndexedIterator

in any context in which the beginning of ranges are known

copy_backward does not satisfy this requirement!

In practice we have found there is no performance penalty for

using the weaker concept

copy_backward_n_indexed

copy_backward_n_indexed, which is often just as useful, is

realizable

template <typename I, typename O>

requires(Readable(I) && IndexedIterator(I) &&

Writable(O) && IndexedIterator(O) &&

ValueType(I) == ValueType(O))

void copy_backward_n_indexed(I f, DistanceType(I) n, O o)

{

typedef DistanceType(I) N;

while (n != N(0)) {

n = n - N(1);

sink(o + n) = source(f + n);

}

}

Relationships between iterator concepts

BI

It FI RI

II

Conclusions

Abelian groups, totally-ordered groups, and Archimedean groups

It also generates concepts describing the fundamental notion of

computer science, iterating through a data structure

We have used three types of refinement, by adding

an operation

an axiom

a tighter complexity requirement

Reference

The Standard Template Library.

HP Laboratories Technical Report 95-11(R.1), November 14, 1995.

Introduced different categories of iterators and their axioms.

Intuition for SegmentedIterator

within certain ranges or segments

hash tables

adjacency list representation of graphs

STL-like deques

It is possible to optimize many algorithms for such structures by

transforming inner loops into nested loops:

a loop over segments

a loop within a segment

This results in a new dimension of the classification of iterators:

homogeneous

segmented

Concept SegmentedIterator (1 of 2)

Definition (requirements, signatures, definition spaces)

SegmentedIterator(T) ⇒

ForwardIterator(T )

The type function SegmentIterator(T ) is defined

ForwardIterator(SegmentIterator(T ))

begin : T → SegmentIterator(T ) is defined

end : T → SegmentIterator(T ) is defined

+ : T × SegmentIterator(T ) ⇒ T is defined

AmortizedConstantTime(begin)

AmortizedConstantTime(end)

AmortizedConstantTime(+)

For any i ∈ [f, l], begin, end, and + are defined

For any w ∈ [begin(i), end(i)], i + w is defined

Concept SegmentedIterator (2 of 2)

Definition (axioms)

SegmentedIterator(T) ⇒

If

[f, l) is a range of segmented iterators

i ∈ [f, l)

w is a segment iterator in [begin(i), end(i))

then the following hold:

1 is_aliased(i, begin(i))

2 begin(i + w) = w

3 i + begin(i) = i

4 i + (begin(i) + 1) = i + 1

5 begin(i) + 1 , end(i) ⇒ begin(i) + 1 = begin(i + 1)

6 begin(i) , end(i)

copy_from_segmented

requires(Readable(I) && SegmentedIterator(I) &&

Writable(O) && Iterator(O) &&

ValueType(I) == ValueType(O))

O copy_from_segmented(I f, I l, O o)

{

while (f + end(f) != l + end(l)) {

// f and l are in different segments

o = copy(begin(f), end(f), o);

f = f + end(f);

}

// f and l are in the same segment

return copy(begin(f), begin(l), o);

}

Project

STL-like deque

SGI STL-like hashed containers

Produce segmented iterator versions of suitable (STL) algorithms

Analyze if the axioms are independent, consistent, and complete

Contents I

1 Introduction

2 Foundations

5 Orderings

6 Combining concepts

Contents II

8 Permutations and rearrangements

Permutations

Permutation groups

Cycle decomposition of a permutation

Rearrangements

Underlying type

Rearranging arbitrary cycles

Reverse permutation

Reverse algorithms

Category dispatch

Conclusions

Reading

9 Rotations

Contents III

10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

Contents IV

17 Index

Onto functions

Definition

A (regular) function f is onto if for all y ∈ Codomain(F), there exists

x ∈ Domain(R) such that y = f(x)

One-to-one functions

Definition

A (regular) function f is one-to-one if for all x, x 0 ∈ Domain(F),

f(x) = f(x 0 ) ⇒ x = x 0

Permutation

Definition

A permutation is a transformation on a finite domain that is one-to-one

and onto

Example of a permutation on [0, 6)

p(0) = 5

p(1) = 2

p(2) = 4

p(3) = 3

p(4) = 1

p(5) = 0

Identity permutation

Definition

A fixed point of a transformation is an element x such that f(x) = x

Definition

The identity permutation on a set S, identityS , maps each element of S to

itself; every element in S is a fixed point of identityS

Composition of permutations

Definition

If p and q are two permutations on a set S, the composition q ◦ p takes

x ∈ S to q(p(x))

Lemma

The composition of permutations is a permutation

Lemma

Composition of permutations is associative

Inverse of a permutation

Lemma

For every permutation p on a set S, there is an inverse permutation p−1

such that p−1 ◦ p = p ◦ p−1 = identityS

Permutation group

The permutations on a set form a group under composition

Lemma

Every group is a subgroup of a permutation group of its elements

where every permutation in the subgroup is generated by multiplying

all the elements by an individual element

Example

× 1 2 3 4

1 1 2 3 4

2 2 4 1 3

3 3 1 4 2

4 4 3 2 1

Multiplication mod 5:

Every row and column of the multiplication table is a permutation

Stepanov, McJones Elements of Programming March 14, 2008 414 / 880

Cycles in a permutation

Definition

A cycle is a circular orbit within a permutation

Definition

A trivial cycle is one with a cycle size of 1

The element in a trivial cycle is a fixed point

Lemma

Every element in a permutation of a finite set belongs to a unique cycle

Example of cycle decomposition

p(0) = 5

p(1) = 2

p(2) = 4

p(3) = 3

p(4) = 1

p(5) = 0

p = (0 5)(1 2 4)(3)

Structure of cycle decomposition

Lemma

Any permutation of a set with n elements contains k 6 n disjoint cycles

Each cycle is itself a permutation of this set, called a cyclic permutation

Disjoint cyclic permutations commute

Every permutation can be represented as a product of its cycles

The inverse of a permutation is the product of the inverses of its cycles

Finite sets

Definition

A finite set S of size n is a set for which there exists a pair of functions

chooseS : [0, n) → S

indexS : S → [0, n)

satisfying

indexS (chooseS (i)) = i

Index permutation of a permutation

Definition

If p is a permutation on a finite set S of size n, there is a corresponding

index permutation p 0 on [0, n) defined as

Lemma

p(x) = chooseS (p 0 (indexS (x)))

index permutations

Rearrangement

Definition

A rearrangement is an algorithm that rearranges the elements of a

mutable range to satisfy a given postcondition

Classifying rearrangements

characteristics:

Characterization of the postcondition

Postcondition kind

Stability

Implementation constraints

Iterator requirements

Mutative versus copying

Complexity

Time

Space

Postcondition kind

position and not on the value itself; for example,

“reverse the range”

bin-based The destination of a value depends only on the result of

applying a k-way bin function to that value, and not

directly on the value itself; for example, “move bad

values before good ones”

A useful subcase is predicate-based algorithms,

where k = 2

ordering-based The destination of a value depends only on the outcome

of applications of an ordering to pairs of values in the

range; for example, “put the smallest value first”

Stability

Definition

A rearrangement is stable if it respects the original order of the range to

the maximal extent possible while satisfying the postcondition

Examples

Stable index partition Move all the elements at even positions in the

sequence before the elements at odd positions,

keeping the original order in both groups

Stable partition Move all the elements with even values before

those with odd values, keeping the original

order in both groups

Stable sort Sort pairs of integers by their first components

so pairs with equal first components remain in

the original order

Iterator requirements

For the same problem, there are often different algorithms for

different iterator requirements

Examples

reverse_forward

reverse_bidirectional

reverse_indexed

Mutative versus copying

Definition

Rearrangements as we have defined them are mutative

Definition

A rearrangement is copying if it sets a writable range to a rearranged

copy of a readable range in way that satisfies a given postcondition

A copying rearrangement could always be obtained by composing

copy with the corresponding mutative rearrangement

Often, however, there are faster algorithms that rearrange while

copying

Space complexity

Definition

A mutative algorithm is in place (or in situ) if it uses an amount of

additional space that is (poly-)logarithmic in the size of the input

Definition

A memory-adaptive algorithm uses as much additional space as it can

acquire to maximize performance

A small percentage of additional space, while theoretically

“linear,” can lead to a large performance improvement

Definition

An algorithm with buffer requires the caller to provide a buffer to be

used by the algorithm

Time complexity

constant k

n log n algorithms arise

Determining the final position may require collecting additional

information, as in sorting

Absence of random access on some iterators requires multipass

algorithms, as with in-place random shuffle for forward iterators

cycle_2

requires(Mutable(I) && Iterator(I))

void cycle_2(I x, I y)

{

ValueType(I) t = source(x);

sink(x) = source(y);

sink(y) = t;

}

Problems with cycle_2

std::vector copies element-by-element instead of interchanging

the headers

It could throw an exception due to unnecessary resource allocation

We should exploit the fact it is needed for implementing mutative

rearrangements, which should not need to construct or destroy

any objects, but just move them around

We will address the issue in Chapters 12 with a notion of an

underlying type

For the time being, imagine that the value types of iterators are

built-in types or C-style structs

cycle_left_3

requires(Mutable(I) && Iterator(I))

void cycle_left_3(I x, I y, I z)

{

ValueType(I) t = source(x);

sink(x) = source(y);

sink(y) = source(z);

sink(z) = t;

}

To-permutation and from-permutation

Definition

Every rearrangement corresponds to two permutations of its

range

A to-permutation mapping an iterator i to the iterator pointing to the

destination of the element at i

A from-permutation mapping an iterator i to the iterator pointing to

the origin of the element moved to i

These two permutations are inverses of each other

do_cycle_from

requires(Mutable(I) && ForwardIterator(I) &&

Transformation(P) && Domain(P) == I)

void do_cycle_from(I i, P p)

{

// Precondition: p is a from-permutation of the range containing i

ValueType(I) t = source(i);

I f = i;

I n = p(i);

while (n != i) {

sink(f) = source(n);

f = n;

n = p(n);

}

sink(f) = t;

}

Strict lower bound for rearrangement

Theorem

The minimum number of assignments required for a rearrangement is

n + cN − cT , where n is the number of elements, cN the number of

nontrivial cycles, and cT the number of trivial cycles

Proof of strict lower bound for rearrangement

a cycle of p of length > 1

The rearrangement must include an assignment of a value of C to

each memory location of C

Since such an assignment overwrites the value at that location,

there must be an additional assignment of some value of C to a

location outside C

Since the cycles are disjoint, these additional assignments are

distinct, so the total number of assignments must be at least

n + cN − cT , where n is the number of elements of p, cN the

number of nontrivial cycles, and cT the number of trivial cycles29

do_cycle_from (applied to nontrivial cycles) make this bound

strict

29

The idea for the proof was provided to us by John Wilkinson

Stepanov, McJones Elements of Programming March 14, 2008 434 / 880

do_cycle_to

Exercise

Implement do_cycle_to and compare the number of assignments it

performs to do_cycle_from

Reverse permutation

Definition

The permutation p on a finite set with n elements defined by an index

permutation p(i) = (n − 1) − i is called the reverse permutation of the set

the number of trivial cycles is n mod 2

bn/2c is the largest possible number of nontrivial cycles in a

permutation

Reverse rearrangement

Definition

The reverse rearrangement of a range is the rearrangement induced by

the reverse permutation

n + cN − cT = 3bn/2c

reverse_n_indexed

requires(Mutable(I) && IndexedIterator(I))

void reverse_n_indexed(I f, DistanceType(I) n)

{

typedef DistanceType(I) N;

N i(0);

while (i < n / N(2)) {

cycle_2(f + i, f + ((n - N(1)) - i));

i = i + N(1);

};

}

If the algorithm is used with forward or bidirectional iterators, it

performs a quadratic number of iterator increments

Since all the cycles are disjoint, this code benefits from forall

Return value of reverse

of elements that were not moved

The middle element when the size of the range is odd

The empty range between the two “middle” elements when the

size of the range is even

However, we do not know of any example when it is useful and,

therefore, return void

reverse_bidirectional

requires(Mutable(I) && BidirectionalIterator(I))

void reverse_bidirectional(I f, I l)

{

while (f != l && f != predecessor(l)) {

--l;

cycle_2(f, l);

++f;

}

}

reverse_n_bidirectional

requires(Mutable(I) && BidirectionalIterator(I))

void reverse_n_bidirectional(I f, I l, DistanceType(I) n)

{

// Precondition: n 6 l − f

typedef DistanceType(I) N;

N i(0);

while (i < n / N(2)) {

--l;

cycle_2(f, l);

++f;

i = i + N(1);

}

}

reverse_indexed

requires(Mutable(I) && IndexedIterator(I))

void reverse_indexed(I f, I l)

{

reverse_n_indexed(f, l - f);

}

Intuition for divide and conquer reverse algorithm

2 Reverse each part

3 Interchange the parts

Illustration of divide and conquer reverse algorithm

[c b a] d [g f e] [j i h] k [n m l]

[g f e d c b a] [n m l k j i h]

n m l k j i h g f e d c b a

swap_ranges_n

requires(Mutable(I1) && Iterator(I1) &&

Mutable(I2) && Iterator(I2) &&

ValueType(I1) == ValueType(I2) &&

Integer(N))

pair<I1, I2> swap_ranges_n(I1 f1, I2 f2, N n)

{

while (n != N(0)) {

cycle_2(f1, f2);

++f1;

++f2;

n = n - N(1);

};

return pair<I1, I2>(f1, f2);

}

reverse_n_recursive

requires(Mutable(I) && ForwardIterator(I))

I reverse_n_recursive(I f, DistanceType(I) n)

{

typedef DistanceType(I) N;

const N h = n / N(2);

const N r = n - N(2) * h;

if (h == N(0))

return f + n;

I m = reverse_n_recursive(f, h);

m += r;

I l = reverse_n_recursive(m, h);

swap_ranges_n(f, m, h);

return l;

}

Correctness of reverse_n_recursive

Lemma

The reverse permutation on [0, n) is the only permutation satisfying

i < j ⇒ p(j) < p(i)

2 The recursive calls inductively establish that the condition holds

within each half

3 swap_ranges_n reestablishes the condition between the halves

(and the skipped middle element, if any)

Complexity of reverse_n_recursive

Lemma

Pblog nc

For a range of length n = i=0 ai 2i , where ai is the ith digit in the

binary representation of n, the number of assignments equals

3 Pblog nc

2 i=0 ai i2i

reverse_n_forward

requires(Mutable(I) && ForwardIterator(I))

void reverse_n_forward(I f, DistanceType(I) n)

{

reverse_n_recursive(f, n);

}

reverse_forward

requires(Mutable(I) && ForwardIterator(I))

void reverse_forward(I f, I l)

{

reverse_n_forward(f, l - f);

}

reverse_copy_n

requires(Mutable(B) && BidirectionalIterator(B) &&

Mutable(I) && Iterator(I) &&

ValueType(B) == ValueType(I))

I reverse_copy_n(B l, DistanceType(I) n, I r)

{

typedef DistanceType(I) N;

while (n != N(0)) {

--l;

sink(r) = source(l);

++r;

n = n - N(1);

}

return r;

}

reverse_n_with_buffer

requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && BidirectionalIterator(B) &&

ValueType(I) == ValueType(B))

I reverse_n_with_buffer(I f, DistanceType(I) n, B b)

{

return reverse_copy_n(copy_n(f, n, b), n, f);

}

reverse_n_adaptive

requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && BidirectionalIterator(B) &&

ValueType(I) == ValueType(B))

I reverse_n_adaptive(I f, DistanceType(I) n, B b, DistanceType(I) n_b)

{

typedef DistanceType(I) N;

const N h = n / N(2);

const N r = n - N(2) * h;

if (h == N(0))

return f + n;

if (n <= n_b)

return reverse_n_with_buffer(f, n, b);

I m = reverse_n_adaptive(f, h, b, n_b);

m += r;

I l = reverse_n_adaptive(m, h, b, n_b);

swap_ranges_n(f, m, h);

return l;

}

Complexity of reverse_n_adaptive

Exercise

Derive a formula for the number of assignments performed by

reverse_n_adaptive for given range and buffer sizes

Conjecture

Conjecture

There is no algorithm that reverses a forward iterator range with

polylogarithmic additional space and in linear time

Selecting algorithms according to requirements

algorithms based on the requirements the types satisfy

In Appendix 2 we show a mechanism called category dispatch for

doing such selection in C++

Future versions of C++ may provide a more natural way of

expressing it

Conclusions

A permutation of a range induces a rearrangement

A taxonomy of rearrangements was presented

Weakening iterator requirements often raises interesting

algorithmic problems

A convention known as iterator category dispatch allows

automatic selection of the appropriate algorithm

We introduced a new class of algorithms, memory-adaptive

algorithms

Reading

Donald Knuth.

Section 1.2.5: Permutations and Factorials.

The Art of Computer Programming, Volume 1, Addison-Wesley, 1997,

pages 45-50.

Contents I

1 Introduction

2 Foundations

5 Orderings

6 Combining concepts

Contents II

8 Permutations and rearrangements

9 Rotations

Rotation permutations

Least common multiple

Cycle structure of rotations

Interface for rotate

rotate for indexed iterators

Loop fusion

rotate for bidirectional iterators

rotate for forward iterators

Conclusions

Reading

Projects

Contents III

10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

Contents IV

17 Index

Definition of rotation

Definition

The permutation p on a finite set with n elements defined by an index

permutation p(i) = (i + k) mod n is called the k-rotation of the set

Lemma

The inverse of a k-rotation on an n-element set is an (n − k)-rotation

Least common multiple

Definition

The least common multiple of integers a and b, written lcm(a, b), is the

smallest integer m such that both a and b divide it

Lemma

lcm(a, b) is a multiple of a and b that divides any other multiple of a

and b

Connecting lcm and gcd

Theorem

a · b = lcm(a, b) · gcd(a, b)

Proof.

Q i

If the prime decompositions of a and b are respectively pu i and

Q vi Q ui +vi Q max(ui ,vi )

pi , then a · b = p , lcm(a, b) = pi , and

Q min(ui ,vii)

gcd(a, b) = pi , and the result follows from the useful

identity x + y = max(x, y) + min(x, y)

(Divisibility lattice of integers)

Definition

An ordered set E is said to be a lattice if every subset consisting of two

elements of E has a least upper bound and a greatest lower bound in E

Example

The set of integers > 1, ordered by the relation “m divides n” between

m and n, is a lattice; the least upper bound of {m, n} is lcm(m, n), and

the greatest lower bound is gcd(m, n)

N. Bourbaki.

Chapter 3, Section 1.11.

Theory of Sets, Springer, 2004, pages 145-146.

Cycle structure of k-rotation on n elements

The length of the cycle is the smallest positive integer m such that

i = (i + mk) mod n

This is equivalent to mk mod n = 0, which shows the length of

the cycle to be independent of i

Since m is the smallest positive number such that mk mod n = 0,

it is obvious that lcm(k, n) = mk

lcm(k,n) kn n

m= k = gcd(k,n)k = gcd(k,n)

The number of cycles, therefore, is gcd(k, n)

Disjoint cycles of k-rotation on n elements

(i + vk) mod n

The distance between them is

|(i+uk) mod n−(i+vk) mod n| = (u−v)k mod n = (u−v)k−pn

where p = quotient((u − v)k, n)

Since both k and n are divisible by d = gcd(k, n), so is the distance

Therefore the distance between different elements in the same

cycle is > d

Elements with indices in [0, d) belong to disjoint cycles

Interface for rotate

interchanging the relative positions of [f, m) and [m, l), where

m = f + ((l − f) − k) = l − k

Experimentally, m is a more useful input than k (and when

forward or bidirectional iterators are involved, it avoids

performing linear-time operations to compute m from k)

Experimentally, returning the iterator m 0 = f + k pointing to the

new position of element at f is useful for many other algorithms30

30

Joseph Tighe suggests returning a pair, m and m 0 , in the order constituting a valid

range; while it is an interesting suggestion and preserves all the information, we do

not yet know of a compelling use of such interface

Stepanov, McJones Elements of Programming March 14, 2008 469 / 880

From-permutation for k-rotation in terms of f, m, l

From-permutation: p−1 (i) = (i + (n − k)) mod n

n=l−f

Since m goes to f, m + k = f + n ⇒ n − k = m − f

k = n − (m − f) = l − m

i < k ⇒ i + (n − k) < n ⇒ p−1 (i) = i + (n − k)

i > k ⇒ p−1 (i) = i + (n − k) − n = i − k

i + (n − k) if i < k

p−1 (i) =

i−k if i > k

A from-permutation for RandomAccessIterator

requires(Mutable(I) && RandomAccessIterator(I))

struct k_rotate_from_permutation_random_access

{

DistanceType(I) k;

DistanceType(I) n_minus_k;

I m_prime;

k_rotate_from_permutation_random_access(I f, I m, I l) :

k(l - m), n_minus_k(m - f), m_prime(f + (l - m)) {}

I operator()(I x) {

if (x < m_prime)

return x + n_minus_k;

else

return x - k;

}

};

A from-permutation for IndexedIterator

The absence of < and − for indexed iterators costs us an extra

operation or two (+ and −)

On modern processors this may not cost any time

template <typename I>

requires(Mutable(I) && IndexedIterator(I))

struct k_rotate_from_permutation_indexed

{

DistanceType(I) k;

DistanceType(I) n_minus_k;

I f;

k_rotate_from_permutation_indexed(I f, I m, I l) :

k(l - m), n_minus_k(m - f), f(f) {}

I operator()(I x) {

DistanceType(I) i = x - f;

if (i < k)

return x + n_minus_k;

else

return f + (i - k);

}

};

rotate_indexed_helper

requires(Mutable(I) && IndexedIterator(I) &&

Transformation(P) && Domain(P) == I)

I rotate_indexed_helper(I f, I m, I l, P p)

{

// Precondition: p is a from-permutation on [f, l)

typedef DistanceType(I) N;

N d = gcd(m - f, l - m);

N i = N(0);

while (i < d) {

do_cycle_from(f + i, p);

++i;

}

return f + (l - m);

};

William Fletcher and Roland Silver.

Algorithm 284: Interchange of Two Blocks of Data.

CACM, Volume 9, Number 5, May 1966, page 326.

Stepanov, McJones Elements of Programming March 14, 2008 473 / 880

rotate_indexed and rotate_random_access

I rotate(I f, I m, I l, indexed_iterator_tag)

{

k_rotate_from_permutation_indexed<I> p(f, m, l);

return rotate_indexed_helper(f, m, l, p);

};

I rotate(I f, I m, I l, random_access_iterator_tag)

{

k_rotate_from_permutation_random_access<I> p(f, m, l);

return rotate_indexed_helper(f, m, l, p);

};

I rotate(I f, I m, I l)

{

if (m == f) return l;

if (m == l) return f;

return rotate(f, m, l, IteratorCategory(I)());

}

Complexity of rotate for IndexedIterator

On average, gcd n

Loop fusion

value of k = n/2

That gives very bad locality of reference

Since the cycles are disjoint, we can improve locality of reference

by operating on adjacent elements in different cycles

This technique is called loop fusion, and was first discussed in this

paper:

A.P. Yershov.

ALPHA–An Automatic Programming System of High

Efficiency.

J. ACM, Volume 13, Number 1, January 1966, pages 17-24.

do_fused_cycles_from_with_buffer

requires(Mutable(I) && IndexedIterator(I) &&

Transformation(P) && Domain(P) == I &&

Mutable(B) && IndexedIterator(B) &&

ValueType(I) == ValueType(B))

void do_fused_cycles_from_with_buffer(I i, P p, B b, DistanceType(I) n)

{

// Precondition: p is a from-permutation on the range containing i

copy_n(i, b, n);

I f = i;

I next = p(i);

while (next != i) {

copy_n(next, f, n);

f = next;

next = p(next);

}

copy_n(b, f, n);

}

rotate_indexed_helper_fused

requires(Mutable(I) && IndexedIterator(I) &&

Transformation(P) && Domain(P) == I)

I rotate_indexed_helper_fused(I f, I m, I l, P p)

{

// Precondition: p is a from-permutation on [f, l)

typedef DistanceType(I) N;

const N fusion_factor(16);

ValueType(I) buffer[fusion_factor];

N d = gcd(m - f, l - m);

N i(0);

while (i + fusion_factor < d) {

do_fused_cycles_from_with_buffer(f + i, p, buffer, fusion_factor);

i = i + fusion_factor;

}

do_fused_cycles_from_with_buffer(f + i, p, buffer, d - i);

return f + (l - m);

};

Issues with fused rotate

programmers need to know

In this particular case, it is not that beneficial since often there are

not too many loops to fuse

gcd(n, k) = 1 with probability ≈ 60%

There are algorithms that do more more assignments but have

good locality of reference and work for weaker iterator

requirements

Connection of rotate and reverse

Lemma

The k-rotation on [0, n) is the only permutation p such that

1 i < n − k ∧ n − k 6 j < n ⇒ p(j) < p(i)

2 i < j < n − k ∨ n − k 6 i < j ⇒ p(i) < p(j)

Applying reverse to subranges [0, n − k) and [n − k, n) and then

applying reverse to the entire range will satisfy both conditions

rotate_three_reverses

requires(Mutable(I) && BidirectionalIterator(I))

void rotate_three_reverses(I f, I m, I l)

{

reverse(f, m);

reverse(m, l);

reverse(f, l);

}

reverse_until

without doing any extra work31

requires(Mutable(I) && BidirectionalIterator(I))

pair<I, I> reverse_until(I f, I m, I l)

{

while (f != m && m != l) {

--l;

cycle_2(f, l);

++f;

}

return pair<I, I>(f, l);

}

31

It was suggested to us by Raymond Lo and Wilson Ho

Stepanov, McJones Elements of Programming March 14, 2008 482 / 880

rotate for BidirectionalIterator

requires(Mutable(I) && BidirectionalIterator(I))

I rotate(I f, I m, I l, bidirectional_iterator_tag)

{

reverse(f, m);

reverse(m, l);

pair<I, I> p = reverse_until(f, m, l);

reverse(p.first, p.second);

if (m == p.first)

return p.second;

else

return p.first;

}

Complexity of rotate for BidirectionalIterator

Lemma

The number of assignments is 3(bn/2c + bk/2c + b(n − k)/2c), which

gives us 3n when n and k are both even and 3(n − 2) in every other

case

swap_ranges

requires(Mutable(I1) && Iterator(I1) &&

Mutable(I2) && Iterator(I2) &&

ValueType(I1) == ValueType(I2))

pair<I1, I2> swap_ranges(I1 f1, I1 l1, I2 f2, I2 l2)

{

while (f1 != l1 && f2 != l2) {

cycle_2(f1, f2);

++f1;

++f2;

};

return pair<I1, I2>(f1, f2);

}

rotates [f, l) about m

Intuition for ForwardIterator rotate algorithm

m = m 0 swap_ranges is sufficient

m 0 < m after swap_ranges, [f, m 0 ) are in the final position; we

need to rotate [m 0 , l) around m

m < m 0 after swap_ranges, [f, m) are in the final position; we

need to rotate [m, l) around m 0

This algorithm was first published by:

David Gries and Harlan Mills.

Swapping Sections.

Technical Report 81-452, Department of Computer Science, Cornell

University, January 1981.

rotate_0

requires(Mutable(I) && ForwardIterator(I))

void rotate_0(I f, I m, I l)

{

I c = m;

while (true) {

pair<I, I> p = swap_ranges(f, m, m, l);

if (p.first == m && p.second == l)

return;

if (p.first == m) {

m = p.second;

f = p.first;

} else {

assert(p.second == l);

f = p.first;

}

}

}

Annotated rotate_0

template <typename I>

requires(Mutable(I) && ForwardIterator(I))

void rotate_0_annotated(I f, I m, I l)

{

DistanceType(I) u = m - f;

DistanceType(I) v = l - m;

I c = m;

while (true) {

pair<I, I> p = swap_ranges(f, m, m, l);

if (p.first == m && p.second == l) { assert(u == v);

return;

} else if (p.first == m) { assert(v > u);

m = p.second;

f = p.first; v = v - u;

} else {

assert(p.second == l); assert(u > v);

f = p.first; u = u - v;

}

}

}

Stepanov, McJones Elements of Programming March 14, 2008 488 / 880

Complexity of rotate for ForwardIterator

Lemma

The number of assignments is 3(n − gcd(n, k))

rotate_unguarded

requires(Mutable(I) && ForwardIterator(I))

void rotate_unguarded(I f, I m, I l)

{

// Precondition: f , m ∧ m , l

I c = m;

while (c != l) {

cycle_2(f, c);

++f;

++c;

if (f == m)

m = c;

if (c == l)

c = m;

}

}

rotate for ForwardIterator

template <typename I>

requires(Mutable(I) && ForwardIterator(I))

I rotate(I f, I m, I l, forward_iterator_tag)

{

// Precondition: f , m ∧ m , l

I m_prime = l;

I c = m;

while (c != l) {

cycle_2(f, c);

++f;

++c;

if (f == m)

m = c;

if (c == l) {

if (m_prime == l) m_prime = f;

c = m;

}

}

return m_prime;

}

Stepanov, McJones Elements of Programming March 14, 2008 491 / 880

Conclusions

adjacent ranges of different sizes

Applying elementary number theory allows us to determine the

cycle structure of rotations

Interesting algorithms exist for forward, bidirectional, and

indexed iterator algorithms

Complexity is often dominated by locality of reference concerns

rather than minimizing operation counts

Reading

Donald Knuth.

Section 1.3.3: Permutations and Factorials.

The Art of Computer Programming, Volume 1, Addison-Wesley, 1997,

pages 164-185.

In particular, see problems 34 and 35.

Project 1: industrial-strength rotate

other two algorithms because it performs approximately 31 as

many assignments, but its locality of reference is poor

Design a benchmark comparing performance of all three

algorithms for different array sizes, element sizes, and rotation

amounts

Based on the results of the benchmark, design a composite

algorithm that appropriately uses one of the three algorithms

depending on its inputs

Project 2: Research in position-based rearrangements

algorithms: reverse and rotate

There are, however, many other examples of such algorithms; you

can find some of them in the section of Knuth given in Reading as

well as in the Collected Algorithms of the ACM (see, for example,

numbers 302, 380, and 467)

Develop a taxonomy of position-based rearrangements, discover

additional algorithms, and produce a library

Contents I

1 Introduction

2 Foundations

5 Orderings

6 Combining concepts

Contents II

8 Permutations and rearrangements

9 Rotations

Increasing ranges

Partition

Searching

Copying merging and sorting

In-place merging and sorting

Adaptive merging and sorting

Conclusions

Project

11 Coordinate structures

Contents III

12 Composite objects

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

17 Index

Increasing ranges

Definition

A range [f, l) is increasing if for every non-last iterator m ∈ [f, l),

¬(source(successor(m)) < source(m))

A range [f, l) is increasing with respect to a strict weak ordering r if

for every non-last iterator m ∈ [f, l),

¬r(source(successor(m)), source(m))

A range [f, l) is key-increasing a with respect to a regular function

key and a strict weak ordering r if for every non-last iterator

m ∈ [f, l), ¬r(key(source(successor(m))), key(source(k)))

a

In database parlance, a key is a part of a record on which sorting is done

Defaulting the key function

key function being the identity function on the value type of the

iterator

find_out_of_order

requires(Readable(I) && ForwardIterator(I) &&

RegularFunction(F) && Domain(F) == ValueType(I) &&

StrictWeakOrdering(R) && Domain(R) == Codomain(F))

I find_out_of_order(I f, I l, F key, R r, forward_iterator_tag)

{

if (f == l) return l;

I n(successor(f));

while (n != l) {

if (r(key(source(n)), key(source(f))))

return n;

f = n;

++n;

}

return l;

}

before we advance

find_out_of_order

requires(Readable(I) && Iterator(I) &&

RegularFunction(F) && Domain(F) == ValueType(I) &&

StrictWeakOrdering(R) && Domain(R) == Codomain(F))

I find_out_of_order(I f, I l, F key, R r, iterator_tag)

{

if (f == l) return l;

I n(successor(f));

Codomain(F) v = key(source(f));

while (n != l) {

Codomain(F) u = key(source(n));

if (r(u, v))

return n;

v = u;

++n;

}

return l;

}

is_increasing

requires(Readable(I) && Iterator(I) &&

RegularFunction(F) && Domain(F) == ValueType(I) &&

StrictWeakOrdering(R) && Domain(R) == Codomain(F))

bool is_increasing(I f, I l, F key, R r)

{

return l == find_out_of_order(f, l, key, r);

}

Partition

Definition

UnaryPredicate(P) ⇒

Predicate(P)

Arity(P) = 1

Definition

A range is partitioned if it is key-increasing with respect to a key

function that is a unary predicate

More generally, a range is k-partitioned if it is key-increasing with

respect to a key function that returns one of k distinct values for

some (compile-time) constant integer k > 0

with key-increasing ranges when the range of the key function is a

bounded set of integers

Stepanov, McJones Elements of Programming March 14, 2008 504 / 880

Partition points

Definition

A partition point in a range [f, l) partitioned by a unary predicate p is an

iterator i ∈ [f, l] such that ¬p(source(j)) for all j ∈ [f, i) and

p(source(k)) for all k ∈ [i, l)

predicate is satisfied, if such a position exists, or the limit of the

range otherwise

Lower bound and upper bound

Definition

In an increasing range [f, l), every value a of the value type of the

range determines two partition points, lower bound and upper bound,

defined by, respectively:

Pa (x) ≡ x < a

Pa0 (x) ≡ ¬(a < x)

sequences and sequences with respect to a strict weak order r

Lemma

lower bound upper bound

Intuition for lower bound and upper bound

to a could occur in the increasing sequence

Similarly, an upper bound is the successor of the last position

where a value equal to a could occur

Therefore, elements equal to a appear only in the semi-open range

from lower bound to upper bound

A sequence with lower bound i and upper bound j for the value a

(note any of the three regions may be empty):

| {z } | {z } | {z }

<a =a a<

Partition algorithms

range

Algorithmically, it is worth providing special-case algorithms for

dealing with partitions

Testing whether a range is partitioned dereferences half as many

iterators as testing whether it is increasing

Partitioning a range of size n is linear while sorting it is n log2 n

Efficient algorithms for dealing with increasing ranges can be

composed from algorithms dealing with partitioned ranges

Quicksort is implemented using a specialized partition algorithm

Finding a lower or upper bound uses the algorithm for finding a

partition point

Testing whether a range is partitioned

Lemma

Given any range and a unary predicate, some prefix of the range is

partitioned by the predicate

by testing whether the prefix equals the range we can determine

whether the range is partitioned

find_if and find_if_not

We need to find the first element satisfying the predicate

template <typename I, typename P>

requires(Readable(I) && Iterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I find_if(I f, I l, P p) {

while (f != l && !p(source(f)))

++f;

return f;

}

Then finding the first element not satisfying the predicate gives

the longest partitioned prefix

template <typename I, typename P>

requires(Readable(I) && Iterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I find_if_not(I f, I l, P p) {

while (f != l && p(source(f)))

++f;

return f;

}

is_partitioned

requires(Readable(I) && Iterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

pair<I, I> find_partitioned_prefix(I f, I l, P p)

{

I i = find_if(f, l, p);

I j = find_if_not(i, l, p);

return pair<I, I>(i, j);

}

prefix

requires(Readable(I) && Iterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

bool is_partitioned(I f, I l, P p)

{

return l == find_partitioned_prefix(f, l, p).second;

}

Quantifier functions

requires(Readable(I) && Iterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

bool all(I f, I l, P p) { return l == find_if_not(f, l, p); }

requires(Readable(I) && Iterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

bool none(I f, I l, P p) { return l == find_if(f, l, p); }

requires(Readable(I) && Iterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

bool not_all(I f, I l, P p) { return !all(f, l, p); }

requires(Readable(I) && Iterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

bool some(I f, I l, P p) { return !none(f, l, p); }

is_partition_point

requires(Readable(I) && Iterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

bool is_partition_point(I f, I m, I l, P p)

{

return none(f, m, p) && all(m, l, p);

}

Importance of finding a partition point

itself is not that useful

On an increasing range, it can be combined with an appropriate

predicate to implement all the varieties of binary search –

allowing us to find a value without examining every position

Intuition for finding a partition point

partitioned range [f, l)

If p(m) is false, we know the partition point of [f, l) is greater than

m, and in fact is equal to the partition point of [successor(m), l)

If p(m) is true, we know the partition point is not greater than m, so

it is equal to the partition point of [f, m)

Note the partition point of [f, m) will equal m if the predicate p is

false for every iterator in [f, m)

Lemma

Choosing m in the middle of the range assures optimal worst-case

performance

of m might give better average performance

partition_point_n

requires(Readable(I) && ForwardIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I partition_point_n(I f, DistanceType(I) n, P p)

{

typedef DistanceType(I) N;

while (n != N(0)) {

N h = half_nonnegative(n);

I m = f + h;

if (!p(source(m))) {

n = n - successor(h); f = successor(m);

} else

n = h;

}

return f;

}

requires(Readable(I) && ForwardIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I partition_point(I f, I l, P p) { return partition_point_n(f, l - f, p); }

Complexity of partition_point

the length of the range is reduced by a factor of 2 at each step

It also performs a logarithmic number of iterator/integer

additions, which for an indexed or random access iterator are

constant-time but which for a forward or bidirectional iterator

(singly- or doubly-linked list) require n iterator increments

partition_point adds another n iterator increments for forward

and bidirectional iterators

It may still be worthwhile to use it on linked lists when the

predicate application is much more expensive than an iterator

increment

Origin of the bisection technique

least as far as the proof of the Intermediate Value Theorem

discovered by Bernard Bolzano32 and, independently, by

Augustin-Louis Cauchy about 4 years later33

The proof of the Bolzano-Cauchy theorem leads us to the

following interpolation algorithm34

32

Bernard Bolzano. Rein analytischer Beweis des Lehrsatzes, daß zwischen je zwey

Werthen, die ein entgegengesetztes Resultat gewähren, wenigstens eine reelle Wurzel der

Gleichung liege. 1817

33

A.-L. Cauchy. Cours D’Analyse de L’Ecole Royale Polytechnique. 1821

34

This book explores the connections between some simple algebraic concepts and

algorithms; we strongly believe it is also possible to define algorithms on analytic

concepts

Stepanov, McJones Elements of Programming March 14, 2008 518 / 880

bisection_interpolation

requires(Transformation(F) && ArchimedeanOrderedField(Domain(F)))

Domain(F) bisection_interpolation(

F f, Domain(F) x0, Domain(F) x1, Domain(F) y0, Domain(F) e)

{

// Precondition: f is continuous and increasing on [x0 , x1 ]

typedef Domain(F) R;

R x;

while (true) {

x = (x0 + x1) / R(2);

R y = f(x);

if (y <= y0) {

x0 = x; if (y0 < y + e) break;

} else {

x1 = x; if (y < y0 + e) break;

}

}

return x;

// Postcondition: x is in the original interval [x0 , x1 ] and |f(x) − y0 | <

}

count_if and count_if_not

template <typename I, typename P>

requires(Readable(I) && Iterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I count_if(I f, I l, P p) {

DistanceType(I) n(0);

while (f != l) {

if (p(source(f))) n = successor(n);

++f;

}

return n;

}

requires(Readable(I) && Iterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I count_if_not(I f, I l, P p) {

DistanceType(I) n(0);

while (f != l) {

if (!source(f)) n = successor(n);

++f;

}

return n;

}

potential_partition_point

requires(Readable(I) && ForwardIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I potential_partition_point(I f, I l, P p)

{

I i = f;

while (f != l) {

if (!p(source(f))) ++i;

++f;

}

return i;

// Postcondition: i = f + count_if_not(f, l, p)

}

Multiplicity of partitioned rearrangements

precede true values, there are u!v! ways to partition a range with u

false values and v true values

A partition algorithm would satisfy all our needs if it had these

properties:

Requires only forward iterators

Stable

In place

Linear time, with minimal number of assignments and predicate

applications

We conjecture no such algorithm exists

We present some partition algorithms to demonstrate the kind of

tradeoffs that must be made

partition_copy35

requires(Readable(I) && Iterator(I) &&

Writable(O0) && Iterator(O0) &&

Writable(O1) && Iterator(O1) &&

UnaryPredicate(P) && Domain(P) == ValueType(I) &&

ValueType(I) == ValueType(O0) &&

ValueType(I) == ValueType(O1))

pair<O0, O1> partition_copy(I f, I l, O0 o0, O1 o1, P p)

{

typedef DistanceType(I) N;

while (f != l) {

if (p(source(f))) {

sink(o1) = source(f); ++o1;

} else {

sink(o0) = source(f); ++o0;

}

++f;

}

return pair<O0, O1>(o0, o1);

}

35

T.K. Lakshman suggested the interface

Stepanov, McJones Elements of Programming March 14, 2008 523 / 880

Properties of partition_copy 37

It is stable: the values of each of the two output ranges are in the

same relative order as in the input range

It is a copying rearrangement36

It is optimal in terms both of number of predicate applications and

assignments (both numbers are n, the size of the range)

36

Note that if the input range is defined by mutable forward iterators, it may be

supplied as one of the output ranges

37

A variation we use later, partition_copy_n, is given in Appendix 2

Stepanov, McJones Elements of Programming March 14, 2008 524 / 880

Semistable partition rearrangements

Definition

A partition rearrangement is semistable if the relative order of elements

not satisfying the predicate is preserved

extent-based sequence (such as the array of Chapter 11 or the

C++ std::vector) of elements satisfying a given predicate

First partition the sequence by the predicate

Erase the elements from the partition point to the end of the

sequence

The sequence now contains the elements not satisfying the

predicate, in their original order 38

38

In Chapter 12 we will explain why this is the right thing to do

Stepanov, McJones Elements of Programming March 14, 2008 525 / 880

semistable_partition39

requires(Mutable(I) && ForwardIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I semistable_partition(I f, I l, P p)

{

I i = find_if(f, l, p); // none(f, i, p)

I j = find_if_not(i, l, p); // none(f, i, p) ∧ all(i, j, p) ∧ (j = l ∨ ¬p(j))

while (j != l) { // none(f, i, p) ∧ all(i, j, p) ∧ p(i) ∧ ¬p(j)

cycle_2(i, j); // none(f, i, p) ∧ all(i + 1, j, p) ∧ ¬p(i) ∧ p(j)

// (none(f, i, p)∧¬p(i))∧(all(i+1, j, p)∧p(j))

++i; ++j; // none(f, i, p) ∧ all(i, j, p)

j = find_if_not(j, l, p); // j = l ∨ ¬p(j)

}

return i;

}

39

The algorithm is due to Nico Lomuto, according to: Jon Bentley. Programming

Pearls. Communications of the ACM, Volume 27, Number 4, April 1984, pages 287-291

Stepanov, McJones Elements of Programming March 14, 2008 526 / 880

Useful lemmas

The annotations on the previous slide follow from some trivial but

useful lemmas

Lemma

After executing i = find_if(f, l, p), none(f, i, p) ∧ (i = l ∨ p(i))

After executing j = find_if_not(f, l, p), all(f, j, p) ∧ (j = l ∨ ¬p(j))

If p(source(i)) ∧ q(source(j)) before executing cycle_2(i, j), then

afterward q(source(i)) ∧ p(source(j))

none(f, l, p) ∧ ¬p(l) ⇒ none(f, l + 1, p)

all(f + 1, l, p) ∧ p(f) ⇒ all(f, l, p)

Properties of semistable_partition

It is in place

Let n = l − f be the number of elements in the range and let w be

the number of elements not satisfying the predicate following the

partitioned prefix40

The predicate is applied n times

cycle_2 is performed w times

The number of iterator increments is n + w

It is totally inappropriate for use in quicksort since it does not

divide a range of equal values at the midpoint and that leads to

quadratic complexity in many practical situations

40

w = count_if_not(find_partitioned_prefix(f, l, p), l, p)

Stepanov, McJones Elements of Programming March 14, 2008 528 / 880

semistable_partition performs too many

assignments

Consider a range t, f, f, f, f, f, f, f

semistable_partition will perform seven calls of cycle_2, while

one suffices

assured_find_if and assured_find_if_not

We can eliminate one of the two tests in the loop for find_if if we

are assured there is an element in the range that satisfies the

predicate

template <typename I, typename P>

requires(Readable(I) && Iterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I assured_find_if(I f, P p) {

// Let l be the end of the implied range starting with f

// Precondition: some(f, l, p)

while (!p(source(f))) ++f;

return f;

}

requires(Readable(I) && Iterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I assured_find_if_not(I f, P p) {

// Let l be the end of the implied range starting with f

// Precondition: not_all(f, l, p)

while (p(source(f))) ++f;

return f;

}

Stepanov, McJones Elements of Programming March 14, 2008 530 / 880

unstable_forward_partition

template <typename I, typename P>

requires(Mutable(I) && ForwardIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I unstable_forward_partition(I f, I l, P p)

{

I i = potential_partition_point(f, l, p);

I j = find_if_not(i, l, p);

while (j != l) {

f = assured_find_if(f, p);

cycle_2(f, j); ++f; ++j;

j = find_if_not(j, l, p);

}

return i;

}

elements before and after the partition point are equal

The code does one extra iterator comparison that could be

eliminated by inlining both calls to find_if_not (the same

optimization applies to many following algorithms)

Stepanov, McJones Elements of Programming March 14, 2008 531 / 880

Properties of unstable_forward_partition

It is in place

The number of times cycle_2 is performed, v, equals the number

of misplaced elements not satisfying the predicate, that is, the

number of elements not satisfying the predicate in the subrange

from the potential partition point to the end of the range41

The algorithm applies the predicate 2n times, therefore it is useful

when the cost of predicate application is insignificant compared to

the cost of cycle_2

This technique of computing the sizes of a set of buckets and then

moving the elements appears in radix sort42

With a bidirectional iterator, we can avoid precomputing the

bucket sizes

41

v = count_if_not(potential_partition_point(f, l, p))

42

See the use of this technique for in-place k-partition Exercise 5.2-13 in Donald

Knuth. The Art of Computer Programming. Volume 3, Addison-Wesley, 1998, page 80;

Solution, page 618

Stepanov, McJones Elements of Programming March 14, 2008 532 / 880

find_backward_if and find_backward_if_not

To indicate “not found”, we return f, which forces us to return

i + 1 if we find a satisfying element at position i

template <typename I, typename P>

requires(Readable(I) && BidirectionalIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I find_backward_if(I f, I l, P p) {

while (true) {

if (f == l) return f;

--l;

if (p(source(l))) return successor(l);

}

}

template <typename I, typename P>

requires(Readable(I) && BidirectionalIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I find_backward_if_not(I f, I l, P p) {

while (true) {

if (f == l) return f;

--l;

if (!p(source(l))) return successor(l);

}

}

Stepanov, McJones Elements of Programming March 14, 2008 533 / 880

unstable_bidirectional_partition

requires(Mutable(I) && BidirectionalIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I unstable_bidirectional_partition(I f, I l, P p)

{

while (true) {

f = find_if(f, l, p);

l = find_backward_if_not(f, l, p);

if (f == l) return f;

--l;

cycle_2(f, l);

++f;

}

return f;

}

Properties of unstable_bidirectional_partition

It is in place

The predicate is applied n times

cycle_2 is called v times (where v is as previously defined for

unstable_forward_partition)

The algorithm moves only those elements that need to be moved,

and puts two elements in their final destination with each call to

cycle_2

The total number of assignments, therefore, is 3v

Minimizing the number of assignments

rearrangement that has only a single cycle, resulting in 2v + 1

assignments

The idea is to:

Save the first misplaced element, creating a “hole”

Repeatedly find a misplaced element on the opposite side of the

potential partition point and move it into the hole, creating a new

hole

Move the saved element into the final hole

assured_find_backward_if and

assured_find_backward_if_not

requires(Readable(I) && BidirectionalIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I assured_find_backward_if(I l, P p) {

do --l; while (!p(source(l)));

return l;

}

requires(Readable(I) && BidirectionalIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I assured_find_backward_if_not(I l, P p) {

do --l; while (p(source(l)));

return l;

}

find_backward_if since we do not need to indicate failure

single_cycle_partition

requires(Readable(I) && BidirectionalIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I single_cycle_partition(I f, I l, P p)

{

f = find_if(f, l, p);

l = find_backward_if_not(f, l, p);

if (f == l) return f;

--l;

ValueType(I) tmp = source(f);

while (true) {

sink(f) = source(l);

f = find_if(successor(f), l, p);

if (f == l) {

sink(l) = tmp;

return f;

}

sink(l) = source(f);

l = assured_find_backward_if_not(l, p);

}

}

Properties of single_cycle_partition

multiple 2-cycles is faster on modern computers with

instruction-level parallelism

The technique, however, is worth careful study

assured_bidirectional_partition

requires(Readable(I) && BidirectionalIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I assured_bidirectional_partition(I f, I l, P p)

{

// Precondition:

// (¬all(f, l, p) ∧ some(f, l, p)) ∨ (¬p(source(f − 1)) ∧ p(source(l)))

while (true) {

f = assured_find_if(f, p);

l = assured_find_backward_if_not(l, p);

if (successor(l) == f) return f;

cycle_2(f, l);

++f; // ¬p(source(f − 1)) ∧ p(source(l))

}

}

Correctness of assured_bidirectional_partition

assured_find_backward_if_not

After these two calls, f , l, since they point to elements with

different predicate values

l − f decreases by at least two each iteration

f and l can cross over by only one position

Suppose they had crossed by more than one, and consider any

iterator m between l and f

If ¬p(source(m)), then f would have stopped there; otherwise, l

would have stopped there

Therefore, there is no such m

If the exit is not taken, f ≺ l, and f and l point, correspondingly, to

satisfying and non-satisfying elements

The cycle_2 call moves these elements to correct positions, and

the increment of f restores the invariant

Stepanov, McJones Elements of Programming March 14, 2008 541 / 880

sentinel_partition

precondition to use an assured function43

requires(Readable(I) && BidirectionalIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I sentinel_partition(I f, I l, P p)

{

f = find_if(f, l, p);

l = find_backward_if_not(f, l, p);

if (f == l) return f;

--l;

cycle_2(f, l);

++f;

return assured_bidirectional_partition(f, l, p);

}

43

It is possible to combine the sentinel and single-cycle techniques, producing

sentinel_single_cycle_partition

Stepanov, McJones Elements of Programming March 14, 2008 542 / 880

The origin of partition algorithms

Hoare:

unstable_bidirectional_partition

single_cycle_partition

sentinel_partition

44

C.A.R. Hoare. Quicksort. The Computer Journal, Volume 5, Number 1, 1962, pages

10-16. This paper, which is a serious contender for the best-ever computer science

paper, should not be confused with his earlier publication: C.A.R. Hoare. Algorithm

63, Partition; Algorithm 64, Quicksort; Algorithm 65, Find. Communications of the

ACM, Volume 4, Number 7, July 1961, pages 321-322. No one should attempt to

implement or teach quicksort without studying this paper.

Stepanov, McJones Elements of Programming March 14, 2008 543 / 880

Regularity of predicate for partition algorithms

function; this assures the same boolean value is returned every

time the predicate is applied to a particular object

This in turn assures that after the partition algorithm returns, a

call to is_partitioned(f, l, p) would return true

However the algorithms using assured finds depend on regularity

in a more fundamental way: to assure they stay within bounds of

the range

Consider, for example, using a nonregular predicate to perform a

random shuffle

Exercise

Exercise

Design an algorithm implementing in-place uniform random

shuffle of a range of forward iterators (hint: recursively partition

the range with a non-regular coin-tossing predicate a )

Prove that the expected number of coin tosses is n log2 n

Prove that in-place uniform random shuffle of a range of forward

iterators cannot be done in linear time

a

The algorithm was discovered by Raymond Lo and Wilson Ho

Intuition for stable partition

A singleton range can be partitioned with one predicate

application and possibly one iterator increment

An arbitrary range can be partitioned with a divide-and-conquer

algorithm:

Divide the range in the middle

Stably partition each half

Combine the results by rotating the range bounded by the first and

second partition points around the middle: **** simple picture *****

This easily extends to a memory adaptive version

Base cases for stable partition

requires(Mutable(I) && ForwardIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

pair<I, I> stable_partition_0(I f, P)

{

return pair<I, I>(f, f);

}

requires(Mutable(I) && ForwardIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

pair<I, I> stable_partition_1(I f, P p)

{

I l = successor(f);

if (p(source(f)))

return pair<I, I>(f, l);

else

return pair<I, I>(l, l);

}

Combining operation for stable partition

requires(Mutable(I) && ForwardIterator(I))

pair<I, I> stable_partition_combine(pair<I, I> r0, pair<I, I> r1)

{

I m = rotate(r0.first, r0.second, r1.first);

return pair<I, I>(m, r1.second);

}

stable_partition_n

requires(Mutable(I) && ForwardIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

pair<I, I> stable_partition_n(I f, DistanceType(I) n, P p)

{

typedef DistanceType(I) N;

if (n == N(0))

return stable_partition_0(f, p);

if (n == N(1))

return stable_partition_1(f, p);

N h = half_nonnegative(n);

pair<I, I> r0 = stable_partition_n(f, h, p);

pair<I, I> r1 = stable_partition_n(r0.second, n - h, p);

return stable_partition_combine(r0, r1);

}

stable_partition_n_with_buffer

requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && ForwardIterator(B) &&

UnaryPredicate(P) && Domain(P) == ValueType(I) &&

ValueType(I) == ValueType(B))

pair<I, I> stable_partition_n_with_buffer(I f, DistanceType(I) n, B b, P p)

{

pair<I, B> r = partition_copy_n(f, n, f, b, p);

return pair<I, I>(r.first, copy(b, r.second, r.first));

}

stable_partition_n_adaptive

requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && ForwardIterator(B) &&

UnaryPredicate(P) && Domain(P) == ValueType(I) &&

ValueType(I) == ValueType(B))

pair<I, I> stable_partition_n_adaptive(

I f, DistanceType(I) n, B b, DistanceType(I) n_b, P p)

{

typedef DistanceType(I) N;

if (n == N(0))

return stable_partition_0(f, p);

if (n == N(1))

return stable_partition_1(f, p);

if (n <= n_b)

return stable_partition_n_with_buffer(f, n, b, p);

N h = half_nonnegative(n);

pair<I, I> r0 = stable_partition_n_adaptive(f, h, b, n_b, p);

pair<I, I> r1 = stable_partition_n_adaptive(r0.second, n - h, b, n_b, p);

return stable_partition_combine(r0, r1);

}

Properties of stable partition

of recursion

The depth of the recursion for stable_partition_n is dlog2 ne

At every recursive level we rotate n/2 elements on the average,

requiring between n/2 and 3n/2 assignments depending on the

iterator category

The total number of assignments is n log2 n/2 for random access

iterators and 3n log2 n/2 for forward and bidirectional iterators

Exercise

Determine the number of assignments for the adaptive version

Interfaces for binary search

related algorithms for determining the lower bound, the upper

bound, or both

The advantage of these as compared to the way binary search is

usually specified in textbooks is that the lower and upper bounds

are always defined, so there is no issue of what to return when the

element is not present

Several algorithms in this chapter will demonstrate the utility of

these interfaces

Implementing lower bound and upper bound

bound for a value a as the partition points of two predicates:

Pa (x) ≡ x < a

Pa0 (x) ≡ ¬(a < x)

Pa and Pa0 to partition_point_n

See Appendix 2 for the C++ versions

Exercise

Implement a function that returns both lower and upper bounds and

does fewer comparisons than the sum of the comparisons that would

be done by calling both lower_bound_n and upper_bound_na

a

A similar STL function is called equal_range

Merging

Definition

Merge is an operation that combines two increasing ranges into a single

increasing range that contains all the elements of both ranges and no

other elements

Definition

A merge is stable if the output range preserves the relative order of

equivalent elements both within each input range, and between the

first and second input range

Lemma

Stability establishes a unique ordering of the elements in the output

range

Merge interfaces

ranges and rearranges the elements in the combined range into

increasing order; a range [f, l) is mergeable if there is an iterator

m ∈ [f, l) such that the subranges [f, m) and [m, l) are increasing

A copying merge operation sets a writable range to the merge of

two readable increasing ranges

A melding merge operation combines two ranges defined by node

iterators by changing their relative order46

45

A mutating merge may or may not be in place depending on how much extra

memory it uses

46

We define node iterators in the next chapter

Stepanov, McJones Elements of Programming March 14, 2008 556 / 880

is_mergeable

requires(Readable(I) && ForwardIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))

bool is_mergeable(I f, I m, I l, R r)

{

return is_increasing(f, m, r) && is_increasing(m, l, r);

}

is_merged

Exercise

Implement this function that checks whether the range starting at o is

equal to the merge of the ranges [f0, l0) and [f1, l1)

template <typename I, typename R>

requires(Readable(I) && ForwardIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))

bool is_merged(I f0, I l0, I f1, I l1, I o, R r);

Algebraic properties of merge

An empty range is its identity element (both left and right)

Merge is associative

Its complexity needs to be taken into consideration when

reassociating

Merge is commutative when stability is not taken into account

Complexity property of copying merge

If ◦ denotes the merge operation and C(x, y) denotes the cost (the

number of assignments) to merge ranges x and y, then C

satisfies47 :

C(x, y) + C(x, z) + C(y, z)

C(x ◦ y, z) =

2

A consequence of this is:

47

In the next chapter we will define a Huffman semigroup, where the cost of the

semigroup operation satisfies this axiom

Stepanov, McJones Elements of Programming March 14, 2008 560 / 880

merge_copy_n

template <typename I0, typename I1, typename O, typename R>

requires(Readable(I0) && Iterator(I0) && Readable(I1) && Iterator(I1) &&

Writable(O) && Iterator(O) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I0) &&

ValueType(I0) == ValueType(O) && ValueType(I1) == ValueType(O))

void merge_copy_n(I0 f0, DistanceType(I0) n0,

I1 f1, DistanceType(I1) n1, O o, R r) {

// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)

// Precondition: the input ranges do not overlap the output range

typedef DistanceType(I0) N0; typedef DistanceType(I1) N1;

while (true) {

if (n0 == N0(0)) { copy_n(f1, n1, o); return; }

if (n1 == N1(0)) { copy_n(f0, n0, o); return; }

if (r(source(f1), source(f0))) {

sink(o) = source(f1); ++f1; n1 = n1 - N1(1);

} else {

sink(o) = source(f0); ++f0; n0 = n0 - N0(1);

}

++o;

}

}

Stepanov, McJones Elements of Programming March 14, 2008 561 / 880

merge_copy_backward_n

template <typename I0, typename I1, typename O, typename R>

requires(Readable(I0) && BidirectionalIterator(I0) &&

Readable(I1) && BidirectionalIterator(I1) &&

Writable(O) && BidirectionalIterator(O) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I0) &&

ValueType(I0) == ValueType(O) && ValueType(I1) == ValueType(O))

void merge_copy_backward_n(I0 l0, DistanceType(I0) n0,

I1 l1, DistanceType(I1) n1, O o, R r) {

// Let f0 = l0 − n0 and f1 = l1 − n1

// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)

typedef DistanceType(I0) N0; typedef DistanceType(I1) N1;

while (true) {

if (n0 == N0(0)) { copy_backward_n(l1, n1, o); return; }

if (n1 == N1(0)) { copy_backward_n(l0, n0, o); return; }

--o;

if (r(source(predecessor(l1)), source(predecessor(l0)))) {

--l0; sink(o) = source(l0); n0 = n0 - N0(1);

} else {

--l1; sink(o) = source(l1); n1 = n1 - N1(1);

}

}

}

Stability requires the order of the if clauses is reversed from the one in merge_copy_n

Stepanov, McJones Elements of Programming March 14, 2008 562 / 880

Return value for copying merges

Exercise

Reimplement merge_copy_n and merge_copy_backward_n with the

correct return values

Complexity of copying merges

Lemma

The number of assignments is always n0 + n1 and the worst-case

number of comparisons is n0 + n1 − 1

Exercise

Determine the average number of comparisons

merge_n_with_buffer for ForwardIterator

requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && ForwardIterator(B) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&

ValueType(I) == ValueType(B))

void merge_n_with_buffer(I f0, DistanceType(I) n0,

I f1, DistanceType(I) n1, B b, R r,

forward_iterator_tag)

{

// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)

// Precondition: the range beginning with b is of size at least min(n0, n1)

if (n0 <= n1) {

copy_n(f0, n0, b);

merge_copy_n(b, n0, f1, n1, f0, r);

} else {

f1 = rotate(f0, f1, f1 + n1);

copy_n(f0, n1, b);

merge_copy_n(f1, n0, b, n1, f0, r);

}

}

merge_n_with_buffer for BidirectionalIterator

requires(Mutable(I) && BidirectionalIterator(I) &&

Mutable(B) && BidirectionalIterator(B) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&

ValueType(I) == ValueType(B))

void merge_n_with_buffer(I f0, DistanceType(I) n0,

I f1, DistanceType(I) n1, B b, R r,

bidirectional_iterator_tag)

{

// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)

// Precondition: the range beginning with b is of size at least min(n0, n1)

if (n0 <= n1) {

copy_n(f0, n0, b);

merge_copy_n(b, n0, f1, n1, f0, r);

} else {

pair<I, B> tmp = copy_n(f1, n1, b);

merge_copy_backward_n(f1, n0, tmp.second, n1, tmp.first, r);

}

}

Concept Merger

Definition

Merger(M) ⇒

The type function IteratorT ype(M) is defined

Procedure(M)

For all m ∈ M, I = IteratorT ype(M), and N = DistanceT ype(I),

m : I × N × I × N → void

There is an implicit function ordering such that for all m ∈ M,

ordering(m) is a strict weak ordering on the value type of the

iterator type of M

For all m ∈ M, a call m(f0, n0, f1, n1) satisfying the precondition

Stepanov, McJones Elements of Programming March 14, 2008 567 / 880

merger_with_buffer

requires(Mutable(I) && BidirectionalIterator(I) &&

Mutable(B) && BidirectionalIterator(B) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&

ValueType(I) == ValueType(B))

struct merger_with_buffer

{

B b;

R r;

merger_with_buffer(B b, R r) : b(b), r(r) { }

void operator()(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1)

{

// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)

// Precondition: the range beginning with b is of size at least min(n0, n1)

merge_n_with_buffer(f0, n0, f1, n1, b, r);

}

};

Sorting algorithms

Definition

A key-sorting (or just sorting) algorithm is a rearrangement those

postcondition is that the range is key-increasing (or just

increasing) with respect to a given strict weak order

A key-sorting (or sorting) algorithm is stable if it does not change

the relative order of values that are equivalent under the

symmetric complement of the given strict weak order

sorting functions in this chapter is not instructive since

transforming a function dealing with an increasing sequence into

one with a key-increasing sequence is straightforward

merge_sort_n

requires(Mutable(I) && ForwardIterator(I) &&

Merger(M) && IteratorType(M) == I)

I merge_sort_n(I f, DistanceType(I) n, M merger)

{

typedef DistanceType(I) N;

if (n < N(2)) return f + n;

N h = half_nonnegative(n);

I m = merge_sort_n(f, h, merger);

I l = merge_sort_n(m, n - h, merger);

merger(f, h, m, n - h);

return l;

}

stable_sort_n_with_buffer

requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && ForwardIterator(B) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&

ValueType(I) == ValueType(B))

I stable_sort_n_with_buffer(I f, DistanceType(I) n, B b, R r)

{

// Precondition: the range beginning with b is of size at least n/2

return merge_sort_n(f, n, merger_with_buffer<I, B, R>(b, r));

}

ForwardIterator version of merge_n_with_buffer will never be

executed

Complexity of stable_sort_n_with_buffer

Each level performs 3n/2 assignments, for a total number of

assignments of 32 ndlog2 ne

At ith level from the bottom, the worst-case number of

comparisons is n − 2ni ; summing over all levels gives

dlog2 ne

X n

ndlog2 ne − ≈ ndlog2 ne − n

2i

i=1

Intuition for stable in-place merge

Theorem

If is_mergeable(f, m, l, r), then for every iterator i ∈ [f, m), there is a

unique j ∈ [m, l) such that after x ← source(i) and

m 0 ← rotate(i, m, j),

[f, l) is stably partitioned, with [f, m 0 ) containing elements less

than or equal to x and [m 0 , l) containing elements greater than or

equal to x

is_mergeable(f, i, m 0 , r) ∧ is_mergeable(m 0 , j, l, r)

Proof.

Consider j = lower_bound(m, l, source(i), r)

i = upper_bound(f, m, source(j), r), and perform

m 0 ← rotate(i, m, successor(j))

Stepanov, McJones Elements of Programming March 14, 2008 573 / 880

A sketch of an algorithm for stable in-place merge

[f, m) and the corresponding j in [m, l), or a j in [m, l) and a

corresponding i in [f, m), rotate appropriately, and then merge

each of the mergeable subranges

Now since j − m 0 = m − i, choosing i halfway between f and m or

j halfway between m and l will cut the minimum of m − f and

l − m in half

These considerations lead to the following algorithm

merge_n_in_place

template <typename I, typename R>

requires(Mutable(I) && ForwardIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))

void merge_n_in_place(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1, R r) {

// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)

typedef DistanceType(I) N;

if (min(n0, n1) == N(0)) return;

I m0, m1;

N h0, h1;

if (n0 < n1) {

h0 = half_nonnegative(n0); m0 = f0 + h0;

m1 = lower_bound_n(f1, n1, source(m0), r); h1 = m1 - f1;

I m = rotate(m0, f1, m1);

merge_n_in_place(f0, h0, m0, h1, r);

merge_n_in_place(successor(m), n0 - successor(h0), m1, n1 - h1, r);

} else {

h1 = half_nonnegative(n1); m1 = f1 + h1;

m0 = upper_bound_n(f0, n0, source(m1), r); h0 = m0 - f0;

I m = rotate(m0, f1, successor(m1));

merge_n_in_place(f0, h0, m0, h1, r);

merge_n_in_place(m, n0 - h0, successor(m1), n1 - successor(h1), r);

}

}

Properties of merge_n_in_place

Lemma

1 Each call of rotate puts one element into its final position

2 The algorithm terminates with a sorted range

3 Each level of recursion reduces m = min(n0, n1) to no more than

bm/2c

4 There are at most blog2 (min(n0, n1))c + 1 recursive levels

also contains a careful complexity analysis

48

Krzysztof Dudziński and Andrzej Dydek. On a Stable Minimum Storage Merging

Algorithm. Information Processing Letters, Volume 12, Number 1, February 1981, pages

5-8.

Stepanov, McJones Elements of Programming March 14, 2008 576 / 880

Complexity of merge_n_in_place: number of

assignments

Theorem

Let

kn be the maximum number of assignments required to rotate a

range of length n

a(n0 , n1 ) be the maximum number of assignments required for a

stable in-place merge of two ranges with lengths n0 and n1 , where

n0 6 n1

Then

a(n0 , n1 ) 6 k(n1 + dn0 /2e) log2 (2n0 ) 6 kn log2 n

(and bidirectional) iterators

Number of assignments of merge_n_in_place49

Proof.

By induction on n0 .

n0 = 1 A single rotation of a range of length at most n1 + 1 accomplishes the merge, so

a(1, n1 ) = r(n1 + 1) 6 k(n1 + 1) = k(n1 + d1/2e) log2 (2).

n0 > 1 We have a(n0 , n1 ) 6 r(n1 + n0 − h0 ) + a(h0 , h1 ) + a(n0 − h0 − 1, n1 − h1 ). By

the induction hypothesis, a(h0 , h1 ) 6 k(h1 + dh0 /2e) log2 (2h0 ) and

It is easily verified that dx/2e + dy/2e 6 d(x + y + 1)/2e for any x and y. Thus

and

a(n0 , n1 ) 6 k(n1 + dn0 /2e)(log2 (n0 ) + 1) = k(n1 + dn0 /2e)(log2 (2n0 )).

49

We are indebted to John Wilkinson for this proof

Stepanov, McJones Elements of Programming March 14, 2008 578 / 880

Complexity of merge_n_in_place: number of

comparisons

Theorem

Let c(n0 , n1 ) be the maximum number of comparisons required for a

stable in-place merge of a range of length n0 with a range of length n1 ,

where n0 6 n1 ; then c(n0 , n1 ) 6 kn1

Exercise

Determine the bound k and prove the theorem

merger_in_place

requires(Mutable(I) && BidirectionalIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))

struct merger_in_place

{

R r;

merger_in_place(R r) : r(r) { }

void operator()(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1)

{

// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)

merge_n_in_place(f0, n0, f1, n1, r);

}

};

stable_sort_n_in_place

requires(Mutable(I) && ForwardIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))

I stable_sort_n_in_place(I f, DistanceType(I) n, R r)

{

return merge_sort_n(f, n, merger_in_place<I, R>(r));

}

merge_n

template <typename I, typename B, typename R>

requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && ForwardIterator(B) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&

ValueType(I) == ValueType(B))

void merge_n(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1,

B b, DistanceType(I) n_b, R r) {

// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)

if (min(n0, n1) <= n_b) { merge_n_with_buffer(f0, n0, f1, n1, b, r); return; }

I m0, m1; DistanceType(I) h0, h1;

if (n0 < n1) {

h0 = half_nonnegative(n0); m0 = f0 + h0;

m1 = lower_bound_n(f1, n1, source(m0), r); h1 = m1 - f1;

I m = rotate(m0, f1, m1);

merge_n(f0, h0, m0, h1, b, n_b, r);

merge_n(successor(m), n0 - successor(h0), m1, n1 - h1, b, n_b, r);

} else {

h1 = half_nonnegative(n1); m1 = f1 + h1;

m0 = upper_bound_n(f0, n0, source(m1), r); h0 = m0 - f0;

I m = rotate(m0, f1, successor(m1));

merge_n(f0, h0, m0, h1, b, n_b, r);

merge_n(m, n0 - h0, successor(m1), n1 - successor(h1), b, n_b, r);

}

}

Stepanov, McJones Elements of Programming March 14, 2008 582 / 880

merger

requires(Mutable(I) && BidirectionalIterator(I) &&

Mutable(B) && BidirectionalIterator(B) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&

ValueType(I) == ValueType(B))

struct merger_adaptive

{

typedef DistanceType(B) N;

B b;

N n_b;

R r;

merger_adaptive(B b, N n_b, R r) : b(b), n_b(n_b), r(r) { }

void operator()(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1)

{

// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)

// Precondition: the range beginning with b is of size at least min(n0, n1)

merge_n(f0, n0, f1, n1, b, n_b, r);

}

};

stable_sort_n

requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && ForwardIterator(B) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&

ValueType(I) == ValueType(B))

I stable_sort_n(I f, DistanceType(I) n, B b, DistanceType(I) n_b, R r)

{

return merge_sort_n(f, n, merger_adaptive<I, B, R>(b, n_b, r));

}

Importance of stable in-place and adaptive merge and

sort

never be used

Its simple extension to the memory-adaptive algorithm leads to a

good practical algorithm when a buffer of as low as 1% is available

and becomes extremely competitive with 10-25%

50

Donald Knuth ranked this problem as 47 out of 50 – just a notch below the Last

Fermat Theorem – in the first edition Volume 3 of his Art of Computer Programming

Stepanov, McJones Elements of Programming March 14, 2008 585 / 880

The origin of computer sorting and binary searching

copying merge sort was John W. Mauchly’s 1946 lecture “Sorting

and Collating”51

51

Martin Campbell-Kelly and Michael R. Williams, editors. The Moore School

Lectures: Theory and Techniques for Design of Electronic Digital Computers. The

MIT Press and Tomash Publishers, 1985.

Stepanov, McJones Elements of Programming March 14, 2008 586 / 880

Conclusions

Mathematicians use definitions and theorems as building blocks

for ever more powerful results

This chapter shows how a difficult problem of stable sorting in

minimal space can be effectively solved using simple building

blocks

Copying (forward and backward)

Rotate (for various iterator categories)

Lower and upper bound

It provides specific examples of why our attention to interfaces

(arguments and return values) is important

It shows that the technique of memory adaptive algorithms,

which we first introduced in a rather artifical context (reverse for

forward iterators), leads to a simple, efficient version of an

important algorithm (stable sort)

Lower and upper bound, merge, and sort allow sequences to be

used efficiently as representations of sets and multisets

Stepanov, McJones Elements of Programming March 14, 2008 587 / 880

Project

Project

Design a benchmark comparing performance of

unstable_bidirectional_partition, single_cycle_partition,

sentinel_partition, and sentinel_single_cycle_partition for

different array sizes, element sizes, and distributions of elements

satisfying the predicate

Design a benchmark to analyze the performance of stable_sort_n

for different array sizes, element sizes, and buffer sizes

Contents I

1 Introduction

2 Foundations

5 Orderings

6 Combining concepts

Contents II

8 Permutations and rearrangements

9 Rotations

11 Coordinate structures

Coordinate structures

Linkable iterators

Linking operations

Link rearrangement

Reversing a linkable range

Examples of linkable iterators

Generalized rearrangements

Partition

Contents III

Merging

Sorting

Temporarily breaking invariants

Bifurcate coordinates

Traversing trees

Similarity, equivalence, and ordering

Conclusion

Projects

12 Composite objects

14 Mathematical notation

Contents IV

15 C++ machinery

16 Acknowledgments

17 Index

Intuition for coordinate structures

generalize pointers and describe linear traversals within any

type-homogenous data structure

To allow traversal within arbitrary data structures, we generalize

iterators to a family of concepts called coordinate structures

While an iterator type has only linear traversals (e.g., i 7→ i + 1 or

i 7→ i + n), and a single static value type, a coordinate structure

may have several interrelated coordinate types, each with its own

value type and diverse traversal functions

Coordinate structures abstract the navigational aspects of data

structures, while containers (introduced in the next chapter)

abstract storage management and ownership

There could be multiple coordinate structures describing the same

set of objects

Definition of coordinate structure

Definition

A concept is a coordinate structure if it consists of:

One or more coordinate types

For each coordinate type Ti , a value type ValueT ype(Ti )

For each coordinate type, dereferencing functions source and/or

sink providing “fast” readable, writable, or mutable access

Access via a coordinate value is as fast as via any other mechanism,

so there is no need to “cache” a low-level pointer or reference

For each coordinate type Ti , one or more traversal functions:

fi,j : Ti × · · · → Tk

Reflections on coordinate structure

related concepts

It is not a concept because it does not have specific coordinate types

or traversal functions

No algorithms can be defined on coordinate structure as such;

meta-algorithms are possible but are outside the scope of this book

Taxonomy of coordinate structures

Related types Most coordinate structures have a single

coordinate type; a few have two or more

Dereference regularity The dereferencing functions on a coordinate

type may be regular (repeatable) or may be

weak, meaning a protocol must be followed

between dereferences

Topology The traversal functions may allow linear,

bifurcating, multi-dimensional, or graph

traversal

Traversal regularity The traversal functions may be regular

(supporting multi-pass algorithms), or weak

(supporting only single-pass algorithms)

Complexity The traversal functions may have linear,

logarithmic, or amortized constant-time

complexity

Changeable topology The topology between coordinate values may

be fixed or mutable

Stepanov, McJones Elements of Programming March 14, 2008 596 / 880

Plan for the chapter

they are concepts of linear fixed-topology coordinates

This chapter introduces coordinate structures illustrating more

dimensions in our taxonomy:

Linkable iterators: changeable topology, with different algorithms

for partition, merging, and sorting

Bifurcated coordinates: nonlinear topologies, with algorithms on

binary trees

Linkable bifurcated coordinates: algorithms for iterative,

constant-space binary tree traversal

Linkable iterators

these relationships between linkable iterators are mutable

A linkable range is a range defined by a pair of linkable iterators

[f, l)

The mutability of the relationships between linkable iterators

requires care to preserve the invariants of ranges:

Iterators in a range are totally ordered by ≺

Multiple input or output ranges of an algorithm are disjoint

The iterators in the input to an algorithm are preserved in its output

Linking operations for linkable iterators

successor and predecessor for a given type

We treat the linking operation differently, passing them as

parameters, because of different ways linking is performed on the

same type

Similarly, in chapter 3 the operation was an explicit parameter to

power

To define the requirements on the type of objects performing

linking operations, we define related concepts ForwardLinker,

BackwardLinker, and BidirectionalLinker

Concept ForwardLinker

Definition

ForwardLinker(S) ⇒

A type function IteratorT ype(S) is defined

ForwardIterator(IteratorT ype(S))

An object of type S is a proper mutator taking two iterators of the

affiliated type

For all set_link ∈ S, immediately after set_link(i, j) is performed,

successor(i) = j

Concept BackwardLinker

Definition

BackwardLinker(S) ⇒

A type function IteratorT ype(S) is defined

BidirectionalIterator(IteratorT ype(S))

An object of type S is a proper mutator taking two iterators of the

affiliated type

For all set_link ∈ S, immediately after set_link(i, j) is performed,

i = predecessor(j)

Concept BidirectionalLinker

Definition

BidirectionalLinker(S) ⇒

ForwardLinker(S)

BackwardLinker(S)

Link rearrangements

producing a linkable range, with the properties:

Precondition (The input range is well-formed, i.e., doesn’t

contain a cycle)

Every iterator in the input range appears in the

output range

Every iterator in the output range appeared in the

input range

Every iterator in the output range designates the

same object as before the rearrangement, and this

object has the same value

successor and predecessor relationships that held

in the input range may or may not hold in the

output range

52

The rearrangements introduced in Chapter 7 rearrange data; link rearrangements

rearrange links

Stepanov, McJones Elements of Programming March 14, 2008 603 / 880

reverse_linkable

requires(ForwardLinker(S))

IteratorType(S) reverse_linkable(IteratorType(S) f, IteratorType(S) l,

S set_link)

{

typedef IteratorType(S) I;

I r = l;

while (f != l) {

I tmp = successor(f);

set_link(f, r);

r = f;

f = tmp;

}

return r;

}

Concept LinkableForwardIterator

Definition

LinkableForwardIterator(I) ⇒

ForwardIterator(I)

For all i ∈ I,

source(i.p).forward_link = successor(i)

For all j ∈ I, immediately after sink(i.p).forward_link ← j,

successor(i) = j

Concept LinkableBidirectionalIterator

Definition

LinkableBidirectionalIterator(I) ⇒

BidirectionalIterator(I) ∧ LinkableForwardIterator(I)

For all j ∈ I,

source(j.p).backward_link = predecessor(j)

For all i ∈ I, immediately after sink(j.p).backward_link ← i,

i = predecessor(j)

forward_linker, backward_linker and

bidirectional_linker

template <typename I> requires(LinkableForwardIterator(I))

struct forward_linker

{

void operator()(I x, I y)

{ sink(x.p).forward_link = y.p; }

};

struct backward_linker

{

void operator()(I x, I y)

{ sink(y.p).backward_link = x.p; }

};

struct bidirectional_linker

{

void operator()(I x, I y)

{ sink(x.p).forward_link = y.p; sink(y.p).backward_link = x.p; }

};

reverse_forward_linkable and

reverse_bidirectional_linkable

requires(LinkableForwardIterator(I))

I reverse_forward_linkable(I f, I l)

{

return reverse_linkable(f, l, l, forward_linker<I>());

}

requires(LinkableBidirectionalIterator(I))

I reverse_bidirectional_linkable(I f, I l)

{

return reverse_linkable(f, l, l, bidirectional_linker<I>());

}

Generalized rearrangements

and producing l > 1 ranges

Copying merge takes two input ranges and produces one output

range

In-place partition may be viewed as taking one input range and

producing two output ranges

In-place rotate may be viewed as taking two input ranges ([f, m)

and [m, l)) and producing two output ranges ([f, m 0 ) and [m 0 , l))

Generalized link rearrangements

Precondition The input ranges must be disjoint

the output ranges

Every iterator in an output range appeared in one

of the input ranges

Every iterator in the output range designates the

same object as before the rearrangement, and this

object has the same value

successor and predecessor relationships that held

in the input range may or may not hold in the

output range

The output ranges are disjoint

partition_linkable_unoptimized

template <typename S, typename P>

requires(ForwardLinker(S) && Readable(IteratorType(S)) &&

UnaryPredicate(P) && Domain(P) == ValueType(IteratorType(S)))

pair<pair<IteratorType(S), IteratorType(S)>,

pair<IteratorType(S), IteratorType(S)> >

partition_linkable_unoptimized(IteratorType(S) f, IteratorType(S) l, P p,

S set_link)

{

typedef IteratorType(S) I;

I h0 = l; I t0 = l;

I h1 = l; I t1 = l;

while (f != l) {

if (p(source(f))) {

if (h1 == l) { h1 = f; t1 = f; } else { set_link(t1, f); t1 = f; }

} else {

if (h0 == l) { h0 = f; t1 = f; } else { set_link(t0, f); t0 = f; }

}

++f;

}

return pair<pair<I, I>, pair<I, I> >

(pair<I, I>(h0, t0), pair<I, I>(h1, t1));

}

Stepanov, McJones Elements of Programming March 14, 2008 611 / 880

Improving on partition_linkable_unoptimized

after the not-satisfying and satisfying objects are found

By maintaining a state indicating whether the predicate was

satisfied for the previous element, set_link calls that do not

change the successor can be avoided

The state can be represented by the location in the code rather

than by an explicit state variable; using goto avoids setting and

testing a state variable53

53

Undisciplined use of goto leads to code that is harder to understand and less

likely to be correct; disciplined use frequently improves clarity and efficiency.

Benchmarking is required to verify that versions with goto are faster on processors

with instruction level parallelism.

Stepanov, McJones Elements of Programming March 14, 2008 612 / 880

partition_linkable signature

requires(ForwardLinker(S) && Readable(IteratorType(S)) &&

UnaryPredicate(P) && Domain(P) == ValueType(IteratorType(S)))

pair<pair<IteratorType(S), IteratorType(S)>,

pair<IteratorType(S), IteratorType(S)> >

partition_linkable(IteratorType(S) f, IteratorType(S) l, P p, S set_link)

{

typedef IteratorType(S) I;

partition_linkable body

entry: I h0 = l; I t0 = l; I h1 = l; I t1 = l;

if (f == l) { goto exit; }

if (p(source(f))) { h1 = f; goto s1; }

else { h0 = f; goto s0; }

s0: t0 = f; ++f;

if (f == l) { goto exit; }

if (p(source(f))) { h1 = f; goto s3; }

else { goto s0; }

s1: t1 = f; ++f;

if (f == l) { goto exit; }

if (p(source(f))) { goto s1; }

else { h0 = f; goto s2; }

s2: t0 = f; ++f;

if (f == l) { goto exit; }

if (p(source(f))) { set_link(t1, f); goto s3; }

else { goto s2; }

s3: t1 = f; ++f;

if (f == l) { goto exit; }

if (p(source(f))) { goto s3; }

else { set_link(t0, f); goto s2; }

exit: return pair<pair<I, I>, pair<I, I> >

(pair<I, I>(h0, t0), pair<I, I>(h1, t1));

}

merge_linkable_nonempty

requires(ForwardLinker(S) && Readable(IteratorType(S)) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))

triple<IteratorType(S), IteratorType(S), IteratorType(S)>

merge_linkable_nonempty(IteratorType(S) f0, IteratorType(S) l0,

IteratorType(S) f1, IteratorType(S) l1, R r, S set_link)

{

typedef IteratorType(S) I;

typedef triple<I, I, I> Triple;

entry: I h, t;

if (r(source(f1), source(f0))) { h = f1; goto s1; }

else { h = f0; goto s0; }

s0: t = f0; ++f0;

if (f0 == l0) { set_link(t, f1); return Triple(h, t, l1); }

if (r(source(f1), source(f0))) { set_link(t, f1); goto s1; }

else { goto s0; }

s1: t = f1; ++f1;

if (f1 == l1) { set_link(t, f0); return Triple(h, t, l0); }

if (r(source(f1), source(f0))) { goto s1; }

else { set_link(t, f0); goto s0; }

}

Interface of merge_linkable_nonempty

returns the beginning and end of the merged linkable range as the

first and third elements of a triple

It returns the last-visited iterator in the output range as the second

element of the triple; this iterator could be used with the following

procedure to link the range to another linkable range

requires(LinkableForwardIterator(I))

void link_at_end(I f0, I l0, I f1)

{

// Precondition: f0 , l0

I next = successor(f0);

while (next != l0) {

f0 = next;

++next;

}

set_successor(f0, f1);

}

Properties of partition_linkable and

merge_linkable_nonempty

every set_link call links to an iterator occurring not earlier in the

original list(s)

sort_linkable_n

template <typename S, typename R>

requires(ForwardLinker(S) && Readable(IteratorType(S)) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))

pair<IteratorType(S), IteratorType(S)>

sort_linkable_n(IteratorType(S) f, DistanceType(IteratorType(S)) n,

R r, S set_link)

{

// Precondition: n > 0

typedef IteratorType(S) I;

typedef triple<I, I, I> Triple;

typedef pair<I, I> Pair;

if (n == 1) return Pair(f, successor(f));

Pair p1 = sort_linkable_n(f, half_nonnegative(n), r, set_link);

Pair p2 = sort_linkable_n(p1.second, n - half_nonnegative(n), r, set_link);

Triple t = merge_linkable_nonempty(p1.first, p1.second,

p2.first, p2.second, r, set_link);

link_at_end(t.second, t.third, p2.second);

return Pair(t.first, p2.second);

}

poor locality of reference limits its usefulness if the linked

structure does not fit into cache memory

Stepanov, McJones Elements of Programming March 14, 2008 618 / 880

Exercise

Exercise

sort_linkable_n and merge_sort_n from Chapter 9 are very similar;

figure out how to unify them

sort_linkable_recursive

requires(ForwardLinker(S) && Readable(IteratorType(S)) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))

IteratorType(S) sort_linkable_recursive(IteratorType(S) f, IteratorType(S) l,

R r, S set_link)

{

if (f == l) return f;

else return sort_linkable_n(f, l - f, r, set_link).first;

}

sort_forward_linkable_recursive

requires(Readable(I) && LinkableForwardIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))

I sort_forward_linkable_recursive(I f, I l, R r)

{

return sort_linkable_recursive(f, l, r, forward_linker<I>());

}

Temporarily breaking invariants

temporarily violated during an update of those variables

For example, there is a point during the execution of swap(x, y)

when both x and y are copies of the same value

Most linked rearrangements do not depend on predecessor links

Maintaining the invariant i = predecessor(successor(i)) requires

extra time; for sorting this requires n log2 n extra time

Parameterizing the ForwardLinker used by a linked rearrangement

allows the caller to control when to restore the predecessor

invariant

restore_backward_links

requires(BackwardLinker(S))

void restore_backward_links(IteratorType(S) f, IteratorType(S) l, S set_link)

{

while (f != l) {

set_link(f, successor(f));

++f;

};

}

sort_bidirectional_linkable_recursive

requires(Readable(I) && LinkableBidirectionalIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))

I sort_bidirectional_linkable_recursive(I f, I l, R r)

{

f = sort_linkable_recursive(f, l, r, forward_linker<I>());

restore_backward_links(f, l, backward_linker<I>());

return f;

}

Forward versus bidirectional linkable structures

and merge (hence merge sort) of forward linkable structures

Adding backward links to a forward linkable structure does not

appear to improve algorithms on the structure as a whole

However, backward links allow constant-time removal and

insertion of elements at an arbitrary location

Since it is the efficiency of insertion and deletion that is often the

reason for choosing linkable structures in the first place,

bidirectional linkage should be seriously considered

Concept WeakBifurcateCoordinate

Definition

WeakBifurcateCoordinate(T ) ⇒

Regular(T )

The type function WeightT ype(T ) is defined

Integer(WeightT ype(T ))

The unary predicates is_empty, has_left_successor and

has_right_successor are defined on T

The actions step_left and step_right, and the corresponding

transformations left_successor and right_successor, are defined

on T

AmortizedConstantTime(step_left) ∧

AmortizedConstantTime(step_right)

Implication of weakness of WeakBifurcateCoordinate

an algorithm takes one of the branches, the other branch can never

be taken

Concept BifurcateCoordinate

Definition

BifurcateCoordinate(T ) ⇒

WeakBifurcateCoordinate(T )

RegularAction(step_left) ∧ RegularAction(step_right)

Concept BidirectionalBifurcateCoordinate (1 of 3)

BidirectionalBifurcateCoordinate(T ) ⇒

BifurcateCoordinate(T )

The unary predicate has_predecessor is defined on T

The action step_up and the corresponding transformation

predecessor are defined on T

RegularAction(step_up)

AmortizedConstantTime(step_up)

Concept BidirectionalBifurcateCoordinate (2 of 3)

Definition (signatures)

template <typename C>

requires(BidirectionalBifurcateCoordinate(C))

bool is_left_successor(C c)

{

// Precondition: has_predecessor(c)

C parent = predecessor(c);

return has_left_successor(parent) && left_successor(parent) == c;

}

requires(BidirectionalBifurcateCoordinate(C))

bool is_right_successor(C c)

{

// Precondition: has_predecessor(c)

C parent = predecessor(c);

return has_right_successor(parent) && right_successor(parent) == c;

}

Concept BidirectionalBifurcateCoordinate (3 of 3)

Definition (axioms)

For any coordinate x ∈ T :

has_left_successor(x) ⇒ predecessor(left_successor(x)) = x

has_right_successor(x) ⇒ predecessor(right_successor(x)) = x

has_predecessor(x) ⇒

is_left_successor(x) ∨ is_right_successor(x)

Reachability of bifurcate coordinates

template <typename C>

requires(BifurcateCoordinate(C))

bool reachable(C x, C y)

{

if (x == y) return true;

if (has_left_successor(x)) return reachable(left_successor(x), y);

if (has_right_successor(x)) return reachable(right_successor(x), y);

return false;

}

requires(BifurcateCoordinate(C))

bool left_reachable(C x, C y)

{

return has_left_successor(x) && reachable(left_successor(x), y);

}

requires(BifurcateCoordinate(C))

bool right_reachable(C x, C y)

{

return has_right_successor(x) && reachable(right_successor(x), y);

}

Properties of bifurcate coordinates

Definition

The descendants of a bifurcate coordinate x are all the coordinates y

such that reachable(x, y)

Definition

The descendants of x are acyclic if for all y in the descendants of x,

¬left_reachable(y, y) ∧ ¬right_reachable(y, y)

Definition

The descendants of x are a tree if they are acyclic and for all y, z in the

descendants of x, ¬left_reachable(y, z) ∨ ¬right_reachable(y, z)

weight_recursive

requires(BifurcateCoordinate(C))

WeightType(C) weight_recursive(C f)

{

WeightType(C) w(1);

if (has_left_successor(f)) w += weight_recursive(left_successor(f));

if (has_right_successor(f)) w += weight_recursive(right_successor(f));

return w;

}

Lemma

The descendants of x are a finite tree if and only if they are a tree and

weight_recursive(x) terminates

height_recursive

requires(BifurcateCoordinate(C))

WeightType(C) height_recursive(C f)

{

WeightType(C) l(0), r(0);

if (has_left_successor(f)) l = height_recursive(left_successor(f));

if (has_right_successor(f)) r = height_recursive(right_successor(f));

return 1 + max(l, r);

}

Lemma

height(x) 6 weight(x)

Recursive tree traversal

requires(BifurcateCoordinate(I) &&

UnaryProcedure(Pre) && Domain(Pre) == ValueType(C) &&

UnaryProcedure(In) && Domain(Pre) == ValueType(C) &&

UnaryProcedure(Post) && Domain(Pre) == ValueType(C))

void traverse_nonempty_recursive(C r, Pre f, In g, Post h)

{

f(source(r));

if (has_left_successor(r))

traverse_nonempty_recursive(left_successor(r), v);

g(source(r));

if (has_right_successor(r))

traverse_nonempty_recursive(right_successor(r), v);

h(source(r));

}

Problems with recursive traversal

can be as large as the weight, which is often unacceptable for

large, unbalanced trees

The interface is not flexible, for example it cannot be used to

compare two trees

We will now develop low-cost iterator-like traversal mechanisms

traverse_step

requires(BidirectionalBifurcateCoordinate(C))

void traverse_step(C& c, visit& v)

{

// Precondition: has_predecessor(c) ∨ v , post

switch (v) {

case pre:

if (has_left_successor(c)) step_left(c);

else v = in;

break;

case in:

if (has_right_successor(c)) { v = pre; step_right(c); }

else v = post;

break;

case post:

if (is_left_successor(c)) v = in;

step_up(c);

}

}

Iterative tree traversal

requires(BidirectionalBifurcateCoordinate(C) &&

UnaryProcedure(Pre) && Domain(Pre) == ValueType(C) &&

UnaryProcedure(In) && Domain(Pre) == ValueType(C) &&

UnaryProcedure(Post) && Domain(Pre) == ValueType(C))

void traverse(C c, Pre f_pre, In f_in, Post f_post)

{

if (is_empty(c)) return;

C root = c;

visit v = pre;

while (true) {

switch (v) {

case pre: f_pre (source(c)); break;

case in: f_in (source(c)); break;

case post: f_post(source(c)); if (c == root) return;

}

traverse_step(c, v);

}

}

Binary trees and iterators

iterating through a range because of the three directions of

movement

traverse_step and traverse provide full access to the structure of

a binary tree

It is possible to write iterator adaptors to traverse_step that

provide sequential access to the nodes of a binary tree in preorder,

inorder, or postorder, but such an iterator loses structural

information

An iterator adapter whose value type pairs a visit with a

coordinate preserves the structural information

(See the Projects section at the end of this chapter for more details)

Advantages of bidirectional binary trees

and deletion of elements

With binary trees the situation is different: without predecessor

links even the simplest traversal algorithms require linear space

Adding predecessor links allows constant-space traversal, and

also makes possible various rebalancing algorithms

The overhead for the extra link, while 50% in the worst case, is

typically much less in practice

Linkable bifurcate coordinates

parameter because of the need to use different linking operations,

such as restoring back links after sort

For linkable bifurcate coordinates there does not appear to be a

need for alternate versions of the linking operations, so we

overload them on the concept

Concept LinkableBifurcateCoordinate

Definition

LinkableBifurcateCoordinate(T ) ⇒

BifurcateCoordinate(T )

Proper mutators set_left_successor and set_right_successor

taking two iterators of type T are defined

Immediately after set_left_successor(i, j) is performed,

left_succesor(i) = j

Immediately after set_right_successor(i, j) is performed,

right_succesor(i) = j

Concept LinkableBidirectionalBifurcateCoordinate

Definition

LinkableBidirectionalBifurcateCoordinate(T ) ⇒

LinkableBifurcateCoordinate(T )

Immediately after set_left_successor(i, j) is performed,

has_left_successor(i) ⇒ predecessor(j) = i

Immediately after set_right_successor(i, j) is performed,

has_right_successor(i) ⇒ predecessor(j) = i

Reversing links

traverse_step is an efficient way to traverse via bidirectional

bifurcating coordinates, but requires the predecessor function

When the predecessor function is not available and recursive

(stack-based) traversal is unacceptable because of unbalanced

trees, there are interesting algorithms that depend on temporarily

storing the link to the predecessor in a link normally containing a

successor; this assures there is a path back to the root 54

54

Link reversal was introduced in: H. Schorr and W.M. Waite. An Efficient and

Machine-Independent Procedure for Garbage Collection in Various List Structures.

Communications of the ACM, Volume 10, Number 8, August 1967, pages 501-506 (it was

independetly discovered by L.P. Deutsch). A version without tag bits was published

in: J.M. Robson. An Improved Algorithm for Traversing Binary Trees Without

Auxiliary Stack. Information Processing Letters, Volume 2, 1973, pages 12-14; and Joseph

M. Morris. Traversing Binary Trees Simply and Cheaply, Information Processing Letters,

Volume 9, Number 5, 1979, pages 197-200. An ingenious technique of “rotating” the

links was published in: Gary Lindstrom. Scanning List Structures Without Stack or

Tag Bits. Information Processing Letters, Volume 2, 1973, pages 47-51; and

independently in: Barry Dwyer. Simple Algorithms for Traversing a Tree Without an

Auxiliary Stack. Information Processing Letters, Volume 2, 1974, pages 143-145.

Stepanov, McJones Elements of Programming March 14, 2008 645 / 880

Exclusive-or’ing links

storage for predecessor links stores in each successor link the

exclusive-or of the predecessor and the corresponding successor55

By additionally requiring that the address of a node’s left

successor is less than the address of its right successor, it is

possible to design a data structure that supports a non-recursive

traversal algorithm that does not modify the tree and uses only a

constant amount of additional storage

55

The exclusive-or technique was first applied to binary trees in: Laurent Siklóssy.

Fast and Read-only Algorithms for Traversing Trees Without an Auxiliary Stack.

Information Processing Letters, Volume 1, 1972, pages 149-152.

Stepanov, McJones Elements of Programming March 14, 2008 646 / 880

Similarity

Definition

Two sets S and S 0 of coordinates from the same coordinate type T of a

coordinate structure are similar under a given one-to-one

correspondence if any traversal function maps corresponding

coordinates to corresponding coordinates

To coordinate structures with more than one coordinate type

To sets of coordinates from different but isomorphic coordinate

structures

Similarity does not depend on the values of the objects pointed to

by the coordinates

Algorithms for testing similarity use only traversal functions but

not dereferencing functions

Examples of similarity

Examples

Two ranges of iterators [f, l) and [f 0 , l 0 ) are similar if l − f = l 0 − f 0

Two trees are similar if both are empty or they have similar left

and right subtrees

Equivalence

Definition

Two sets S and S 0 of coordinates from the same coordinate type T of a

coordinate structure represent equivalent value sets under a given

one-to-one correspondence and a given equivalence relation if they are

similar and dereferencing corresponding coordinates gives equivalent

objects

lexicographic_equivalence

requires(Readable(I0) && Iterator(I0) &&

Readable(I1) && Iterator(I1) &&

ValueType(I0) == ValueType(I1) &&

EquivalenceRelation(R) && Domain(R) == ValueType(I0))

bool lexicographic_equivalence(I0 f0, I0 l0, I1 f1, I1 l1, R r)

{

while (true) {

if (f0 == l0 && f1 == l1) return true;

if (f0 == l0 || f1 == l1) return false;

if (!r(source(f0), source(f1))) return false;

++f0; ++f1;

}

}

lexicographic_ordering

requires(Readable(I0) && Iterator(I0) &&

Readable(I1) && Iterator(I1) &&

ValueType(I0) == ValueType(I1) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I0))

bool lexicographic_ordering(I0 f0, I0 l0, I1 f1, I1 l1, R r)

{

while (true) {

if (f1 == l1) return false;

if (f0 == l0) return true;

if (r(source(f0), source(f1))) return true;

if (r(source(f0), source(f1))) return false;

++f0; ++f1;

}

}

of an ordering

bifurcate_similarity

requires(BidirectionalBifurcateCoordinate(C))

bool bifurcate_similarity(C c0, C c1)

{

if (is_empty(c0) || is_empty(c1)) return is_empty(c0) && is_empty(c1);

C root0 = c0; C root1 = c1;

visit v0 = pre; visit v1 = pre;

while (true) {

traverse_step(c0, v0); traverse_step(c1, v1);

if (d0 != d1) return false;

if (c0 == root0 && v0 == post) return true;

}

}

bifurcate_equivalence

requires(BidirectionalBifurcateCoordinate(C) &&

EquivalenceRelation(R) && Domain(R) == ValueType(C))

bool bifurcate_equivalence(C c0, C c1, R r)

{

if (is_empty(c0) || is_empty(c1)) return is_empty(c0) && is_empty(c1);

C root0 = c0; C root1 = c1;

visit v0 = pre; visit v1 = pre;

while (true) {

if (v0 == pre && !r(source(c0), source(c1))) return false;

traverse_step(c0, v0); traverse_step(c1, v1);

if (v0 != v1) return false;

if (c0 == root0 && v0 == post) return true;

}

}

bifurcate_ordering

requires(BidirectionalBifurcateCoordinate(C) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(C))

bool bifurcate_ordering(C c0, C c1, R r)

{

if (is_empty(c0) || is_empty(c1)) return is_empty(c0) && !is_empty(c1);

C root0 = c0; C root1 = c1;

visit v0 = pre; visit v1 = pre;

while (true) {

if (v0 == pre) {

if (r(source(c0), source(c1))) return true;

if (r(source(c1), source(c0))) return false;

}

traverse_step(c0, v0); traverse_step(c1, v1);

if (v0 != v1) return v0 > v1;

if (c0 == root0 && v0 == post) return false;

}

}

Conclusions

generalizations:

Linkable iterators allow the topology between iterators to be

changed rather than changing the objects to which the iterators

point

Bifurcate coordinates extend from linear to binary tree topology

There is, of course, a wide variety of different coordinate structures

No computational device (e.g., goto) is harmful per se; it could and

should be used when appropriate

An invariant is associated with a scope, such as a loop, an

algorithm, or a datatype

Outside the scope, the invariant must hold

Inside the scope, it is permissible to violate the invariant

Proper maintenance of the invariant requires attention to

concurrency and exceptions

Projects

Project 1

Design an adapter that converts traverse_step into an iterator, taking

into consideration the following issues:

What should the value type be?

What iterator concept should be modeled?

What should the limit iterator be to allow traversal of a subtree

rooted at an arbitrary coordinate?

Project 2

Extend ordering to an arbitrary coordinate structure by defining an

appropriate meta-algorithm

Contents I

1 Introduction

2 Foundations

5 Orderings

6 Combining concepts

Contents II

8 Permutations and rearrangements

9 Rotations

11 Coordinate structures

12 Composite objects

Composite objects, regular types, and containers

Equality on composite objects

Collocated types

Distributed types

Construction

Contents III

Memory allocation and deallocation

Explicit construction and destruction

Lists

Arrays

Containers as elements

Relational concepts

Compatibility

Generalized copy algorithms

Underlying type

Generalized permutation algorithms

More generalized permutation algorithms

Generalized partition algorithms

Underlying iterator

More generalized partition algorithms

underlying_compare

Contents IV

More generalized merge and sort algorithms

Composite objects

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

17 Index

Definition of composite object (1 of 4)

Definition

A type T belongs to a concept that is a composite object concept if:

Parts Objects of type T are made up of other objects called

parts

Subparts Subsidiary definition: x is a subpart of y if x is a part of y

or x is a subpart of a part of y

Definition of composite object (2 of 4)

Definition

Connected T has an affiliated coordinate structure

Every object of type T has a distinguished starting

address

It is possible to reach every part of an object of type T

using traversals in the coordinate structure, beginning

at the starting address

Definition of composite object (3 of 4)

Definition

Noncircular No object is a subpart of itself

Disjoint Subsidiary definition: Two objects are disjoint if they

have no subpart in common

If x and y are objects, either one is a subpart of the

other or they are disjoint

Definition of composite object (4 of 4)

Definition

Complete destruction Destroying an object of type T causes the

destruction of every part of the object (and,

therefore, every subpart)

Reflections on composite objects

a family of related concepts

No algorithms can be defined on composite objects as such;

meta-algorithms are possible but are outside the scope of this book

Examples of composite objects

Example

A tuple: traversal via member access

An array: traversal via random access iterator

A binary tree: traversal via bifurcate coordinate

A complex number: traversal via member access

Regularity and composite objects

The requirements on the element type allowing objects to be

sorted should be satisfied by arrays themselves, so we can sort

an array of arrays

objects in such a way that they satisfy the requirements for regular

types

Equality

Assignment (respecting equality)

Default construction (making an object assignable/destructable)

Copy construction (equivalent to default construction followed by

assignment)

Total ordering (natural or default)

Destruction

Container

Definition

A container is a composite object whose interpretation is a collection of

the interpretations of the parts of the container

Example

A tuple

An array

A binary tree

The role of containers

consisting only of sets

The computer science analog of sets is containers, and they come

in many varieties

Coordinate structure of a container

Definition

Each container type defines a corresponding coordinate structure and

functions giving one or more specific coordinates to initiate traversals

or access

Example

Tuple: member access

Array: random-access iterators and begin and end

Binary tree: bifurcate coordinates and root

Equality on containers

Definition

Two containers are equal if corresponding parts are equal

Example

Array equality: use lexicographic_equivalence with equality on

the element type

Binary tree equality: use bifircate_equivalence with equality on

the element type

taken into account by equality; we will see examples later in this

chapter

Structural equality

in terms of consistent observed behavior

There is a related notion of structural equality

Definition

Two composite objects are structurally equal if their states are equal

Objects that are not structurally equal could still be equal in the

Leibniz sense (such objects do not have unique representations)

Structural equality versus equality

Examples

Complex numbers x + iy and x 0 + iy 0 are equal if x = x 0 ∧ y = y 0

0

Complex numbers reiϕ and r 0 eiϕ are equal if

r = r0 ∧ ϕ ≡ ϕ0 (mod 2π)

They are structurally equal (same state) ⇒

They are equal (same behavior) ⇒

They are equivalent (same under some equivalence relation)

Leibniz equality is difficult to implement for certain

types

Example

Implementing Leibniz equality for priority queues requires comparing

successive values from both queues

This consumes the elements compared, requiring copies to be

made of the two queues

It requires n log n time

Collocated types

Definition

A collocated type is one all of whose parts are adjacent in memory

Example

Given types T0 , . . . , Tk−1 , we can construct the heterogeneous type

structT0 ,...,Tk−1 with useful special cases

pairT0 ,T1

tripleT0 ,T1 ,T2

If all the Ti are the same, we can define the homogeneous type

arrayk,T

Properties of collocated types

construction, and destruction of a collocated type is elementwise;

for equality and ordering it is the lexicographic techniques

defined in the previous chapter56

Collocated types have constant area, so they can be fully allocated

at compile time

There are several situations when a collocated type is not optimal

When the area of an object is not constant, that is, its area is fixed or

dynamic

When the object might be frequently moved (e.g., an element of an

array being sorted)

56

In some situations C++ automatically generates elementwise constructors,

destructor, and assignment for struct types; equality and ordering are never

automatically generated. Constant-size arrays in C++ are not first-class citizens and

one must define a type such as the array_k in Appendix 2 to make them be regular.

Stepanov, McJones Elements of Programming March 14, 2008 676 / 880

Distributed types

Definition

A distributed type has parts that are not adjacent:

Data representation of the externally-visible behavior of the

type

Connectors traversal through all the data

Header the origin for all traversal

Padding memory locations included to maintain hardware

alignment or optimize cache behavior

Reserve memory locations reserved for future growth

Other locks, software caches, handles for system resources, etc.

Ownership: containers versus iterators

container (connectors) from pointers representing intermediate

positions within an algorithm (iterators or coordinates)

Example: list

The functions begin and end, when applied to a list, each return a list

iterator

The function successor, when applied to a list iterator, returns another

list iterator; there is no function like successor defined on list

The destructor of the listT type is responsible for iterating through all

the elements, destroying each object of type T , and freeing the storage of

its containing list node

Destroying a list iterator has no impact on any list object

list: example of container versus iterator

struct list_node

{

T value;

list_node* forward_link;

// Destructor does nothing

};

struct list_iterator

{

list_node<T>* p;

// Destructor does nothing

};

struct list

{

list_iterator<T> first;

// Destructor erases all elements

};

Primary objects

Definition

A primary object is one that is not a subpart of another object; an object

that is not primary is secondary

A primary object may have automatic storage duration

A distributed primary object may have parts with dynamic

storage duration

Construction

storage

Construction of a secondary object takes place in a subpart of a

primary object

Example

***** Show code with global object, local object, and insert constructor

?????

(Languages without ownership)

Many languages follow the example of Lisp57 use the same type

for both a list as a container and a list iterator

This design decision means it is not possible for the programmer

to explicitly free a list since there might be other variables

containing list objects with overlapping storage

57

John McCarthy. Recursive Functions of Symbolic Expressions and Their

Computation by Machine, Part I. Communications of the ACM, Volume 3, Number 4,

1960, pages 184-195

Stepanov, McJones Elements of Programming March 14, 2008 682 / 880

remote

requires(Regular(T))

struct remote

{

T* p;

remote() : p(0) {}

remote(const T& x) : p(new T(x)) { }

remote(const remote& x) : p(new T(source(x.p))) { }

~remote() { delete p; }

friend bool operator==(const remote& x, const remote& y)

{ return source(x.p) == source(y.p); }

friend void swap(remote& x, remote& y) { swap(x.p, y.p); }

void operator=(remote& x) { swap(sink(this), x); }

friend bool operator<(const remote& x, const remote& y)

{ return source(x.p) < source(y.p); }

};

Allocation and deallocation

void* raw_allocate(size_t n)

{

return ::operator new(n);

}

requires(Regular(T))

T* allocate(size_t n)

{

return (T*)raw_allocate(n * sizeof(T));

}

void deallocate(void* p)

{

::operator delete(p);

}

construct_copy and destroy

requires(Regular(T))

struct construct_copy

{

const T* p;

construct_copy(const T& x) : p(&x) {}

void operator()(T& q)

{

new (&q) T(source(p));

}

};

requires(Regular(T))

void destroy(T& p)

{

sink(&p).~T();

}

list_node and list_iterator

requires(Regular(T))

struct list_node

{

T value;

list_node* forward_link;

list_node(const T& v, list_node* f) : value(v), forward_link(f) {}

};

requires(Regular(T))

struct list_iterator

{

list_node<T>* p;

list_iterator() : p(0) {}

list_iterator(list_node<T>* p) : p(p) {}

};

Functions for list_iterator

void operator++(list_iterator<T>& i) { i.p = source(i.p).forward_link; }

void set_successor(list_iterator<T> i, list_iterator<T> j)

{ sink(i.p).forward_link = j.p; }

bool operator==(list_iterator<T> i, list_iterator<T> j)

{ return i.p == j.p; }

const T& source(list_iterator<T> i) { return source(i.p).value; }

T& sink(list_iterator<T> i) { return sink(i.p).value; }

Type functions for list_iterator

requires(Regular(T))

struct value_type< list_iterator<T> >

{

typedef T type;

};

requires(Regular(T))

struct distance_type< list_iterator<T> >

{

typedef DistanceType(list_node<T>*) type;

};

requires(Regular(T))

struct iterator_category< list_iterator<T> >

{

typedef forward_iterator_tag category;

};

Insert functions for list_node

requires(Regular(T) && ConstructorObject(F) && Domain(F) == T)

list_iterator<T> insert_construct(list_iterator<T> j, F f)

{

list_iterator<T> i(allocate< list_node<T> >(1));

f(sink(i));

set_successor(i, j);

return i;

}

requires(Regular(T))

list_iterator<T> insert(list_iterator<T> i, const T& x)

{

return insert_construct(i, construct_copy<T>(x));

}

More insert functions for list_node

requires(Regular(T) && ConstructorObject(F) && Domain(F) == T)

list_iterator<T> insert_after_construct(list_iterator<T> i, F f)

{

list_iterator<T> j = insert_construct(successor(i), f);

set_successor(i, j);

return j;

}

requires(Regular(T))

list_iterator<T> insert_after(list_iterator<T> i, const T& x)

{

return insert_after_construct(i, construct_copy<T>(x));

}

list_insert_after_iterator

requires(Regular(T))

struct list_insert_after_iterator

{

list_iterator<T> h;

list_iterator<T> t;

list_insert_after_iterator() { }

list_insert_after_iterator(list_iterator<T> h, list_iterator<T> t)

: h(h), t(t) { }

void operator=(const T& x)

{

if (t == list_iterator<T>()) {

h = insert(h, x);

t = h;

} else

t = insert_after(t, x);

}

void operator++() { }

};

(Equality for list_insert_after_iterator)

requires(Regular(T))

bool operator==(const list_insert_after_iterator<T>& i,

const list_insert_after_iterator<T>& j)

{

return i.h == j.h && i.t == j.t;

}

Type functions for list_insert_after_iterator

requires(Regular(T))

struct value_type< list_insert_after_iterator<T> >

{

typedef T type;

};

requires(Regular(T))

struct distance_type< list_insert_after_iterator<T> >

{

typedef DistanceType(list_iterator<T>) type;

};

requires(Regular(T))

struct iterator_category< list_insert_after_iterator<T> >

{

typedef iterator_tag category;

};

Erase functions for list_node

requires(Regular(T))

list_iterator<T> erase_first(list_iterator<T> i)

{

list_iterator<T> j = successor(i);

destroy(sink(i));

deallocate(i.p);

return j;

}

requires(Regular(T))

void erase_after(list_iterator<T> i)

{

set_successor(i, erase_first(successor(i)));

}

list container

requires(Regular(T))

struct list

{

list_iterator<T> first;

list() : first(0) {}

template <typename I>

requires(Readable(I) && Iterator(I) && ValueType(I) == T)

list(I f, I l);

list(const list& x);

~list();

friend void swap(list& x, list& y) { swap(x.first, y.first); }

void operator=(list x) { swap(sink(this), x); }

};

Type functions for list

Definition

The type function IteratorT ype(C) is defined on any container type C;

it returns a type such that Iterator(IteratorT ype(C))

Type functions for list

requires(Regular(T))

struct iterator_type< list<T> >

{

typedef list_iterator<T> type;

};

Functions for list

template <typename T> requires(Regular(T))

IteratorType(list<T>) begin(const list<T>& x)

{

return x.first;

}

IteratorType(list<T>) end(const list<T>& x)

{

return list_iterator<T>();

}

DistanceType(IteratorType(C)) size(const C& x)

{

return end(x) - begin(x);

}

requires(Container(C))

bool is_empty(const C& x)

{

return begin(x) == end(x);

}

Stepanov, McJones Elements of Programming March 14, 2008 698 / 880

Constructors for list

template <typename T, typename I>

requires(Regular(T) && Iterator(I) &&

ValueType(I) == T)

void insert(list<T>& x, IteratorType(list<T>) i, I f, I l)

{

x.first = copy(f, l, list_insert_after_iterator<T>(begin(x), i)).h;

}

template <typename I>

requires(Regular(T) && Readable(I) && Iterator(I) &&

ValueType(I) == T)

list<T>::list(I f, I l) : first(0)

{

insert(sink(this), end(sink(this)), f, l);

}

requires(Regular(T))

list<T>::list(const list& x) : first(0)

{

insert(sink(this), end(sink(this)), begin(x), end(x));

}

Destructor for list

requires(Regular(T))

void pop_all(list<T>& x)

{

while (!is_empty(x))

x.first = erase_first(begin(x));

}

requires(Regular(T))

list<T>::~list()

{

pop_all(sink(this));

}

Equality and less than for list

template <typename T>

requires(Regular(T))

struct equality

{

bool operator()(const T& x, const T& y) { return x == y; }

};

requires(Regular(T))

bool operator==(const list<T>& x, const list<T>& y)

{

return lexicographic_equivalence(begin(x), end(x), begin(y), end(y),

equality<T>());

}

requires(Regular(T))

bool operator<(const list<T>& x, const list<T>& y)

{

return lexicographic_ordering(begin(x), end(x), begin(y), end(y), less<T>());

}

Stepanov, McJones Elements of Programming March 14, 2008 701 / 880

partition for list

requires(Regular(T) && UnaryPredicate(P) && Domain(P) == T)

void partition_list(list<T>& x, list<T>& y, P p)

{

typedef IteratorType(list<T>) I;

pair< pair<I, I>, pair<I, I> > pp =

partition_linkable(begin(x), end(x), p, forward_linker<I>());

x.first = pp.m0.m0;

if (pp.m1.m0 != end(x)) {

set_successor(pp.m1.m1, begin(y));

y.first = pp.m1.first;

}

}

merge for list

requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)

void merge(list<T>& x, list<T>& y, R r)

{

if (is_empty(y)) return;

if (is_empty(x)) x = y;

else

x.first = merge_linkable_nonempty(

begin(x), end(x), begin(y), end(y),

r, forward_linker<IteratorType(list<T>)>()).m0;

y.first = end(y);

}

sort for list

requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)

void sort_recursive(list<T>& x, R r)

{

x.first = sort_forward_linkable_recursive(begin(x), end(x), r);

}

requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)

void sort(list<T>& x, R r)

{

x.first = sort_forward_linkable<32>(begin(x), end(x), r);

}

array_header

requires(Regular(T))

struct array_header

{

T* m;

T* l;

T a;

};

[f, m) are constructed elements

[m, l) are unconstructed (reserved) elements

allocate_array_header

requires(Regular(T))

array_header<T>* allocate_array(DistanceType(T*) n)

{

typedef array_header<T>* P;

size_t bsize = size_t(predecessor(n)) * sizeof(T);

P p = P(raw_allocate(sizeof(array_header<T>) + bsize));

T* f = &sink(p).a;

sink(p).m = f;

sink(p).l = f + n;

return p;

}

deallocate_array_header

requires(Regular(T))

void deallocate_array(array_header<T>* p)

{

deallocate(p);

}

array container

requires(Regular(T))

struct array

{

typedef DistanceType(IteratorType(array<T>)) N;

array_header<T>* p;

array() : p(0) {}

array(N c); // size is 0 and capacity is c

array(N s, N c, const T& x); // size is s, capacity is c, all elements equal to x

template <typename I>

requires(Readable(I) && Iterator(I) && ValueType(I) == T)

array(I f, I l);

array(const array& x);

~array();

friend void swap(array& x, array& y) { swap(x.p, y.p); }

void operator=(array x) { swap(sink(this), x); }

};

requires(Regular(T))

void reserve(array<T>& x, DistanceType(IteratorType(array<T>)) n);

Type functions for array

requires(Regular(T))

struct iterator_type< array<T> >

{

typedef T* type;

};

Functions for array

template <typename T>

requires(Regular(T))

IteratorType(array<T>) begin(const array<T>& x)

{

if (x.p == 0) return IteratorType(array<T>)(0);

return IteratorType(array<T>)(&source(x.p).a);

}

requires(Regular(T))

IteratorType(array<T>) end(const array<T>& x)

{

if (x.p == 0) return IteratorType(array<T>)(0);

return IteratorType(array<T>)(source(x.p).m);

}

requires(Regular(T))

IteratorType(array<T>) end_of_storage(const array<T>& x)

{

if (x.p == 0) return IteratorType(array<T>)(0);

return IteratorType(array<T>)(source(x.p).l);

}

More functions for array

requires(Regular(T))

DistanceType(IteratorType(array<T>)) capacity(const array<T>& x)

{

return end_of_storage(x) - begin(x);

}

requires(Regular(T))

bool is_full(const array<T>& x)

{

return end(x) == end_of_storage(x);

}

push_construct

requires(Regular(T) && ConstructorObject(F) && Domain(F) == T)

T& push_construct(array<T>& x, F f)

{

typedef DistanceType(IteratorType(array<T>)) N;

N n = size(x);

if (n == capacity(x))

reserve(x, max(N(1), n + n));

f(sink(source(x.p).m));

++sink(x.p).m;

return sink(end(x) - 1);

}

push

requires(Regular(T))

T& push(array<T>& x, const T& y)

{

return push_construct(x, construct_copy<T>(y));

}

array_push_iterator

requires(Regular(T))

struct array_push_iterator

{

array<T>* p;

array_push_iterator() {}

array_push_iterator(array<T>& x) : p(&x) {}

friend bool operator==(array_push_iterator x, array_push_iterator y)

{

return x.p == y.p;

}

friend void operator++(array_push_iterator& x) {}

void operator=(const T& x)

{

push(sink(p), x);

}

};

Type functions for array_push_iterator

requires(Regular(T))

struct value_type< array_push_iterator<T> >

{

typedef T type;

};

requires(Regular(T))

struct distance_type< array_push_iterator<T> >

{

typedef DistanceType(IteratorType(array<T>)) type;

};

requires(Regular(T))

struct iterator_category< array_push_iterator<T> >

{

typedef iterator_tag category;

};

push for a range

requires(Regular(T) &&

Readable(I) && Iterator(I) && ValueType(I) == T)

void push(array<T>& x, I f, I l)

{

copy(f, l, array_push_iterator<T>(x));

}

Pop functions for array

requires(Regular(T))

void pop(array<T>& x)

{

--sink(x.p).m;

destroy(sink(source(x.p).m));

if (is_empty(x)) {

deallocate_array(x.p);

x.p = 0;

}

}

requires(Regular(T))

void pop_all(array<T>& x)

{

while (!is_empty(x)) pop(x);

}

Constructors for array

requires(Regular(T))

array<T>::array(DistanceType(IteratorType(array<T>)) c) :

p(allocate_array<T>(c))

{

}

requires(Regular(T))

array<T>::array(DistanceType(IteratorType(array<T>)) s,

DistanceType(IteratorType(array<T>)) c,

const T& x) :

p(allocate_array<T>(c))

{

while (s != N(0)) { push(sink(this), x); --s; }

}

More constructors for array

template <typename I>

requires(Regular(T) && Readable(I) && Iterator(I) && ValueType(I) == T)

array<T>::array(I f, I l) : p(0)

{

push<T, I>(sink(this), f, l);

}

requires(Regular(T))

array<T>::array(const array& x) : p(allocate_array<T>(size(x)))

{

push<T>(sink(this), begin(x), end(x));

}

Destructor for array

requires(Regular(T))

array<T>::~array()

{

pop_all(sink(this));

}

reserve

requires(Regular(T))

void reserve(array<T>& x, DistanceType(IteratorType(array<T>)) n)

{

if (n < size(x) || n == capacity(x)) return;

array<T> tmp(n);

copy(begin(x), end(x), array_push_iterator<T>(tmp));

swap(tmp, x);

}

Equality and less than for array

requires(Regular(T))

bool operator==(const array<T>& x, const array<T>& y)

{

return lexicographic_equivalence(begin(x), end(x), begin(y), end(y),

equality<T>());

}

requires(Regular(T))

bool operator<(const array<T>& x, const array<T>& y)

{

return lexicographic_ordering(begin(x), end(x), begin(y), end(y), less<T>());

}

insert for array

requires(Regular(T) &&

Readable(I) && Iterator(I) && ValueType(I) == T)

void insert(array<T>& x, IteratorType(array<T>) i, I f, I l)

{

DistanceType(IteratorType(array<T>)) o_f = i - begin(x);

DistanceType(IteratorType(array<T>)) o_m = end(x) - begin(x);

push(x, f, l);

rotate(begin(x) + o_f, begin(x) + o_m, end(x));

}

sort for array

requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)

void sort(array<T>& x, R r)

{

typedef DistanceType(IteratorType(array<T>)) N;

N n = size(x) / N(10);

array<T> buffer(n, n, T());

stable_sort_n(begin(x), size(x), begin(buffer), n, r);

}

Introduction

together when the elements of the containers are built-in types or

C-style structs

However performance suffers when the elements are themselves

larger containers

Sorting an array of arrays

In this section we present a solution to this problem using the

notions of underlying type and type compatibility

We rewrite versions of various algorithms from chapters 6-10

using this approach and define the underlying type for the

containers in chapter 11

Relational concepts (from Chapter 1)

Definition

A relational concept is a concept defined on two types

Concept SameGenus (from Chapter 1)

Definition

SameGenus(T , T 0 ) ⇒

The genus of T and the genus of T 0 are equal

Example

SameGenus(int, long)

Concept PartiallyCompatible (1 of 3) (from Chapter 6)

PartiallyCompatible(T , U) ⇒

SameGenus(T , U)

The partial equality predicates (=) on T × U and U × T are defined

The (potentially implicit) unary predicates is_representableU (T )

and is_representableT (U) are defined

The partial assignments (←) on T × U and U × T are defined

The morphisms U(T ) and T (U) are defined

Concept PartiallyCompatible (2 of 3) (from Chapter 6)

PartiallyCompatible(T , U) ⇒

For all t ∈ T and for all u ∈ U,

is_representableU (t) ∧ is_representableT (u) ⇒

is_defined(=, t, u) ∧ is_defined(=, u, t)

For all t ∈ T and for all u ∈ U,

is_representableT (u) ⇒ is_defined(←, t, u)

For all t ∈ T and for all u ∈ U,

is_representableU (t) ∧ is_representableT (u) ⇒

is_defined(U(·), t) ∧ is_defined(T (·), u)

Concept PartiallyCompatible (3 of 3) (from Chapter 6)

Definition (axioms)

PartiallyCompatible(T , U) ⇒

Where = is defined, it is true if and only if its arguments have the

same interpretation

← depends on the type but not the value of its first argument;

where it is defined, it makes the first argument equal to the second

without modifying the second

Where the morphisms U(T ) and T (U) are defined, each constructs

a new element of the target type that has the same interpretation

as its argument

Properties of PartiallyCompatible types (from Chapter 6)

Lemma

PartiallyCompatible(T , U) ⇒ PartiallyCompatible(U, T )

Lemma

If PartiallyCompatible(T , U), PartiallyCompatible(U, V),

PartiallyCompatible(T , V), then (∀t ∈ T )(∀u ∈ U)(∀v ∈ V):

t=u⇒u=t

(t = u ∧ u = v) ⇒ t = v

t←u⇒t=u

T t(u) ⇒ t = u

predicate holds immediately after execution of the statement)

Concept Compatible (from Chapter 6)

Definition

Compatible(T , U) ⇒

PartiallyCompatible(T , U)

For all t ∈ T and for all u ∈ U,

is_representableU (t) ∧ is_representableT (u)

Type requirements of copy (from Chapter 6)

#if 0 // *****

requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&

PartiallyCompatible(ValueType(I), ValueType(O)))

O copy(I f, I l, O r);

Iterator(I) ensures ++ is defined

Writable(O) ensures sink is defined

Iterator(O) ensures ++ is defined

PartiallyCompatible(ValueT ype(I), ValueT ype(O)) ensures

assignment is defined

Preconditions and postconditions of copy (from

Chapter 6)

template <typename I, typename O>

requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&

PartiallyCompatible(ValueType(I), ValueType(O)))

O copy(I f, I l, O r);

The output range must be writable and of at least

the size of the input range

Each value of the input range must be

representable in the output value type

If the ith iterator in the input range is aliased to the

jth iterator of the output, then i 6 j (informally,

every value is used before it is overwritten)

Postconditions The sequence of values in the output range is

equal to the sequence of original values in the

input range

Stepanov, McJones Elements of Programming March 14, 2008 734 / 880

Implementation of copy (from Chapter 6)

requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&

PartiallyCompatible(ValueType(I), ValueType(O)))

O copy(I f, I l, O r)

{

while (f != l) {

sink(r) = source(f);

++f;

++r;

}

return r;

}

known to the caller, who might find it useful

It is worth the small constant time to return it

Example using copy (from Chapter 6)

requires(Integer(N) && Writable(O) && Iterator(O) &&

PartiallyCompatible(N, ValueType(O)))

O iota(N n, O r) // like APL ι

{

return copy(N(0), n, r);

}

copy_bounded (from Chapter 6)

requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&

PartiallyCompatible(ValueType(I), ValueType(O)))

pair<I, O> copy_bounded(I f, I l, O r_f, O r_l)

{

while (f != l && r_f != r_l) {

sink(r_f) = source(f);

++f;

++r_f;

}

return pair<I, O>(f, r_f);

}

While the ends of both ranges are known to the caller, returning

the pair allows determining which range is smaller and where in

the larger range copying stopped

copy_n (from Chapter 6)

requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&

PartiallyCompatible(ValueType(I), ValueType(O)))

pair<I, O> copy_n(I f, DistanceType(I) n, O r)

{

typedef DistanceType(I) N;

while (n != N(0)) {

sink(r) = source(f);

++f;

++r;

n = n - N(1);

}

return pair<I, O>(f, r);

}

copy_k (from Chapter 6)

requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&

PartiallyCompatible(ValueType(I), ValueType(O)))

void copy_k(I& f, O& r)

{

copy_k<k - 1>(f, r);

sink(r) = source(f);

++f; ++r;

}

void copy_k<0>(I&, O&) { }

copy_n_unrolled (from Chapter 6)

requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&

PartiallyCompatible(ValueType(I), ValueType(O)))

pair<I, O> copy_n_unrolled(I f, DistanceType(I) n, I r)

{

typedef DistanceType(I) N;

const int k = 4; // unroll factor

while (n >= N(k)) {

copy_k<k>(f, r);

n = n - N(k);

}

return copy_n(f, n, r);

}

copy_backward (from Chapter 6)

ith iterator in the input range is aliased to the jth iterator of the

output, then j 6 i

requires(Readable(I) && BidirectionalIterator(I) &&

Writable(O) && BidirectionalIterator(O) &&

PartiallyCompatible(ValueType(I), ValueType(O)))

O copy_backward(I f, I l, O r)

{

while (f != l) {

--l;

--r;

sink(r) = source(l);

}

return r;

}

copy_k_indexed (from Chapter 6)

requires(Readable(I) && IndexedIterator(I) &&

Writable(O) && IndexedIterator(O) &&

PartiallyCompatible(ValueType(I), ValueType(O)))

void basic_copy_k_indexed(I f, O r)

{

basic_copy_k<k - 1>()(f, r);

sink(r + (k - 1)) = source(f + (k - 1));

}

void basic_copy_k_indexed<0>(I, O) { }

requires(Readable(I) && IndexedIterator(I) &&

Writable(O) && IndexedIterator(O) && ValueType(I) == ValueType(O))

void copy_k_indexed(I& f, O& r)

{

basic_copy_k<k>(f, r);

f += k; r += k;

}

copy_n_unrolled_indexed (from Chapter 6)

requires(Readable(I) && IndexedIterator(I) &&

Writable(O) && IndexedIterator(O) &&

PartiallyCompatible(ValueType(I), ValueType(O)))

pair<I, O> copy_n_unrolled_indexed(I f, DistanceType(I) n, O r)

{

typedef DistanceType(I) N;

const int k = 4; // unroll factor

while (n >= N(k)) {

copy_k_indexed<k>(f, r);

n = n - N(k);

}

return copy_n(f, n, r);

}

copy_parallel for disjoint ranges (from Chapter 6)

input and output ranges do not alias

A special forall executes its body for all values of its iteration

variable in arbitrary order, possibly concurrently

template <typename I, typename O>

requires(Readable(I) && IndexedIterator(I) &&

Writable(O) && IndexedIterator(O) &&

PartiallyCompatible(ValueType(I), ValueType(O)))

void copy_parallel(I f, DistanceType(I) n, O r)

{

typedef DistanceType(I) N;

forall(N i, N(0), n) {

sink(r + i) = source(f + i);

}

}

copy_backward_n (from Chapter 6)

copy_backward_n, which is often just as useful, is realizable

requires(Readable(I) && IndexedIterator(I) &&

Writable(O) && IndexedIterator(O) &&

PartiallyCompatible(ValueType(I), ValueType(O)))

O copy_backward_n(I f, DistanceType(I) n, O r)

{

typedef DistanceType(I) N;

while (n != N(0)) {

n = n - N(1);

sink(r + n) = source(f + n);

}

return r;

}

copy_from_segmented (from Chapter 6)

requires(Readable(I) && SegmentedIterator(I) &&

Writable(O) && Iterator(O) &&

PartiallyCompatible(ValueType(I), ValueType(O)))

O copy_from_segmented(I f, I l, O r)

{

while (f + end(f) != l + end(l)) {

// f and l are in different segments

r = copy(begin(f), end(f), r);

f = f + end(f);

}

// f and l are in the same segment

return copy(begin(f), begin(l), r);

}

Underlying type (from Chapter 7)

Definition

For any type T , the underlying type U is an affiliated type satisfying:

T and U are compatible

References with value types T and U may be reinterpretively cast

into each other

Construction of type U and assignment to type U never throw an

exception

An object of type U may only be used to hold temporary values

while implementing a rearrangement of a range of T objects

A reference to an object of type U may be reinterpretively cast to a

reference to T and passed as a const T& parameter

We denote the underlying type of T as UnderlyingT ype(T )

Motivation for underlying type (from Chapter 7)

to save time

As we shall see when implementing containers, underlying type

allows multiple headers to point to the same data while

performing rearrangements

UnderlyingT ype type function (from Chapter 7)

template <typename T> requires(Regular(T))

struct underlying_type

{

typedef T type; // default

};

const UnderlyingType(ValueType(I))&

underlying_source(I x)

{

return reinterpret_cast<

const UnderlyingType(ValueType(I))&>(source(x));

}

UnderlyingType(ValueType(I))&

underlying_sink(I x)

{

return reinterpret_cast<

UnderlyingType(ValueType(I))&>(sink(x));

}

cycle_2 (from Chapter 7)

requires(Mutable(I) && Iterator(I))

void cycle_2(I x, I y)

{

UnderlyingType(ValueType(I)) t = underlying_source(x);

underlying_sink(x) = underlying_source(y);

underlying_sink(y) = t;

}

Underlying type and invariants (from Chapter 7)

UnderlyingT ype(T ) to:

Be within a critical section: that is, avoid concurrent access

Avoid any exceptions being thrown

Restore the class invariants of T by the end of the sequence

Reflection

Disciplined violation of invariants to enhance performance is perfectly

legitimate

cycle_left_3 (from Chapter 7)

requires(Mutable(I) && Iterator(I))

void cycle_left_3(I x, I y, I z)

{

UnderlyingType(ValueType(I)) t = underlying_source(x);

underlying_sink(x) = underlying_source(y);

underlying_sink(y) = underlying_source(z);

underlying_sink(z) = t;

}

swap (from Chapter 7)

underlying type

requires(Regular(T))

void swap(T& x, T& y)

{

UnderlyingType(T) tmp;

tmp = UnderlyingRef(T)(x);

UnderlyingRef(T)(x) = UnderlyingRef(T)(y);

UnderlyingRef(T)(y) = tmp;

}

do_cycle_from (from Chapter 7)

requires(Mutable(I) && ForwardIterator(I) &&

Transformation(P) && Domain(P) == I)

void do_cycle_from(I i, P p)

{

// Precondition: p is a from-permutation of the range containing i

UnderlyingType(ValueType(I)) t = underlying_source(i);

I f = i;

I n = p(i);

while (n != i) {

underlying_sink(f) = underlying_source(n);

f = n;

n = p(n);

}

underlying_sink(f) = t;

}

swap_ranges_n (from Chapter 7)

requires(Mutable(I1) && Iterator(I1) &&

Mutable(I2) && Iterator(I2) &&

PartiallyCompatible(ValueType(I1), ValueType(I2)) &&

Integer(N))

pair<I1, I2> swap_ranges_n(I1 f1, I2 f2, N n)

{

while (n != N(0)) {

cycle_2(f1, f2);

++f1;

++f2;

n = n - N(1);

};

return pair<I1, I2>(f1, f2);

}

reverse_n_with_buffer (from Chapter 7)

requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && BidirectionalIterator(B) &&

UnderlyingType(ValueType(I)) == ValueType(B))

I reverse_n_with_buffer(I f, DistanceType(I) n, B b)

{

return reverse_copy_n(copy_n(f, n, b), n, f);

}

reverse_n_adaptive (from Chapter 7)

requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && BidirectionalIterator(B) &&

UnderlyingType(ValueType(I)) == ValueType(B))

I reverse_n_adaptive(I f, DistanceType(I) n, B b, DistanceType(I) n_b)

{

typedef DistanceType(I) N;

const N h = n / N(2);

const N r = n - N(2) * h;

if (h == N(0))

return f + n;

if (n <= n_b)

return reverse_n_with_buffer(f, n, b);

I m = reverse_n_adaptive(f, h, b, n_b);

m += r;

I l = reverse_n_adaptive(m, h, b, n_b);

swap_ranges_n(f, m, h);

return l;

}

do_fused_cycles_from_with_buffer (from Chapter

8)

requires(Mutable(I) && ForwardIterator(I) &&

Transformation(P) && Domain(P) == I &&

Mutable(B) && ForwardIterator(B) &&

UnderlyingType(ValueType(I)) == ValueType(B))

void do_fused_cycles_from_with_buffer(I i, P p, B b, DistanceType(I) n)

{

// Precondition: p is a from-permutation on the range containing i

copy_n(i, b, n);

I f = i;

I next = p(i);

while (next != i) {

copy_n(next, f, n);

f = next;

next = p(next);

}

copy_n(b, f, n);

}

rotate_indexed_helper_fused (from Chapter 8)

requires(Mutable(I) && IndexedIterator(I) &&

Transformation(P) && Domain(P) == I)

I rotate_indexed_helper_fused(I f, I m, I l, P p)

{

// Precondition: p is a from-permutation on [f, l)

typedef DistanceType(I) N;

const N fusion_factor(16);

UnderlyingType(ValueType(I)) buffer[fusion_factor];

N d = gcd(m - f, l - m);

N i(0);

while (i + fusion_factor < d) {

do_fused_cycles_from_with_buffer(f + i, p, buffer, fusion_factor);

i = i + fusion_factor;

}

do_fused_cycles_from_with_buffer(f + i, p, buffer, d - i);

return f + (l - m);

};

swap_ranges (from Chapter 8)

requires(Mutable(I1) && Iterator(I1) &&

Mutable(I2) && Iterator(I2) &&

PartiallyCompatible(ValueType(I1), ValueType(I2)))

pair<I1, I2> swap_ranges(I1 f1, I1 l1, I2 f2, I2 l2)

{

while (f1 != l1 && f2 != l2) {

cycle_2(f1, f2);

++f1;

++f2;

};

return pair<I1, I2>(f1, f2);

}

rotates [f, l) about m

partition_copy_n (from Chapter 9)

requires(Readable(I) && Iterator(I) &&

Writable(O0) && Iterator(O0) &&

Writable(O1) && Iterator(O1) &&

UnaryPredicate(P) && Domain(P) == ValueType(I) &&

PartiallyCompatible(ValueType(I), ValueType(O0)) &&

PartiallyCompatible(ValueType(I), ValueType(O1)))

pair<O0, O1> partition_copy_n(I f, DistanceType(I) n, O0 r0, O1 r1, P p)

{

typedef DistanceType(I) N;

while (n != N(0)) {

if (p(source(f))) {

sink(r1) = source(f); ++r1;

} else {

sink(r0) = source(f); ++r0;

}

++f;

n = n - N(1);

}

return pair<O0, O1>(r0, r1);

}

single_cycle_partition (from Chapter 9)

requires(Readable(I) && BidirectionalIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I single_cycle_partition(I f, I l, P p)

{

f = find_if(f, l, p);

l = find_backward_if_not(f, l, p);

if (f == l) return f;

UnderlyingType(ValueType(I)) tmp = source(f);

while (true) {

--l;

sink(f) = source(l);

f = find_if(successor(f), l, p);

if (f == l) {

sink(l) = tmp;

return f;

}

sink(l) = source(f);

l = assured_find_backward_if_not(l, p);

}

}

underlying_forward_iterator (from Chapter 9)

requires(ForwardIterator(I))

struct underlying_forward_iterator

{

typedef UnderlyingType(ValueType(I)) U;

typedef DistanceType(I) N;

typedef underlying_forward_iterator UFI;

I i;

underlying_forward_iterator() {}

underlying_forward_iterator(const I& x) : i(x) {}

operator I() { return i; }

void operator++() { ++i; }

UFI operator+(N n) { return i + n; }

friend N operator-(UFI x, UFI y) { return x.i - y.i; }

friend bool operator==(const UFI& x, const UFI& y) { return x.i == y.i; }

friend const U& source(const UFI& x) { return underlying_source(x.i); }

friend U& sink(UFI& x) { return underlying_sink(x.i); }

};

#define UFI(I) underlying_forward_iterator<I>

underlying_bidirectional_iterator (from Chapter 9)

requires(BidirectionalIterator(I))

struct underlying_bidirectional_iterator

{

typedef UnderlyingType(ValueType(I)) U;

typedef DistanceType(I) N;

typedef underlying_bidirectional_iterator UBI;

I i;

underlying_bidirectional_iterator() {}

underlying_bidirectional_iterator(const I& x) : i(x) {}

operator I() { return i; }

void operator++() { ++i; }

void operator--() { --i; }

UBI operator+(N n) { return i + n; }

friend N operator-(UBI x, UBI y) { return x.i - y.i; }

friend bool operator==(const UBI& x, const UBI& y) { return x.i == y.i; }

friend const U& source(const UBI& x) { return underlying_source(x.i); }

friend U& sink(UBI& x) { return underlying_sink(x.i); }

};

#define UBI(I) underlying_bidirectional_iterator<I>

Type functions for underlying_forward_iterator

(from Chapter 9)

template <typename I>

requires(ForwardIterator(I))

struct value_type<underlying_forward_iterator<I> >

{

typedef UnderlyingType(ValueType(I)) type;

};

requires(ForwardIterator(I))

struct distance_type<underlying_forward_iterator<I> >

{

typedef DistanceType(I) type;

};

requires(ForwardIterator(I))

struct iterator_category<underlying_forward_iterator<I> >

{

typedef forward_iterator_tag category;

};

stable_partition_n_with_buffer (from Chapter 9)

requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && ForwardIterator(B) &&

UnaryPredicate(P) && Domain(P) == ValueType(I) &&

UnderlyingType(ValueType(I)) == ValueType(B))

pair<I, I> stable_partition_n_with_buffer(I f, DistanceType(I) n, B b, P p)

{

pair<UFI(I), B> r = partition_copy_n(UFI(I)(f), n, UFI(I)(f), b, p);

return pair<I, I>(I(r.first), I(copy(b, r.m1, r.first)));

}

stable_partition_n_adaptive (from Chapter 9)

requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && ForwardIterator(B) &&

UnaryPredicate(P) && Domain(P) == ValueType(I) &&

UnderlyingType(ValueType(I)) == ValueType(B))

pair<I, I> stable_partition_n_adaptive(

I f, DistanceType(I) n, B b, DistanceType(I) n_b, P p)

{

typedef DistanceType(I) N;

if (n == N(0))

return stable_partition_0(f, p);

if (n == N(1))

return stable_partition_1(f, p);

if (n <= n_b)

return stable_partition_n_with_buffer(f, n, b, p);

N h = half_nonnegative(n);

pair<I, I> r0 = stable_partition_n_adaptive(f, h, b, n_b, p);

pair<I, I> r1 = stable_partition_n_adaptive(r0.m1, n - h, b, n_b, p);

return stable_partition_combine(r0, r1);

}

merge_copy_n (from Chapter 9)

requires(Readable(I0) && Iterator(I0) && Readable(I1) && Iterator(I1) &&

Writable(O) && Iterator(O) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&

PartiallyCompatible(ValueType(I0), ValueType(O)) &&

PartiallyCompatible(ValueType(I1), ValueType(O)))

void merge_copy_n(I0 f0, DistanceType(I0) n0,

I1 f1, DistanceType(I1) n1, O o, R r)

{

typedef DistanceType(I0) N0; typedef DistanceType(I1) N1;

while (true) {

if (n0 == N0(0)) { copy_n(f1, n1, o); return; }

if (n1 == N1(0)) { copy_n(f0, n0, o); return; }

if (r(source(f1), source(f0))) {

sink(o) = source(f1); ++f1; n1 = n1 - N1(1);

} else {

sink(o) = source(f0); ++f0; n0 = n0 - N0(1);

}

++o;

}

}

underlying_compare (from Chapter 9)

requires(Regular(T) && Relation(R) && Domain(R) == T)

struct underlying_compare

{

typedef UnderlyingType(T) U;

R r;

underlying_compare(R r) : r(r) {}

bool operator()(const U& x, const U& y)

{

return r(reinterpret_cast<const T&>(x), reinterpret_cast<const T&>(y));

}

};

merge_n_with_buffer (from Chapter 9)

requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && ForwardIterator(B) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&

UnderlyingType(ValueType(I)) == ValueType(B))

void merge_n_with_buffer(

I f0, DistanceType(I) n0, I f1, DistanceType(I) n1, B b, R r)

{

typedef underlying_compare<ValueType(I), R> UR;

copy_n(UFI(I)(f0), n0, b);

merge_copy_n(b, n0, UFI(I)(f1), n1, UFI(I)(f0), UR(r));

}

merge_n_adaptive (from Chapter 9)

template <typename I, typename B, typename R>

requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && ForwardIterator(B) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&

UnderlyingType(ValueType(I)) == ValueType(B))

void merge_n_adaptive(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1,

B b, DistanceType(I) n_b, R r)

{

typedef DistanceType(I) N;

if (n0 + n1 < N(8)) // ***** MEASURE AND TUNE

return insertion_merge_n(f0, n0, f1, n1, r);

if (n0 <= n_b)

return merge_n_with_buffer(f0, n0, f1, n1, b, r);

I i, j;

if (n0 > n1) {

i = f0 + half_nonnegative(n0);

j = lower_bound_n(f1, n1, source(i), r);

} else {

j = f1 + half_nonnegative(n1);

i = upper_bound_n(f0, n0, source(j), r);

}

I m = rotate(i, f1, j);

merge_n_adaptive(f0, i - f0, i, m - i, b, n_b, r);

merge_n_adaptive(m, j - m, j, n1 - (j - f1), b, n_b, r);

} Stepanov, McJones Elements of Programming March 14, 2008 771 / 880

stable_sort_n_adaptive (from Chapter 9)

requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && ForwardIterator(B) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&

UnderlyingType(ValueType(I)) == ValueType(B))

I stable_sort_n_adaptive(

I f, DistanceType(I) n, B b, DistanceType(I) n_b, R r)

{

typedef DistanceType(I) N;

if (n < N(16))

return binary_insertion_sort_n(f, n, r);

N h = half_nonnegative(n);

I m = stable_sort_n_adaptive(f, h, b, n_b, r);

I l = stable_sort_n_adaptive(m, n - h, b, n_b, r);

merge_n_adaptive(f, n / 2, m, n - h, b, n_b, r);

return l;

}

Type functions for list (from Chapter 11)

struct underlying_type< list<T> >

{

typedef list_iterator<T> type; // or IteratorType(list<T>)

};

requires(Container(C))

struct iterator_type

{

typedef C type;

};

#define IteratorType(C) typename iterator_type<C>::type

struct iterator_type< list<T> >

{

typedef list_iterator<T> type;

};

Type functions for array (from Chapter 11)

struct underlying_type< array<T> >

{

typedef struct { array_header<T>* p; } type;

};

struct iterator_type< array<T> >

{

typedef T* type;

};

reserve (from Chapter 11)

requires(Regular(T))

void reserve(array<T>& a, DistanceType(IteratorType(array<T>)) n)

{

if (n < size(a) || n == capacity(a)) return;

typedef UnderlyingType<T> U;

typedef UFI(IteratorType(array<T>)) UI;

array<U> tmp(n)

copy(UI(begin(a)), UI(end(a)), array_push_iterator<U>); // never throws

swap(tmp, a);

}

sort for array (from Chapter 11)

requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)

void sort(array<T>& x, R r)

{

typedef DistanceType(IteratorType(array<T>)) N;

typedef UnderlyingType(T) U;

N n = size(x) / N(10);

array<U> buffer(n, n, U());

stable_sort_n_adaptive(begin(x), size(x), begin(buffer), n, r);

}

#endif // *****

Conclusions

...

Contents I

1 Introduction

2 Foundations

5 Orderings

6 Combining concepts

Contents II

8 Permutations and rearrangements

9 Rotations

11 Coordinate structures

12 Composite objects

Reduction

Sorting

Stable partition redux

Contents III

Conclusions

Reading

Project

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

17 Index

Concept PartialSemigroupOperation

Definition

PartialSemigroupOperation(Op) ⇒

BinaryOperation(Op)

For all op ∈ Op and for all a, b, c, ∈ Domain(Op), if

is_defined(op, a, b) ∧ is_defined(op, b, c)

then

is_defined(op, op(a, b), c) ∧ is_defined(op, a, op(b, c)) ∧

op(op(a, b), c) = op(a, op(b, c))

reduce_nonempty

requires(Readable(I) && Iterator(I) &&

SemigroupOperation(Op) && Domain(Op) == ValueType(I))

Domain(Op) reduce_nonempty(I f, I l, Op op)

{

// Precondition: f , l

Domain(Op) r = source(f);

while (true) {

++f;

if (f == l) return r;

r = op(r, source(f));

}

}

reduce

requires(Readable(I) && Iterator(I) &&

MonoidOperation(Op) && Domain(Op) == ValueType(I))

Domain(Op) reduce(I f, I l, Op op, const Domain(Op)& z)

{

if (f == l) return z;

return reduce_nonempty(f, l, op);

}

other than identity_element(op)

***** Explain that there can be many monoids within Domain(Op)

?????

find_not

requires(Readable(I) && Iterator(I))

I find_not(I f, I l, const ValueType(I)& x)

{

while (f != l && source(f) == x) ++f;

return f;

}

reduce_nonzeroes

requires(Readable(I) && Iterator(I) &&

MonoidOperation(Op) && Domain(Op) == ValueType(I))

Domain(Op) reduce_nonzeroes(I f, I l, Op op, const Domain(Op)& z)

{

f = find_not(f, l, z);

if (f == l) return z;

Domain(Op) r = source(f);

while (true) {

f = find_not(successor(f), l, z);

if (f == l) return r;

r = op(r, source(f));

}

}

add_to_counter

template <typename I, typename Op>

requires(Mutable(I) && ForwardIterator(I) &&

BinaryOperation(Op) && Domain(Op) == ValueType(I))

Domain(Op) add_to_counter(I f, I l, Op op, Domain(Op) x, const Domain(Op)& z)

{

if (x == z) return z;

while (f != l) {

if (source(f) != z) {

x = op(source(f), x);

sink(f) = z;

} else {

sink(f) = x;

return z;

}

++f;

}

return x;

}

The intent is to call it many times with the same mutable counter

Thus we specify the requirement as ForwardIterator

Stepanov, McJones Elements of Programming March 14, 2008 786 / 880

tranpose_operation

requires(BinaryOperation(Op))

struct transpose_operation

{

Op op;

transpose_operation(Op op) : op(op) { }

typedef Domain(Op) T;

T operator()(const T& x, const T& y)

{

return op(y, x);

}

};

codomain and domain

reduce_balanced

requires(Readable(I) && ForwardIterator(I) &&

MonoidOperation(Op) && Domain(Op) == ValueType(I))

Domain(Op) reduce_balanced(I f, I l, Op op, const Domain(Op)& z)

{

// Precondition: 2k > l − f

typedef array_k<k, Domain(Op)> C;

C counter;

IteratorType(C) c_f = begin(counter), c_l = c_f;

while (f != l) {

Domain(Op) carry = add_to_counter(c_f, c_l, op, source(f), z);

if (carry != z) {

sink(c_l) = carry;

++c_l;

};

++f;

}

return reduce_nonzeroes(c_f, c_l, transpose_operation<Op>(op), z);

}

merger_linkable

requires(ForwardLinker(S) && Readable(IteratorType(S)) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))

struct merger_linkable

{

typedef IteratorType(S) I;

I l;

R r;

S set_link;

merger_linkable(I l, R r, S set_link) : l(l), r(r), set_link(set_link) {}

I operator()(I x, I y)

{

return merge_linkable_nonempty(x, l, y, l, r, set_link).first;

}

};

sort_linkable

template <int k, typename S, typename R>

requires(ForwardLinker(S) && Readable(IteratorType(S)) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))

IteratorType(S) sort_linkable(IteratorType(S) f, IteratorType(S) l, R r, S set_link

{

// Precondition: 2k > l − f

typedef merger_linkable<S, R> Op;

Op op(l, r, set_link);

typedef IteratorType(S) I;

typedef array_k<k, I> C;

C counter;

IteratorType(C) c_f = begin(counter), c_l = c_f;

while (f != l) {

I old_f = f;

++f;

set_link(old_f, l);

I carry = add_to_counter(c_f, c_l, op, old_f, l);

if (carry != l) {

sink(c_l) = carry;

++c_l;

}

}

return reduce_nonzeroes(c_f, c_l, transpose_operation<Op>(op), l);

}

Stepanov, McJones Elements of Programming March 14, 2008 790 / 880

sort_forward_linkable

requires(Readable(I) && LinkableForwardIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))

I sort_forward_linkable(I f, I l, R r)

{

return sort_linkable<32>(f, l, r, forward_linker<I>());

}

sort_bidirectional_linkable

requires(Readable(I) && LinkableBidirectionalIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))

I sort_bidirectional_linkable(I f, I l, R r)

{

f = sort_linkable<32>(f, l, r, forward_linker<I>());

restore_backward_links(f, l, backward_linker<I>());

return f;

}

combine_ranges

requires(ForwardIterator(I)) // but not Readable(I)

struct combine_ranges

{

typedef pair<I, I> Pair;

Pair operator()(const Pair& x, const Pair& y) const

{

return Pair(

rotate(x.first, x.second, y.first),

y.second);

}

};

partition_trivial

requires(ForwardIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

struct partition_trivial

{

P p;

partition_trivial(const P & p) : p(p) {}

pair<I, I> operator()(I i) const {

// return stable_partition_1<I, P>(i, p);

if (p(source(i)))

return pair<I, I>(i, i);

else

return pair<I, I>(i, successor(i));

}

};

value_iterator

template <typename I, typename F>

requires(Incrementable(I) && Transformation(F) && Domain(F) == I)

struct value_iterator

{

I i;

F f;

value_iterator() {}

value_iterator(const I& i, const F& f) : i(i), f(f) {}

void operator++() {

++i;

}

friend Codomain(F) source(const value_iterator& x) {

return x.f(x.i);

}

friend bool operator==(const value_iterator& x, const value_iterator& y) {

// Precondition: x.f = y.f

return x.i == y.i;

}

friend bool operator!=(const value_iterator& x, const value_iterator& y) {

return !(x == y);

}

};

stable_partition_iterative

requires(ForwardIterator(I) && UnaryPredicate(P) && ValueType(I) == Domain(P))

I stable_partition_inplace_iterative(I f, I l, P p)

{

typedef partition_trivial<I, P> Fun;

typedef value_iterator<I, Fun> RangeIterator;

typedef combine_ranges<I> Op;

Fun fun(p);

RangeIterator f1(f, fun);

RangeIterator l1(l, fun);

combine_ranges<I> op;

pair<I, I> z(l, l);

return reduce_balanced<32, RangeIterator, Op>(f1, l1, op, z).first;

}

Conclusions

Reading

Project

Project

***** Write an iterative version of stable_partition_n and

stable_partition_n_adaptive from Chapter 9 using

reduce_balanced

***** Give hint that many z’s per op will be required ?????

***** Compare the performance of the corresponding recursive

and iterative versions

Contents I

1 Introduction

2 Foundations

5 Orderings

6 Combining concepts

Contents II

8 Permutations and rearrangements

9 Rotations

11 Coordinate structures

12 Composite objects

14 Mathematical notation

Contents III

15 C++ machinery

16 Acknowledgments

17 Index

Connectives

Definition

If P and Q are propositions, then so are ¬P (read as “not P”), P ∨ Q (“P

or Q”), P ∧ Q (“P and Q”), P ⇒ Q (“P implies Q”), and P ⇔ Q (“P is

equivalent to Q”), with these truth tables:

F F T F F T T

F T T T F T F

T F F T F F F

T T F T T T T

P iff Q

Existential and universal quantifiers

Definition

If P is a proposition and x is a variable, then (∃x)P is a proposition

(read as "there exists x such that P")

Definition

If P is a proposition and x is a variable, then (∀x)P is a proposition

(read as “for all x, P”); (∀x)P ⇔ (¬(∃x)¬P)

Sets and functions

a ∈ X (“a is an element of X”)

X ⊂ Y (“X is a subset of Y”)

{a0 , . . . , an } (“the finite set with elements a0 , . . . , and an ”)

{a ∈ X|P(a)} (“the subset of X for which the predicate P holds”)

X ∪ Y (“the union of X and Y”)

X ∩ Y (“the intersection of X and Y”)

X − Y (“the complement of Y in X”)

X × Y (“the direct product of X and Y”)

f : X → Y (“f is a function from X to Y”)

X is the domain of f

Y is the codomain of f

Contents I

1 Introduction

2 Foundations

5 Orderings

6 Combining concepts

Contents II

8 Permutations and rearrangements

9 Rotations

11 Coordinate structures

12 Composite objects

14 Mathematical notation

Contents III

15 C++ machinery

Foundations

DistanceT ype type function

Special cases of Integer procedures

Default implementations for additive and multiplicative types

Orderings

Iterators

Category dispatch, with reverse and reverse_n examples

Increasing sequences

Linkable ranges

Bifurcate coordinates

Collocated types

fill_iterator

16 Acknowledgments

Contents IV

17 Index

Syntax for requires and models

#define requires(...)

#define models(...)

ArgumentT ype, Domain, and Codomain type

functions

function:

template <typename T, int i>

requires(Procedure(T))

struct argument_type;

#define Domain(T) ArgumentType(T, 0)

requires(Procedure(T))

struct codomain_type;

// because of the use of the keyword typename

DistanceT ype type function

requires(Countable(T)) /∗ ????? ∗/

struct distance_type

{

typedef size_t type;

};

requires(Regular(T))

struct distance_type<T*>

{

typedef ptrdiff_t type;

};

#define DistanceType(T) typename distance_type< T >::type

template <>

struct distance_type<short>

{

typedef unsigned short type;

};

Default implementations of special cases of Integer

procedures

// The successor defined in Chapter 6 for Iterator suffices

template <typename I> requires(Integer(I))

I predecessor(I a) { --a; return a; }

template <typename I> requires(Integer(I))

I half_nonnegative(const I& a) { return a >> I(1); }

template <typename I> requires(Integer(I))

void halve_nonnegative(I& a) { a >>= I(1); }

template <typename I> requires(Integer(I))

I binary_scale_down_nonnegative(const I& a, const I& k) { return a >> k; }

template <typename I> requires(Integer(I))

I binary_scale_up_nonnegative(const I& a, const I& k) { return a << k; }

template <typename I> requires(Integer(I))

bool is_positive(const I& a) { return I(0) < a; }

template <typename I> requires(Integer(I))

bool is_negative(const I& a) { return a < I(0); }

template <typename I> requires(Integer(I))

bool is_zero(const I& a) { return a == I(0); }

template <typename I> requires(Integer(I))

bool is_even(const I& a) { return (a & I(1)) == I(0); }

template <typename I> requires(Integer(I))

bool is_odd(const I& a) { return (a & I(1)) != I(0); }

Default identity and inverse for additive types

requires(AdditiveSemigroup(T))

struct plus {

T operator()(const T& x, const T& y) const { return x + y; }

};

requires(AdditiveMonoid(T))

T identity_element(const plus<T>&) { return T(0); }

requires(AdditiveGroup(T))

struct negate {

T operator()(const T& x) const { return -x; }

};

requires(AdditiveGroup(T))

negate<T> inverse_operation(const plus<T>&) {

return negate<T>();

}

Define ArgumentT ype(plusT , 0)

requires(AdditiveSemigroup(T))

struct argument_type<plus<T>, 0>

{

typedef T type;

};

Default identity and inverse for multiplicative types

requires(MultiplicativeSemigroup(T))

struct multiplies {

T operator()(const T& x, const T& y) const { return x * y; }

};

requires(MultiplicativeMonoid(T))

T identity_element(const multiplies<T>&) { return T(1); }

requires(MultiplicativeGroup(T))

struct reciprocal {

T operator()(const T& x) const { return T(1) / x; }

};

requires(MultiplicativeGroup(T))

reciprocal<T> inverse_operation(const multiplies<T>&) {

return reciprocal<T>();

}

Default ordering for StrictTotallyOrdered

requires(StrictTotallyOrdered(T))

struct less

{

bool operator()(const T& x, const T& y) const { return x < y; }

};

requires(StrictTotallyOrdered(T))

struct argument_type<less<T>, 0>

{

typedef T type;

};

Default order selection for StrictTotallyOrdered

requires(StrictTotallyOrdered(T))

T& min(T& a, T& b)

{

return min(a, b, less<T>());

}

Dereferenceable functions for T ∗

template <typename T>

requires(Regular(T))

struct value_type;

requires(Regular(T))

struct value_type<T*> { typedef T type; };

requires(Regular(T))

const T& source(T* x) { return *x; }

requires(Regular(T))

T& sink(T* x) { return *x; }

More aliasing tests

template <typename T0, typename T1>

requires(Writable(T0) && Writable(T1) && ValueType(T0) == ValueType(T1))

bool is_aliased_sinks(const T0& x0, const T1& x1)

{

// Return true ⇔

// For all U with Readable(U) and for all y ∈ U,

// is_aliased(x0, y) == is_aliased(x1, y)

// For many types, this works:

typedef const ValueType(T0)* P;

return P(&sink(x0)) == P(&sink(x1));

}

requires(Readable(T0) && Readable(T1) && ValueType(T0) == ValueType(T1))

bool is_aliased_sources(const T0& x0, const T1& x1)

{

// Return true ⇔

// For all U with Writable(U) and for all y ∈ U,

// is_aliased(y, x0) == is_aliased(y, x1)

// For many types, this works:

typedef const ValueType(T0)* P;

return P(&source(x0)) == P(&source(x1));

}

Default Dereferenceable functions for T

requires(Regular(T))

struct value_type

{

typedef T type;

};

requires(Regular(T))

const T& source(const T& x)

{

return x;

}

requires(Regular(T))

T& sink(T& x)

{

return x;

}

Mechanics of category dispatch

A type function is defined to obtain a type tag from a type

Overloading is used to select from multiple signatures differing

only by a tag type

Iterator tag types

BI

It FI RI

II

struct forward_iterator_tag {};

struct bidirectional_iterator_tag {};

struct indexed_iterator_tag {};

struct random_access_iterator_tag {};

IteratorCategory type function

requires(Iterator(T))

struct iterator_category

{

typedef iterator_tag category;

};

requires(Regular(T))

struct iterator_category<T*>

{

typedef random_access_iterator_tag category;

};

Category dispatch for reverse_n and reverse

requires(Mutable(I) && ForwardIterator(I))

void reverse_n(I f, DistanceType(I) n)

{

reverse_n(f, n, IteratorCategory(I)());

}

requires(Mutable(I) && ForwardIterator(I))

void reverse(I f, I l)

{

reverse(f, l, IteratorCategory(I)());

}

Category dispatch cases for reverse_n

template <typename I> requires(Mutable(I) && ForwardIterator(I))

void reverse_n(I f, DistanceType(I) n, forward_iterator_tag)

{

reverse_n_forward(f, n);

}

void reverse_n(I f, DistanceType(I) n, bidirectional_iterator_tag)

{

reverse_n_bidirectional(f, n);

}

void reverse_n(I f, DistanceType(I) n, indexed_iterator_tag)

{

reverse_n_indexed(f, n);

}

void reverse_n(I f, DistanceType(I) n, random_access_iterator_tag)

{

reverse_n_indexed(f, n);

}

Category dispatch cases for reverse

template <typename I> requires(Mutable(I) && ForwardIterator(I))

void reverse(I f, I l, forward_iterator_tag)

{

reverse_forward(f, l);

}

void reverse(I f, I l, bidirectional_iterator_tag)

{

reverse_bidirectional(f, l);

}

void reverse(I f, I l, indexed_iterator_tag)

{

reverse_indexed(f, l);

}

void reverse(I f, I l, random_access_iterator_tag)

{

reverse_indexed(f, l);

}

find_out_of_order

requires(Readable(I) && ForwardIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))

I find_out_of_order(I f, I l, R r, forward_iterator_tag)

{

if (f == l) return l;

I n(successor(f));

while (n != l) {

if (r(source(n), source(f)))

return n;

f = n;

++n;

}

return l;

}

find_out_of_order

requires(Readable(I) && Iterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))

I find_out_of_order(I f, I l, R r, iterator_tag)

{

if (f == l) return l;

I n(successor(f));

ValueType(I) v = source(f);

while (n != l) {

if (r(source(n), v))

return n;

v = source(n);

++n;

}

return l;

}

Category dispatch for find_out_of_order

template <typename I, typename R>

requires(Readable(I) && RandomAccessIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))

I find_out_of_order(I f, I l, R r, random_access_iterator_tag)

{

return find_out_of_order(f, l, r, forward_iterator_tag());

}

requires(Readable(I) && BidirectionalIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))

I find_out_of_order(I f, I l, R r, bidirectional_iterator_tag)

{

return find_out_of_order(f, l, r, forward_iterator_tag());

}

requires(Readable(I) && Iterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))

I find_out_of_order(I f, I l, R r)

{

return find_out_of_order(f, l, r, IteratorCategory(I)());

}

is_increasing

requires(Readable(I) && Iterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))

bool is_increasing(I f, I l, R r)

{

return l == find_out_of_order(f, l, r);

}

partition_copy_n

requires(Readable(I) && Iterator(I) &&

Writable(O0) && Iterator(O0) &&

Writable(O1) && Iterator(O1) &&

UnaryPredicate(P) && Domain(P) == ValueType(I) &&

ValueType(I) == ValueType(O0) &&

ValueType(I) == ValueType(O1))

pair<O0, O1> partition_copy_n(I f, DistanceType(I) n, O0 o0, O1 o1, P p)

{

typedef DistanceType(I) N;

while (n != N(0)) {

if (p(source(f))) {

sink(o1) = source(f); ++o1;

} else {

sink(o0) = source(f); ++o0;

}

++f;

n = n - N(1);

}

return pair<O0, O1>(o0, o1);

}

unstable_indexed_partition

template <typename I, typename P>

requires(Mutable(I) && IndexedIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))

I unstable_indexed_partition(I f, I l, P p)

{

DistanceType(I) i = 0;

DistanceType(I) j = l - f;

while (true) {

while (true) {

if (i == j) return f + i;

if (p(source(f + i))) break;

i = i + 1;

}

while (true) {

j = j - 1;

if (i == j) return f + j + 1;

if (!p(source(f + j))) break;

}

cycle_2(f + i, f + j);

i = i + 1;

}

}

lower_bound_n

requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)

struct greater_than_or_equal_to_a

{

const T& a;

R r;

greater_than_or_equal_to_a(const T& a, R r) : a(a), r(r) { }

bool operator()(const T& x) const { return !r(x, a); }

};

requires(Mutable(I) && ForwardIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))

I lower_bound_n(I f, DistanceType(I) n, const ValueType(I)& a, R r)

{

greater_than_or_equal_to_a<ValueType(I), R> predicate(a, r);

return partition_point_n(f, n, predicate);

}

upper_bound_n

requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)

struct greater_than_a

{

const T& a;

R r;

greater_than_a(const T& a, R r) : a(a), r(r) { }

bool operator()(const T& x) const { return r(a, x); }

};

requires(Mutable(I) && ForwardIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))

I upper_bound_n(I f, DistanceType(I) n, const ValueType(I)& a, R r)

{

greater_than_a<ValueType(I), R> predicate(a, r);

return partition_point_n(f, n, predicate);

}

Category dispatch for merge_n_with_buffer

template <typename I, typename B, typename R>

requires(Mutable(I) && RandomAccessIterator(I) &&

Mutable(B) && RandomAccessIterator(B) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&

ValueType(I) == ValueType(B))

void merge_n_with_buffer(I f0, DistanceType(I) n0,

I f1, DistanceType(I) n1, B b, R r,

random_access_iterator_tag)

{

merge_n_with_buffer(f0, n0, f1, n1, b, r, bidirectional_iterator_tag());

}

requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && ForwardIterator(B) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&

ValueType(I) == ValueType(B))

void merge_n_with_buffer(I f0, DistanceType(I) n0,

I f1, DistanceType(I) n1, B b, R r)

{

merge_n_with_buffer(f0, n0, f1, n1, b, r, IteratorCategory(I)());

}

IteratorT ype type function

requires(ImplementsIteratorType(T))

struct iterator_type;

IteratorT ype for linkers

struct forward_linker;

struct iterator_type< forward_linker<I> > { typedef I type; };

struct backward_linker;

struct iterator_type< backward_linker<I> > { typedef I type; };

struct bidirectional_linker;

struct iterator_type< bidirectional_linker<I> > { typedef I type; };

Domain for transpose_operation

requires(BinaryOperation(Op))

struct transpose_operation;

requires(BinaryOperation(Op))

struct argument_type< transpose_operation<Op>, 0 >

{

typedef Domain(Op) type;

};

Domain for merger_linkable

requires(ForwardLinker(S) && Readable(IteratorType(S)) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))

struct merger_linkable;

requires(ForwardLinker(S) && Readable(IteratorType(S)) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))

struct argument_type<merger_linkable<S, R>, 0>

{

typedef IteratorType(S) type;

};

Domain for combine_ranges

requires(ForwardIterator(I)) // but not Readable(I)

struct combine_ranges;

requires(ForwardIterator(I)) // but not Readable(I)

struct argument_type<combine_ranges<I>, 0>

{

typedef pair<I, I> type;

};

Codomain for partition_trivial

struct partition_trivial;

struct codomain_type< partition_trivial<I, P> >

{

typedef pair<I, I> type;

};

WeightT ype type function

requires(WeakForwardBifurcateCoordinate(T))

struct weight_type

{

typedef size_t type; // ***** What should the default be ?????

};

requires(WeakForwardBifurcateCoordinate(T))

struct weight_type<T*>

{

typedef ptrdiff_t type;

};

#define WeightType(T) typename weight_type< T >::type // ***** ?????

left_successor and right_successor for

WeakForwardBifurcateCoordinate

requires(WeakForwardBifurcateCoordinate(C))

C left_successor(C f)

{

move_left(f);

return f;

}

requires(WeakForwardBifurcateCoordinate(C))

C right_successor(C f)

{

move_right(f);

return f;

}

pair

requires(Regular(T0) && Regular(T1))

struct pair

{

T0 m0;

T1 m1;

pair() {}

pair(T0 m0, T1 m1) : m0(m0), m1(m1) { }

pair(const pair& x) : m0(x.m0), m1(x.m1) { }

void operator=(const pair& x) { m0 = x.m0; m1 = x.m1; }

friend void bool operator==(const pair& x, const pair& y)

{ return x.m0 == y.m0 && x.m1 == y.m1; }

friend void bool operator<(const pair& x, const pair& y)

{ return x.m0 < y.m0 || (!y.m0 < x.m0 && x.m1 < y.m1); }

};

triple

requires(Regular(T0) && Regular(T1) && Regular(T2))

struct triple

{

T0 m0;

T1 m1;

T2 m2;

triple() {}

triple(T0 m0, T1 m1, T2 m2) :

m0(m0), m1(m1), m2(m2) {}

triple(const triple& x) : m0(x.m0), m1(x.m1), m2(x.m2) { }

void operator=(const triple& x)

{ m0 = x.m0; m1 = x.m1; m2 = x.m2}

friend void bool operator==(const triple& x, const triple& y)

{ return x.m0 == y.m0 && x.m1 == y.m1 && x.m2 == y.m2; }

friend void bool operator<(const triple& x, const triple<T0, T1, T2>& y)

{ return x.m0 < y.m0 || (!y.m0 < x.m0 && x.m1 < y.m1 ||

(!y.m1 < x.m1 && x.m2 == y.m2)); }

};

lexicographic_equalityk

requires(Readable(I0) && ForwardIterator(I0) &&

Readable(I1) && ForwardIterator(I1) &&

ValueType(I0) == ValueType(I1))

struct lexicographic_equality_k

{

bool operator()(I0 f0, I1 f1)

{

if (source(f0) != source(f1)) return false;

return lexicographic_equality_k<k - 1, I0, I1>()(f0 + 1, f1 + 1);

}

};

struct lexicographic_equality_k<0, I0, I1>

{

bool operator()(I0, I1)

{

return true;

}

};

lexicographic_orderingk

template <int k, typename I0, typename I1>

requires(Readable(I0) && ForwardIterator(I0) &&

Readable(I1) && ForwardIterator(I1) &&

ValueType(I0) == ValueType(I1))

struct lexicographic_ordering_k

{

bool operator()(I0 f0, I1 f1)

{

if (source(f0) < source(f1)) return true;

if (source(f0) > source(f1)) return false;

return lexicographic_equality_k<k - 1, I0, I1>()(f0 + 1, f1 + 1);

}

};

struct lexicographic_ordering_k<0, I0, I1>

{

bool operator()(I0, I1)

{

return false;

}

};

arrayk

***** Need const and non-const versions of begin and end ?????

***** What about subscript: a[i] ?????

template <int k, typename T>

requires(Regular(T))

struct array_k

{

typedef T* I; // IteratorT ype(arrayk,T )

T a[k];

array_k() { }

array_k(const array_k& x) { basic_copy_k_indexed<k, I, O>(&x.a, &a); }

friend void operator=(const array_k& x)

{ basic_copy_k_indexed<k, I, O>(&x.a, &a); }

friend size_t size(const array_k& x) { return size_t(k); }

friend I begin(array_k& x) { return &x.a[0]; }

friend I end(array_k& x) { return &x.a[k]; }

friend bool operator==(const array_k& x, const array_k& y)

{ return lexicographic_equality_k<k, I, I>(begin(x), begin(y)); }

friend bool operator<(const array_k& x, const array_k& y)

{ return lexicographic_ordering_k<k, I, I>(begin(x), begin(y)); }

};

Forward declaration for array_k data structure

requires(Regular(T))

struct array_k;

IteratorT ype for array_k

requires(Regular(T))

struct iterator_type< array_k<k, T> >

{

typedef T* type;

};

fill_iterator

requires(Regular(T) && Integer(N))

struct fill_iterator

{

const T* v;

N n;

fill_iterator() {}

fill_iterator(const T& v, N n) : v(&v), n(n) {}

friend void operator++(fill_iterator& x) { ++x.n; }

friend const T& source(fill_iterator& x) { return source(x.v); }

friend void operator+=(fill_iterator& x, N n) { x.n += n; }

friend N operator-(fill_iterator x, fill_iterator y) { return x.n - y.n; }

friend bool operator==(fill_iterator x, fill_iterator y) {

return x.n == y.n;

}

};

Type functions for fill_iterator

requires(Regular(T) && Integer(N))

struct value_type< fill_iterator<T, N> >

{

typedef T type;

};

requires(Regular(T) && Integer(N))

struct distance_type< fill_iterator<T, N> >

{

typedef N type;

};

requires(Regular(T) && Integer(N))

struct iterator_category< fill_iterator<T, N> >

{

typedef indexed_iterator_tag category;

};

Acknowledgments

Andrei Alexandrescu,Matt Austern, Dave Abrahams, Jon Brandt,

Boris Fomitchev, Kevlin Henney, Jussi Ketonen, Karl Malbrain,

Larry Masinter, David Musser, Dave Parent, Sean Parent,

Dmitry Polukhin, Jon Reid, Mark Ruzon, Geoff Scott,

David Simons, Tony Van Eerd, Walter Vannini, John Wilkinson,

and Oleg Zabluda

Index I

++

definition space on range, 361

6

standard definition, 242

>

standard definition, 242

>

standard definition, 242

absorption

for min, max, 251

abstraction, 19

action

duality with transformation, 343

Action concept, 342

activity on entities, 19

acyclic

descendants of bifurcate coordinate, 633

AdditiveGroup concept, 200

AdditiveMonoid concept, 198

AdditiveSemigroup concept, 196

algorithm

memory-adaptive, 426

with buffer, 426

algorithmically-induced structure, 276

aliasing, 338, 385

amortized complexity, 347

AmortizedConstantTime concept, 347

Index II

and (∧), 803

Archimedean monoid

discrete, 287

ArchimedeanMonoid concept, 276

ArchimedeanOrderedField concept, 519

area

constant, 82

fixed, 82

of an object, 82

variable, 82

area function, 82

argument, 40

ArgumentT ype type function, 55

Arity type attribute, 54

assignment, 49

for Regular, 65

associativity, 130

exploited by power, 145

of min, max, 251

of permutation composition, 412

asymmetric relation, 218

attribute of entity, 19

average cost, 84

Axiom of Archimedes, 276, 277

BackwardLinker concept, 601

Banning, John, 358

bidirectional binary trees, advantages, 641

Index III

BidirectionalBifurcateCoordinate concept, 629

BidirectionalIterator concept, 374

BidirectionalLinker concept, 602

BifurcateCoordinate concept, 628

bignum, 139

bin-based rearrangement, 422

binary gcd, 327

binary search, 553

BinaryOperation concept, 129

bisection technique, 518

Bolzano, Bernard, 518

bounded integer type, 322

buffer

with buffer algorithm, 426

category dispatch

mechanism, 822

Cauchy, Augustin Louis, 518

closed bounded range, 357

closed counted range, 357

closed interval, 243

clusters

of derived procedures, 238

code transformations

enabled by regular types, 81, 181

codomain of function, 805

codomain of procedure, 42

Codomain type function, 55

Index IV

CommutativeOperation concept, 194

CommutativeSemiringconcept, 298

commutativity

of min, max, 251

Compatible concept, 732

complement (−), 805

complement of converse of relation, 239

complement of relation, 239

complexity

algorithm selection, 175

amortized, 347

counting operations, 176

in specifications, 175

of generalized algorithm, 175

slow_power versus power, 177, 178

to determine feasibility, 175

composition

of permutations, 412

concept, 56

ArchimedeanMonoid, 276

Action, 342

AdditiveGroup, 200

AdditiveMonoid, 198

AdditiveSemigroup, 196

AmortizedConstantTime, 347

ArchimedeanOrderedField, 519

BackwardLinker, 601

BidirectionalBifurcateCoordinate, 629

BidirectionalIterator, 374

Index V

BidirectionalLinker, 602

BifurcateCoordinate, 628

BinaryOperation, 129

CancellableOperation, 195

CommutativeOperation, 194

CommutativeSemiring, 298

Compatible, 732

Dereferenceable, 340

DiscreteArchimedeanMonoid, 287

DiscreteArchimedeanRing, 319

DiscreteArchimedeanSemiring, 317

EquivalenceRelation, 219

EuclideanMonoid, 293

EuclideanSemimodule, 304

EuclideanSemiring, 306

examples from C++, 59

ForwardIterator, 371

ForwardLinker, 600

Function, 60

GroupOperation, 137

HomogeneousFunction, 61

IndexedIterator, 380

Integer, 139

Iterator, 348

LinkableBidirectionalBifurcateCoordinate, 644

LinkableBidirectionalIterator, 606

LinkableBifurcateCoordinate, 643

LinkableForwardIterator, 605

Merger, 567

Index VI

MonoidOperation, 136

MultiplicativeGroup, 201

MultiplicativeMonoid, 199

MultiplicativeSemigroup, 197

Mutable, 339

NonnegativeDiscreteArchimedeanSemiring, 318

Operation, 88

OrderedAdditiveGroup, 271

OrderedAdditiveMonoid, 271

OrderedAdditiveSemigroup, 271

OrderedCancellableMonoid, 273

Ordering, 215

PartiallyCompatible, 728

PartialSemigroupOperation, 781

Predicate, 214

Procedure, 60

ProperProcedure, 60

QRSemimodule, 302

RandomAccessIterator, 387

Readable, 336

refinement, 57

Regular, 63

RegularAction, 346

RegularFunction, 79

Relation, 214

relational concept, 726

SameGenus, 727

SegmentedIterator, 399

SemigroupOperation, 130

Index VII

Semimodule, 300

StrictTotallyOrdered, 237

StrictTotalOrdering, 223

StrictWeakOrdering, 226

Transformation, 89

type concept, 58

UnaryPredicate, 504

univalent, 320

WeakBifurcateCoordinate, 626

weakening, 57

WeaklyHalvableMonoid, 283

Writable, 337

constant area, 82

constructor, 52

type constructor, 53

container, 668

converse of relation, 239

coordinate structure, 594

copy constructor

for Regular, 68

copying rearrangement, 425

cost

average, 84

fixed, 84

uniform, 84

worst-case, 84

cost function, 84

cost of a procedure call, 84

cycle in a permutation, 415

Index VIII

cyclic permutation, 417

default constructor

for Regular, 67

default ordering, 236

definition space of procedure, 41

Dereferenceable concept, 340

derived relations, 239

descendants

of bifurcate coordinate, 633

description, 18

destructor, 50

for Regular, 66

DifferenceT ype type function, 387

direct product (×), 805

discrete Archimedean monoid, 287

DiscreteArchimedeanMonoid concept, 287

DiscreteArchimedeanRing concept, 319

DiscreteArchimedeanSemiring concept, 317

discreteness, 317

distance type, 110

DistanceT ype type function, 348

distributivity

of semiring, 298

divisibility lattice of integers, 466

divisibility on an Archimedean monoid, 285

division

Egyptian, 279

division on multiplicative group, 201

Index IX

domain of function, 805

Domain type function, 61

Dudziński, Krzysztof, 576

Dydek, Andrzej, 576

element (∈), 805

empty range, 360

entity

activity on, 19

attribute, 19

part, 19

particulars, 20

relationship, 19

equality

for Regular, 64

in the real world, 33

of objects, 38

on containers, 670, 671

properties of, 31

unique on a type, 234

equality (=), 30

equational reasoning, 32

in mathematics, 32

EquivalenceRelation concept, 219

equivalent (⇔), 803

essential object, 39

Euclidean domain, 297

Euclidean function, 306

Index X

EuclideanMonoid concept, 293

EuclideanSemimodule concept, 304

EuclideanSemiring concept, 306

exclusive-or of links, 646

exists (∃), 804

Fibonacci

matrices, 185

numbers, 182

Fiduccia, Charles M., 206

finite set, 418

finite tree

descendents of bifurcate coordinate, 634

first in a range, 358

fixed area, 82

fixed cost, 84

fixed point, 411

for all (∀), 804

ForwardIterator concept, 371

ForwardLinker concept, 600

from-permutation, 431

function

one-to-one, 408

onto, 407

function (→), 805

Function concept, 60

function procedure, 47

Index XI

Gaussian integers

defined, 166

example of Euclidean domain, 297

Stein’s algorithm, 328

gcd

and lcm, 465

binary, 327

greatest common divisor, 288

subtractive, 290

generalized link rearrangement, 610

generalized procedure, 71

generalized rearrangement, 609

genus, 35

global state, 40

goto

considered not harmful, 655

greatest common divisor, 288

group

of permutations, 414

GroupOperation concept, 137

Ho, Wilson, 482, 545

Hoare, C.A.R., 543

HomogeneousFunction concept, 61

idea, 18

idempotency

Index XII

of min, max, 251

identity

in the real world, 29

of objects, 29

identity permutation, 411

implies (⇒), 803

improper procedure, 46

in place, 426

in situ, 426

increasing order, 248

increasing range, 499

index permutation, 419

indexed iterator

equivalent to random access iterator, 391

IndexedIterator concept, 380

inequality

standard definition, 241

Integer concept, 139

interface

for rotate, 469

useful variations, 180

interoperability, 62

interpretatation of object, 25

intersection (∩), 805

interval, 243

invariant, 655

invariants

breaking, 622

inverse

Index XIII

of permutation, 413, 417

is_defined predicate, 43

isomorphic types, 320

iterator

linkable, 598

Iterator concept, 348

k-partition, 504

key function, 499, 569

key-increasing range, 499

key-sorting, 569

last in a range, 358

lattice, 466

lcm

and gcd, 465

least common multiple, 464

least common multiple, 464

Leibniz’s Law, 30

lexicographic ordering, 225, 229

limit in a range, 358

linear ordering

importance of, 213

linear recurrences, 206

link rearrangement, 603

linkable iterator, 598

linkable range, 598

Index XIV

linkable structures

forward versus bidirectional, 625

LinkableBidirectionalBifurcateCoordinate concept, 644

LinkableBidirectionalIterator concept, 606

LinkableBifurcateCoordinate concept, 643

LinkableForwardIterator concept, 605

linking operation, 599

links

exclusive-or of, 646

reversing, 645

Lo, Raymond, 482, 545

local state, 40

locality of reference, 476, 618

located sequence, 356

Lomuto, Nico, 526

loop fusion, 476

lower bound, 506

implementation, 554

memory, 335

memory-adaptive algorithm, 426

merge, 555

algebraic properties, 559

complexity property, 560

copying, 556

melding, 556

mutating, 556

stability, 555

Index XV

mergeable range, 556

Merger concept, 567

models

models (|=), 56

type models concept, 56

MonoidOperation concept, 136

MultiplicativeGroup concept, 201

MultiplicativeMonoid concept, 199

MultiplicativeSemigroup concept, 197

Mutable concept, 339

mutable range, 362

mutative rearrangement, 425

mutator procedure, 47

< reserved for, 237

Noether, Emmy, 307

non-terminating procedure, 44

nonessential object, 39

NonnegativeDiscreteArchimedeanSemiring concept, 318

not (¬), 803

not last in a range, 358

object, 24, 25

interpretation, 25

state, 25

one-to-one function, 408

onto function, 407

Index XVI

open interval, 243

Operation concept, 88

optimization

enabled by regular types, 81

loop fusion, 476

using segmented iterator, 398

or (∨), 803

orbit, 92

OrderedAdditiveGroup concept, 271

OrderedAdditiveMonoid concept, 271

OrderedAdditiveSemigroup concept, 271

OrderedCancellableMonoid concept, 273

ordering

reflexive, 217

strict, 217

weakly reflexive, 217

Ordering concept, 215

ordering-based rearrangement, 422

out-of-order execution, 385

overlapping ranges, 377

overloading, 190, 599, 642

own state, 40

partial

procedure, 43

representation, 27

partial ordering

example, 231

topological sort defined on, 233

Index XVII

partially-formed, 51

PartiallyCompatible concept, 728

PartialSemigroupOperation concept, 781

particulars of entity, 20

partition, 504

k-partition, 504

partition algorithm, 508

partition algorithms, origin of, 543

partition point, 505

finding, 515

lower bound, 506

upper bound, 506

partition rearrangement

semistable, 525

partitioned prefix of a range, 509

parts of an entity, 19

permutation, 409

composition, 412

cycle, 415

cyclic, 417

from, 431

identity, 411

index, 419

inverse, 413, 417

product of its cycles, 417

reverse, 436

rotation, 463

to, 431

permutation group, 414

Index XVIII

pointers

exclusive-or of, 646

polynomials

example of Euclidean domain, 297

position-based rearrangement, 422

powers

non-positive exponent, 138

of same element commute, 132

positive exponent, 131

Predicate concept, 214

procedure, 40

function, 47

generalized, 71

improper, 46

mutator, 47

non-terminating, 44

partial, 43

proper, 46

semi-terminating, 44

terminating, 44

total, 43

Procedure concept, 60

proper procedure, 46

ProperProcedure concept, 60

quicksort, 528, 543

Index XIX

random access iterator

equivalent to indexed iterator, 391

RandomAccessIterator concept, 387

range, 356

backward movement, 533

closed bounded, 357

closed counted, 357

empty, 360

first, 358

increasing, 499

k-partition, 504

key-increasing, 499

last, 358

limit, 358

linkable, 598

lower bound, 506

mergeable, 556

mutable, 362

not last, 358

overlapping, 377

partition, 504

partition point, 505

partitioned prefix, 509

readable, 362

semi-open bounded, 357

semi-open counted, 357

size, 359

uniform random shuffle, 545

upper bound, 506

Index XX

writable, 362

reachability of bifurcate coordinates, 632

reachable, 90

Readable concept, 336

readable range, 362

rearrangement, 420

bin-based, 422

copying, 425

generalized, 609

generalized link, 610

key-sorting, 569

link, 603

mutative, 425

ordering-based, 422

partition, 508

position-based, 422

reverse, 437

rotation, 469

sorting, 569

time complexity, 427, 434

refinement of concept, 57

reflexive ordering, 217

reflexivity of equality, 31

regular

allows code transformations, 81, 181

required for predicate with assured find, 544

Regular concept, 63

RegularAction concept, 346

RegularFunction concept, 79

Index XXI

relation

complement, 239

complement of converse, 239

converse, 239

symmetric, 218

symmetric complement of, 222

Relation concept, 214

relational concept, 726

relations

derived, 239

relationship between entities, 19

representation, 27

partial, 27

total, 27

unique, 28

requires clause, 71, 72

result space of procedure, 42

returning useful information, 365, 367, 394, 439, 469, 482, 523, 533, 537, 553, 563, 735, 737

reverse

permutation, 436

rearrangement, 437

reversing links, 645

rotation

permutation, 463

rearrangement, 469

Russian peasant algorithm, 144

same genus, 35

SameGenus concept, 727

Index XXII

Scalar type function, 300

segmented iterator

intuition, 398

SegmentedIterator concept, 399

SegmentIterator type function, 399

semi

usage convention, 232

semi-open bounded range, 357

semi-open counted range, 357

semi-terminating procedure, 44

SemigroupOperation concept, 130

Semimodule concept, 300

semistable partition rearrangement, 525

sequence

located, 356

sequence of iterators, 355

sets, 805

sink, 337

size

of range, 359

sort (type) of value, 21

sorting

stability, 569

sorting algorithm, 569

space complexity

forward iterator reverse conjecture, 455

in place, 426

memory adaptive, 426

with buffer, 426

Index XXIII

stability, 247

of min, max, 250

of rearrangement, 423

of sorting algorithm, 569

stable adaptive merge and sort, importance of, 585

stable in-place merge and sort, importance of, 585

stable merge, 555

state of object, 25

Stein, Josef, 327

storage efficiency, 83

strict lower bound, 434

strict ordering, 217

strictly increasing order, 248

StrictTotallyOrdered concept, 237

StrictTotalOrdering concept, 223

StrictWeakOrdering concept, 226

subset (⊂), 805

subtraction on additive group, 200

subtractive gcd, 290

symmetric complement of a relation, 222

symmetric relation, 218

symmetry of equality, 31

terminating procedure, 44

thing, 18

Tighe, Joseph, 469

time complexity

of rearrangement, 427, 434

to-permutation, 431

Index XXIV

total

procedure, 43

representation, 27

trait class, 811

transformation

duality with action, 343

fixed point of, 411

self-composable, 90

Transformation concept, 89

transitivity, 215

transitivity of equality, 31

traversal

of tree, recursive, 637

tree

descendants of bifurcate coordinate, 633

trichotomy law

for total ordering, 224

for weak ordering, 228

trivial cycle, 415

type, 36

constant area, 82

fixed area, 82

intuition, 37

isomorphism, 320

variable area, 82

type attribute, 54

Arity, 54

type concept, 58

type constructor, 53

Index XXV

type expression, 72

type function, 55

ArgumentT ype, 55

Codomain, 55

DifferenceT ype, 387

DistanceT ype, 348

Domain, 61

example, 109

Scalar, 300

SegmentIterator, 399

simulated in C++ via trait class, 811

UBI, 764

UFI, 763

UnderlyingT ype, 747

ValueT ype, 336, 337

WeightT ype, 626

UFI type function, 763

UnaryPredicate concept, 504

underlying type, 429, 747

UnderlyingT ype type function, 747

uniform cost, 84

uniform random shuffle, 545

union (∪), 805

unique representation, 28

univalent concept, 320

upper bound, 506

implementation, 554

Index XXVI

value, 19

sort (type), 21

value type, 336

ValueT ype type function, 336, 337

van der Waerden, Bartel Leendert, 307

variable area, 82

weak

usage convention, 232

WeakBifurcateCoordinate concept, 626

weakening of concept, 57

weakly reflexive ordering, 217

WeaklyHalvableMonoid concept, 283

WeightT ype type function, 626

well-formed object state, 26

with buffer algorithm, 426

worst-case cost, 84

Writable concept, 337

writable range, 362

- NetSuite Data SheetUploaded bybook_carlos
- Jason Brownlee - Clever AlgorithmsUploaded byCosmin Parvulescu
- Learning C# by Developing Games with Unity 5.x - Second Edition - Sample ChapterUploaded byPackt Publishing
- Cpp TutorialUploaded bydan_todoran
- roboti vUploaded bybriuliana
- Programming Concepts in C++ _Version 3Uploaded byAshi Sharma
- 40443498 Programming in CUploaded byrmiller6377227
- BcaUploaded byshiv kapila
- Wrapper Class ExampleUploaded byMuneesh Bhalla
- Working With Classes and Properties for Game Development in VBUploaded byZaeni Marjiyanto, A.md
- CS101x S443 Constructor and Destructor Functions IIT BombayUploaded bysiriuslot
- C++_Coding_StyleUploaded bynazmin rahman
- 5-OOP2Uploaded byEyas A-e ElHashmi
- lklklklklUploaded byUsmanSing
- Chapter 3 Lesson 2 Functions.docxUploaded byJohn
- defensesm pseudocodeUploaded byapi-397509789
- Operation Manual T-EM 1-2 en 3Uploaded byJayesh Kumar
- Functions S10Uploaded byJoe KL
- chapter 3- functions sir shahzadUploaded byapi-394738731
- UNIT IIUploaded byapjkalam1
- Expanding UniverseUploaded byjiderjustin
- The Language of Technical ComputingUploaded byyogesh sharma
- Chapter 02_Global SettingsUploaded bytopankaj
- The ProgramUploaded byguillermo_17kts
- j2c++uk(s12)Uploaded byMartin Jørgnsen
- Missing ManualUploaded bynebos
- Lab Manual 4Uploaded byMohd Akid
- 25422733 C Programming and Data Structures Lab ManualUploaded byAnanth Nath
- oopstask1Uploaded bySurya Vijayakumar
- CS304_FINAL_SPRING2006.pdfUploaded bymc190204637 AAFIAH TAJAMMAL

- Starting High Inertia LoadsUploaded byviswajd
- Adv Plsql2Uploaded byranusofi
- Numeric LiteralsUploaded byTran Nam
- CaryTucker CVUploaded bystephmhall
- Format of Lab Report Example 8609Uploaded byherrk1
- Butterfly Diagram - WikipediaUploaded byHashimIdrees
- Peka Form 5 2 (Exp No 1.3)Uploaded byNaguib Zakaria
- L11 Hermite Bicubic Surface PatchUploaded byAnonymous dGnj3bZ
- Antenna TheoryUploaded byImran Wali
- Sampling TheoryUploaded byPrima Hp
- Is LM TutorialUploaded byRAJAT SHARMA
- Cfd-ivings and GantUploaded bygirish19
- TEACHERS PERFORMANCE MANAGEMENT SYSTEM AT ISOMORPHIC HIGHER EDUCATIONAL INSTITUTIONS.Uploaded byIJAR Journal
- bernhofen-lect2-2Uploaded bymajka077677
- Astrorex D22 catlougeUploaded byManisankar Dhabal
- Exercise quadratic equationsUploaded byzaedmohd
- Wave Databases & Shipping RoutesUploaded byAnonymous qT66pyK
- Scaffold.pptUploaded byliandu
- Calculation of Shear CenterUploaded byflorin
- Input & OutputUploaded bymas akhbaruddin
- aeroZ_UKUploaded bypunkers17
- Twister TutorialUploaded byNawaal Ali
- Thesis Writing - CopyUploaded byHamka Hidayah
- Cojinetes para fricciónUploaded byRaúl García
- GateUploaded byAnonymous Y3GhNf
- K-2413 (Computer Science and Application) (Paper III)Uploaded bySunitha Kishore
- Circles TheoryUploaded bykishangopi123
- CPR Compact Position Report Format FaaUploaded byAustrianSimon
- relatividad generalUploaded byYhomn Acer
- Assignment 5Uploaded byXu Ka