You are on page 1of 880

Elements of Programming

Alexander Stepanov Paul McJones

March 14, 2008

Stepanov, McJones Elements of Programming March 14, 2008 1 / 880


Contents I

1 Introduction

2 Foundations

3 Transformations and their orbits

4 Algorithms on algebraic structures

5 Orderings

6 Combining concepts

7 Refining concepts of iterators

Stepanov, McJones Elements of Programming March 14, 2008 2 / 880


Contents II
8 Permutations and rearrangements

9 Rotations

10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects

13 Iterative algorithms for divide-and-conquer

14 Mathematical notation

Stepanov, McJones Elements of Programming March 14, 2008 3 / 880


Contents III
15 C++ machinery

16 Acknowledgments

17 Index

Stepanov, McJones Elements of Programming March 14, 2008 4 / 880


Approach

Programs are mathematical objects


Mathematics provides tools for
reasoning
deriving
composing
optimizing
Mathematics applied to programming addresses
security
reliability
performance
productivity

Stepanov, McJones Elements of Programming March 14, 2008 5 / 880


Blending programming and mathematics

1 Start with a practical algorithm


2 Define mathematical requirements on which it depends
3 Implement it abstractly and efficiently
4 Place it in the general taxonomy of requirements and algorithms

Stepanov, McJones Elements of Programming March 14, 2008 6 / 880


Intended audience

This book is for those who seek deeper understanding of


programming and are willing to invest substantial effort working
through the material
The book is not encyclopedic and is designed to be read
sequentially, in its entirety1
While this book does not have formal prerequisites, programming
maturity and a readiness to deal with serious mathematics are
assumed
We present a unified approach to programming spanning from the
highest level of abstraction down to the machine

1
Perhaps more than once
Stepanov, McJones Elements of Programming March 14, 2008 7 / 880
The choice of C++

We use a small subset of C++ as a good approximation to the ideal


teaching language
Programs in this book deal with both memory addresses and
abstract mathematical theories
C defines an abstract machine that serves as a foundation for
modern processor architectures2
C++ extends C with mechanisms for writing abstract versions of
algorithms and data structures without sacrificing efficiency

2
C does not address the issues of memory hierarchies, instruction-level parallelism,
and thread-level parallelism; research is needed to incorporate these features into the
system programming language of the future
Stepanov, McJones Elements of Programming March 14, 2008 8 / 880
The relationship to C++ STL

The book and C++ STL are results of the same research
The book departs from it significantly by providing improved
interfaces in a number of places
The book does not attempt to provide a full set of library
interfaces or deal with all the issues a real library must address

Stepanov, McJones Elements of Programming March 14, 2008 9 / 880


The machine model

Our machine model is abstracted from C/C++


Sequential (single-threaded) stacked-based execution
A linear address space, not necessarily all accessible
Memory allocation and deallocation under programmer control
Our programs and techniques can be used as building blocks for
systems with concurrent execution and automatic storage
deallocation

Stepanov, McJones Elements of Programming March 14, 2008 10 / 880


C++ usage conventions

We avoid implicit conversions to make our code strongly typed


with respect to concepts 3
We do not use postincrement ++ nor do we depend on the return
value of preincrement ++
We omit inline since its use depends on compiler, target
platform, and the code

3
Therefore all the constructors defined in the book should be considered to be
explicit
Stepanov, McJones Elements of Programming March 14, 2008 11 / 880
Our use of mathematics

We present only enough mathematics to provide the appropriate


settings for the programs in the book
We strive for the level of rigor typical in modern mathematical
textbooks, while attempting to preserve mathematical and
algorithmic intuitions
All the lemmas in this book can be proved via straightforward
derivations from the provided definitions
We urge the reader to prove every lemma to ensure true
comprehension

Stepanov, McJones Elements of Programming March 14, 2008 12 / 880


Outline of the book

Part I: Algorithms on mathematical abstractions


Chapter 1 - Foundations
Chapter 2 - Transformations and their orbits
Chapter 3 - Algorithms on algebraic structures
Chapter 4 - Orderings
Chapter 5 - Combining concepts
Part II: Algorithms on abstractions of memory
Chapter 6 - Refining concepts of iterators
Chapter 7 - Permutations and rearrangements
Chapter 8 - Rotations
Chapter 9 - Algorithms on increasing ranges
Chapter 10 - Coordinate structures
Chapter 11 - Composite objects
Chapter 12 - Iterative algorithms for divide-and-conquer
Appendix 1 - Mathematical notation
Appendix 2 - C++ machinery

Stepanov, McJones Elements of Programming March 14, 2008 13 / 880


Contents I

1 Introduction

2 Foundations
Meaning
Objects
Relations on objects
Procedures
Assignment, destruction, and construction
Relationships between types
Concept Regular
Procedures as objects
Generalized procedures
Parameter passing
Regular functions

Stepanov, McJones Elements of Programming March 14, 2008 14 / 880


Contents II
Complexity

3 Transformations and their orbits

4 Algorithms on algebraic structures

5 Orderings

6 Combining concepts

7 Refining concepts of iterators

8 Permutations and rearrangements

Stepanov, McJones Elements of Programming March 14, 2008 15 / 880


Contents III
9 Rotations

10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects

13 Iterative algorithms for divide-and-conquer

14 Mathematical notation

15 C++ machinery

Stepanov, McJones Elements of Programming March 14, 2008 16 / 880


Contents IV
16 Acknowledgments

17 Index

Stepanov, McJones Elements of Programming March 14, 2008 17 / 880


Things, ideas, and descriptions

Humans observe and interact with real things


We observe things: our minds form ideas (mental images) of
them; we act: our ideas affect things or create new things
We create descriptions of our ideas to preserve them,
communicate them, or allow them to participate in manual and
automatic processes
Note that a description is itself a thing

Stepanov, McJones Elements of Programming March 14, 2008 18 / 880


Universe of ideas

A value is an abstract idea (logical or mathematical) such as false


or 2 or a geometrical point
An entity is an idea of a thing such as a car or an email message
An entity has attributes with values, such as job level or salary of an
employee
An entity can be composed of other entities called parts, such as the
engine of a car or the body of an email message
An entity can be in relationships with other entities, such as the
supervisor of an employee
An activity is an idea of changes performed on entities that
changes their attributes or creates new entities, such as promoting
a given employee
An abstraction is what is common to a class of entities
An abstraction of values, such as Integer
An abstraction of entities, such as Employee
An abstraction of activities, such as Promotion
Stepanov, McJones Elements of Programming March 14, 2008 19 / 880
Entities

When we form an idea of an entity, we grasp:


Its existence
Its nature
Activities affecting it
Its particulars: attributes, parts, and relationships
In computer programs
Existence corresponds to object identity
Nature corresponds to object type
Activities correspond to procedures
Particulars correspond to object state, object components, and
object relationships

Stepanov, McJones Elements of Programming March 14, 2008 20 / 880


Values

Similar values (natural numbers, real numbers, polygons, etc.)


belong to different sorts

Stepanov, McJones Elements of Programming March 14, 2008 21 / 880


The universality of mathematics

“Mathematics, in general, is the science of quantity; or, the science


which investigates the means of measuring quantity” 4
Mathematical reasoning, including discrete arithmetic reasoning
and continuous spatial reasoning, is a fundamental, innate ability
of the human mind

4
Leonard Euler. Elements of Algebra. Third Edition, Longman, London, 1822.
Stepanov, McJones Elements of Programming March 14, 2008 22 / 880
Continuity and discreteness

Ancient mathematicians (the early Pythagoreans) discovered


space can’t be represented using discrete quantities
The arithmetization of geometry required the introduction of the
real numbers
The continuity of physical processes allows us to effectively
approximate real numbers with discrete, finite-precision quantities
Scientists and engineers have encountered no problem where
careful approximation has failed to work
This is the key to the applicability of digital computers

Stepanov, McJones Elements of Programming March 14, 2008 23 / 880


Computer descriptions of entities

As we said earlier, we understand the world in terms of entities


Computer programmers map these entities into objects containing
discrete, bounded quantities approximating the values of the
attributes
The meaning of a program is in the mind, not in the computer

Stepanov, McJones Elements of Programming March 14, 2008 24 / 880


Objects

Definition
An object is a set of memory locations and other resources together
with primitive procedures defined on it that provide its
computational basis
The state of an object is the addresses of its memory locations
together with the content of those locations at a particular point in
time
The interpretation of an object is a mapping from its state to an
entity
An interpretation maps states in such a way that procedures on the
object are consistent with the corresponding activities on entities

Stepanov, McJones Elements of Programming March 14, 2008 25 / 880


Well-formed object states

Definition
An object is in a well-formed state if there is an entity corresponding to it

Examples
Every state of a computer word is well-formed when interpreted
as an integer
A pointer containing an illegal address is not well-formed
An IEEE 754 floating point NaN (Not a Number) is not
well-formed when interpreted as a real number

Stepanov, McJones Elements of Programming March 14, 2008 26 / 880


Partial and total representations

Definition
A representation for an entity of a given nature is an object state
corresponding to it; the entity is representable if such a state exists

Definition
A representation is partial if not every entity is representable by an
object state; a representation is total if it is not partial

Example
The type int is a partial representation of integers

Stepanov, McJones Elements of Programming March 14, 2008 27 / 880


Unique representations

Definition
A representation is unique if there is at most one object state
corresponding to every set of particulars of the entity

Examples
A representation of a truth value as a byte that interprets zero as
false and nonzero as true is not unique
Signed-magnitude representation does not provide a unique
representation of zero

Stepanov, McJones Elements of Programming March 14, 2008 28 / 880


Identity

In the real world, identity is conceptually straightforward, but


often slippery to deal with in specific situations
It was a major astronomical discovery that the Morning Star and
Evening Star are identical
The question of whether the author of Hamlet and William
Shakespeare identical is still debated
When does replacing parts change the identity?
A computer programmer must determine how to map the identity
of real things to objects
Multiple techniques are possible for representing object identity
A memory address
An index into a data structure
A unique value of an attribute (e.g., Social Security Number)

Stepanov, McJones Elements of Programming March 14, 2008 29 / 880


Equality

Leibniz’s law
x = y if, and only if, everything that may be said about any one of the
things x or y may also be said about the other a
a
Alfred Tarski, Introduction to Logic and to the Methodology of Science, Oxford, 1965

Stepanov, McJones Elements of Programming March 14, 2008 30 / 880


Properties of equality

Formalization of Leibniz’s Law:

x = y ⇔ ∀P(P(x) ⇔ P(y))

The following properties follow:


reflexivity x = x
symmetry (x = y) ⇔ (y = x)
transitivity ((x = y) ∧ (y = z)) ⇒ (x = z)

Stepanov, McJones Elements of Programming March 14, 2008 31 / 880


Equality in mathematics

Equational reasoning is the basis of mathematics

2
Arithmetic: 4 = 21
Geometry: 4ABC = 4ACB
Algebra: (a + b)2 = a2 + 2ab + b2

Stepanov, McJones Elements of Programming March 14, 2008 32 / 880


Equality in the real world

Equality generally implies equality of essential attributes


Not all attributes are necessarily essential
Essentialness of attributes often depends on context
All US quarters are equal to a fare collector, but not to a coin
collector
Equality is not the same as identity
A thing’s attributes can change without affecting its identity
Distinct things can have equal attributes and, therefore, be equal
themselves

Stepanov, McJones Elements of Programming March 14, 2008 33 / 880


Equality in programming

This book extends Leibniz’s law from unchanging values to


mutable software objects
Leibniz’s law is the fundamental property of equality, but is not a
constructive definition
It establishes the consequences of the equality of two integers
It does not describe how to compare them for equality
Developing constructive implementations of equality is one of the
tasks of this book

Stepanov, McJones Elements of Programming March 14, 2008 34 / 880


Genus

Definition
Two objects are of the same genus if they represent entities with the
same nature under their corresponding interpretations

Examples
Polar and Cartesian representations of a complex number are of
the same genus
Two’s-complement and signed-magnitude representations of
integers are of the same genus
16-bit and 32-bit unsigned representations of natural numbers are
of the same genus

Stepanov, McJones Elements of Programming March 14, 2008 35 / 880


Type

Definition
Two objects have the same type if they share the same procedures and
the same interpretation

Stepanov, McJones Elements of Programming March 14, 2008 36 / 880


Intuition for type

The English word type derives from the Greek word túpoc (tupos):
“the impress made by the blow, what is formed, what leaves its
impress, the form-giving form, hence form generally as outline”5
The technical sense of tupos was “. . . the pattern in conformity to
which a thing must be made” 6
In programming, type is a fundamental idea preceding any notion
of type in programming languages
A type is a design for objects, as an engineering drawing is a
design for manufactured objects

5
Kittel and Friedrich. Theological Dictionary of the New Testament, Volume 8,
Eerdmans, 1972, page 246
6
Thayer and Smith. “Greek Lexicon entry for Tupos”. “The New Testament Greek
Lexicon”
Stepanov, McJones Elements of Programming March 14, 2008 37 / 880
Equality of objects

Definition
Two objects of the same genus are equal if their states represent entities
with equal essential particulars

Like Leibniz’s Law, this is not a constructive definition; it,


however, allows us to apply the mathematical understanding of
equality on abstract values to the world of programming

Stepanov, McJones Elements of Programming March 14, 2008 38 / 880


Nonessential objects

Definition
A nonessential object is one whose value does not affect the
computation

Example
An object used only to instrument or tune the performance is
nonessential

Stepanov, McJones Elements of Programming March 14, 2008 39 / 880


How procedures interact with objects

A procedure interacts with the state of the computation through


four kinds of objects:
Arguments are objects given to a procedure at its point of invocation
Local state is objects created and accessed during a single invocation
of the procedure
Global state is objects accessible to this and other procedures across
multiple invocations
Own state is objects accessible only to this procedure but shared
across multiple invocations

Stepanov, McJones Elements of Programming March 14, 2008 40 / 880


Domain, argument types, and definition space

In mathematics, a function typically takes a single argument, or an


operation such as addition takes two arguments
In programming, it is common for a procedure to take many
arguments
Rather than defining the domain of a procedure as the direct
product of the types of its arguments, we refer to the argument
types of a procedure
In the next chapter we extend the definition of domain to
homogeneous functions (all of whose arguments are of the same
types)

Definition
The definition space for a procedure is that subset of the direct product
of its argument types to which the procedure is intended to be applied

Stepanov, McJones Elements of Programming March 14, 2008 41 / 880


Codomain and result space

Definition
The codomain for a procedure is the type it returns

Definition
The result space for a procedure is the set of values from its codomain
returned by the procedure for some inputs from its definition space

Stepanov, McJones Elements of Programming March 14, 2008 42 / 880


Total and partial procedures

Definition
A procedure is partial if its definition space is a subset of the direct
product of the types of its arguments; it is total if they are equala
a
In mathematical usage, “partial” includes “total”; for example, total ordering
implies partial ordering

There is a (potentially implicit) predicate, is_defined, that is true


when applied to a procedure and set of arguments lying within
the definition space of that procedure
Preconditions are used to specify the definition spaces for
procedures
Before a partial procedure is called, either its precondition must be
satisfied, or the call must be guarded by a call on is_defined7
7
For example of the former, see weak_remainder in Chapter 5, and for an example
of the latter, see collision_point in Chapter 2
Stepanov, McJones Elements of Programming March 14, 2008 43 / 880
Terminating and non-terminating procedures

Definition
A procedure is terminating, semi-terminating, or non-terminating
depending on whether it always terminates, sometimes terminates, or
never terminates on its definition space

A partial procedure may terminate even for arguments outside its


definition space
All the procedures in this book are terminating unless explicitly
stated otherwise
A terminating procedure is known as an algorithm

Stepanov, McJones Elements of Programming March 14, 2008 44 / 880


Stateful and indexed procedures

Definition
A procedure is stateful, indexed, or stateless depending on whether its
own state is mutable, immutable, or empty

Stepanov, McJones Elements of Programming March 14, 2008 45 / 880


Proper and improper procedures

Definition
A procedure is proper if it does not modify any essential objects in its
global state; otherwise it is improper

All the procedures in this book are proper

Stepanov, McJones Elements of Programming March 14, 2008 46 / 880


Functions and mutators

Definition
A function is a procedure that does not modify its arguments and
returns a newly constructed object called its result; a mutator is a
procedure whose main purpose is to modify its arguments

Stepanov, McJones Elements of Programming March 14, 2008 47 / 880


Procedure examples

Addition is a total function on integers, but is partial on int


Division is a partial function on integers since its definition space
omits zero as a value for its second argument
A daemon is a non-terminating procedure
The Knuth-Bendix procedure in automatic theorem proving is
semi-terminating
Sorting is a terminating proper mutator
A pseudo-random number generator that keeps its state in an own
variable is a stateful proper function
Multiplication mod k for a fixed k is an indexed proper function
with k being its index
Setting a global graphics state variable is an improper mutator

Stepanov, McJones Elements of Programming March 14, 2008 48 / 880


Assignment

Definition
An assignment is a mutator taking two objects of the same genus whose
effect is to make the first object equal to the second without modifying
the second

The final interpretation of the first object must equal the


interpretation of the second object

Stepanov, McJones Elements of Programming March 14, 2008 49 / 880


Destruction

Definition
A destructor is a mutator causing the cessation of an object’s existence
After a destructor has been invoked on an object, no procedure
can be applied to it and its former memory locations and
resources may be reused for other purposes

Stepanov, McJones Elements of Programming March 14, 2008 50 / 880


Partially-formed state

Definition
An object is in a partially-formed state if it can be assigned to or
destroyed

For an object that is partially-formed but not well-formed, the


effect of any procedure other than assignment (only on the left
side) and destruction is not defined

Stepanov, McJones Elements of Programming March 14, 2008 51 / 880


Construction

Definition
A constructor is a mutator transforming memory locations into an
object

The possible behaviors range from doing nothing to establishing


complex object state

Stepanov, McJones Elements of Programming March 14, 2008 52 / 880


Type constructors

Definition
A type constructor is a mechanism for creating a new type from one or
more existing types

Examples
In C++, T* is the type “pointer to T”, for any type T
In C++, struct{T0 , ..., Tn−1 } is an n-ary type constructor
In Chapter 10 we show how to construct types corresponding to
data structures such as lists and arrays

Stepanov, McJones Elements of Programming March 14, 2008 53 / 880


Type attributes

Definition
A type attribute is a mapping from a type to a value describing some
characteristic of the type

Examples
The size of an object in bytes
The alignment of an object
The number of members in a struct

Definition
If F is a procedure type, Arity(F) returns its number of arguments

Stepanov, McJones Elements of Programming March 14, 2008 54 / 880


Type functions
Definition
A type function is a mapping from a type to an affiliated type

Examples
Given “pointer to T ”, the value type T
The result type of the difference of two pointers of a given type
The type of the ith member of a struct type (counting from 0)

Definition
If F is a procedure type and i < Arity(F), ArgumentT ype(F, i)
returns the type of the ith argument (counting from 0)
If F is a procedure type, Codomain(F) returns the type of the
result

In Appendix 2 we show how to define some type functions in C++


Stepanov, McJones Elements of Programming March 14, 2008 55 / 880
Concepts

Definition
A concept is a predicate on types stated in terms of properties of
procedures, type attributes, and type functions defined on the types

Definition
A concept is said to be modeled by specific types, or the types model
the concept, if the properties of the concept are satisfied for those
types
To assert that a concept C is modeled by types t0 , . . . , tn−1 , we
write C(t0 , . . . , tn−1 )

Stepanov, McJones Elements of Programming March 14, 2008 56 / 880


Refinement and weakening of concepts

Definition
Concept C 0 refines concept C if whenever C 0 is satisfied for a set of
types, C is also satisfied for those types a
We say C weakens C 0 if C 0 refines C

a
Technically, it is sufficient for C to hold for a subset of the types, possibly reordered

Stepanov, McJones Elements of Programming March 14, 2008 57 / 880


Type concepts

Definition
A type concept is a concept defined on one type

Stepanov, McJones Elements of Programming March 14, 2008 58 / 880


Examples of type concepts in C++

Examples
Integral type
Unsigned integral type
Modeled by: unsigned, unsigned long, unsigned char
Signed integral type
Modeled by: int, long, char
Pointer type
Modeled by: int*, char*
Sequence
Modeled by: std::vector<int>, std::list<int>
Bidirectional iterator
Modeled by: std::list<int>::iterator, int*

Stepanov, McJones Elements of Programming March 14, 2008 59 / 880


Primitive type concepts

Procedure
ProperProcedure
Function

Stepanov, McJones Elements of Programming March 14, 2008 60 / 880


Concept HomogeneousFunction

Definition
HomogeneousFunction(F) ⇒
Function(F)
Arity(F) > 0
For all i, j such that 0 6 i, j < Arity(F),
ArgumentT ype(F, i) = ArgumentT ype(F, j)

Definition
The type function Domain(F) is defined on any function type F
satisfying HomogeneousFunction(F):

Domain(F) ≡ ArgumentT ype(F, 0)

Stepanov, McJones Elements of Programming March 14, 2008 61 / 880


Requirements for interoperability of components
We want a sort procedure that works on an array of elements
The requirements on the element type allowing objects to be
sorted should be satisfied by arrays themselves, so we can sort an
array of arrays
All built-in types (e.g., int, float) should satisfy these common
requirements
The requirements are based on equational reasoning
Since the requirements assure regularity of behavior and
interoperability, we call the type concept corresponding to them
regular and the types modeling it regular types 8
8
Regular types were first introduced in:
James C. Dehnert and Alexander A. Stepanov.
Fundamentals of Generic Programming.
Report of the Dagstuhl Seminar on Generic Programming, Schloss Dagstuhl,
Germany, April 1998; also appears in Lecture Notes in Computer Science (LNCS)
volume 1766, pages 1-11.

Stepanov, McJones Elements of Programming March 14, 2008 62 / 880


Concept Regular

A Regular type has9 :


Equality
Assignment
Destructor
Default constructor
Copy constructor
Default total ordering (defined in Chapter 4)

9
In Chapter 12 we extend Regular with the notion of underlying type and its
affiliated types and functions
Stepanov, McJones Elements of Programming March 14, 2008 63 / 880
Equality for Regular

A regular type must implement equality


Equality should be defined on well-formed object states; two
objects are equal if and only if they represent the same abstract
value
Equality is written as a = b and inequality is written as a , b

Stepanov, McJones Elements of Programming March 14, 2008 64 / 880


Assignment for Regular

A regular type must implement assignment


Assignment is written as a ← b;

Stepanov, McJones Elements of Programming March 14, 2008 65 / 880


Destructor for Regular

A regular type must implement a destructor

Stepanov, McJones Elements of Programming March 14, 2008 66 / 880


Default constructor for Regular

A regular type must implement a default constructor


This is a constructor that takes no arguments and leaves the object
in a partially-formed state
The default construction for a local object a of type T is written T a;

Stepanov, McJones Elements of Programming March 14, 2008 67 / 880


Copy constructor for Regular

A regular type must implement a copy constructor


This is a constructor that takes an additional argument of the same
type and constructs a new object equal to it
The effect of the copy constructor is equivalent to default
construction followed by assignment
The copy constructor for a local object a of type T with initial
value b is written T a ← b;

Stepanov, McJones Elements of Programming March 14, 2008 68 / 880


Additional functions and affiliated types for Regular

In later chapters we introduce additional functions and affiliated


types required for a Regular type

Stepanov, McJones Elements of Programming March 14, 2008 69 / 880


Procedures as objects

A procedure is itself an object, and thus can be constructed,


destructed, assigned, copied, and tested for equality
These operations on a procedure are all operations on its own state
Two procedures of different type that perform the same
computation belong to the same genus
For example, heapsort and quicksort

Stepanov, McJones Elements of Programming March 14, 2008 70 / 880


Generalized procedures

Definition
A generalized procedure is a procedure defined in terms of type
requirements
Each type requirement has a formal parameter name, introduced
via a typename clause following the template keyword
Requirements on the types are specified via the requires clause,
whose argument is an expression built up from actual types, formal
types, applications of type functions, type equality, concepts, and
their logical connectivesa

a
In this book we use only conjunction

Stepanov, McJones Elements of Programming March 14, 2008 71 / 880


Syntax of type expressions and requires clauses

<type expression> ::= <type> | <formal type> |


<type function> ( <type expression> )
<type predicate> ::= <type expression> == <type expression> |
<type concept> ( <type expression )
<requires expression> ::= <type predicate> |
<requires expression> && <type predicate>
<requires clause> ::= requires( <requires expression> )

A <type expression> can be used to declare procedure


arguments and local variables
Our requires clause is implemented in Appendix 2 with a simple
macro10

10
In the future C++ is expected to include support for concepts with different syntax
Stepanov, McJones Elements of Programming March 14, 2008 72 / 880
Example of a generalized procedure

template <typename Op>


requires(SemigroupOperation(Op))
Domain(Op) square(const Domain(Op)& x)
{
return op(x, x);
}

Stepanov, McJones Elements of Programming March 14, 2008 73 / 880


Passing parameters to procedures

We distinguish three cases of how a procedure uses an object


passed to it as a parameter:
For reading The procedure depends only on the initial value of
the object, not on its identity
For writing The procedure mutates the object
For selecting The procedure selects one of several objects
(without mutating any of them) and returns the
selected object (not a copy)

Stepanov, McJones Elements of Programming March 14, 2008 74 / 880


Passing parameters for reading
To pass a parameter for reading in C++, we use one of two
techniques
If the size of the parameter is small, we pass it by value (which
makes a local copy)
Otherwise, we pass it by const reference
If the procedure needs a local copy it can mutate, we pass it by
value even when it is large

template <typename I>


requires(Readable(I) && Iterator(I))
I find(I f, I l, const ValueType(I)& x)
{
while (f != l && source(f) != x)
++f;
return f;
}

Regular types allow us to use call by value: there is a copy


constructor, and it creates a copy equal to the original
Stepanov, McJones Elements of Programming March 14, 2008 75 / 880
Passing parameters for writing

To pass a parameter for writing in C++, we pass it by reference

template <typename T>


requires(Regular(T))
void swap(T& x, T& y)
{
T tmp = x;
x = y;
y = tmp;
}

(In a later chapter we show a more efficient way to write swap)

Stepanov, McJones Elements of Programming March 14, 2008 76 / 880


Passing parameters for selecting

To pass a parameter for selecting in C++, we must provide both


by reference and by const reference versions of the procedure:
template <typename T>
requires(StrictTotallyOrdered(T))
T& min(T& a, T& b)
{
if (b < a) return b;
else return a;
}

template <typename T>


requires(StrictTotallyOrdered(T))
const T& min(const T& a, const T& b)
{
if (b < a) return b;
else return a;
}

Stepanov, McJones Elements of Programming March 14, 2008 77 / 880


More on selecting

A function such as min can be used on the left side of an


assignment when all its parameters are writable
Increase the smaller of two variables by 100: min(a, b) += 100;
Note the return type of the two versions of the procedure must
agree with the selecting parameters with respect to constness
The reasons why C++ requires both versions lie outside the scope
of this book
(In this book we often show only the non-const version)

Stepanov, McJones Elements of Programming March 14, 2008 78 / 880


Concept RegularFunction

Definition
RegularFunction(F) ⇒
Function(F)
Regular(F)
For all f, g ∈ F such that f = g and for all 0 6 i < n:
Regular(ArgumentT ype(F, i))
For all xi , yi ∈ ArgumentT ype(F, i) such that xi = yi ,

f(x0 , . . . , xn−1 ) = g(y0 , . . . , yn−1 )

where n = Arity(F)

By extension, we call a generalized function regular if all of its


instantiations are regular

Stepanov, McJones Elements of Programming March 14, 2008 79 / 880


Reasons for non-regular functions

In mathematics, all functions are regular as a consequence of


Leibniz’s Law
In programming, there are situations when it’s not so
When a function returns the address of a memory location
The “address of” operator (& in C++)
When the function returns a value determined by the state of the
real world
Input from a device or another process
When a stateful function returns a value depending on its own state
A pseudo-random number generator
When a function returns a nonessential attribute of an object
Amount of preallocated memory for a data structure

Stepanov, McJones Elements of Programming March 14, 2008 80 / 880


Program optimization

Regular types and regular functions allow us to many standard


transformations, such as:
Common subexpression elimination
Loop hoisting
Copy propagation

Example
If f and g are regular, we can optimize f(g(x), g(x)) by evaluating g(x)
only once

Stepanov, McJones Elements of Programming March 14, 2008 81 / 880


Area of an object

The area of an object is the total number of bytes it occupies11


A type is of constant area if every object of that type has the same
area
A type is of fixed area if every object of that type has an an area that
does not change during its lifetime
A type is of dynamic area if the area of an object of the type can
vary during the lifetime of the object
For every type, there is a (potentially implicit) function, area, that
returns the area of an object of that type

11
In a binary computer we should in principle measure areas in bits, but unless we
used packed representations we deal in bytes
Stepanov, McJones Elements of Programming March 14, 2008 82 / 880
Storage efficiency

Definition
dlog2 56s(x)e
The byte storage efficiency of a fixed-area object x is area(x) , where
s(x) is the number of distinct states of x

Example
The byte storage efficiency of struct{bool} is 1
The byte storage efficiency of struct{bool,bool} is 1/2
The byte storage efficiency of struct{bool,bool,bool} is 1/3

Stepanov, McJones Elements of Programming March 14, 2008 83 / 880


Cost of a procedure call

The cost of a procedure call is the time required for its execution
The worst-case cost of a procedure call for arguments of given areas
is the maximum cost across all procedure calls with arguments of
these areas
The average cost of a procedure call for arguments of given areas is
the average cost across all procedure calls with arguments of these
areas
A procedure has uniform cost if the cost of a procedure call
depends only on the areas of its arguments
A procedure has fixed cost if the cost of a procedure call does not
depend on its arguments
There is a (possibly implicit) function, cost, that returns the cost
when applied to a procedure and set of arguments lying within
the definition space of that procedure

Stepanov, McJones Elements of Programming March 14, 2008 84 / 880


Contents I

1 Introduction

2 Foundations

3 Transformations and their orbits


Transformations
Orbits
Applications
Examples
Conclusions
Project

4 Algorithms on algebraic structures

Stepanov, McJones Elements of Programming March 14, 2008 85 / 880


Contents II
5 Orderings

6 Combining concepts

7 Refining concepts of iterators

8 Permutations and rearrangements

9 Rotations

10 Algorithms on increasing ranges

11 Coordinate structures

Stepanov, McJones Elements of Programming March 14, 2008 86 / 880


Contents III
12 Composite objects

13 Iterative algorithms for divide-and-conquer

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

17 Index

Stepanov, McJones Elements of Programming March 14, 2008 87 / 880


Concept Operation

Definition
Operation(Op) ⇒
RegularFunction(Op)
HomogeneousFunction(Op)
Codomain(Op) = Domain(Op)

Example
Unary abs : double → double
Binary + : double × double → double
Ternary multiply_add : double × double × double → double

Stepanov, McJones Elements of Programming March 14, 2008 88 / 880


Concept Transformation

Definition
Transformation(F) ⇒
Operation(F)
Arity(F) = 1
The predicate is_defined : F × Domain(F) → bool is defined

Example
Numbers Square root
Geometry Plane rotations; space translations
Combinatorics Reversal of a sequence

Stepanov, McJones Elements of Programming March 14, 2008 89 / 880


Composition and reachability

Transformations are self-composable


f(x), f(f(x)), . . .
Self-composability allows us to define iterative algorithms

Definition
f0 (x) = x
fn+1 (x) = f(fn (x))

Definition
y is reachable from x under a transformation f if y = x or there is a z
reachable from x and f(z) = y

Stepanov, McJones Elements of Programming March 14, 2008 90 / 880


Cyclic and terminal elements

Definition
x is cyclic under f if f(x) is defined and x is reachable from f(x)

Definition
x is terminal under f if and only if f is not defined at x

Stepanov, McJones Elements of Programming March 14, 2008 91 / 880


Orbits

Definitions
An orbit of x under a transformation f is the set of all elements
reachable from x under f

Lemma
An orbit does not contain both a cyclic and a terminal element

Lemma
An orbit contains at most one terminal element

Stepanov, McJones Elements of Programming March 14, 2008 92 / 880


Classification of orbits

Definitions
An orbit of x under f is:
infinite if it has no cyclic or terminal elements
terminating if it has a terminal element
circular if x is cyclic
ρ-shaped if x is not cyclic and its orbit contains a cyclic element
An orbit of x is finite if it is not infinite

Stepanov, McJones Elements of Programming March 14, 2008 93 / 880


Finite orbits

terminating

circular

ρ-shaped

Stepanov, McJones Elements of Programming March 14, 2008 94 / 880


Structure of orbits

Definition
The orbit cycle is the set of cyclic elements in the orbit

Definition
The orbit handle is the complement of the orbit cycle with respect to the
orbit

Definition
The connection point is the first cyclic element

Stepanov, McJones Elements of Programming March 14, 2008 95 / 880


Sizes

Definitions
The orbit size o of an orbit is the number of distinct elements in it
The handle size h of an orbit is the number of elements in the orbit
handle
The cycle size c of an orbit is the number of elements in the orbit
cycle

Lemma
o=h+c

Stepanov, McJones Elements of Programming March 14, 2008 96 / 880


Finite orbit assumption

Finiteness of a transformation is equivalent to the halting problem


Thus no algorithm can determine whether an orbit is finite or
infinite for an arbitrary transformation
For many transformations, finiteness is easily provable
There is an implicit precondition of orbit finiteness for all the
algorithms in this chapter

Stepanov, McJones Elements of Programming March 14, 2008 97 / 880


Algorithmic intuition for cycle detection

If two cars, a fast car and a slow car, start along a path, the fast car
will catch up with the slow car if and only if there is a cycle
If there is no cycle, the fast car will reach the end of the path before
the slow car
If there is a cycle, by the time the slow car enters the cycle, the fast
car will already be there, and will catch up eventually

Stepanov, McJones Elements of Programming March 14, 2008 98 / 880


Collision point

Definition
The collision point of a transformation f and a starting point x is the
unique y such that
y = fn (x) = f2n+1 (x)
and n > 0 is the smallest integer satisfying this condition

Stepanov, McJones Elements of Programming March 14, 2008 99 / 880


collision_point

template <typename F>


requires(Transformation(F))
Domain(F) collision_point(const Domain(F)& x, F f)
{
Domain(F) fast = x;
Domain(F) slow = x; // n ← 0 (completed iterations)
while (true) { // slow = fn (x) ∧ fast = f2n (x)
if (!is_defined(f, fast)) break;
fast = f(fast); // slow = fn (x) ∧ fast = f2n+1 (x)
A: if (fast == slow) break;
if (!is_defined(f, fast)) break;
fast = f(fast); // slow = fn (x) ∧ fast = f2n+2 (x)
slow = f(slow); // slow = fn+1 (x) ∧ fast = f2n+2 (x)
// n ← n + 1
}
return fast;
// Postcondition: fast is either the terminal point or the collision point of f and x
}

Stepanov, McJones Elements of Programming March 14, 2008 100 / 880


Proof of termination of collision_point

The movement of fast is guarded by a call of is_defined


The movement of slow is unguarded, because by the regularity of
f, slow will be traversing the same orbit as fast
If there is no cycle, is_defined will eventually return false because
of finiteness
If there is a cycle, slow will eventually reach the connection point
(the first element in the cycle):
Consider the distance d from fast to slow at the point labeled A
once slow enters the cycle
06d<c
If d = 0 the procedure terminates
Otherwise the distance decreases by one on each iteration
By induction, the procedure always terminates
When it terminates, slow has moved a total of h + d steps

Stepanov, McJones Elements of Programming March 14, 2008 101 / 880


Position of collision_point when a cycle exists
The annotations show that when there is a cycle and n > 0 is the
number of completed iterations, then fn (x) = f2n+1 (x)
n is the smallest such integer since we checked the condition for
every i < n
We express n = h + d where d is the distance slow moves after
reaching the connection point
Therefore h + d + qc = 2h + 2d + 1 for some q > 0 that counts the
number of cycles completed by fast when slow enters the cycle
Simplifying gives qc = h + d + 1
Represent h in terms of its quotient and remainder when divided
by c: h = mc + r for 0 6 r < c
Substitution gives qc = mc + r + d + 1 or d = (q − m)c − r − 1
0 6 d < c implies q − m = 1, so d = c − r − 1
The distance from the collision point to the connection point is
e=c−d=r+1
Stepanov, McJones Elements of Programming March 14, 2008 102 / 880
Distinguishing circular from ρ-shaped case

As a corollary of e = r + 1, it follows that in case of a circular orbit


h = 0, which implies r = 0 and so e = 1
So the distance from the collision point to the beginning of the
orbit is equal to 1
Circularity therefore can be checked with the following simple
procedure (we leave implementation of similar checks for
terminating and ρ-shaped to the reader)

Stepanov, McJones Elements of Programming March 14, 2008 103 / 880


is_circular

template <typename F>


requires(Transformation(F))
bool is_circular(const Domain(F)& x, F f)
{
Domain(F) y = collision_point(x, f);
return is_defined(f, y) && x == f(y);
}

Stepanov, McJones Elements of Programming March 14, 2008 104 / 880


Finding connection point

Let xw = xh+c−r−1+1 = xh+c−r be the element one past the


collision point
The element h steps beyond xw is xw+h = xh+c−r+h
Substituting h = mc + r gives xh+c−r+mc+r = xh+(m+1)c = xh
since it is m + 1 times around the cycle from xh
This suggests an algorithm for determining xh , the connection
point

Stepanov, McJones Elements of Programming March 14, 2008 105 / 880


connection_point

template <typename F>


requires(Transformation(F))
Domain(F) convergent_point(Domain(F) x, Domain(F) y, F f)
{
// Precondition: the orbits of x and y converge
while (x != y) {
x = f(x);
y = f(y);
}
return x;
}

template <typename F>


requires(Transformation(F))
Domain(F) connection_point(const Domain(F)& x, F f)
{
Domain(F) y = collision_point(x, f);
if (!is_defined(f, y)) return y;
return convergent_point(x, f(y), f);
}

Stepanov, McJones Elements of Programming March 14, 2008 106 / 880


Complexity

The complexity analysis of these algorithms trivially follows from


the termination proofs and is left as an exercise for the reader

Stepanov, McJones Elements of Programming March 14, 2008 107 / 880


Applications

Determining whether a linked structure contains a cycle


See for example the Common Lisp function list-length
Determining the length and the period of a random number
generator – see for example:
Donald Knuth.
Exercise 3.1.6.
The Art of Computer Programming, Volume 2, 3rd edition, 1998,
page 7.
Credits two-pointer orbit detection to R.W. Floyd.
Our code works for both cases

Stepanov, McJones Elements of Programming March 14, 2008 108 / 880


Problem: representing sizes

What type to use for o, h, and c?


We need to define a type function to obtain an integer type big
enough to encode the orbit size for a given type T

Stepanov, McJones Elements of Programming March 14, 2008 109 / 880


Definition of distance type

Definition
The distance type for a type T is an integer type a that allows us to
encode the maximum number of transformations from one
element of T into another
If a type occupies k bits, its distance type can be represented with
an unsigned integer type occupying k bits
This avoids an infinite tower of types
(It is difficult to have a count type that could count the number of
elements in any collection of type T , because that would require
an extra value)
The type function DistanceT ype(T ) returns the distance type of
T ; for the implementation in C++, see Appendix 2
a
We will discuss integer types briefly in the next chapter and in depth in Chapter 4

Stepanov, McJones Elements of Programming March 14, 2008 110 / 880


distance

template <typename F>


requires(Transformation(F))
DistanceType(Domain(F)) distance(const Domain(F)& x, const Domain(F)& y, F f)
{
// Precondition: y is reachable from x under f

typedef DistanceType(Domain(F)) D;
Domain(F) z = x;
D n = D(0);
while (z != y) {
z = f(z);
n = n + D(1);
}
return n;
}

Stepanov, McJones Elements of Programming March 14, 2008 111 / 880


Representing sizes

Note that o, h, and c may not fit in the distance type



 o−1
But each of these does fit: h−1

c−1
For non-terminating cases, h fits also

Stepanov, McJones Elements of Programming March 14, 2008 112 / 880


orbit_structure

template <typename F>


requires(Transformation(F))
triple<DistanceType(Domain(F)), DistanceType(Domain(F)), Domain(F)>
orbit_structure(const Domain(F)& x, F f)
{
typedef DistanceType(Domain(F)) D;
Domain(F) y = connection_point(x, f);
D m = distance(x, y, f);
D n(0);
if (is_defined(f, y))
n = distance(f(y), y, f);
// Terminating: m = h − 1 ∧ n = 0
// Otherwise: m = h ∧ n = c − 1
return triple<D, D, Domain(F)>(m, n, y);
}

Stepanov, McJones Elements of Programming March 14, 2008 113 / 880


Postcondition of orbit_structure

Case first second third


terminating h−1 0 terminal
circular 0 c−1 x
ρ-shaped h c−1 connection

Stepanov, McJones Elements of Programming March 14, 2008 114 / 880


Testing random number generators

Exercise
Given the C++ function object
struct random_function
{
int operator()(int x)
{
srand(x);
return rand();
}
};

use orbit_structure on enough different initial elements to decide


whether the cycle length generated by std::rand on your platform is
satisfactory

Stepanov, McJones Elements of Programming March 14, 2008 115 / 880


Conclusions

Practical algorithms can be described using basic mathematical


theories
Nomenclature helps
e.g., orbit kinds and sizes
Equational reasoning on types and functions is essential
Abstract code facilitates reasoning and complexity analysis

Stepanov, McJones Elements of Programming March 14, 2008 116 / 880


Project

There are other algorithms for orbit analysis:


R.T. Sedgwick, T.G. Szymanski and A.C. Yao.
The complexity of finding cycles in periodic functions.
Proc. 11th SIGACT Meeting, 1979, pages 376-390.
Richard P. Brent.
An improved Monte Carlo factorization algorithm.
BIT, Volume 20, 1980, pages 176-184.
Leon S. Levy.
An improved list-searching algorithm.
Information Processing Letters, Volume 15, Issue 1, August 1982,
pages 43-45.
Write generic versions of these algorithms and compare their
efficiencies

Stepanov, McJones Elements of Programming March 14, 2008 117 / 880


Contents I

1 Introduction

2 Foundations

3 Transformations and their orbits

4 Algorithms on algebraic structures


Algebra and abstraction
Power on semigroups, monoids, and groups
Slow power algorithm
Stepwise derivation of fast power algorithm
Concept for the exponent
Complexity
Applications

Stepanov, McJones Elements of Programming March 14, 2008 118 / 880


Contents II
Overloading
Conclusions
Reference
Projects

5 Orderings

6 Combining concepts

7 Refining concepts of iterators

8 Permutations and rearrangements

9 Rotations

Stepanov, McJones Elements of Programming March 14, 2008 119 / 880


Contents III
10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects

13 Iterative algorithms for divide-and-conquer

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

Stepanov, McJones Elements of Programming March 14, 2008 120 / 880


Contents IV
17 Index

Stepanov, McJones Elements of Programming March 14, 2008 121 / 880


Elementary algebra

Since its inception by the Arabs in the ninth century, algebra has
been viewed as the theory of operations on numbers irrespective
of their particular kinds (such as integers or rationals) or values
The common algebraic operations are addition, subtraction,
multiplication, and division
These operations possess fundamental properties such as
associativity of addition or distributivity of multiplication over
addition

Stepanov, McJones Elements of Programming March 14, 2008 122 / 880


Higher algebra

Through the eighteenth and nineteenth century algebra was


extended to include aggregates of numbers (such as matrices and
polynomials) together with the common algebraic operations on
them
The operations on aggregates turned out to satisfy the same
properties as the operations on numbers

Stepanov, McJones Elements of Programming March 14, 2008 123 / 880


Precursors of abstract algebra

By the late nineteenth century mathematicians (Jordan, Klein, Lie,


etc.) observed that a cluster of properties called a group (namely
an associative, invertible binary operation) appears in several
different domains (such as theory of equations and geometry)
At a slightly later time, mathematicians (Dedekind, Weber and
Hilbert) observed that a cluster of properties called a ring
connecting addition and multiplication also appears in multiple
contexts
By the early twentieth century, people learned to recognize
groups, rings, vector spaces, and other similar clusters of
properties

Stepanov, McJones Elements of Programming March 14, 2008 124 / 880


Abstract algebra

In the 1920s a fundamentally new point of view on algebra was


developed by several German mathematicians (Noether, Artin,
etc.)
The new approach was to take a well-understood cluster of
properties (such as a group) as axioms, and explore their
consequences independently of any underlying setting
As a result, the same algebraic techniques can be used in any
setting where these axioms are satisfied, leading to effective
applications in domains ranging from quantum mechanics to
crystallography
Computer science has found specialized use of these techniques in
areas such as automata theory and coding theory

Stepanov, McJones Elements of Programming March 14, 2008 125 / 880


Abstract mathematics

As with abstract algebra, from 1930 until about 1970 a large


portion of mathematics was reinterpreted putting classical results
in their most abstract settings
This process led to
an increase in soundness
reuse of techniques across different domains
Many of the known theories from abstract mathematics
(especially algebra) provide a natural setting for practical
algorithms, with a corresponding increase in soundness and reuse

Stepanov, McJones Elements of Programming March 14, 2008 126 / 880


Abstraction

By the time of Aristotle—if not even earlier—it had been recognized


that every formal discipline rested on what might be called
“the principle of voluntary denial of complete knowledge”:
abstraction or generalization literally indicates a systematic
discarding of certain aspects of the studied objects. 12

12
Jean Dieudonné. Linear Algebra and Geometry. Herman, Paris, 1969, page 17
Stepanov, McJones Elements of Programming March 14, 2008 127 / 880
Grounding abstraction

Concentrating only on abstractions could lead to forgetting the


underlying realities and the problems to be solved
Abstraction in programming should always start with the best
existing and practically useful algorithms and data structures

Stepanov, McJones Elements of Programming March 14, 2008 128 / 880


Concept BinaryOperation

Definition
BinaryOperation(Op) ⇒
Operation(Op)
Arity(Op) = 2

Stepanov, McJones Elements of Programming March 14, 2008 129 / 880


Concept SemigroupOperation

Definition
SemigroupOperation(Op) ⇒
BinaryOperation(Op)
For all op ∈ Op and for all a, b, c ∈ Domain(Op):

op(a, op(b, c)) = op(op(a, b), c)

The property is associativity


A common convention is to represent a semigroup operation as
the infix operator ◦; for example: a ◦ (b ◦ c) = (a ◦ b) ◦ c

Stepanov, McJones Elements of Programming March 14, 2008 130 / 880


Powers

a2 = a ◦ a
a3 = a ◦ a ◦ a

a1 = a
an = a
| ◦ a ◦{z· · · ◦ a}
n

an ◦ am = an+m

Stepanov, McJones Elements of Programming March 14, 2008 131 / 880


Properties of powers

Lemma
an ◦ am = am ◦ an = an+m (powers of the same element commute)

Lemma
(an )m = anm

Stepanov, McJones Elements of Programming March 14, 2008 132 / 880


Frobenius’s theorem

Definition
An element x has finite order under a semigroup operation if there exist
integers 0 < n < m such that xn = xm

Definition
An element x is an idempotent element under a semigroup operation if
x = x2

Theorem
An element of finite order has an idempotent power a
a
Georg Ferdinand Frobenius. Über endliche Gruppen. Berlin 1895. In:
Sitzungesberichte der Königlich Preussischen Akademie der Wissenschaften zu Berlin.
Phys.-math. Classe 1895, pages 163-194

Stepanov, McJones Elements of Programming March 14, 2008 133 / 880


Proof of Frobenius’s theorem

template <typename Op>


requires(SemigroupOperation(Op))
struct multiply_transformation
{
Domain(Op) x;
Op op;
multiply_transformation(Domain(Op) x, Op op) : x(x), op(op) {}
Domain(Op) operator()(const Domain(Op)& y)
{
return op(y, x);
}
};

template <typename Op>


requires(SemigroupOperation(Op))
Domain(Op) idempotent_power(Domain(Op) x, Op op)
{
return collision_point(x, multiply_transformation<Op>(x, op));
}

Stepanov, McJones Elements of Programming March 14, 2008 134 / 880


Explanation of proof of Frobenius’s theorem

Assume x is an element of finite order under a semigroup


operation op
Let

g(z) = multiply_transformation(op, x) = op(z, x) = z ◦ x

Let y = idempotent_power(x, op)


Since x is an element of finite order, its orbit under g has a cycle
By the postcondition of collision_point, y = gn (x) for the
smallest n > 0 such that gn (x) = g2n+1 (x), where
g1 (x) = g(x) = x ◦ x = x2
Thus gn (x) = xn+1 and g2n+1 (x) = x2n+2 = x2(n+1) = (xn+1 )2
Therefore y = gn (x) = xn+1 is the idempotent power of x

Stepanov, McJones Elements of Programming March 14, 2008 135 / 880


Concept MonoidOperation

Definition
MonoidOperation(Op) ⇒
SemigroupOperation(Op)
identity_element : Op → Domain(Op) is defined
For all op ∈ Op, for all a ∈ Domain(Op), and
for e = identity_element(op):

op(a, e) = a = op(e, a)

Stepanov, McJones Elements of Programming March 14, 2008 136 / 880


Concept GroupOperation

Definition
GroupOperation(Op) ⇒
MonoidOperation(Op)
inverse_operation : Op → (Domain(Op) → Domain(Op)) is
defined
For all op ∈ Op, for all a ∈ Domain(Op), and for
g = inverse_operation(op):

op(a, g(a)) = identity_element(op) = op(g(a), a)

A common convention is to represent the inverse of a as a−1

Stepanov, McJones Elements of Programming March 14, 2008 137 / 880


Extending powers to non-positive exponents

a0 = e
an ◦ a0 = an+0 = an
a−n = (an )−1
an ◦ a−n = an−n = a0

Lemma
(a−1 )n = a−n

Stepanov, McJones Elements of Programming March 14, 2008 138 / 880


Concept Integer

We will study various concepts describing integers in Chapter 4


In the meantime we will rely on the intuitive understanding of
what integers are
Models include:
all C++ integral types, signed and unsigned
bignums (type allowing arbitrary-precision integers)
Operations are + - * / % with their standard semantics
All integer types contain constants 0 and 1
Initially we use the constant 2, but later in the chapter we introduce
special-case functions that allow us to avoid its use
In C++ the constants 0, 1, and 2 of an integer type I are represented
as I(0), I(1), and I(2)

Stepanov, McJones Elements of Programming March 14, 2008 139 / 880


slow_power_positive

template <typename I, typename Op>


requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) slow_power_positive(Domain(Op) a, I n, Op op)
{
// Precondition: n > I(0)
if (n == I(1))
return a;
else
return op(a, slow_power_positive(a, n - I(1), op));
}

Stepanov, McJones Elements of Programming March 14, 2008 140 / 880


slow_power_nonnegative

template <typename I, typename Op>


requires(Integer(I) && MonoidOperation(Op))
Domain(Op) slow_power_nonnegative(Domain(Op) a, I n, Op op)
{
// Precondition: n > I(0)
if (n == I(0))
return identity_element(op);
else
return slow_power_positive(a, n, op);
}

Stepanov, McJones Elements of Programming March 14, 2008 141 / 880


Observations

When computing an , we perform n − 1 operations


We never perform the operation with the identity element
Except when a is the identity element
It is not worth adding an extra check for the case of a being the
identity element
This would slow down the average case
The caller can do this if it is important

Stepanov, McJones Elements of Programming March 14, 2008 142 / 880


slow_power

template <typename I, typename Op>


requires(Integer(I) && GroupOperation(Op))
Domain(Op) slow_power(Domain(Op) a, I n, Op op)
{
if (n < I(0))
{
a = inverse_operation(op)(a);
n = -n;
}
return slow_power_nonnegative(a, n, op);
}

Stepanov, McJones Elements of Programming March 14, 2008 143 / 880


Ancient Egyptian discovery

Egyptian scribe Ahmes described (circa 1650 BC) a different


power algorithm that gives an efficient way to multiply 13
It is sometimes called the Russian peasant algorithm 14

13
Gay Robins and Charles Shute. The Rhind Mathematical Papyrus, British Museum
Press, 1987
14
The oldest reference to Russian origin we have found appears in: Sir Thomas
Heath. A History of Greek Mathematics, Volume I, Clarendon Press, 1921, page 53.
Reprint: Dover, 1981. Heath writes: “I have been told that there is a method in use
to-day (some say in Russia, but I have not been able to verify this) . . . ” We have not
been able to verify this either.
Stepanov, McJones Elements of Programming March 14, 2008 144 / 880
Exploiting associativity

For example: aaaaaaa = ((aa)a)((aa)a)a


In general: an = (a2 )n/2 an mod 2
This translates into the following procedure

Stepanov, McJones Elements of Programming March 14, 2008 145 / 880


Initial (recursive) version

template <typename I, typename Op>


requires(Integer(I) && MonoidOperation(Op))
Domain(Op) power_nonnegative_0(Domain(Op) a, I n, Op op)
{
// Precondition: n > I(0)
if (n == I(0))
return identity_element(op);
else if (n == I(1))
return a;
else
return op(power_nonnegative_0(op(a, a), n / I(2), op),
power_nonnegative_0(a, n % I(2), op));
}

Stepanov, McJones Elements of Programming March 14, 2008 146 / 880


Transforming a program

We will now demonstrate a sequence of transformations that


improve the performance of this algorithm without changing its
asymptotic complexity
For the rest of the book we will typically only show final or
almost-final versions

Stepanov, McJones Elements of Programming March 14, 2008 147 / 880


Eliminating unnecessary multiplication by identity

template <typename I, typename Op>


requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_positive_0(Domain(Op) a, I n, Op op)
{
// Precondition: n > I(0)
if (n == I(1))
return a;
else if (n % I(2) == I(0))
return power_positive_0(op(a, a), n / I(2), op);
else
return op(power_positive_0(op(a, a), n / I(2), op), a);
}

Stepanov, McJones Elements of Programming March 14, 2008 148 / 880


Eliminating common subexpression

template <typename I, typename Op>


requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_positive_1(Domain(Op) a, I n, Op op)
{
// Precondition: n > I(0)
if (n == I(1))
return a;
Domain(Op) result = power_positive_1(op(a, a), n / I(2), op);
if (n % I(2) != I(0))
result = op(result, a);
return result;
}

Stepanov, McJones Elements of Programming March 14, 2008 149 / 880


Introducing accumulation variable

template <typename I, typename Op>


requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_nonnegative_0(Domain(Op) r, Domain(Op) a, I n, Op op)
{
// Precondition: n > I(0)
if (n == I(0))
return r;
if (n % I(2) != I(0)) r = op(r, a);
return power_accumulate_nonnegative_0(r, op(a, a), n / I(2), op);
}

Recursion invariant: ran

Stepanov, McJones Elements of Programming March 14, 2008 150 / 880


Almost iterative

template <typename I, typename Op>


requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_nonnegative_1(Domain(Op) r, Domain(Op) a, I n, Op op)
{
// Precondition: n > I(0)
if (n == I(0))
return r;
if (n % I(2) != I(0)) r = op(r, a);
a = op(a, a);
n = n / I(2);
return power_accumulate_nonnegative_1(r, a, n, op);
}

Recursion invariant: ran

Stepanov, McJones Elements of Programming March 14, 2008 151 / 880


Iterative

template <typename I, typename Op>


requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_nonnegative_2(Domain(Op) r, Domain(Op) a, I n, Op op)
{
// Precondition: n > I(0)
while (true) {
if (n == I(0))
return r;
if (n % I(2) != I(0)) r = op(r, a);
a = op(a, a); // wasted on last iteration
n = n / I(2);
}
}

Loop invariant: ran

Stepanov, McJones Elements of Programming March 14, 2008 152 / 880


Rotating the loop: reordering

template <typename I, typename Op>


requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_nonnegative_3(Domain(Op) r, Domain(Op) a, I n, Op op)
{
// Precondition: n > I(0)
while (true) {
if (n == I(0))
return r;
if (n % I(2) != I(0)) r = op(r, a);
n = n / I(2); // reorder
a = op(a, a); // independent statements
}
}

Loop invariant: ran

Stepanov, McJones Elements of Programming March 14, 2008 153 / 880


Rotating the loop: duplicating exit

template <typename I, typename Op>


requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_nonnegative_4(Domain(Op) r, Domain(Op) a, I n, Op op)
{
// Precondition: n > I(0)
while (true) {
if (n == I(0))
return r;
if (n % I(2) != I(0)) r = op(r, a);
n = n / I(2);
if (n == I(0))
return r; // early exit
a = op(a, a);
}
}

Loop invariant: ran

Stepanov, McJones Elements of Programming March 14, 2008 154 / 880


Rotating the loop: hoisting exit

template <typename I, typename Op>


requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_nonnegative_5(Domain(Op) r, Domain(Op) a, I n, Op op)
{
// Precondition: n > I(0)
if (n == I(0))
return r; // moved first test out of loop
while (true) {
if (n % I(2) != I(0)) r = op(r, a);
n = n / I(2);
if (n == I(0))
return r;
a = op(a, a);
}
}

Loop invariant: ran

Stepanov, McJones Elements of Programming March 14, 2008 155 / 880


Dependency between conditions

n will reach 0 only from 1


1 is odd
The second condition will only be true when the first condition is
true:
if (n % I(2) != I(0)) r = op(r, a);
n = n / I(2);
if (n == I(0))
return r;

Stepanov, McJones Elements of Programming March 14, 2008 156 / 880


Utilizing dependency

template <typename I, typename Op>


requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_nonnegative_6(Domain(Op) r, Domain(Op) a, I n, Op op)
{
// Precondition: n > I(0)
if (n == I(0))
return r;
while (true) {
bool odd = n % I(2) != I(0);
n = n / I(2);
if (odd) {
r = op(r, a);
if (n == I(0))
return r;
}
a = op(a, a);
}
}

Stepanov, McJones Elements of Programming March 14, 2008 157 / 880


Specializing for positive n

template <typename I, typename Op>


requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_positive_0(Domain(Op) r, Domain(Op) a, I n, Op op)
{
// Precondition: n > I(0)
while (true) {
bool odd = n % I(2) != I(0);
n = n / I(2);
if (odd) {
r = op(r, a);
if (n == I(0))
return r;
}
a = op(a, a);
}
}

Stepanov, McJones Elements of Programming March 14, 2008 158 / 880


(Nearly final) power_accumulate_nonnegative

template <typename I, typename Op>


requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_nonnegative_7(Domain(Op) r, Domain(Op) a, I n, Op op)
{
// Precondition: n > I(0)
if (n == I(0))
return r;
else
return power_accumulate_positive_0(r, a, n, op);
}

Stepanov, McJones Elements of Programming March 14, 2008 159 / 880


Eliminating accumulation variable

template <typename I, typename Op>


requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_positive_2(Domain(Op) a, I n, Op op)
{
// Precondition: n > I(0)
n = n - I(1);
return power_accumulate_nonnegative_7(a, a, n, op);
}

Stepanov, McJones Elements of Programming March 14, 2008 160 / 880


Observation

When n is 16, we do 7 operations where only 4 are needed


When n is odd, this code is fine

Stepanov, McJones Elements of Programming March 14, 2008 161 / 880


Factoring out powers of 2

template <typename I, typename Op>


requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_positive_3(Domain(Op) a, I n, Op op)
{
// Precondition: n > I(0)
while (n % I(2) == I(0)) {
a = op(a, a);
n = n / I(2);
}
n = n - I(1);
if (n == I(0))
return a;
else
return power_accumulate_positive_0(a, a, n, op);
}

Stepanov, McJones Elements of Programming March 14, 2008 162 / 880


Observation

power_accumulate_positive_0 checks if n is odd on the first pass


We know n is even at that point
One extra operation!

Stepanov, McJones Elements of Programming March 14, 2008 163 / 880


Unrolling one iteration of
power_accumulate_positive_0

template <typename I, typename Op>


requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_positive_4(Domain(Op) a, I n, Op op)
{
// Precondition: n > I(0)
while (n % I(2) == I(0)) {
a = op(a, a);
n = n / I(2);
}
n = n / I(2);
if (n == I(0))
return a;
else
return power_accumulate_positive_0(a, op(a, a), n, op);
}

Stepanov, McJones Elements of Programming March 14, 2008 164 / 880


Operations on exponent

In the final version we used these operations:


n - I(1)
n = n / T(2);
n % T(2) == T(0)
n == T(0)
General / and % are very expensive
For all integral types, if we know n is non-negative, we can use
shifts and masks

Stepanov, McJones Elements of Programming March 14, 2008 165 / 880


Special cases of procedures

It is frequently useful to identify commonly-occuring expressions


involving procedures and constants of a type by defining
special-case procedures
Often these special cases can be implemented more efficiently
than the general case
For built-in types there may exist machine instructions for the
special cases (e.g., shifting instead of multiplication by a power of
two)
For complex user-defined types there are often even more
significant opportunities for optimizing special cases
Division of two arbitrary polynomials is more difficult than
division of a polynomial by x
Division of two Gaussian integers (numbers
√ of the form a + bi
where a and b are integers and i = −1) is more difficult than
division of a Gaussian integer by 1 + i

Stepanov, McJones Elements of Programming March 14, 2008 166 / 880


Special cases of Integer procedures
Integer(I) ∧ i ∈ I ⇒
successor(i) ≡ i + 1
predecessor(i) ≡ i − 1
half_nonnegative(i) ≡ bi/2c
binary_scale_down_nonnegative(i, k) ≡ i/2k
binary_scale_up_nonnegative(i, k) ≡ 2k i
is_positive(i) ≡ i > 0
is_negative(i) ≡ i < 0
is_zero(i) ≡ i = 0
is_even(i) ≡ (i mod 2) = 0
is_odd(i) ≡ (i mod 2) , 0

See default C++ implementations in Appendix 2


Stepanov, McJones Elements of Programming March 14, 2008 167 / 880
power_accumulate_positive

template <typename I, typename Op>


requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_positive(Domain(Op) r, Domain(Op) a, I n, Op op)
{
// Precondition: is_positive(n)
while (true) {
bool odd = is_odd(n);
halve_nonnegative(n);
if (odd) {
r = op(r, a);
if (is_zero(n))
return r;
}
a = op(a, a);
}
}

Stepanov, McJones Elements of Programming March 14, 2008 168 / 880


power_accumulate_nonnegative

template <typename I, typename Op>


requires(Integer(I) && MonoidOperation(Op))
Domain(Op) power_accumulate_nonnegative(Domain(Op) r, Domain(Op) a, I n, Op op)
{
// Precondition: ¬is_negative(n)
if (is_zero(n))
return r;
else
return power_accumulate_positive(r, a, n, op);
}

Stepanov, McJones Elements of Programming March 14, 2008 169 / 880


power_positive

template <typename I, typename Op>


requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_positive(Domain(Op) a, I n, Op op)
{
// Precondition: is_positive(n)
while (is_even(n)) {
a = op(a, a);
halve_nonnegative(n);
}
halve_nonnegative(n);
if (is_zero(n))
return a;
else
return power_accumulate_positive(a, op(a, a), n, op);
}

Stepanov, McJones Elements of Programming March 14, 2008 170 / 880


power_nonnegative

template <typename I, typename Op>


requires(Integer(I) && MonoidOperation(Op))
Domain(Op) power_nonnegative(Domain(Op) a, I n, Op op)
{
// Precondition: ¬is_negative(n)
if (is_zero(n))
return identity_element(op);
else
return power_positive(a, n, op);
}

Stepanov, McJones Elements of Programming March 14, 2008 171 / 880


power

template <typename I, typename Op>


requires(Integer(I) && GroupOperation(Op))
Domain(Op) power(Domain(Op) a, I n, Op op)
{
if (is_negative(n)) {
a = inverse_operation(op)(a);
n = -n;
}
return power_nonnegative(a, n, op);
}

Stepanov, McJones Elements of Programming March 14, 2008 172 / 880


The number of operations performed by power

Let u = number of significant bits in n > 0


Let v = number of one-bits in n
Let p = number of operations performed
Then p = (u − 1) + (v − 1) = u + v − 2 6 2blog2 nc

Exercise
Prove that power achieves this complexity

Stepanov, McJones Elements of Programming March 14, 2008 173 / 880


The operation count of power is not optimal

For n = 15, u = 4 and v = 4, so u + v − 2 = 6


But 15 = 3 × 5, so a15 = (a3 )5
a3 takes 2 operations
a5 takes 3 operations
(a3 )5 takes 5 total
There are faster ways for 23, 27, 39, 43, . . .
See Project 3

Stepanov, McJones Elements of Programming March 14, 2008 174 / 880


Choosing algorithms based on complexity

Complexity is an important part of specifications


For example, a stack is not a stack if the cost of push grows linearly
with the size of the stack
When only one algorithm for solving a problem is available, the
main issue is whether the algorithm is feasible for a particular
situation (set of arguments determining problem size, functional
parameters, etc.)
When there are more than one algorithm for solving a problem, we
need to analyze which algorithm to use for a particular situation
The analysis depends on the complexity properties of the
alternative algorithms combined with the complexity properties
of the types to which these algorithms are applied

Stepanov, McJones Elements of Programming March 14, 2008 175 / 880


Specifying complexity

Knuth pioneered the analysis of algorithms based on counting the


machine instructions performed by a MIX program
Since we write generalized procedures defined in terms of type
requirements and sometimes taking function objects as
arguments, we need to count applications of the function objects
and the operations on the types

Stepanov, McJones Elements of Programming March 14, 2008 176 / 880


Complexity of slow_power versus power

The total cost is the sum of the cost of the operations on the
exponent, the cost of the semigroup operation, and the cost of the
control structure
For all the models of Integer we are familiar with, the logarithmic
number of exponent operations performed by power will be faster
than the linear number of operations performed by slow_power,
except for very small n
For most models of the semigroup operation, power will be faster,
but there are exceptions

Stepanov, McJones Elements of Programming March 14, 2008 177 / 880


When is power faster than slow_power?

Suppose cost(op, xi , xj ) = k(x) · (i · j)α


If α = 0, we have a fixed-cost operation and power is always
much faster than slow_power
Examples: fixed-precision multiplication; matrix multiplication
with fixed-precision elements
If α 6 1, power is always faster15
Examples: multiplication of univariate polynomials with
fixed-precision coefficients; multiplication of bignums
If α > 2, slow_power is always faster16
Examples: multiplication of bivariate polynomials with
fixed-precision coefficients; multiplication of univariate
polynomials with bignum coefficients
15
R.L. Graham, A.C.-C. Yao, and F.-F. Yao. Addition chains with multiplicative
costs. Discrete Mathematics, Volume 23, Number 2, 1978, pages 115-119.
16
D.P. McCarthy. Effect of Improved Multiplication Efficiency on Exponentation
Algorithms Derived from Addition Chains. Mathematics of Computation, Volume 46,
Number 174, April 1986, pages 603-608.
Stepanov, McJones Elements of Programming March 14, 2008 178 / 880
Applications

The main application of fast exponentiation is cryptography


Donald Knuth.
Secret factors.
The Art of Computer Programming, Vol. 2, pages 403-407.
RSA.
Donald Knuth.
Improved primality tests.
The Art of Computer Programming, Vol. 2, pages 394-396.
Primality testing.
Manindra Agrawal, Neeraj Kayal, and Nitin Saxena.
PRIMES is in P.
Annals of Mathematics, 160 (2004), pages 781-793.
Primality testing.

Stepanov, McJones Elements of Programming March 14, 2008 179 / 880


Reflections on deriving useful interfaces

power_accumulate_nonnegative is useful whenever one wants


to compute axn
Even power_accumulate_positive is useful in a situation when it
is known that n > 0

Reflection
While developing a component, it is often possible to discover
additional useful interfaces

Stepanov, McJones Elements of Programming March 14, 2008 180 / 880


Reflection on code transformations

Reflection
Compilers can perform code transformations when the semantics
of the operations are known
Currently this is only for built-in types
Someday we will be able to tell the compiler the semantics of our
operations

Stepanov, McJones Elements of Programming March 14, 2008 181 / 880


Fibonacci numbers

Definition

f0 = 0
f1 = 1
fn+2 = fn+1 + fn

Example
0, 1, 1, 2, 3, 5, 8, 13, . . .

Stepanov, McJones Elements of Programming March 14, 2008 182 / 880


Recursive way to calculate fn

template <typename I>


requires(Integer(I))
I fibonacci_recursive(I n)
{
if (n == I(0))
return I(0);
else if (n == I(1))
return I(1);
else
return fibonacci_recursive(n - I(1))
+ fibonacci_recursive(n - I(2));
}

Stepanov, McJones Elements of Programming March 14, 2008 183 / 880


Iterative way to calculate fn

template <typename I>


requires(Integer(I))
I fibonacci_iterative(I n)
{
if (n == I(0))
return I(0);
I fib_i = I(0);
I fib_j = I(1);
while (n != I(1)) {
I next = fib_i + fib_j;
fib_i = fib_j;
fib_j = next;
n = n - I(1);
}
return fib_j;
}

Stepanov, McJones Elements of Programming March 14, 2008 184 / 880


Fibonacci matrices

 
1 1
F1 =
1 0
 
fn+1 fn
Fn =
fn fn−1
   
fn+1 + fn fn + fn−1 fn+2 fn+1
F1 Fn = = = Fn+1
fn+1 fn fn+1 fn
Fn = F1 F1 . . . F1 = Fn
| {z } 1
n
m n
Fm Fn = F1 F1 = Fm+n
1 = Fm+n

Stepanov, McJones Elements of Programming March 14, 2008 185 / 880


Fibonacci matrices

 
fm+1 fm
Fm =
fm fm−1
 
fn+1 fn
Fn =
fn fn−1
 
fm+1 fn+1 + fm fn fm+1 fn + fm fn−1
Fm Fn =
fm fn+1 + fm−1 fn fm fn + fm−1 fn−1

Stepanov, McJones Elements of Programming March 14, 2008 186 / 880


Observations

We can represent the matrix with a pair corresponding to the


bottom row
pair::first is fn and pair::second is fn−1
The identity fn+1 = fn−1 + fn allows us to compute the missing
term from the top row of Fn needed to compute the bottom row of
Fm Fn

Stepanov, McJones Elements of Programming March 14, 2008 187 / 880


fibonacci_multiplies

template <typename I>


requires(Integer(I))
struct fibonacci_multiplies
{
typedef pair<I, I> D;
D operator()(const D& x, const D& y) const
{
return D(
x.first * (y.first + y.second) + x.second * y.first,
x.first * y.first + x.second * y.second);
}
};

Stepanov, McJones Elements of Programming March 14, 2008 188 / 880


fibonacci

template <typename I>


requires(Integer(I))
I fibonacci(I n)
{
// Precondition: n > I(0)
if (n == I(0))
return I(0);
fibonacci_multiplies<I> op;
return power_positive(pair<I, I>(I(1), I(0)), n, op).first;
}

Stepanov, McJones Elements of Programming March 14, 2008 189 / 880


Overloading
Definition
Overloading an operator symbol or function name means using it for
operations or functions on several types

Reflection
Progress in mathematics often involves extending an operator to a new
domain

Example
+ on
natural numbers
integers
rationals
polynomials
matrices
Stepanov, McJones Elements of Programming March 14, 2008 190 / 880
Overloading should preserve semantics

In all of these examples


+ is associative
+ is commutative
+ obeys the cancellation law
a+c=b+c⇒a=b
× distributes over +

Stepanov, McJones Elements of Programming March 14, 2008 191 / 880


Type parameterization results in overloading

When we define template functions such as collision_point and


power, we are overloading the function name
If the actual type parameters satisfy the requirements the
semantics is preserved since the same algorithm is used

Stepanov, McJones Elements of Programming March 14, 2008 192 / 880


Two ways operations are provided to power

The group operation is passed as a parameter


This allows power to be used with different operations on the same
type, and the operation can have a state such as a modulus for use
in encryption
The integer operations are provided through overloading of -, /,
%, and ==
Using explicit operators is convenient whenever there are several
related operations on the type and no other correct interpretation of
the operations on the type

Stepanov, McJones Elements of Programming March 14, 2008 193 / 880


Concept CommutativeOperation

Definition
CommutativeOperation(Op) ⇒
BinaryOperation(Op)
For all op ∈ Op and for all a, b ∈ Domain(Op),

op(a, b) = op(b, a)

Stepanov, McJones Elements of Programming March 14, 2008 194 / 880


Concept CancellableOperation

Definition
CancellableOperation(Op) ⇒
BinaryOperation(Op)
For all op ∈ Op and for all a, b, c ∈ Domain(Op),

op(a, c) = op(b, c) ⇒ a = b
op(c, a) = op(c, b) ⇒ a = b

Stepanov, McJones Elements of Programming March 14, 2008 195 / 880


Concept AdditiveSemigroup

Definition
AdditiveSemigroup(T ) ⇒
The binary operation infix + is defined on T
SemigroupOperation(+)
CommutativeOperation(+) a
a
It would have been nice to require the cancellation law, but digital logic designers
use + for an operation that is not cancellable

Stepanov, McJones Elements of Programming March 14, 2008 196 / 880


Concept MultiplicativeSemigroup

Definition
MultiplicativeSemigroup(T ) ⇒
The binary operation infix × is defined on T
SemigroupOperation(×)

Stepanov, McJones Elements of Programming March 14, 2008 197 / 880


Concept AdditiveMonoid

Definition
AdditiveMonoid(T ) ⇒
AdditiveSemigroup(T )
MonoidOperation(+)
The constant T (0) is defined
identity_element(+) = T (0)

Stepanov, McJones Elements of Programming March 14, 2008 198 / 880


Concept MultiplicativeMonoid

Definition
MultiplicativeMonoid(T ) ⇒
MultiplicativeSemigroup(T )
MonoidOperation(×)
The constant T (1) is defined
identity_element(×) = T (1)

Stepanov, McJones Elements of Programming March 14, 2008 199 / 880


Concept AdditiveGroup

Definition
AdditiveGroup(T ) ⇒
AdditiveMonoid(T )
GroupOperation(+)
The unary operation negation (prefix −) is defined on T
inverse_operation(+) = negation

Definition
For an additive group, subtraction is defined as a − b = a + (−b)

Stepanov, McJones Elements of Programming March 14, 2008 200 / 880


Concept MultiplicativeGroup

Definition
MultiplicativeGroup(T ) ⇒
MultiplicativeMonoid(T )
GroupOperation(×)
The unary operation reciprocal is defined on T
inverse_operation(×) = reciprocal

Definition
For a multiplicative group, division is defined as
a/b = a × reciprocal(b)

Stepanov, McJones Elements of Programming March 14, 2008 201 / 880


Power for multiplicative types
template <typename G, typename I>
requires(MultiplicativeSemigroup(G) && Integer(I))
G power_positive(G a, I n)
{
return power_positive(a, n, multiplies<G>());
}

template <typename G, typename I>


requires(MultiplicativeMonoid(G) && Integer(I))
G power_nonnegative(G a, I n)
{
return power_nonnegative(a, n, multiplies<G>());
}

template <typename G, typename I>


requires(MultiplicativeGroup(G) && Integer(I))
G power(G a, I n)
{
return power(a, n, multiplies<G>());
}

See Appendix 2 for the enabling C++ definitions


Stepanov, McJones Elements of Programming March 14, 2008 202 / 880
Conclusions

Algorithms are generic by nature


They can be used with different models satisfying the same
requirements
Algorithms are affiliated with algebraic structures: semigroup,
monoid, . . .
Stepwise refinement leads from mathematical definitions to
efficient code
Special-case procedures can make code more efficient and even
more generic
Mathematics leads to surprising algorithms: fibonacci

Stepanov, McJones Elements of Programming March 14, 2008 203 / 880


Reference

D. Kapur, D.R. Musser, and A.A. Stepanov.


Operators and Algebraic Structures.
Proceedings of the 1981 conference on Functional programming
languages and computer architecture, pages 59-63.
One of the first presentations of algorithms on algebraic structures.

Stepanov, McJones Elements of Programming March 14, 2008 204 / 880


Project 1

Project
Our definition of the Fibonacci sequence starts from zero and goes up;
extend the definition and our code to work for negative indices a
a
Suggested by Oleg Zabluda

Stepanov, McJones Elements of Programming March 14, 2008 205 / 880


Project 2

Charles M. Fiduccia.
An efficient formula for linear recurrences.
SIAM Journal of Computing, Volume 14, Number 1, February 1985,
page 106-112.
Generalizes the fibonacci algorithm to arbitrary linear recurrences.

Project
Create a library implementing Fiduccia’s algorithm

Stepanov, McJones Elements of Programming March 14, 2008 206 / 880


Project 3

Donald E. Knuth.
Addition chains.
The Art of Computer Programming, Volume 2: Seminumerical
Algorithms, 3rd edition, Addison-Wesley, San Francisco, 1998,
pages 465-481.
Describes a way to do minimal-operation exponentiation using
addition chains.
Project
Implement a useful library doing power optimally for exponents
known at compile time

Stepanov, McJones Elements of Programming March 14, 2008 207 / 880


Project 4

Project
Floating-point multiplication and addition are not associative, so
may give different results when they are used as the operation for
slow_power and power; establish whether slow_power or power
gives a more accurate result for:
1 Raising a floating-point number to an integral power
2 Multiplying a floating-point number by an integral factor

Stepanov, McJones Elements of Programming March 14, 2008 208 / 880


Contents I

1 Introduction

2 Foundations

3 Transformations and their orbits

4 Algorithms on algebraic structures

5 Orderings
Motivation
Relation; transitivity; ordering
Strict versus reflexive
Symmetric versus asymmetric

Stepanov, McJones Elements of Programming March 14, 2008 209 / 880


Contents II
Equivalence; equality
Symmetric complement; total ordering; weak ordering; partial
ordering
Multiple orderings; natural total ordering; default ordering;
overloading
Intervals
Order selection algorithms
Conclusions
Reference
Project

6 Combining concepts

7 Refining concepts of iterators

Stepanov, McJones Elements of Programming March 14, 2008 210 / 880


Contents III
8 Permutations and rearrangements

9 Rotations

10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects

13 Iterative algorithms for divide-and-conquer

14 Mathematical notation

Stepanov, McJones Elements of Programming March 14, 2008 211 / 880


Contents IV
15 C++ machinery

16 Acknowledgments

17 Index

Stepanov, McJones Elements of Programming March 14, 2008 212 / 880


The importance of linear ordering

Equality allows comparing individual elements easily


Searching a collection using only equality requires linear time
Intersecting two collections using only equality requires time
proportional to the product of the sizes of the collections
A linear ordering allows organizing elements so that their
collections can be
matched (intersection, subset, union) in linear time
searched in logarithmic time

Stepanov, McJones Elements of Programming March 14, 2008 213 / 880


Concepts Predicate and Relation

Definition
Predicate(Op) ⇒
RegularFunction(Op)
Codomain(Op) = bool

Definition
Relation(Op) ⇒
Predicate(Op)
HomogeneousFunction(Op)
Arity(Op) = 2
(Other books use this term for more general non-unary predicates)

Stepanov, McJones Elements of Programming March 14, 2008 214 / 880


Concept Ordering

Definition
Ordering(Op) ⇒
Relation(Op)
For all op ∈ Op and for all a, b, c ∈ Domain(Op)

op(a, b) ∧ op(b, c) ⇒ op(a, c)

This property is called transitivity

Stepanov, McJones Elements of Programming March 14, 2008 215 / 880


Examples of ordering

Examples
Equality
Equality of the first character
Reachability in an orbit
Divisibility

Stepanov, McJones Elements of Programming March 14, 2008 216 / 880


Strict versus reflexive

Definition
An ordering r is strict if ¬r(a, a)
An ordering r is reflexive if r(a, a)
An ordering r is weakly reflexive if r(a, b) ⇒ r(a, a) ∧ r(b, b)

Examples
Reflexive: all examples on the previous slide
Strict: proper factor

Stepanov, McJones Elements of Programming March 14, 2008 217 / 880


Symmetric versus asymmetric

Definition
A relation r is symmetric if r(a, b) ⇒ r(b, a)
A relation r is asymmetric if r(a, b) ⇒ ¬r(b, a)

Lemma
A strict ordering is asymmetric

Lemma
A symmetric ordering is weakly reflexive

Examples
Symmetric: sibling
Asymmetric: parent

Stepanov, McJones Elements of Programming March 14, 2008 218 / 880


Concept EquivalenceRelation

Definition
EquivalenceRelation(R) ⇒
Ordering(R)
Reflexive(R)
Symmetric(R)

Examples
Equality
Geometric congruence
a ≡ b (mod n)

Stepanov, McJones Elements of Programming March 14, 2008 219 / 880


Equivalence and equality

Lemma
If r is an equivalence relation, a = b ⇒ r(a, b)

Stepanov, McJones Elements of Programming March 14, 2008 220 / 880


Key function

Definition
If T is a regular type and r ∈ R is an equivalence relation on T , a
function f ∈ F : T → T 0 is a key function for r if
r(x, x 0 ) ⇔ f(x) = f(x 0 )
Given the choice of a particular key function f, we say f(x) is a
canonical representation of the elements equivalent to x

One way to implement an equivalence relation is to define a key


function for it and then apply equality to the results

Stepanov, McJones Elements of Programming March 14, 2008 221 / 880


symmetric_complement

template <typename R>


requires(Relation(R))
struct symmetric_complement
{
R r;
symmetric_complement(R r) : r(r) {}

bool operator()(const Domain(R)& a, const Domain(R)& b)


{
return !r(a, b) && !r(b, a);
}
};

Stepanov, McJones Elements of Programming March 14, 2008 222 / 880


Concept StrictTotalOrdering

Definition
StrictTotalOrdering(R) ⇒
Ordering(R)
For all r ∈ R and for all a, b ∈ Domain(R)

symmetric_complementr (a, b) ⇔ a = b

Stepanov, McJones Elements of Programming March 14, 2008 223 / 880


Properties of total ordering

Lemma
A total ordering is strict

Lemma
A total ordering is asymmetric

Lemma
The trichotomy law holds: r(a, b) ∨ r(b, a) ∨ (a = b)

Lemma
Exactly one clause of the trichotomy law holds

Stepanov, McJones Elements of Programming March 14, 2008 224 / 880


Extending strict total orderings to a direct product

Strict total orderings r0 ∈ R0 , . . . , rn ∈ Rn can be extended to a


strict total ordering on the direct product
Domain(R0 ) × . . . × Domain(Rn ) via the lexicographic ordering r̄
defined by:

r̄((x0 , . . . , xn ), (y0 , . . . , yn )) ≡
(∃k ∈ [0, n])(x0 = y0 ∧ . . . ∧ xk−1 = yk−1 ∧ rk (xk , yk ))

Lemma
r̄ is a strict total ordering

Stepanov, McJones Elements of Programming March 14, 2008 225 / 880


Concept StrictWeakOrdering

Definition
StrictWeakOrdering(R) ⇒
Ordering(R)
EquivalenceRelation(symmetric_complementR )

Stepanov, McJones Elements of Programming March 14, 2008 226 / 880


Weak ordering is a weakening of total ordering

Lemma
StrictTotalOrdering(R) ⇒ StrictWeakOrdering(R)

Stepanov, McJones Elements of Programming March 14, 2008 227 / 880


Properties of weak ordering

Lemma
A weak ordering is strict

Lemma
A weak ordering is asymmetric

Lemma
The trichotomy law holds:
r(a, b) ∨ r(b, a) ∨ symmetric_complementr (a, b)

Lemma
Exactly one clause of the trichotomy law holds

Stepanov, McJones Elements of Programming March 14, 2008 228 / 880


Extending strict weak orderings to a direct product

Strict weak orderings can be extended to a direct product via


lexicographic ordering, just as with strict total orderings

Stepanov, McJones Elements of Programming March 14, 2008 229 / 880


Defining a strict weak ordering using a key function

A key function f on a set T and a strict total ordering r on the


codomain of f define a weak ordering r̃(x, y) ⇔ r(f(x), f(y))

Example
Weak ordering on ith component of a direct product

template <typename F, typename R>


requires(Function(F) && Arity(F) == 1 &&
StrictTotalOrdering(R) && Codomain(F) == Domain(R))
struct key_ordering models(StrictWeakOrdering)
{
F f;
R r;
key_ordering(F f, R r) : f(f), r(r) { }
bool operator()(const Domain(F)& x, const Domain(F)& y) const
{
return r(f(x), f(y));
}
};

Stepanov, McJones Elements of Programming March 14, 2008 230 / 880


Not every strict ordering is weak

Suppose a relation ∝ on {a, b, c, d, e} is a ∝ b ∝ c ∝ d, a ∝ e ∝ d:


b c
a d
e

The symmetric complement  of ∝ is b  e, e  b, e  c, c  e


If  were an equivalence relation it would have to include b  c
and c  b by transitivity

Stepanov, McJones Elements of Programming March 14, 2008 231 / 880


Mathematical conventions for usage of weak and semi

Weak refers to weakening (which includes dropping) an axiom


A strict weak ordering replaces equality with equivalence
Semi refers to dropping an operation
A semigroup lacks the inverse operation

Stepanov, McJones Elements of Programming March 14, 2008 232 / 880


Partial ordering

There are algorithms that deal with orderings that are not weak:
algorithms on partially-ordered sets
The most important is topological sort:
Donald Knuth.
Topological sorting.
The Art of Computer Programming, Volume 1, Addison-Wesley,
1997, pages 261-268.
Operations as simple as max and min do not make sense without
a weak order

Stepanov, McJones Elements of Programming March 14, 2008 233 / 880


Multiple orderings on a type

There is one equality relation on a type T


There can be many equivalence relations
Similarly there can be many orderings

Stepanov, McJones Elements of Programming March 14, 2008 234 / 880


Natural total ordering

Definition
The total ordering on a type that is consistent with algebraic operations
on it is called the natural total ordering

Example
Examples of consistent axioms for natural ordering:
Incrementable a < successor(a)
Incrementable a < b ⇒ successor(a) < successor(b)
Ordered group a < b ⇒ a + c < b + c
Ordered ring (a < b) ∧ (0 < c) ⇒ ca < cb

Stepanov, McJones Elements of Programming March 14, 2008 235 / 880


Default ordering

Sometimes a type does not have a natural total ordering


Complex numbers
Iterators on linked lists
We still want a total ordering to enable logarithmic searching, so
we always define the default ordering for regular types
Lexicographic ordering for complex numbers
Address-based ordering for iterators on linked lists
When the natural order exists, it coincides with the default
ordering

Definition
less<T> defines the default ordering for T

Stepanov, McJones Elements of Programming March 14, 2008 236 / 880


Concept StrictTotallyOrdered

Definition
StrictTotallyOrdered(T ) ⇒
Regular(T )
The relation < is defined on T
StrictTotalOrdering(<)

< is reserved for the natural total ordering

Stepanov, McJones Elements of Programming March 14, 2008 237 / 880


Clusters of derived procedures

There are procedures that naturally come in clusters


If some of the cluster are defined, the definitions of the others
naturally follow

Example
In an additive group, negation and subtraction constitute such a cluster

Stepanov, McJones Elements of Programming March 14, 2008 238 / 880


Derived relations

Given any relation, say, r(a, b), there are


complement ¬r(a, b)
converse r(b, a)
complement of converse ¬r(b, a)
Given a symmetric relation, say r(a, b), since the converse is equal
to the original relation, the only derivable relation is the
complement, ¬r(a, b)

Stepanov, McJones Elements of Programming March 14, 2008 239 / 880


Properties of derived relations

Lemma
Given an ordering, its complement, its converse, and the
complement of its converse are orderings
If an ordering is strict, its converse is also strict and its
complement and complement of converse are reflexive
If an ordering is reflexive, its converse is also reflexive and its
complement and complement of converse are strict
There exist C++ template function object classes complement,
converse, and complement_of_converse which, given a relation,
construct the corresponding derived relations

Stepanov, McJones Elements of Programming March 14, 2008 240 / 880


Equality and inequality

For every regular type, we have = and ,


We assure consistency by always defining equality and relying on
the following template for inequality

template <typename T>


requires(Regular(T))
bool operator!=(const T& a, const T& b)
{
return !(a == b);
}

Stepanov, McJones Elements of Programming March 14, 2008 241 / 880


<, >, 6, and >

For every totally ordered type, we have <, >, 6, and >
We assure consistency by always defining < and relying on the
following templates for the others

template <typename T>


requires(StrictTotallyOrdered(T))
bool operator>(const T& a, const T& b) { return b < a; }

template <typename T>


requires(StrictTotallyOrdered(T))
bool operator<=(const T& a, const T& b) { return !(b < a); }

template <typename T>


requires(StrictTotallyOrdered(T))
bool operator>=(const T& a, const T& b) { return !(a < b); }

Stepanov, McJones Elements of Programming March 14, 2008 242 / 880


Intervals

Definition
A closed interval [a, b] is the set of all elements x such that a 6 x 6 b
An open interval (a, b) is the set of all elements x such that
a<x<b
A half-open on right interval [a, b) is the set of all elements x such
that a 6 x < b
A half-open on left interval (a, b] is the set of all elements x such that
a<x6b
A half-open interval is our short-hand for half-open on right
These definitions generalize to weak orderings

Stepanov, McJones Elements of Programming March 14, 2008 243 / 880


min and max

template <typename R>


requires(StrictWeakOrdering(R))
Domain(R)& min(Domain(R)& a, Domain(R)& b, R r)
{
if (r(b, a)) return b;
else return a;
}

template <typename R>


requires(StrictWeakOrdering(R))
Domain(R)& max(Domain(R)& a, Domain(R)& b, R r)
{
if (r(b, a)) return a;
else return b;
}

Stepanov, McJones Elements of Programming March 14, 2008 244 / 880


Other min and max

template <typename R>


requires(StrictWeakOrdering(R))
Domain(R)& other_min(Domain(R)& a, Domain(R)& b, R r)
{
if (r(a, b)) return a;
else return b;
}

template <typename R>


requires(StrictWeakOrdering(R))
Domain(R)& other_max(Domain(R)& a, Domain(R)& b, R r)
{
if (r(a, b)) return b;
else return a;
}

We will see shortly why min and max are preferable to


other_min and other_max 17

17
The C++ standard library uses min and other_max
Stepanov, McJones Elements of Programming March 14, 2008 245 / 880
A convention for order selection functions

For the order selection procedures in this section, there are four
useful versions
Since the parameters are being passed for selecting, versions taking
both references and const references are needed
For convenience, we want versions for totally ordered types (with
<) as well as for an explicitly-supplied ordering
From now on, we show only the non-const reference version with
an explict ordering; see Appendix 2 for the other versions
The code below shows how to obtain a version for a totally
ordered type given a version for an explicitly supplied ordering

Stepanov, McJones Elements of Programming March 14, 2008 246 / 880


Stability

Definition
An algorithm is stable if it respects the original order of equivalent
elements

Stability is not relevant with a total ordering since equivalent


elements are equal and, therefore, indistinguishable

Example
Stable sort
Multiple passes compose naturally
Sorting by first name and then by last name results in expected order
Stable partition
Even/odd partition of {1, 2, 3, 4, 5} results in {1, 3, 5, 2, 4}, not
{1, 5, 3, 4, 2} as produced by a fast partitioning algorithm
Stability sometimes increases the complexity
Stepanov, McJones Elements of Programming March 14, 2008 247 / 880
Increasing order

Definition

increasing
A sequence . . . , x, . . . , x 0 , . . . is in order with
strictly increasing

¬r(x 0 , x) a
respect to a strict weak ordering r if
r(x, x 0 )
a
While some would use the terms non-decreasing and increasing, we follow Nicolas
Bourbaki, Theory of Sets, III.1.5, page 138.

In this book we always assume increasing order for sorted


sequences

Stepanov, McJones Elements of Programming March 14, 2008 248 / 880


sort_2

template <typename R>


requires(StrictWeakOrdering(R))
void sort_2(Domain(R)& a, Domain(R)& b, R r)
{
if (r(b, a)) swap(a, b);
}

sort_2 is stable
It does the minimal amount of work since it does not swap
equivalent elements

Stepanov, McJones Elements of Programming March 14, 2008 249 / 880


Stability of min and max

Natural postcondition for sort_2:


a0 = a; b0 = b;
sort_2(a, b, r);
assert(a == min(a0, b0, r) && b == max(a0, b0, r));

min and max satisfy it, while other_min and other_max do not

Stepanov, McJones Elements of Programming March 14, 2008 250 / 880


Properties of min and max for StrictTotalOrdering r

Lemma
associativity minr (minr (a, b), c) = minr (a, minr (b, c))
associativity maxr (maxr (a, b), c) = maxr (a, maxr (b, c))
commutativity minr (a, b) = minr (b, a)
commutativity maxr (a, b) = maxr (b, a)
absorption minr (a, maxr (a, b)) = a
absorption maxr (minr (a, b), b) = b
idempotency minr (a, a) = a
idempotency maxr (a, a) = a

Stepanov, McJones Elements of Programming March 14, 2008 251 / 880


Properties of min and max for StrictWeakOrdering r

Lemma
associativity minr (minr (a, b), c) = minr (a, minr (b, c))
associativity maxr (maxr (a, b), c) = maxr (a, maxr (b, c))
commutativity symmetric_complementr (minr (a, b), minr (b, a))
commutativity symmetric_complementr (maxr (a, b), maxr (b, a))

Exercise
What are the absorption and idempotency laws for minr and maxr ?

Stepanov, McJones Elements of Programming March 14, 2008 252 / 880


min_3 and max_3

template <typename R>


requires(StrictWeakOrdering(R))
Domain(R)& min_3(Domain(R)& a, Domain(R)& b, Domain(R)& c, R r)
{
return min(min(a, b, r), c, r);
}

template <typename R>


requires(StrictWeakOrdering(R))
Domain(R)& max_3(Domain(R)& a, Domain(R)& b, Domain(R)& c, R r)
{
return max(max(a, b, r), c, r);
}

Stepanov, McJones Elements of Programming March 14, 2008 253 / 880


Implementing order selection

There are more complicated cases than min and max such as
median of 3 and select 2nd of 4
Writing order selections is somewhat complicated and can be
helped by decomposition into simpler subproblems

Stepanov, McJones Elements of Programming March 14, 2008 254 / 880


median_3

template <typename R>


requires(StrictWeakOrdering(R))
Domain(R)& median_3(Domain(R)& a, Domain(R)& b, Domain(R)& c, R r)
{
if (r(b, a)) return median_3_stage1(b, a, c, r);
else return median_3_stage1(a, b, c, r);
}

template <typename R>


requires(StrictWeakOrdering(R))
Domain(R)& median_3_stage1(Domain(R)& a, Domain(R)& b, Domain(R)& c, R r)
{
// Precondition: a, b are sorted
if (!r(c, b)) return b; // a, b, c are sorted
else return max(a, c, r); // b is not the median
}

Stepanov, McJones Elements of Programming March 14, 2008 255 / 880


Stability of median_3

Exercise
Define the appropriate stability property for median_3

Lemma
median_3 is stable

Stepanov, McJones Elements of Programming March 14, 2008 256 / 880


Complexity of median_3

median_3 does 3 comparisons in the worst case


The function does 2 comparisons only when c is max_3(a, b, c)
and that happens in one-third of the cases
The average number of comparison is 2 32

Stepanov, McJones Elements of Programming March 14, 2008 257 / 880


Selecting second smallest

Finding second smallest implies finding smallest


If after finding second smallest, two candidates for smallest remain,
then the second smallest is not second smallest!
Finding second smallest from n elements requires at least
n + log n − 2 comparisons
Finding second out of four requires 4 comparisons

Stepanov, McJones Elements of Programming March 14, 2008 258 / 880


select_2nd_4
template <typename R> requires(StrictWeakOrdering(R))
Domain(R)&
select_2nd_4(Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, R r) {
if (r(b, a)) return select_2nd_4_stage1(b, a, c, d, r);
else return select_2nd_4_stage1(a, b, c, d, r);
}

template <typename R> requires(StrictWeakOrdering(R))


Domain(R)&
select_2nd_4_stage1(Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, R r) {
// Precondition: a 6 b
if (r(d, c)) return select_2nd_4_stage2(a, b, d, c, r);
else return select_2nd_4_stage2(a, b, c, d, r);
}

template <typename R> requires(StrictWeakOrdering(R))


Domain(R)&
select_2nd_4_stage2(Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, R r) {
// Precondition: a 6 b ∧ c 6 d
if (r(c, a)) return min(a, d, r);
else return min(b, c, r);
}

Stepanov, McJones Elements of Programming March 14, 2008 259 / 880


Exercises

Exercise
Is select_2nd_4 stable?

Exercise
Implement select_3rd_4

Stepanov, McJones Elements of Programming March 14, 2008 260 / 880


median_5
template <typename R> requires(StrictWeakOrdering(R))
Domain(R)& median_5(
Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, Domain(R)& e, R r) {
if (r(b, a)) return median_5_stage1(b, a, c, d, e, r);
else return median_5_stage1(a, b, c, d, e, r);
}

template <typename R> requires(StrictWeakOrdering(R))


Domain(R)& median_5_stage1(
Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, Domain(R)& e, R r) {
// Precondition: a 6 b
if (r(d, c)) return median_5_stage2(a, b, d, c, e, r);
else return median_5_stage2(a, b, c, d, e, r);
}

template <typename R> requires(StrictWeakOrdering(R))


Domain(R)& median_5_stage2(
Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, Domain(R)& e, R r) {
// Precondition: a 6 b ∧ c 6 d
if (r(c, a)) return select_2nd_4_stage1(a, b, d, e, r);
else return select_2nd_4_stage1(c, d, b, e, r);
}

Stepanov, McJones Elements of Programming March 14, 2008 261 / 880


Exercises

Exercise
Show that median_5 is not stable

Exercise
Design a stable version of median_5

Exercise
Find an algorithm for median of 5 that does slightly fewer
comparisons on average

Stepanov, McJones Elements of Programming March 14, 2008 262 / 880


Conclusions

Weak orderings allow efficient algorithms on collections of


elements
The axioms of ordering provide the interface to connect specific
orderings with general purpose algorithms
Overloaded operators should preserve their semantics in
mathematics

Stepanov, McJones Elements of Programming March 14, 2008 263 / 880


Reference

Our treatment of ordering is based on:


N. Bourbaki.
Chapter 3, Section 1: Order relations. Ordered Sets.
Theory of Sets, Springer, 2004, pages 131-148.

Stepanov, McJones Elements of Programming March 14, 2008 264 / 880


Project

Using material from


Donald E. Knuth.
Section 5.3: Optimum Sorting.
The Art of Computer Programming, Volume 3: Sorting and
Searching, 2nd edition, Addison-Wesley, 1998.
create a library for generic minimum-comparison networks for
sorting, merging, and selection
For sorting and merging networks, minimize not only the number
of comparisons, but the number of data movements
Assure stability of these networks

Stepanov, McJones Elements of Programming March 14, 2008 265 / 880


Contents I

1 Introduction

2 Foundations

3 Transformations and their orbits

4 Algorithms on algebraic structures

5 Orderings

6 Combining concepts
Combining algebraic structures and ordering
Remainder

Stepanov, McJones Elements of Programming March 14, 2008 266 / 880


Contents II
Greatest common divisor
Extending the domain of greatest common divisor
Quotient
Quotient and remainder for negative quantities
Integers
Conclusions
References
Project

7 Refining concepts of iterators

8 Permutations and rearrangements

9 Rotations

Stepanov, McJones Elements of Programming March 14, 2008 267 / 880


Contents III
10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects

13 Iterative algorithms for divide-and-conquer

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

Stepanov, McJones Elements of Programming March 14, 2008 268 / 880


Contents IV
17 Index

Stepanov, McJones Elements of Programming March 14, 2008 269 / 880


Axioms combine concepts

Combining different structures on the same type


Combining different types

Stepanov, McJones Elements of Programming March 14, 2008 270 / 880


Ordered structures

Definition
OrderedAdditiveSemigroup(T ) ⇒
AdditiveSemigroup(T )
StrictTotallyOrdered(T )
For all x, y, z ∈ T ,
x<y⇒x+z<y+z

Definition
OrderedAdditiveMonoid(T ) ⇒ Monoid(T ) ∧ OrderedAdditiveSemigroup(T )

Definition
OrderedAdditiveGroup(T ) ⇒ Group(T ) ∧ OrderedAdditiveMonoid(T )

Stepanov, McJones Elements of Programming March 14, 2008 271 / 880


Absolute value

template <typename T>


requires(OrderedAdditiveGroup(T))
T abs(const T& x)
{
if (x < T(0)) return -x;
else return x;
}

The correctness of abs depends on

x < T (0) ⇒ x + (−x) < T (0) + (−x) ⇒ T (0) < −x

We use the notation |x| for the absolute value of x

Stepanov, McJones Elements of Programming March 14, 2008 272 / 880


Concept OrderedCancellableMonoid

Definition
OrderedCancellableMonoid(T ) ⇒
OrderedAdditiveMonoid(T )
The partial binary infix operator − is defined on T
For all a, b ∈ T

b 6 a ⇒ is_defined(−, a, b) ∧ (a − b) + b = a

Stepanov, McJones Elements of Programming March 14, 2008 273 / 880


Division with remainder

As repeated addition induces multiplication,


repeated subtraction induces division with remainder
Interestingly, it is easier to introduce the remainder computation,
leaving the quotient for later

Stepanov, McJones Elements of Programming March 14, 2008 274 / 880


weak_remainder

template <typename T>


requires(OrderedCancellableMonoid(T))
T weak_remainder(T a, T b)
{
// Precondition: a > T (0) ∧ b > T (0)
while (b <= a) a = a - b;
return a;
}

The concept OrderedCancellableMonoid is not strong enough to


prove termination of weak_remainder; we need another
property to ensure there are no unreachable (infinite) elements

Stepanov, McJones Elements of Programming March 14, 2008 275 / 880


Concept ArchimedeanMonoid

Definition
ArchimedeanMonoid(T) ⇒
OrderedCancellableMonoid(T )
For all a and all b > 0 ∈ T , weak_remainder(a, b) terminates

The property is called the Axiom of Archimedes

Reflection
A concept can be defined with an axiom stating that a particular
procedure terminates

Stepanov, McJones Elements of Programming March 14, 2008 276 / 880


The Axiom of Archimedes

The Axiom of Archimedes is usually written as “there exists an


integer m such that a < m · b”
Archimedes originally wrote: “... that the excess by which the
greater of (two) unequal areas exceeds the less can, by being
added to itself, be made to exceed any given finite area.” 18
Archimedes attributes this lemma to Eudoxus 19

18
T.L. Heath, translator. The Works of Archimedes, Preface to Quadrature of the
Parabola. Dover, 2002, page 234
19
Sir Thomas Heath. A History of Greek Mathematics, Volume I. Dover, 1981, page 327
Stepanov, McJones Elements of Programming March 14, 2008 277 / 880
Examples of Archimedean monoids

Line segments in Euclidean geometry


Rational numbers
Binary fractions { 2nk }
Ternary fractions { 3nk }

Stepanov, McJones Elements of Programming March 14, 2008 278 / 880


Intuition for a fast algorithm for remainder

Repeated doubling leads to fast (logarithmic complexity) power


A related algorithm is possible for remainder
As with power, the Egyptians used this algorithm to do division
with remainder

Stepanov, McJones Elements of Programming March 14, 2008 279 / 880


Deriving a fast algorithm for remainder

If c is a monoid element and m is a nonnegative integer, m · c


stands for c| + ·{z
· · + c}
m times
Let a = n · b + r, where r = weak_remainder(a, b)
Let n = 2q + p, where q = bn/2c and p = n mod 2
a = q · (2 · b) + (p · b + r) where (p · b + r) < 2 · b
remainder(a, b) = r =


 a if a < b


a − b if a − b < b
remainder(a, 2 · b))
 if remainder(a, 2 · b) < b


remainder(a, 2 · b)) − b otherwise

Stepanov, McJones Elements of Programming March 14, 2008 280 / 880


remainder_nonnegative

template <typename T>


requires(ArchimedeanMonoid(T))
T remainder_nonnegative(T a, T b)
{
// Precondition: a > T (0) ∧ b > T (0)
if (a < b) return a;
if (a - b < b) return a - b;
a = remainder_nonnegative(a, b + b);
if (a < b) return a;
else return a - b;
}

The first comparison is not needed in the recursive calls and could
be eliminated by the introduction of a helper function
a − b < b serves as a guard to ensure b + b will not overflow

Stepanov, McJones Elements of Programming March 14, 2008 281 / 880


Complexity of remainder_nonnegative

It is trivial to see that remainder_nonnegative is logarithmic

Exercise
Determine the exact counts of different operations

Stepanov, McJones Elements of Programming March 14, 2008 282 / 880


Concept WeaklyHalvableMonoid

In many situations while the monoid does not provide general


division by an integer, it has a division by 2
While general k-section of an angle by ruler and compass can not
be done, bisection is trivial
Divisibility by 2 gives an iterative version of remainder

Definition
WeaklyHalvableMonoid(T ) ⇒
ArchimedeanMonoid(T )
The partial unary operation half_nonnegative is defined on T +
where T + = {x ∈ T |x > T (0)}
For all a, b ∈ T +

a = b + b ⇒ half_nonnegative(a) = b

Stepanov, McJones Elements of Programming March 14, 2008 283 / 880


remainder_nonnegative_iterative 20

template <typename T>


requires(WeaklyHalvableMonoid(T))
T remainder_nonnegative_iterative(T a, const T& b)
{
// Precondition: a > T (0) ∧ b > T (0)
if (a < b) return a;
T c = b;
while (a - c >= c) c = c + c;
a = a - c;
while (c != b) {
c = half_nonnegative(c);
if (c <= a) a = a - c;
}
return a;
}

20
Edsger W. Dijkstra attributes this algorithm to N.G. de Bruijn on page 13 of Notes
on Structured Programming, in: O.-J. Dahl, E. W. Dijkstra and C.A.R. Hoare.
Structured Programming. Academic Press, London and New York, 1972. The algorithm
also appears in: Niklaus Wirth. Systematic Programming: An Introduction.
Prentice-Hall, 1973, program 7.22, page 38.
Stepanov, McJones Elements of Programming March 14, 2008 284 / 880
Divisibility on an Archimedean monoid T

Definition
For a > T (0) and b > T (0), b divides a ⇔ remainder(a, b) = T (0)

Stepanov, McJones Elements of Programming March 14, 2008 285 / 880


Properties of divisibility on an Archimedean monoid

Lemma
In an Archimedean monoid T with positive x, a, b
b divides a ⇒ b 6 a
b > a ∧ x divides a ∧ x divides b ⇒ x divides (b − a)
x divides a ∧ x divides b ⇒ x divides remainder(a, b)

Stepanov, McJones Elements of Programming March 14, 2008 286 / 880


Concept DiscreteArchimedeanMonoid

Definition
DiscreteArchimedeanMonoid(T) ⇒
ArchimedeanMonoid(T )
(∃u ∈ T )(∀x ∈ T )x < u ⇒ ¬(T (0) < x)

Such an element u is called a unit

Lemma
Every element of a discrete Archimedean monoid is divisible by the
unit

Stepanov, McJones Elements of Programming March 14, 2008 287 / 880


Greatest common divisor

Definition
The greatest common divisor of a and b, denoted by gcd(a, b), is a
divisor of a and b that is divisible by any other divisor of a and b

This definition works for Archimedean monoid, and, as we shall


see later, for other structures
Note the definition does not depend on ordering but is expressed
strictly in terms of divisibility

Stepanov, McJones Elements of Programming March 14, 2008 288 / 880


Properties of greatest common divisor

Lemma
In an Archimedean monoid with positive x, a, b, the following hold
x divides a ∧ x divides b ⇒ x 6 gcd(a, b)
gcd(a, b) is unique
gcd(a, a) = a

Stepanov, McJones Elements of Programming March 14, 2008 289 / 880


subtractive_gcd
template <typename T>
requires(ArchimedeanMonoid(T))
T subtractive_gcd(T a, T b)
{
// Precondition: a > T (0) ∧ b > T (0)
while (true) {
if (b < a)
a = a - b;
else if (a < b)
b = b - a;
else
return a;
}
}

This is known as Euclid’s algorithm 21


It is fairly certain that the algorithm had been known at least a
century before Euclid
21
Sir Thomas Heath, translator. Euclid’s Elements. Dover, 1956: Volume 2, Book VII,
Propositions 1 and 2, pages 296-300; Volume 3, Book X, Propositions 1-3, pages 14-22
Stepanov, McJones Elements of Programming March 14, 2008 290 / 880
Correctness of subtractive_gcd

Proof.
1 Each subtraction preserves the property of being divisible by any
divisor of the original a and b
2 If the variables become equal, they divide the original a and b
3 Therefore it returns a divisor of a and b that is divisible by any
other divisor
4 In other words, it returns the greatest common divisor


Stepanov, McJones Elements of Programming March 14, 2008 291 / 880


Termination of subtractive_gcd

Lemma
It always terminates for integers and rationals

Lemma
It does not always terminate for reals

Stepanov, McJones Elements of Programming March 14, 2008 292 / 880


Concept EuclideanMonoid

Definition
EuclideanMonoid(T ) ⇒
ArchimedeanMonoid(T )
For all a > T (0), b > T (0) ∈ T , subtractive_gcd(a, b) terminates

Lemma
Every discrete Archimedean monoid is Euclidean

Stepanov, McJones Elements of Programming March 14, 2008 293 / 880


Extending subtractive_gcd to zero

It is straightforward to extend subtractive_gcd to the case where


one of its arguments is zero

template <typename T>


requires(EuclideanMonoid(T))
T subtractive_gcd_with_zero(T a, T b)
{
// Precondition: a > T (0) ∧ b > T (0) ∧ ¬(a = T (0) ∧ b = T (0))
while (true) {
if (b == T(0)) return a;
while (b < a) a = a - b;
if (a == T(0)) return b;
while (a < b) b = b - a;
}
}

Stepanov, McJones Elements of Programming March 14, 2008 294 / 880


Speeding up subtractive gcd

Each of the inner while statements in subtractive_gcd_with_zero


is equivalent to a call of slow_remainder
By using our (logarithmic) remainder algorithm, we can speed up
the case where a and b are very different in magnitude

Stepanov, McJones Elements of Programming March 14, 2008 295 / 880


fast_subtractive_gcd

template <typename T>


requires(EuclideanMonoid(T))
T fast_subtractive_gcd(T a, T b)
{
// Precondition: a > T (0) ∧ b > T (0) ∧ ¬(a = T (0) ∧ b = T (0))
while (true) {
if (b == T(0)) return a;
a = remainder_nonnegative(a, b);
if (a == T(0)) return b;
b = remainder_nonnegative(b, a);
}
}

Stepanov, McJones Elements of Programming March 14, 2008 296 / 880


Generalizing the greatest common divisor algorithm

On a Euclidean monoid, the computation of greatest common


divisor is based on a particular remainder algorithm
If a structure has a properly-defined remainder, it does not have to
be a Euclidean monoid
Polynomials
Gaussian integers
Mathematicians refer to these other types as Euclidean domains
Our goal is to define a concept unifying Euclidean monoids and
Euclidean domains, so we can use the same gcd algorithm for both

Stepanov, McJones Elements of Programming March 14, 2008 297 / 880


Concept CommutativeSemiring

Definition
CommutativeSemiring(T ) ⇒
AdditiveMonoid(T )
MultiplicativeSemigroup(T ) ∧ CommutativeOperation(×)
For any a, T (0) × a = T (0)
The constant T (1) is defined
For any a , T (0), T (1) × a = a
For all a, b, c ∈ T , a × (b + c) = a × b + a × c

The last property is distributivity

Stepanov, McJones Elements of Programming March 14, 2008 298 / 880


Examples of CommutativeSemiring

Example
Nonnegative integers constitute a commutative semiring

Stepanov, McJones Elements of Programming March 14, 2008 299 / 880


Concept Semimodule

Definition
Semimodule(T ) ⇒
AdditiveMonoid(T )
The type function Scalar(T ) is defined
CommutativeSemiring(Scalar(T ))
The binary infix function · : Scalar(T ) × T → T is defined
For all α, β ∈ Scalar(T ) and for all x, y ∈ T ,

α · (β · x) = (α × β) · x
(α + β) · x = α · x + β · x
α · (x + y) = α · x + α · y
Scalar(T )(1) · x = x

Stepanov, McJones Elements of Programming March 14, 2008 300 / 880


Examples of Semimodule

Example
The set {(a, b)} of two-dimensional vectors with nonnegative real
coefficients constitutes a semimodule over nonnegative integers
Polynomials with nonnegative integer coefficients constitute a
semimodule over nonnegative integers

Stepanov, McJones Elements of Programming March 14, 2008 301 / 880


Concept QRSemimodule

Definition
QRSemimodule(T) ⇒
Semimodule(T)
The partial binary operation remainder is defined on T
The partial binary infix function
quotient : T × T → Scalar(T ) is defined
For all a ∈ T , b , T (0) ∈ T ,

a = quotient(a, b) · b + remainder(a, b)

Stepanov, McJones Elements of Programming March 14, 2008 302 / 880


gcd for QRSemimodule

template <typename T>


requires(QRSemimodule(T))
T gcd(T a, T b)
{
// Precondition: ¬(a = T (0) ∧ b = T (0))
while (true) {
if (b == T(0)) return a;
a = remainder(a, b);
if (a == T(0)) return b;
b = remainder(b, a);
}
}

This procedure does not always terminate

Stepanov, McJones Elements of Programming March 14, 2008 303 / 880


Concept EuclideanSemimodule

Definition
EuclideanSemimodule(T) ⇒
QRSemimodule(T)
For all a, b ∈ T such that ¬(a = T (0) ∧ b = T (0)), gcd(a, b)
terminates

Stepanov, McJones Elements of Programming March 14, 2008 304 / 880


gcd

template <typename T>


requires(EuclideanSemimodule(T))
T gcd(T a, T b)
{
// Precondition: ¬(a = T (0) ∧ b = T (0))
while (true) {
if (b == T(0)) return a;
a = remainder(a, b);
if (a == T(0)) return b;
b = remainder(b, a);
}
}

Stepanov, McJones Elements of Programming March 14, 2008 305 / 880


Concept EuclideanSemiring
Definition
EuclideanSemiring(T ) ⇒
CommutativeSemiring(T )
For all a, b ∈ T , a × b = T (0) ⇒ a = T (0) ∨ b = T (0)
A Euclidean function w : T → N is defined
For all a, b ∈ T , w(a × b) > w(a)
Partial binary operations remainder and quotient
are defined on T
For all a ∈ T , b , T (0) ∈ T ,

remainder(a, b) , T (0) ⇒ w(remainder(a, b)) < w(b)

The fact that w decreases with application of remainder assures


that gcd terminates
Stepanov, McJones Elements of Programming March 14, 2008 306 / 880
Euclidean semiring is Euclidean semimodule

Every commutative semiring is a semimodule over itself


This implies every Euclidean semiring is a Euclidean semimodule
This implies gcd can be used both for Euclidean monoids and the
traditional concept of Euclidean semirings 22 , such as polynomials
over reals

22
Technically, the traditional concept introduced by Noether and van der Waerden
is the Euclidean ring, but semirings suffice
Stepanov, McJones Elements of Programming March 14, 2008 307 / 880
Deriving a fast algorithm for quotient and remainder

Let a = n · b + r, where r = weak_remainder(a, b)


Let n = 2q + p, where q = bn/2c and p = n mod 2
a = q · (2 · b) + (p · b + r) where (p · b + r) < 2 · b
remainder(a, b) = r =


 if a < b
a

a − b if a − b < b

 remainder(a, 2 · b)) if remainder(a, 2 · b) < b


remainder(a, 2 · b)) − b otherwise
quotient(a,
 b) =

 0 if a < b


1 if a − b < b

 2 · quotient(a, 2 · b)) if remainder(a, 2 · b) < b


2 · quotient(a, 2 · b)) + 1 otherwise

Stepanov, McJones Elements of Programming March 14, 2008 308 / 880


quotient_remainder_nonnegative

template <typename I, typename T>


requires(Integer(I) && ArchimedeanMonoid(T))
pair<I, T> quotient_remainder_nonnegative(T a, T b)
{
// Precondition: a > T (0) ∧ b > T (0)
if (a < b)
return pair<I, T>(I(0), a);
if (a - b < b)
return pair<I, T>(I(1), a - b);
pair<I, T> q = quotient_remainder_nonnegative(a, b + b);
I n = q.first + q.first;
a = q.second;
if (a < b)
return pair<I, T>(n, a);
else
return pair<I, T>(n + I(1), a - b);
}

Stepanov, McJones Elements of Programming March 14, 2008 309 / 880


quotient_nonnegative

While we can compute remainder without quotient, we don’t


know how to compute quotient without remainder
Most computer instruction sets include an instruction that
computes both quotient and remainder
It would be nice if programming languages provided a binding
for this instruction
template <typename I, typename T>
requires(Integer(I) && ArchimedeanMonoid(T))
I quotient_nonnegative(T a, T b)
{
return quotient_remainder_nonnegative<I>(a, b).first;
}

Stepanov, McJones Elements of Programming March 14, 2008 310 / 880


quotient_remainder_nonnegative_iterative

template <typename I, typename T>


requires(Integer(I) && WeaklyHalvableMonoid(T))
pair<I, T> quotient_remainder_nonnegative_iterative(T a, const T& b)
{
// Precondition: a > T (0) ∧ b > T (0)
if (a < b) return pair<I, T>(I(0), a);
T c = b;
while (a - c >= c) c = c + c;
a = a - c;
I n = I(1);
while (c != b) {
n = n + n;
c = half_nonnegative(c);
if (c <= a) {
a = a - c;
n = n + I(1);
}
}
return pair<I, T>(n, a);
}

Stepanov, McJones Elements of Programming March 14, 2008 311 / 880


Requirements for extending quotient and remainder
to negative quantities

Quotient and remainder on signed quantities must satisfy these


properties:
quotient(a, b) is an integer
a = b · quotient(a, b) + remainder(a, b)
|remainder(a, b)| < |b|
For any integer n, remainder(a, b) = remainder(a + n · b, b)

Stepanov, McJones Elements of Programming March 14, 2008 312 / 880


Problems with current implementations of quotient
and remainder

Our requirements are not satisfied by many programming


languages, in which quotient truncates toward zero 23
Truncation violates our fourth requirement 24
In addition, truncation is an inferior way of rounding because it
sends twice as many values to zero as to any other integer, thus
leading to a nonuniform distribution

23
For an excellent discussion of this and other problems, see: Raymond T. Boute.
The Euclidean Definition of the Functions div and mod. ACM Transactions on
Programming Languages and Systems, Volume 14, Number 2, April 1992, pages 127-144
24
This requirement is equivalent to the classical mathematical definition of
congruence: “If two numbers a and b have the same remainder r relative to the same
modulus k they will be called congruent relative to the modulus k (following Gauss).”
—P. G. L. Dirichlet, Lectures on Number Theory, American Mathematical Society, 1999
Stepanov, McJones Elements of Programming March 14, 2008 313 / 880
Adapting nonnegative remainder and quotient

The next two functions are adaptors that convert remainder and
quotient_remainder correctly defined for nonnegative inputs to
produce “correct” results for positive or negative inputs

Stepanov, McJones Elements of Programming March 14, 2008 314 / 880


remainder

template <typename T, typename Op>


requires(OrderedAdditiveGroup(T) && BinaryOperation(Op) && Domain(Op) == T)
T remainder(T a, T b, Op rem)
{
// Precondition: b , 0
T r;
if (a < T(0))
if (b < T(0)) {
r = -rem(-a, -b);
} else {
r = rem(-a, b); if (r != T(0)) r = b - r;
}
else
if (b < T(0)) {
r = rem(a, -b); if (r != T(0)) r = b + r;
} else {
r = rem(a, b);
}
return r;
}

Stepanov, McJones Elements of Programming March 14, 2008 315 / 880


quotient_remainder

template <typename I, typename T, typename Op>


requires(Integer(I) && OrderedAdditiveGroup(T) &&
HomogeneousFunction(Op) && Arity(Op) == 2 && Domain(Op) == T && Codomain(Op) == pair<I, T>)
pair<I, T> quotient_remainder(T a, T b, Op quo_rem)
{
// Precondition: b , 0
pair<I, T> q_r;
if (a < 0) {
if (b < 0) { q_r = quo_rem(-a, -b); q_r.second = -q_r.second; }
else {
q_r = quo_rem(-a, b);
if (q_r.second != 0) { q_r.second = b - q_r.second; q_r.first = q_r.first + 1; }
q_r.first = -q_r.first;
}
} else {
if (b < 0) {
q_r = quo_rem( a, -b);
if (q_r.second != 0) { q_r.second = b + q_r.second; q_r.first = q_r.first + 1; }
q_r.first = -q_r.first;
}
else
q_r = quo_rem( a, b);
}
return q_r;
}

Stepanov, McJones Elements of Programming March 14, 2008 316 / 880


Concept DiscreteArchimedeanSemiring

Definition
DiscreteArchimedeanSemiring(T ) ⇒
CommutativeSemiring(T )
ArchimedeanMonoid(T )
For all a, b, c ∈ T , a < b ∧ 0 < c ⇒ a × c < b × c
For all a ∈ T , a < T (1) ⇒ ¬(T (0) < a)

The last property is discreteness

Stepanov, McJones Elements of Programming March 14, 2008 317 / 880


Concept NonnegativeDiscreteArchimedeanSemiring

Definition
NonnegativeDiscreteArchimedeanSemiring(T ) ⇒
NonnegativeDiscreteArchimedeanSemiring(T )
For all a ∈ T , T (0) 6 a

Stepanov, McJones Elements of Programming March 14, 2008 318 / 880


Concept DiscreteArchimedeanRing

Definition
DiscreteArchimedeanRing(T ) ⇒
DiscreteArchimedeanSemiring(T )
AdditiveGroup(T )

Stepanov, McJones Elements of Programming March 14, 2008 319 / 880


Univalent concepts

Definition
Two types T1 and T2 are isomorphic if it is possible to write conversion
functions from T1 to T2 and from T2 to T1 that preserve the procedures
and their semantics

Definition
A concept is univalent if any types satisfying it are isomorphic

NonnegativeDiscreteArchimedeanSemiring is univalent; types


satisfying it are isomorphic to N, the natural numbers 25
DiscreteArchimedeanRing is univalent; types satisfying it are
isomorphic to Z, the integers

25
We follow Peano and include 0 in the natural numbers: Giuseppe Peano.
Formulario Mathematico, Edizioni Cremonese, Roma, 1960, page 27
Stepanov, McJones Elements of Programming March 14, 2008 320 / 880
Computer integer types

Computer instruction sets and programming languages provide


partial implementations of natural numbers and integers
While providing a detailed formal specification for computer
integers lies outside the scope of this book, there are some
important points we must make
Few if any programming languages provide access to the full
power of hardware operations for addition, subtraction,
multiplication, quotient, and remainder
We sketch appropriate interfaces

Stepanov, McJones Elements of Programming March 14, 2008 321 / 880


Bounded unsigned and signed integer types

Definition
A bounded unsigned integer type, Un , where n = 8, 16, 32, 64, . . ., is an
unsigned integer type capable of representing a value in the interval
[0, 2n )

Definition
A bounded signed integer type, Sn , where n = 8, 16, 32, 64, . . ., is a
signed integer type capable of representing a value in the interval
[−2n−1 , 2n−1 )

Stepanov, McJones Elements of Programming March 14, 2008 322 / 880


Operations for bounded unsigned and signed types

Programming languages typically provide operations restricted


within a single type
Arithmetic operations such as multiplication and addition return
results that are larger than the type of their operands
While computer instructions return full results, a programmer in a
higher-level language has no access to them
Extended operations such as the ones below should be provided

sum_extended : Un × Un × U1 → U1 × Un
difference_extended : Un × Un × U1 → U1 × Un
product_extended : Un × Un → U2n
quotient_remainder_extended : U2n × Un → Un × Un

Stepanov, McJones Elements of Programming March 14, 2008 323 / 880


Arithmetic in a programming language

A study of the instruction sets for modern computer architectures


shows the functionality that should be encompassed
A good abstraction of these instruction sets is provided by
Knuth’s MMIX 26

26
Donald E. Knuth. The Art of Computer Programming, Volume 1, Fascicle 1, MMIX : A
RISC Computer for the New Millenium, Addison-Wesley, 2005, pages 1-28
Stepanov, McJones Elements of Programming March 14, 2008 324 / 880
Conclusions

Our task is to combine algorithms and mathematical structures


into a seamless whole by describing algorithms in abstract terms
and adjusting theories to fit algorithmic requirements
If this chapter seems too mathematical, it should be noted that the
algorithms and mathematics in it are modern restatements of
results that are thousands of years old

Stepanov, McJones Elements of Programming March 14, 2008 325 / 880


References

Donald Knuth.
The Greatest Common Divisor, and Division of Polynomials.
The Art of Computer Programming, Volume 2, 3rd edition, 1998,
Sections 4.5.2 and 4.6.1, pages 333-356 and 420-439.
Exhaustive coverage of both Euclidean and Stein (binary) gcd.
Pierre Samuel.
About Euclidean Rings.
Journal of Algebra, Volume 19, 1971, pages 282-301.
Exhaustive and relatively elementary treatment of Euclidean rings.

Stepanov, McJones Elements of Programming March 14, 2008 326 / 880


binary_gcd_nonnegative
In 1961, Josef Stein discovered the following gcd algorithm 27 :
template <typename T>
requires(Integer(T))
T binary_gcd_nonnegative(T a, T b)
{
if (is_zero(a)) return b;
if (is_zero(b)) return a;
int d = 0;
while (is_even(a) && is_even(b)) {
halve_nonnegative(a); halve_nonnegative(b); d = d + 1;
}
while (is_even(a)) halve_nonnegative(a);
while (is_even(b)) halve_nonnegative(b);
while (true)
if (a < b) {
b = b - a; do { halve_nonnegative(b); } while (is_even(b));
} else if (b < a) {
a = a - b; do { halve_nonnegative(a); } while (is_even(a));
} else return binary_scale_up_nonnegative(a, d);
}
27
Josef Stein. Computational problems associated with Racah algebra. J. Comput.
Phys., Volume 1, 1967, pages 397-405
Stepanov, McJones Elements of Programming March 14, 2008 327 / 880
Generalizations of binary_gcd_nonnegative

While Stein’s algorithm might at first appear to be a “hack”


dependent on binary integers, it is actually much deeper
The key observation is that 2 is the smallest integer prime, and the
only nonzero remainder mod2 is 1, a unit or invertible ring
element
It is possible to generalize it to other domains by using smallest
primes in those domains, such as the monomial x for polynomials
over reals or 1 + i for Gaussian integers
Polynomials See Knuth (Exercise 4.6.1.6, page 435 and
Solution, page 673)
Gaussian integers See Weilert
Other algebraic integer rings See Damgård and Frandsen,
and Agarwal and Frandsen

Stepanov, McJones Elements of Programming March 14, 2008 328 / 880


Project

Project
Find the correct abstract setting for binary binary_gcd_nonnegative
(Stein domain)

Stepanov, McJones Elements of Programming March 14, 2008 329 / 880


Additional references for binary_gcd_nonnegative

Andre Weilert.
(1+ i)-ary GCD Computation in Z[i] as an Analogue of the Binary
GCD Algorithm.
J. Symbolic Computation (2000) 30, pages 605-617.
Ivan Bjerre Damgård and Gudmund Skovbjerg Frandsen.
Efficient algorithms for GCD and cubic residuosity in the ring of
Eisenstein integers.
Proceedings of the 14th International Symposium on Fundamentals of
Computation Theory, Lecture Notes in Computer Science 2751,
Springer-Verlag (2003), pages 109-117.
Saurabh Agarwal and Gudmund Skovbjerg Frandsen.
Binary GCD Like Algorithms for Some Complex Quadratic Rings.
ANTS 2004, pages 57-71.

Stepanov, McJones Elements of Programming March 14, 2008 330 / 880


Contents I

1 Introduction

2 Foundations

3 Transformations and their orbits

4 Algorithms on algebraic structures

5 Orderings

6 Combining concepts

7 Refining concepts of iterators

Stepanov, McJones Elements of Programming March 14, 2008 331 / 880


Contents II
Memory and dereferencing
Actions
Iterators
Ranges
Forward iterators
Bidirectional iterators
Indexed iterators
Random access iterators
Conclusions
Reference
Project

8 Permutations and rearrangements

9 Rotations

Stepanov, McJones Elements of Programming March 14, 2008 332 / 880


Contents III
10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects

13 Iterative algorithms for divide-and-conquer

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

Stepanov, McJones Elements of Programming March 14, 2008 333 / 880


Contents IV
17 Index

Stepanov, McJones Elements of Programming March 14, 2008 334 / 880


Memory

Intuition
Memory is a set of locations, each with an address and a value
Getting or setting the value given an address is “fast”
The association of a value with a location is changeable

Examples
The physical and virtual address spaces of a computer
The locations visited by an algorithm
The locations owned by a data structure

Stepanov, McJones Elements of Programming March 14, 2008 335 / 880


Concept Readable

Definition
Readable(T ) ⇒
The type function ValueT ype(T ) is defined
Regular(ValueT ype(T ))
source : T → ValueT ype(T ) is defined a
a
source returns its result by value, by reference, or by constant reference

Stepanov, McJones Elements of Programming March 14, 2008 336 / 880


Concept Writable

Definition
Writable(T ) ⇒
The type function ValueT ype(T ) is defined
Regular(ValueT ype(T ))
If x ∈ T and v ∈ ValueT ype(T ), then sink(x) ← v is defined
No other use of sink(x) can be justified by the concept Writable,
although other uses may be supported by a specific type modeling
Writable
For a particular state of an object x, only a single assignment to
sink(x) can be justified by the concept Writable; a specific type
might provide a protocol allowing subsequent assignments to
sink(x)

Stepanov, McJones Elements of Programming March 14, 2008 337 / 880


is_aliased

template <typename T, typename U>


requires(Writable(T) && Readable(U) && ValueType(T) == ValueType(U))
bool is_aliased(const T& x, const U& y)
{
// return true ⇔ immediately after executing sink(x) ← v, source(y) = v
// For many types, this works:
typedef const ValueType(T)* P;
return P(&sink(x)) == P(&source(y));
}

Stepanov, McJones Elements of Programming March 14, 2008 338 / 880


Concept Mutable

Definition
Mutable(T ) ⇒
Readable(T ) ∧ Writable(T )
For all x ∈ T , is_aliased(x, x)

Stepanov, McJones Elements of Programming March 14, 2008 339 / 880


Concept Dereferenceable

Definition
Dereferenceable(T ) ⇒
Readable(T ) ∨ Writable(T ) ∨ Mutable(T )

Dereferenceable is a useful concept for describing types even


though it is not used in programs (since it’s not clear which of
source or sink could be used)

Stepanov, McJones Elements of Programming March 14, 2008 340 / 880


Extending Dereferenceable functions to all regular types

Reflection
It is useful if dereferencing an object x that doesn’t refer to another
object returns x itself
Therefore we assume that unless otherwise defined,
ValueT ype(T ) = T and source and sink return the object to
which they are applied

Stepanov, McJones Elements of Programming March 14, 2008 341 / 880


Concept Action

Definition
Action(T ) ⇒
ProperProcedure(T )
Arity(T ) = 1
T takes its argument by reference

Stepanov, McJones Elements of Programming March 14, 2008 342 / 880


Duality of actions and transformations

For every action there is a corresponding transformation, and vice


versa
The following two functions demonstrate this equivalence
The complexity of an action when implemented independently of
the corresponding transformation could be smaller
Example: interchange the first and last elements of a sequence

Stepanov, McJones Elements of Programming March 14, 2008 343 / 880


transformation_from_action

template <typename A>


requires(Action(A))
struct transformation_from_action
{
A a;
transformation_from_action() {}
transformation_from_action(const A& a) : a(a) {}
Domain(A) operator()(Domain(A) x) { a(x); return x; }
};

Stepanov, McJones Elements of Programming March 14, 2008 344 / 880


action_from_transformation

template <typename F>


requires(Transformation(F))
struct action_from_transformation
{
F f;
action_from_transformation() {}
action_from_transformation(const F& f) : f(f) {}
void operator()(Domain(F)& x) { x = f(x); }
};

Stepanov, McJones Elements of Programming March 14, 2008 345 / 880


Concept RegularAction

Definition
RegularAction(T ) ⇒
Action(T )
RegularFunction(transformation_from_actionT )

Stepanov, McJones Elements of Programming March 14, 2008 346 / 880


Concept AmortizedConstantTime

Definition
The amortized complexity of an operation is the complexity averaged
over a worst-case sequence of operations a
a
For an extensive treatment of amortized complexity, see: Robert Endre Tarjan.
Amortized Computational Complexity. SIAM Journal on Algebraic and Discrete
Methods, Volume 6, Number 2, April 1985, pages 306-318

Definition
AmortizedConstantTime(T ) ⇒
ProperProcedure(T )
The amortized time complexity of it is constant and “small”

Formalizing complexity specifications of abstract operations is a


subject for future research
Stepanov, McJones Elements of Programming March 14, 2008 347 / 880
Concept Iterator 28

Definition
Iterator(T ) ⇒
Regular(T )
The type function DistanceT ype(T ) is defined
Integer(DistanceT ype(T ))
The prefix operator ++ is defined on T
Action(++)
AmortizedConstantTime(++)

28
Our treatment of iterators departs significantly from that in the STL
Stepanov, McJones Elements of Programming March 14, 2008 348 / 880
Iterator protocols

++ is not necessarily regular


If there are two copies of an iterator and one is incremented, the
other may become invalid
Thus an iterator is too weak to be used in a multipass algorithm
Assignment to a sink is not idempotent: a call to ++ must separate
two assignments to an iterator
The asymmetry between readable and writable iterators is
intentional
The ability to read from source more than once allows us to write
simple, useful functions like find_if
No corresponding benefits seem to accrue in the case of sink

Stepanov, McJones Elements of Programming March 14, 2008 349 / 880


Examples of Iterator

An iterator where ++ advances an input stream


An iterator where ++ advances an output stream
An iterator on a singly-linked list
An iterator on a doubly-linked list
An iterator on a one-dimensional array
int∗
int

Stepanov, McJones Elements of Programming March 14, 2008 350 / 880


successor and - for Iterator

template <typename I>


requires(Iterator(I))
I successor(I f)
{
++f;
return f;
}

template <typename I>


requires(Iterator(I))
DistanceType(I) operator-(I l, I f)
{
return distance(f, l, successor<I>);
}

Stepanov, McJones Elements of Programming March 14, 2008 351 / 880


+= and + for Iterator

template <typename I>


requires(Iterator(I))
void operator+=(I& f, DistanceType(I) n)
{
typedef DistanceType(I) N;
while (n != N(0)) {
++f;
n = n - N(1);
}
}

template <typename I>


requires(Iterator(I))
I operator+(I f, DistanceType(I) n)
{
f += n;
return f;
}

Stepanov, McJones Elements of Programming March 14, 2008 352 / 880


Orbit of successor

Since successor is a transformation, it defines an orbit starting at


any iterator
In this book, we only deal with terminating iterator orbits
Therefore, if, for i > 0, successori (x) is defined, it is not equal to x

Stepanov, McJones Elements of Programming March 14, 2008 353 / 880


The natural ordering on an orbit under successor

There exists a natural total ordering ≺ on the iterators in the orbit


of x under successor defined by

x ≺ y ⇔ (∃i > 0)y = successori (x)

≺ and the corresponding nonstrict  are used in specifications


such as preconditions and postconditions of algorithms
For many pairs of values of an iterator type, ≺ is not defined, so
there is often no effective way to write code implementing ≺
There is no efficient way to determine if one node precedes another
in a linked structure (the nodes might not even be linked together)

Stepanov, McJones Elements of Programming March 14, 2008 354 / 880


Sequences of iterators

It is useful to have standard terminology and notation for


sequences and subsequences of iterators
To describe the input and/or output of an algorithm
To describe ranges in a data structure
It is useful to describe a (possibly empty) sequence of iterators
starting at a particular iterator

Example
Binary search looks for the sequence of iterators whose values are
equal to a given value
This sequence is empty if there are no such values
The location of this empty sequence carries information: the values
in the sequence approximating the given value

Stepanov, McJones Elements of Programming March 14, 2008 355 / 880


Ranges

Such a located sequence of iterators is fully specified by a starting


iterator and a nonnegative integer
It is often convenient to specify a sequence by specifying a starting
iterator together with another one that bounds the sequence
A specification of a such a located sequence is called a range
Ranges can be either counted or bounded, and either semi-open or
closed

Stepanov, McJones Elements of Programming March 14, 2008 356 / 880


Different kinds of ranges
Definition
A semi-open counted range Ji, nM, where n > 0 is an integer, denotes the
sequence of iterators {j|i  j ≺ successorn (i)}

Definition
A closed counted range Ji, nK, where n > 0 is an integer, denotes the
sequence of iterators {k|i  k  successorn−1 (i)}

Definition
A semi-open bounded range [i, j) denotes the sequence of iterators
{k|i  k ≺ j}

Definition
A closed bounded range [i, j] denotes the sequence of iterators
{k|i  k  j}
Stepanov, McJones Elements of Programming March 14, 2008 357 / 880
Terminology for iterators in a range

Definition
An iterator f in a range [i, j) is called the first if f = i ∧ f , j
An iterator l in a range [i, j) is called the limit if l = j
An iterator k in a range [i, j) is called the lasta if successor(k) = j
Otherwise an iterator in the range is called not last
a
The terminology distinguishing limit and last was suggested by John Banning

Stepanov, McJones Elements of Programming March 14, 2008 358 / 880


Size of a range

Definition
The size of a counted range Ji, nM or Ji, nK is n
The size of a semi-open bounded range [i, j) is
distance(i, j, successor)
The size of a closed bounded range [i, j] is
distance(i, j, successor) + 1

The size of a range is used in preconditions and postconditions of


algorithms even when it cannot be effectively computed because
of an iterator too weak for a multipass algorithm

Stepanov, McJones Elements of Programming March 14, 2008 359 / 880


Empty ranges

Definition
An empty semi-open range is specified by Ji, 0M or [i, i) for some
iterator i
There are no empty closed ranges

Stepanov, McJones Elements of Programming March 14, 2008 360 / 880


Definition space of ++ on ranges

Lemma
++ and successor are defined for every iterator in a semi-open range,
and for every iterator except the last in a closed range

Stepanov, McJones Elements of Programming March 14, 2008 361 / 880


Readable, writable, and mutable ranges

Definition
A range r is readable, writable, or mutable if, correspondingly, source,
sink, or both of them are defined on all the iterators in the range

Stepanov, McJones Elements of Programming March 14, 2008 362 / 880


Type requirements of copy

template <typename I, typename O>


requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
ValueType(I) == ValueType(O))
O copy(I f, I l, O o);

Readable(I) ensures source is defined


Iterator(I) ensures ++ is defined
Writable(O) ensures sink is defined
Iterator(O) ensures ++ is defined
ValueT ype(I) = ValueT ype(O) ensures assignment is defined

Stepanov, McJones Elements of Programming March 14, 2008 363 / 880


Precondition and postcondition of copy

template <typename I, typename O>


requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
ValueType(I) == ValueType(O))
O copy(I f, I l, O o);

Precondition The input range must be readable


The output range must be writable and of at least
the size of the input range
If the ith iterator in the input range is aliased to the
jth iterator of the output, then i 6 j (informally,
every value is used before it is overwritten)
Postcondition The sequence of values in the output range is equal
to the sequence of original values in the input range

Stepanov, McJones Elements of Programming March 14, 2008 364 / 880


Implementation of copy

template <typename I, typename O>


requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
ValueType(I) == ValueType(O))
O copy(I f, I l, O o)
{
while (f != l) {
sink(o) = source(f);
++f;
++o;
}
return o;
}

We return the end of the output range because it might not be


known to the caller, who might find it useful
It is worth the small constant time to return it

Stepanov, McJones Elements of Programming March 14, 2008 365 / 880


Example using copy

template <typename O>


requires(Writable(O) && Iterator(O) && Integer(ValueType(O)))
O iota(ValueType(O) n, O o) // like APL ι
{
return copy(ValueType(O)(0), n, o);
}

Stepanov, McJones Elements of Programming March 14, 2008 366 / 880


copy_bounded

template <typename I, typename O>


requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
ValueType(I) == ValueType(O))
pair<I, O> copy_bounded(I f, I l, O o_f, O o_l)
{
while (f != l && o_f != o_l) {
sink(o_f) = source(f);
++f;
++o_f;
}
return pair<I, O>(f, o_f);
}

While the ends of both ranges are known to the caller, returning
the pair allows determining which range is smaller and where in
the larger range copying stopped

Stepanov, McJones Elements of Programming March 14, 2008 367 / 880


copy_n

template <typename I, typename O>


requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
ValueType(I) == ValueType(O))
pair<I, O> copy_n(I f, DistanceType(I) n, O o)
{
typedef DistanceType(I) N;
while (n != N(0)) {
sink(o) = source(f);
++f;
++o;
n = n - N(1);
}
return pair<I, O>(f, o);
}

Stepanov, McJones Elements of Programming March 14, 2008 368 / 880


copy_k

template <int k, typename I, typename O>


requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
ValueType(I) == ValueType(O))
struct copy_k
{
void operator()(I& f, O& r)
{
copy_k<k - 1, I, O>(f, r);
sink(r) = source(f);
++f; ++r;
}
};

template <typename I, typename O>


struct copy_k<0, I, O> {
void operator()(I&, O&) { }
};

Stepanov, McJones Elements of Programming March 14, 2008 369 / 880


copy_n_unrolled

template <typename I, typename O>


requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
ValueType(I) == ValueType(O))
pair<I, O> copy_n_unrolled(I f, DistanceType(I) n, I r)
{
typedef DistanceType(I) N;
const int k = 4; // unroll factor
while (n >= N(k)) {
copy_k<k, I, O>()(f, r);
n = n - N(k);
}
return copy_n(f, n, r);
}

Stepanov, McJones Elements of Programming March 14, 2008 370 / 880


Concept ForwardIterator

Definition
ForwardIterator(T ) ⇒
Iterator(T )
RegularAction(++)

Iterator and ForwardIterator differ only by an axiom; there are no


new operations

Stepanov, McJones Elements of Programming March 14, 2008 371 / 880


Examples of ForwardIterator

An iterator on a singly-linked list


An iterator on a doubly-linked list
An iterator on a one-dimensional array
int∗
int

Stepanov, McJones Elements of Programming March 14, 2008 372 / 880


Advantages of regularity of ++

Reflection
ForwardIterator allows
algorithms that maintain more than one iterator into a range
multipass algorithms

Stepanov, McJones Elements of Programming March 14, 2008 373 / 880


Concept BidirectionalIterator

Definition
BidirectionalIterator(T ) ⇒
ForwardIterator(T )
The prefix operator -- and the corresponding transformation
predecessor are defined
RegularAction(--)
AmortizedConstantTime(--)
For any iterator k in a semi-open range [i, j):

k , j ⇒ predecessor(successor(k)) = k
k , i ⇒ successor(predecessor(k)) = k

Stepanov, McJones Elements of Programming March 14, 2008 374 / 880


Examples of BidirectionalIterator

An iterator on a doubly-linked list


An iterator on a one-dimensional array
int∗
int

Stepanov, McJones Elements of Programming March 14, 2008 375 / 880


-= and - for BidirectionalIterator

template <typename I>


requires(BidirectionalIterator(I))
void operator-=(I& f, DistanceType(I) n)
{
typedef DistanceType(I) N;
while (n != N(0)) {
--f;
n = n - N(1);
}
}

template <typename I>


requires(BidirectionalIterator(I))
I operator-(I f, DistanceType(I) n)
{
f -= n;
return f;
}

Stepanov, McJones Elements of Programming March 14, 2008 376 / 880


Using copy with overlapping ranges

If we need to copy a range to a destination that begins within the


source range, copy cannot be used because its aliasing
precondition is violated

Stepanov, McJones Elements of Programming March 14, 2008 377 / 880


copy_backward

The main difference from copy is the aliasing precondition: if the


ith iterator in the input range is aliased to the jth iterator of the
output, then j 6 i

template <typename I, typename O>


requires(Readable(I) && BidirectionalIterator(I) &&
Writable(O) && BidirectionalIterator(O) &&
ValueType(I) == ValueType(O))
O copy_backward(I f, I l, O o)
{
while (f != l) {
--l;
--o;
sink(o) = source(l);
}
return o;
}

Stepanov, McJones Elements of Programming March 14, 2008 378 / 880


copy_backward_n

template <typename I, typename O>


requires(Readable(I) && BidirectionalIterator(I) &&
Writable(O) && BidirectionalIterator(O) &&
ValueType(I) == ValueType(O))
pair<I, O> copy_backward_n(I l, DistanceType(I) n, O o)
{
typedef DistanceType(I) N;
while (n != N(0)) {
--l;
--o;
sink(o) = source(l);
n = n - N(1);
}
return pair<I, O>(l, o);
}

Stepanov, McJones Elements of Programming March 14, 2008 379 / 880


Concept IndexedIterator

Definition
IndexedIterator(T ) ⇒
ForwardIterator(T )
The mutating assignment += is defined on T × DistanceT ype(T )
- : T × T → DistanceT ype(T ) is defined
AmortizedConstantTime(+=)
AmortizedConstantTime(-)

+= and -, which were defined for Iterator, are now required to be


primitive

Stepanov, McJones Elements of Programming March 14, 2008 380 / 880


+ for IndexedIterator

The default implementation of + : I × DistanceT ype(I) → I in


terms of += for Iterator becomes constant-time for IndexedIterator

Stepanov, McJones Elements of Programming March 14, 2008 381 / 880


Examples of IndexedIterator

An iterator on a one-dimensional array


int∗
int

Stepanov, McJones Elements of Programming March 14, 2008 382 / 880


copy_k_indexed
template <int k, typename I, typename O>
requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&
ValueType(I) == ValueType(O))
struct basic_copy_k_indexed {
void operator()(I f, O o)
{
basic_copy_k_indexed<k - 1, I, O>()(f, o);
sink(o + (k - 1)) = source(f + (k - 1));
}
};

template <typename I, typename O>


struct basic_copy_k_indexed<0, I, O>
{ void operator()(I, O) { } };

template <typename I, typename O, int k>


requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&
ValueType(I) == ValueType(O))
void copy_k_indexed(I& f, O& o) {
basic_copy_k_indexed<k, I, O>()(f, o);
f += k; o += k;
}
Stepanov, McJones Elements of Programming March 14, 2008 383 / 880
copy_n_unrolled_indexed

template <typename I, typename O>


requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&
ValueType(I) == ValueType(O))
pair<I, O> copy_n_unrolled_indexed(I f, DistanceType(I) n, O o)
{
typedef DistanceType(I) N;
const int k = 4; // unroll factor
while (n >= N(k)) {
copy_k_indexed<k, I, O>(f, o);
n = n - N(k);
}
return copy_n(f, n, o);
}

Stepanov, McJones Elements of Programming March 14, 2008 384 / 880


Out-of-order execution

Out-of-order execution of adjacent iterations speeds up the


execution of copy
A compiler cannot achieve this because it cannot determine that
the source and destination ranges do not alias
Programmers know when this is the case, and should be able to
specify when such interleaved execution is legitimate

Stepanov, McJones Elements of Programming March 14, 2008 385 / 880


copy_parallel for disjoint ranges

The main difference from copy is the aliasing precondition: the


input and output ranges do not alias
A special forall executes its body for all values of its iteration
variable in arbitrary order, possibly concurrently
template <typename I, typename O>
requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&
ValueType(I) == ValueType(O))
void copy_parallel(I f, DistanceType(I) n, O o)
{
typedef DistanceType(I) N;
forall(N i, N(0), n) {
sink(o + i) = source(f + i);
}
}

Stepanov, McJones Elements of Programming March 14, 2008 386 / 880


Concept RandomAccessIterator (1 of 2)

Definition (requirements, signatures)


RandomAccessIterator(T ) ⇒
BidirectionalIterator(T ) ∧ IndexedIterator(T )
StrictTotallyOrdered(T )
The type function DifferenceT ype(T ) is defined
Integer(DifferenceT ype(T ))
DifferenceT ype(T ) is large enough to contain distances and their
negations
The mutating assignments += and -= are defined on
T × DifferenceT ype(T )
+ : T × DifferenceT ype(T ) → T is defined
- : T × DifferenceT ype(T ) → T is defined
- : T × T → DifferenceT ype(T ) is defined

Stepanov, McJones Elements of Programming March 14, 2008 387 / 880


Concept RandomAccessIterator (2 of 2)

Definition (complexity, definition spaces)


RandomAccessIterator(T ) ⇒
AmortizedConstantTime(<)
AmortizedConstantTime(-=)
AmortizedConstantTime(- : T × DifferenceT ype(T ) → T )
+=, -=, +, and - : T × DifferenceT ype(T ) → T accept negative
values for their second argument
AmortizedConstantTime(- : T × T → DifferenceT ype(T ))
- : T × T → DifferenceT ype(T )) accepts iterators in either order

Stepanov, McJones Elements of Programming March 14, 2008 388 / 880


Axioms for RandomAccessIterator

Exercise
Write an appropriate set of axioms relating the operations to each other

Stepanov, McJones Elements of Programming March 14, 2008 389 / 880


Examples of RandomAccessIterator

int∗
int

Stepanov, McJones Elements of Programming March 14, 2008 390 / 880


Equivalence of
RandomAccessIterator and IndexedIterator

Theorem
For any function defined on a range of random access iterators, there is
another function defined on indexed iterators with the same
asymptotic complexity

Stepanov, McJones Elements of Programming March 14, 2008 391 / 880


Proof of the equivalence

Assuming these definitions:

U ≡ DistanceT ype(I)
W ≡ sign_extendedU

where sign_extended is a templated struct adding a sign bit and


appropriate integer operations, we rewrite the function with these
substitutions:

Replace With
DifferenceType(I) W
i < j i - f < j - f
i += n when n < 0 i = f + ((i - f) - U(-n))
i -= n i += -n
i - j W(i - f) - W(j - f)

Stepanov, McJones Elements of Programming March 14, 2008 392 / 880


Reflection on RandomAccessIterator and IndexedIterator

The theorem shows the theoretical equivalence of these concepts


in any context in which the beginning of ranges are known
copy_backward does not satisfy this requirement!
In practice we have found there is no performance penalty for
using the weaker concept

Stepanov, McJones Elements of Programming March 14, 2008 393 / 880


copy_backward_n_indexed

copy_backward is not realizable for indexed iterators


copy_backward_n_indexed, which is often just as useful, is
realizable
template <typename I, typename O>
requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&
ValueType(I) == ValueType(O))
void copy_backward_n_indexed(I f, DistanceType(I) n, O o)
{
typedef DistanceType(I) N;
while (n != N(0)) {
n = n - N(1);
sink(o + n) = source(f + n);
}
}

There is no useful information to return

Stepanov, McJones Elements of Programming March 14, 2008 394 / 880


Relationships between iterator concepts

BI
It FI RI
II

Stepanov, McJones Elements of Programming March 14, 2008 395 / 880


Conclusions

Refinement generates mathematical structures such as groups,


Abelian groups, totally-ordered groups, and Archimedean groups
It also generates concepts describing the fundamental notion of
computer science, iterating through a data structure
We have used three types of refinement, by adding
an operation
an axiom
a tighter complexity requirement

Stepanov, McJones Elements of Programming March 14, 2008 396 / 880


Reference

Alexander Stepanov and Meng Lee.


The Standard Template Library.
HP Laboratories Technical Report 95-11(R.1), November 14, 1995.
Introduced different categories of iterators and their axioms.

Stepanov, McJones Elements of Programming March 14, 2008 397 / 880


Intuition for SegmentedIterator

There are data structures where ++ can be implemented faster


within certain ranges or segments
hash tables
adjacency list representation of graphs
STL-like deques
It is possible to optimize many algorithms for such structures by
transforming inner loops into nested loops:
a loop over segments
a loop within a segment
This results in a new dimension of the classification of iterators:
homogeneous
segmented

Stepanov, McJones Elements of Programming March 14, 2008 398 / 880


Concept SegmentedIterator (1 of 2)
Definition (requirements, signatures, definition spaces)
SegmentedIterator(T) ⇒
ForwardIterator(T )
The type function SegmentIterator(T ) is defined
ForwardIterator(SegmentIterator(T ))
begin : T → SegmentIterator(T ) is defined
end : T → SegmentIterator(T ) is defined
+ : T × SegmentIterator(T ) ⇒ T is defined
AmortizedConstantTime(begin)
AmortizedConstantTime(end)
AmortizedConstantTime(+)
For any i ∈ [f, l], begin, end, and + are defined
For any w ∈ [begin(i), end(i)], i + w is defined

Stepanov, McJones Elements of Programming March 14, 2008 399 / 880


Concept SegmentedIterator (2 of 2)

Definition (axioms)
SegmentedIterator(T) ⇒
If
[f, l) is a range of segmented iterators
i ∈ [f, l)
w is a segment iterator in [begin(i), end(i))
then the following hold:
1 is_aliased(i, begin(i))
2 begin(i + w) = w
3 i + begin(i) = i
4 i + (begin(i) + 1) = i + 1
5 begin(i) + 1 , end(i) ⇒ begin(i) + 1 = begin(i + 1)
6 begin(i) , end(i)

Stepanov, McJones Elements of Programming March 14, 2008 400 / 880


copy_from_segmented

template <typename I, typename O>


requires(Readable(I) && SegmentedIterator(I) &&
Writable(O) && Iterator(O) &&
ValueType(I) == ValueType(O))
O copy_from_segmented(I f, I l, O o)
{
while (f + end(f) != l + end(l)) {
// f and l are in different segments
o = copy(begin(f), end(f), o);
f = f + end(f);
}
// f and l are in the same segment
return copy(begin(f), begin(l), o);
}

Stepanov, McJones Elements of Programming March 14, 2008 401 / 880


Project

Implement segmented iterators for one or more data structures


STL-like deque
SGI STL-like hashed containers
Produce segmented iterator versions of suitable (STL) algorithms
Analyze if the axioms are independent, consistent, and complete

Stepanov, McJones Elements of Programming March 14, 2008 402 / 880


Contents I

1 Introduction

2 Foundations

3 Transformations and their orbits

4 Algorithms on algebraic structures

5 Orderings

6 Combining concepts

7 Refining concepts of iterators

Stepanov, McJones Elements of Programming March 14, 2008 403 / 880


Contents II
8 Permutations and rearrangements
Permutations
Permutation groups
Cycle decomposition of a permutation
Rearrangements
Underlying type
Rearranging arbitrary cycles
Reverse permutation
Reverse algorithms
Category dispatch
Conclusions
Reading

9 Rotations

Stepanov, McJones Elements of Programming March 14, 2008 404 / 880


Contents III
10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects

13 Iterative algorithms for divide-and-conquer

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

Stepanov, McJones Elements of Programming March 14, 2008 405 / 880


Contents IV
17 Index

Stepanov, McJones Elements of Programming March 14, 2008 406 / 880


Onto functions

Definition
A (regular) function f is onto if for all y ∈ Codomain(F), there exists
x ∈ Domain(R) such that y = f(x)

Stepanov, McJones Elements of Programming March 14, 2008 407 / 880


One-to-one functions

Definition
A (regular) function f is one-to-one if for all x, x 0 ∈ Domain(F),
f(x) = f(x 0 ) ⇒ x = x 0

Stepanov, McJones Elements of Programming March 14, 2008 408 / 880


Permutation

Definition
A permutation is a transformation on a finite domain that is one-to-one
and onto

Stepanov, McJones Elements of Programming March 14, 2008 409 / 880


Example of a permutation on [0, 6)

p(0) = 5
p(1) = 2
p(2) = 4
p(3) = 3
p(4) = 1
p(5) = 0

Stepanov, McJones Elements of Programming March 14, 2008 410 / 880


Identity permutation

Definition
A fixed point of a transformation is an element x such that f(x) = x

Definition
The identity permutation on a set S, identityS , maps each element of S to
itself; every element in S is a fixed point of identityS

Stepanov, McJones Elements of Programming March 14, 2008 411 / 880


Composition of permutations

Definition
If p and q are two permutations on a set S, the composition q ◦ p takes
x ∈ S to q(p(x))

Lemma
The composition of permutations is a permutation

Lemma
Composition of permutations is associative

Stepanov, McJones Elements of Programming March 14, 2008 412 / 880


Inverse of a permutation

Lemma
For every permutation p on a set S, there is an inverse permutation p−1
such that p−1 ◦ p = p ◦ p−1 = identityS

Stepanov, McJones Elements of Programming March 14, 2008 413 / 880


Permutation group
The permutations on a set form a group under composition

Lemma
Every group is a subgroup of a permutation group of its elements
where every permutation in the subgroup is generated by multiplying
all the elements by an individual element

Example
× 1 2 3 4
1 1 2 3 4
2 2 4 1 3
3 3 1 4 2
4 4 3 2 1

Multiplication mod 5:
Every row and column of the multiplication table is a permutation
Stepanov, McJones Elements of Programming March 14, 2008 414 / 880
Cycles in a permutation

Definition
A cycle is a circular orbit within a permutation

Definition
A trivial cycle is one with a cycle size of 1
The element in a trivial cycle is a fixed point

Lemma
Every element in a permutation of a finite set belongs to a unique cycle

Stepanov, McJones Elements of Programming March 14, 2008 415 / 880


Example of cycle decomposition

p(0) = 5
p(1) = 2
p(2) = 4
p(3) = 3
p(4) = 1
p(5) = 0

p = (0 5)(1 2 4)(3)

Stepanov, McJones Elements of Programming March 14, 2008 416 / 880


Structure of cycle decomposition

Lemma
Any permutation of a set with n elements contains k 6 n disjoint cycles
Each cycle is itself a permutation of this set, called a cyclic permutation
Disjoint cyclic permutations commute
Every permutation can be represented as a product of its cycles
The inverse of a permutation is the product of the inverses of its cycles

Stepanov, McJones Elements of Programming March 14, 2008 417 / 880


Finite sets

Definition
A finite set S of size n is a set for which there exists a pair of functions

chooseS : [0, n) → S
indexS : S → [0, n)

satisfying

chooseS (indexS (x)) = x


indexS (chooseS (i)) = i

Stepanov, McJones Elements of Programming March 14, 2008 418 / 880


Index permutation of a permutation

Definition
If p is a permutation on a finite set S of size n, there is a corresponding
index permutation p 0 on [0, n) defined as

p 0 (i) = indexS (p(chooseS (i)))

Lemma
p(x) = chooseS (p 0 (indexS (x)))

We will frequently define permutations by the corresponding


index permutations

Stepanov, McJones Elements of Programming March 14, 2008 419 / 880


Rearrangement

Definition
A rearrangement is an algorithm that rearranges the elements of a
mutable range to satisfy a given postcondition

Stepanov, McJones Elements of Programming March 14, 2008 420 / 880


Classifying rearrangements

Rearrangements are classified according to the following fundamental


characteristics:
Characterization of the postcondition
Postcondition kind
Stability
Implementation constraints
Iterator requirements
Mutative versus copying
Complexity
Time
Space

Stepanov, McJones Elements of Programming March 14, 2008 421 / 880


Postcondition kind

position-based The destination of a value depends only on its original


position and not on the value itself; for example,
“reverse the range”
bin-based The destination of a value depends only on the result of
applying a k-way bin function to that value, and not
directly on the value itself; for example, “move bad
values before good ones”
A useful subcase is predicate-based algorithms,
where k = 2
ordering-based The destination of a value depends only on the outcome
of applications of an ordering to pairs of values in the
range; for example, “put the smallest value first”

Stepanov, McJones Elements of Programming March 14, 2008 422 / 880


Stability

Definition
A rearrangement is stable if it respects the original order of the range to
the maximal extent possible while satisfying the postcondition

Examples
Stable index partition Move all the elements at even positions in the
sequence before the elements at odd positions,
keeping the original order in both groups
Stable partition Move all the elements with even values before
those with odd values, keeping the original
order in both groups
Stable sort Sort pairs of integers by their first components
so pairs with equal first components remain in
the original order

Stepanov, McJones Elements of Programming March 14, 2008 423 / 880


Iterator requirements

For the same problem, there are often different algorithms for
different iterator requirements

Examples
reverse_forward
reverse_bidirectional
reverse_indexed

Stepanov, McJones Elements of Programming March 14, 2008 424 / 880


Mutative versus copying

Definition
Rearrangements as we have defined them are mutative

Definition
A rearrangement is copying if it sets a writable range to a rearranged
copy of a readable range in way that satisfies a given postcondition
A copying rearrangement could always be obtained by composing
copy with the corresponding mutative rearrangement
Often, however, there are faster algorithms that rearrange while
copying

Stepanov, McJones Elements of Programming March 14, 2008 425 / 880


Space complexity

Definition
A mutative algorithm is in place (or in situ) if it uses an amount of
additional space that is (poly-)logarithmic in the size of the input

Definition
A memory-adaptive algorithm uses as much additional space as it can
acquire to maximize performance
A small percentage of additional space, while theoretically
“linear,” can lead to a large performance improvement

Definition
An algorithm with buffer requires the caller to provide a buffer to be
used by the algorithm

Stepanov, McJones Elements of Programming March 14, 2008 426 / 880


Time complexity

Rearranging n elements might take n + k assignments for some


constant k
n log n algorithms arise
Determining the final position may require collecting additional
information, as in sorting
Absence of random access on some iterators requires multipass
algorithms, as with in-place random shuffle for forward iterators

Stepanov, McJones Elements of Programming March 14, 2008 427 / 880


cycle_2

template <typename I>


requires(Mutable(I) && Iterator(I))
void cycle_2(I x, I y)
{
ValueType(I) t = source(x);
sink(x) = source(y);
sink(y) = t;
}

Stepanov, McJones Elements of Programming March 14, 2008 428 / 880


Problems with cycle_2

It could be much slower than necessary


std::vector copies element-by-element instead of interchanging
the headers
It could throw an exception due to unnecessary resource allocation
We should exploit the fact it is needed for implementing mutative
rearrangements, which should not need to construct or destroy
any objects, but just move them around
We will address the issue in Chapters 12 with a notion of an
underlying type
For the time being, imagine that the value types of iterators are
built-in types or C-style structs

Stepanov, McJones Elements of Programming March 14, 2008 429 / 880


cycle_left_3

template <typename I>


requires(Mutable(I) && Iterator(I))
void cycle_left_3(I x, I y, I z)
{
ValueType(I) t = source(x);
sink(x) = source(y);
sink(y) = source(z);
sink(z) = t;
}

Stepanov, McJones Elements of Programming March 14, 2008 430 / 880


To-permutation and from-permutation

Definition
Every rearrangement corresponds to two permutations of its
range
A to-permutation mapping an iterator i to the iterator pointing to the
destination of the element at i
A from-permutation mapping an iterator i to the iterator pointing to
the origin of the element moved to i
These two permutations are inverses of each other

Stepanov, McJones Elements of Programming March 14, 2008 431 / 880


do_cycle_from

template <typename I, typename P>


requires(Mutable(I) && ForwardIterator(I) &&
Transformation(P) && Domain(P) == I)
void do_cycle_from(I i, P p)
{
// Precondition: p is a from-permutation of the range containing i
ValueType(I) t = source(i);
I f = i;
I n = p(i);
while (n != i) {
sink(f) = source(n);
f = n;
n = p(n);
}
sink(f) = t;
}

Stepanov, McJones Elements of Programming March 14, 2008 432 / 880


Strict lower bound for rearrangement

Theorem
The minimum number of assignments required for a rearrangement is
n + cN − cT , where n is the number of elements, cN the number of
nontrivial cycles, and cT the number of trivial cycles

Stepanov, McJones Elements of Programming March 14, 2008 433 / 880


Proof of strict lower bound for rearrangement

Consider a rearrangement realizing a permutation p, and let C be


a cycle of p of length > 1
The rearrangement must include an assignment of a value of C to
each memory location of C
Since such an assignment overwrites the value at that location,
there must be an additional assignment of some value of C to a
location outside C
Since the cycles are disjoint, these additional assignments are
distinct, so the total number of assignments must be at least
n + cN − cT , where n is the number of elements of p, cN the
number of nontrivial cycles, and cT the number of trivial cycles29
do_cycle_from (applied to nontrivial cycles) make this bound
strict

29
The idea for the proof was provided to us by John Wilkinson
Stepanov, McJones Elements of Programming March 14, 2008 434 / 880
do_cycle_to

Exercise
Implement do_cycle_to and compare the number of assignments it
performs to do_cycle_from

Stepanov, McJones Elements of Programming March 14, 2008 435 / 880


Reverse permutation

Definition
The permutation p on a finite set with n elements defined by an index
permutation p(i) = (n − 1) − i is called the reverse permutation of the set

The number of nontrivial cycles in a reverse permutation is bn/2c;


the number of trivial cycles is n mod 2
bn/2c is the largest possible number of nontrivial cycles in a
permutation

Stepanov, McJones Elements of Programming March 14, 2008 436 / 880


Reverse rearrangement

Definition
The reverse rearrangement of a range is the rearrangement induced by
the reverse permutation

The lower bound formula gives the number of assignments as


n + cN − cT = 3bn/2c

Stepanov, McJones Elements of Programming March 14, 2008 437 / 880


reverse_n_indexed

The definition of reverse directly gives this:

template <typename I>


requires(Mutable(I) && IndexedIterator(I))
void reverse_n_indexed(I f, DistanceType(I) n)
{
typedef DistanceType(I) N;
N i(0);
while (i < n / N(2)) {
cycle_2(f + i, f + ((n - N(1)) - i));
i = i + N(1);
};
}

The code does the lower bound number of assignments


If the algorithm is used with forward or bidirectional iterators, it
performs a quadratic number of iterator increments
Since all the cycles are disjoint, this code benefits from forall

Stepanov, McJones Elements of Programming March 14, 2008 438 / 880


Return value of reverse

It is tempting to define the reverse algorithms to return the range


of elements that were not moved
The middle element when the size of the range is odd
The empty range between the two “middle” elements when the
size of the range is even
However, we do not know of any example when it is useful and,
therefore, return void

Stepanov, McJones Elements of Programming March 14, 2008 439 / 880


reverse_bidirectional

template <typename I>


requires(Mutable(I) && BidirectionalIterator(I))
void reverse_bidirectional(I f, I l)
{
while (f != l && f != predecessor(l)) {
--l;
cycle_2(f, l);
++f;
}
}

Stepanov, McJones Elements of Programming March 14, 2008 440 / 880


reverse_n_bidirectional

template <typename I>


requires(Mutable(I) && BidirectionalIterator(I))
void reverse_n_bidirectional(I f, I l, DistanceType(I) n)
{
// Precondition: n 6 l − f
typedef DistanceType(I) N;
N i(0);
while (i < n / N(2)) {
--l;
cycle_2(f, l);
++f;
i = i + N(1);
}
}

Passing n < l − f effectively gives reverse_until_n

Stepanov, McJones Elements of Programming March 14, 2008 441 / 880


reverse_indexed

template <typename I>


requires(Mutable(I) && IndexedIterator(I))
void reverse_indexed(I f, I l)
{
reverse_n_indexed(f, l - f);
}

Stepanov, McJones Elements of Programming March 14, 2008 442 / 880


Intuition for divide and conquer reverse algorithm

1 Split the range into two parts


2 Reverse each part
3 Interchange the parts

Stepanov, McJones Elements of Programming March 14, 2008 443 / 880


Illustration of divide and conquer reverse algorithm

[a] b [c] d [e] f [g] [h] i [j] k [l] m [n]

[c b a] d [g f e] [j i h] k [n m l]

[g f e d c b a] [n m l k j i h]

n m l k j i h g f e d c b a

Stepanov, McJones Elements of Programming March 14, 2008 444 / 880


swap_ranges_n

template <typename I1, typename I2, typename N>


requires(Mutable(I1) && Iterator(I1) &&
Mutable(I2) && Iterator(I2) &&
ValueType(I1) == ValueType(I2) &&
Integer(N))
pair<I1, I2> swap_ranges_n(I1 f1, I2 f2, N n)
{
while (n != N(0)) {
cycle_2(f1, f2);
++f1;
++f2;
n = n - N(1);
};
return pair<I1, I2>(f1, f2);
}

Stepanov, McJones Elements of Programming March 14, 2008 445 / 880


reverse_n_recursive

template <typename I>


requires(Mutable(I) && ForwardIterator(I))
I reverse_n_recursive(I f, DistanceType(I) n)
{
typedef DistanceType(I) N;
const N h = n / N(2);
const N r = n - N(2) * h;
if (h == N(0))
return f + n;
I m = reverse_n_recursive(f, h);
m += r;
I l = reverse_n_recursive(m, h);
swap_ranges_n(f, m, h);
return l;
}

Stepanov, McJones Elements of Programming March 14, 2008 446 / 880


Correctness of reverse_n_recursive

Lemma
The reverse permutation on [0, n) is the only permutation satisfying
i < j ⇒ p(j) < p(i)

1 This condition obviously holds for ranges of size 1


2 The recursive calls inductively establish that the condition holds
within each half
3 swap_ranges_n reestablishes the condition between the halves
(and the skipped middle element, if any)

Stepanov, McJones Elements of Programming March 14, 2008 447 / 880


Complexity of reverse_n_recursive

Lemma
Pblog nc
For a range of length n = i=0 ai 2i , where ai is the ith digit in the
binary representation of n, the number of assignments equals
3 Pblog nc
2 i=0 ai i2i

Stepanov, McJones Elements of Programming March 14, 2008 448 / 880


reverse_n_forward

template <typename I>


requires(Mutable(I) && ForwardIterator(I))
void reverse_n_forward(I f, DistanceType(I) n)
{
reverse_n_recursive(f, n);
}

Stepanov, McJones Elements of Programming March 14, 2008 449 / 880


reverse_forward

template <typename I>


requires(Mutable(I) && ForwardIterator(I))
void reverse_forward(I f, I l)
{
reverse_n_forward(f, l - f);
}

Stepanov, McJones Elements of Programming March 14, 2008 450 / 880


reverse_copy_n

template <typename B, typename I>


requires(Mutable(B) && BidirectionalIterator(B) &&
Mutable(I) && Iterator(I) &&
ValueType(B) == ValueType(I))
I reverse_copy_n(B l, DistanceType(I) n, I r)
{
typedef DistanceType(I) N;
while (n != N(0)) {
--l;
sink(r) = source(l);
++r;
n = n - N(1);
}
return r;
}

Stepanov, McJones Elements of Programming March 14, 2008 451 / 880


reverse_n_with_buffer

template <typename I, typename B>


requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && BidirectionalIterator(B) &&
ValueType(I) == ValueType(B))
I reverse_n_with_buffer(I f, DistanceType(I) n, B b)
{
return reverse_copy_n(copy_n(f, n, b), n, f);
}

Stepanov, McJones Elements of Programming March 14, 2008 452 / 880


reverse_n_adaptive

template <typename I, typename B>


requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && BidirectionalIterator(B) &&
ValueType(I) == ValueType(B))
I reverse_n_adaptive(I f, DistanceType(I) n, B b, DistanceType(I) n_b)
{
typedef DistanceType(I) N;
const N h = n / N(2);
const N r = n - N(2) * h;
if (h == N(0))
return f + n;
if (n <= n_b)
return reverse_n_with_buffer(f, n, b);
I m = reverse_n_adaptive(f, h, b, n_b);
m += r;
I l = reverse_n_adaptive(m, h, b, n_b);
swap_ranges_n(f, m, h);
return l;
}

Stepanov, McJones Elements of Programming March 14, 2008 453 / 880


Complexity of reverse_n_adaptive

Exercise
Derive a formula for the number of assignments performed by
reverse_n_adaptive for given range and buffer sizes

Stepanov, McJones Elements of Programming March 14, 2008 454 / 880


Conjecture

Conjecture
There is no algorithm that reverses a forward iterator range with
polylogarithmic additional space and in linear time

Stepanov, McJones Elements of Programming March 14, 2008 455 / 880


Selecting algorithms according to requirements

It is important to be able to automatically select from a family of


algorithms based on the requirements the types satisfy
In Appendix 2 we show a mechanism called category dispatch for
doing such selection in C++
Future versions of C++ may provide a more natural way of
expressing it

Stepanov, McJones Elements of Programming March 14, 2008 456 / 880


Conclusions

Permutations lead us back to algebra, specifically group theory


A permutation of a range induces a rearrangement
A taxonomy of rearrangements was presented
Weakening iterator requirements often raises interesting
algorithmic problems
A convention known as iterator category dispatch allows
automatic selection of the appropriate algorithm
We introduced a new class of algorithms, memory-adaptive
algorithms

Stepanov, McJones Elements of Programming March 14, 2008 457 / 880


Reading

Donald Knuth.
Section 1.2.5: Permutations and Factorials.
The Art of Computer Programming, Volume 1, Addison-Wesley, 1997,
pages 45-50.

Stepanov, McJones Elements of Programming March 14, 2008 458 / 880


Contents I

1 Introduction

2 Foundations

3 Transformations and their orbits

4 Algorithms on algebraic structures

5 Orderings

6 Combining concepts

7 Refining concepts of iterators

Stepanov, McJones Elements of Programming March 14, 2008 459 / 880


Contents II
8 Permutations and rearrangements

9 Rotations
Rotation permutations
Least common multiple
Cycle structure of rotations
Interface for rotate
rotate for indexed iterators
Loop fusion
rotate for bidirectional iterators
rotate for forward iterators
Conclusions
Reading
Projects

Stepanov, McJones Elements of Programming March 14, 2008 460 / 880


Contents III
10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects

13 Iterative algorithms for divide-and-conquer

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

Stepanov, McJones Elements of Programming March 14, 2008 461 / 880


Contents IV
17 Index

Stepanov, McJones Elements of Programming March 14, 2008 462 / 880


Definition of rotation

Definition
The permutation p on a finite set with n elements defined by an index
permutation p(i) = (i + k) mod n is called the k-rotation of the set

Lemma
The inverse of a k-rotation on an n-element set is an (n − k)-rotation

Stepanov, McJones Elements of Programming March 14, 2008 463 / 880


Least common multiple

Definition
The least common multiple of integers a and b, written lcm(a, b), is the
smallest integer m such that both a and b divide it

Lemma
lcm(a, b) is a multiple of a and b that divides any other multiple of a
and b

Stepanov, McJones Elements of Programming March 14, 2008 464 / 880


Connecting lcm and gcd

Theorem
a · b = lcm(a, b) · gcd(a, b)

Proof.
Q i
If the prime decompositions of a and b are respectively pu i and
Q vi Q ui +vi Q max(ui ,vi )
pi , then a · b = p , lcm(a, b) = pi , and
Q min(ui ,vii)
gcd(a, b) = pi , and the result follows from the useful
identity x + y = max(x, y) + min(x, y) 

Stepanov, McJones Elements of Programming March 14, 2008 465 / 880


(Divisibility lattice of integers)

Definition
An ordered set E is said to be a lattice if every subset consisting of two
elements of E has a least upper bound and a greatest lower bound in E

Example
The set of integers > 1, ordered by the relation “m divides n” between
m and n, is a lattice; the least upper bound of {m, n} is lcm(m, n), and
the greatest lower bound is gcd(m, n)

The definition and example are adapted from:


N. Bourbaki.
Chapter 3, Section 1.11.
Theory of Sets, Springer, 2004, pages 145-146.

Stepanov, McJones Elements of Programming March 14, 2008 466 / 880


Cycle structure of k-rotation on n elements

An element with index i is in the cycle

{i, (i + k) mod n, (i + 2k) mod n, . . .} = {(i + uk) mod n}

The length of the cycle is the smallest positive integer m such that
i = (i + mk) mod n
This is equivalent to mk mod n = 0, which shows the length of
the cycle to be independent of i
Since m is the smallest positive number such that mk mod n = 0,
it is obvious that lcm(k, n) = mk
lcm(k,n) kn n
m= k = gcd(k,n)k = gcd(k,n)
The number of cycles, therefore, is gcd(k, n)

Stepanov, McJones Elements of Programming March 14, 2008 467 / 880


Disjoint cycles of k-rotation on n elements

Consider two different elements in a cycle, (i + uk) mod n and


(i + vk) mod n
The distance between them is
|(i+uk) mod n−(i+vk) mod n| = (u−v)k mod n = (u−v)k−pn
where p = quotient((u − v)k, n)
Since both k and n are divisible by d = gcd(k, n), so is the distance
Therefore the distance between different elements in the same
cycle is > d
Elements with indices in [0, d) belong to disjoint cycles

Stepanov, McJones Elements of Programming March 14, 2008 468 / 880


Interface for rotate

k-rotation rearrangement of a range [f, l) is equivalent to


interchanging the relative positions of [f, m) and [m, l), where
m = f + ((l − f) − k) = l − k
Experimentally, m is a more useful input than k (and when
forward or bidirectional iterators are involved, it avoids
performing linear-time operations to compute m from k)
Experimentally, returning the iterator m 0 = f + k pointing to the
new position of element at f is useful for many other algorithms30

30
Joseph Tighe suggests returning a pair, m and m 0 , in the order constituting a valid
range; while it is an interesting suggestion and preserves all the information, we do
not yet know of a compelling use of such interface
Stepanov, McJones Elements of Programming March 14, 2008 469 / 880
From-permutation for k-rotation in terms of f, m, l

To-permutation: p(i) = (i + k) mod n


From-permutation: p−1 (i) = (i + (n − k)) mod n
n=l−f
Since m goes to f, m + k = f + n ⇒ n − k = m − f
k = n − (m − f) = l − m
i < k ⇒ i + (n − k) < n ⇒ p−1 (i) = i + (n − k)
i > k ⇒ p−1 (i) = i + (n − k) − n = i − k

i + (n − k) if i < k
p−1 (i) =
i−k if i > k

Stepanov, McJones Elements of Programming March 14, 2008 470 / 880


A from-permutation for RandomAccessIterator

template <typename I>


requires(Mutable(I) && RandomAccessIterator(I))
struct k_rotate_from_permutation_random_access
{
DistanceType(I) k;
DistanceType(I) n_minus_k;
I m_prime;
k_rotate_from_permutation_random_access(I f, I m, I l) :
k(l - m), n_minus_k(m - f), m_prime(f + (l - m)) {}
I operator()(I x) {
if (x < m_prime)
return x + n_minus_k;
else
return x - k;
}
};

Stepanov, McJones Elements of Programming March 14, 2008 471 / 880


A from-permutation for IndexedIterator
The absence of < and − for indexed iterators costs us an extra
operation or two (+ and −)
On modern processors this may not cost any time
template <typename I>
requires(Mutable(I) && IndexedIterator(I))
struct k_rotate_from_permutation_indexed
{
DistanceType(I) k;
DistanceType(I) n_minus_k;
I f;
k_rotate_from_permutation_indexed(I f, I m, I l) :
k(l - m), n_minus_k(m - f), f(f) {}
I operator()(I x) {
DistanceType(I) i = x - f;
if (i < k)
return x + n_minus_k;
else
return f + (i - k);
}
};

Stepanov, McJones Elements of Programming March 14, 2008 472 / 880


rotate_indexed_helper

template <typename I, typename P>


requires(Mutable(I) && IndexedIterator(I) &&
Transformation(P) && Domain(P) == I)
I rotate_indexed_helper(I f, I m, I l, P p)
{
// Precondition: p is a from-permutation on [f, l)
typedef DistanceType(I) N;
N d = gcd(m - f, l - m);
N i = N(0);
while (i < d) {
do_cycle_from(f + i, p);
++i;
}
return f + (l - m);
};

This algorithm was first published by:


William Fletcher and Roland Silver.
Algorithm 284: Interchange of Two Blocks of Data.
CACM, Volume 9, Number 5, May 1966, page 326.
Stepanov, McJones Elements of Programming March 14, 2008 473 / 880
rotate_indexed and rotate_random_access

template <typename I> requires(Mutable(I) && IndexedIterator(I))


I rotate(I f, I m, I l, indexed_iterator_tag)
{
k_rotate_from_permutation_indexed<I> p(f, m, l);
return rotate_indexed_helper(f, m, l, p);
};

template <typename I> requires(Mutable(I) && RandomAccessIterator(I))


I rotate(I f, I m, I l, random_access_iterator_tag)
{
k_rotate_from_permutation_random_access<I> p(f, m, l);
return rotate_indexed_helper(f, m, l, p);
};

template <typename I> requires(Mutable(I) && ForwardIterator(I))


I rotate(I f, I m, I l)
{
if (m == f) return l;
if (m == l) return f;
return rotate(f, m, l, IteratorCategory(I)());
}

Stepanov, McJones Elements of Programming March 14, 2008 474 / 880


Complexity of rotate for IndexedIterator

The number of assignments is n + c − t = n + gcd(n, k)


On average, gcd  n

Stepanov, McJones Elements of Programming March 14, 2008 475 / 880


Loop fusion

Assuming uniform distribution of rotation points, the expected


value of k = n/2
That gives very bad locality of reference
Since the cycles are disjoint, we can improve locality of reference
by operating on adjacent elements in different cycles
This technique is called loop fusion, and was first discussed in this
paper:
A.P. Yershov.
ALPHA–An Automatic Programming System of High
Efficiency.
J. ACM, Volume 13, Number 1, January 1966, pages 17-24.

Stepanov, McJones Elements of Programming March 14, 2008 476 / 880


do_fused_cycles_from_with_buffer

template <typename I, typename P, typename B>


requires(Mutable(I) && IndexedIterator(I) &&
Transformation(P) && Domain(P) == I &&
Mutable(B) && IndexedIterator(B) &&
ValueType(I) == ValueType(B))
void do_fused_cycles_from_with_buffer(I i, P p, B b, DistanceType(I) n)
{
// Precondition: p is a from-permutation on the range containing i
copy_n(i, b, n);
I f = i;
I next = p(i);
while (next != i) {
copy_n(next, f, n);
f = next;
next = p(next);
}
copy_n(b, f, n);
}

Stepanov, McJones Elements of Programming March 14, 2008 477 / 880


rotate_indexed_helper_fused

template <typename I, typename P>


requires(Mutable(I) && IndexedIterator(I) &&
Transformation(P) && Domain(P) == I)
I rotate_indexed_helper_fused(I f, I m, I l, P p)
{
// Precondition: p is a from-permutation on [f, l)
typedef DistanceType(I) N;
const N fusion_factor(16);
ValueType(I) buffer[fusion_factor];
N d = gcd(m - f, l - m);
N i(0);
while (i + fusion_factor < d) {
do_fused_cycles_from_with_buffer(f + i, p, buffer, fusion_factor);
i = i + fusion_factor;
}
do_fused_cycles_from_with_buffer(f + i, p, buffer, d - i);
return f + (l - m);
};

Stepanov, McJones Elements of Programming March 14, 2008 478 / 880


Issues with fused rotate

Loop fusion is an important optimization technique that


programmers need to know
In this particular case, it is not that beneficial since often there are
not too many loops to fuse
gcd(n, k) = 1 with probability ≈ 60%
There are algorithms that do more more assignments but have
good locality of reference and work for weaker iterator
requirements

Stepanov, McJones Elements of Programming March 14, 2008 479 / 880


Connection of rotate and reverse

Lemma
The k-rotation on [0, n) is the only permutation p such that
1 i < n − k ∧ n − k 6 j < n ⇒ p(j) < p(i)
2 i < j < n − k ∨ n − k 6 i < j ⇒ p(i) < p(j)

The reverse permutation will satisfy condition 1, but not 2


Applying reverse to subranges [0, n − k) and [n − k, n) and then
applying reverse to the entire range will satisfy both conditions

Stepanov, McJones Elements of Programming March 14, 2008 480 / 880


rotate_three_reverses

This insight gives us the following:

template <typename I>


requires(Mutable(I) && BidirectionalIterator(I))
void rotate_three_reverses(I f, I m, I l)
{
reverse(f, m);
reverse(m, l);
reverse(f, l);
}

The only difficulty is finding the return value, m 0

Stepanov, McJones Elements of Programming March 14, 2008 481 / 880


reverse_until

The following auxiliary function allows us to find the return value


without doing any extra work31

template <typename I>


requires(Mutable(I) && BidirectionalIterator(I))
pair<I, I> reverse_until(I f, I m, I l)
{
while (f != m && m != l) {
--l;
cycle_2(f, l);
++f;
}
return pair<I, I>(f, l);
}

31
It was suggested to us by Raymond Lo and Wilson Ho
Stepanov, McJones Elements of Programming March 14, 2008 482 / 880
rotate for BidirectionalIterator

template <typename I>


requires(Mutable(I) && BidirectionalIterator(I))
I rotate(I f, I m, I l, bidirectional_iterator_tag)
{
reverse(f, m);
reverse(m, l);
pair<I, I> p = reverse_until(f, m, l);
reverse(p.first, p.second);
if (m == p.first)
return p.second;
else
return p.first;
}

Stepanov, McJones Elements of Programming March 14, 2008 483 / 880


Complexity of rotate for BidirectionalIterator

Lemma
The number of assignments is 3(bn/2c + bk/2c + b(n − k)/2c), which
gives us 3n when n and k are both even and 3(n − 2) in every other
case

Stepanov, McJones Elements of Programming March 14, 2008 484 / 880


swap_ranges

template <typename I1, typename I2>


requires(Mutable(I1) && Iterator(I1) &&
Mutable(I2) && Iterator(I2) &&
ValueType(I1) == ValueType(I2))
pair<I1, I2> swap_ranges(I1 f1, I1 l1, I2 f2, I2 l2)
{
while (f1 != l1 && f2 != l2) {
cycle_2(f1, f2);
++f1;
++f2;
};
return pair<I1, I2>(f1, f2);
}

For m ∈ [f, l) such that l − m = m − f, swap_ranges(f, m, m, l)


rotates [f, l) about m

Stepanov, McJones Elements of Programming March 14, 2008 485 / 880


Intuition for ForwardIterator rotate algorithm

m = m 0 swap_ranges is sufficient
m 0 < m after swap_ranges, [f, m 0 ) are in the final position; we
need to rotate [m 0 , l) around m
m < m 0 after swap_ranges, [f, m) are in the final position; we
need to rotate [m, l) around m 0
This algorithm was first published by:
David Gries and Harlan Mills.
Swapping Sections.
Technical Report 81-452, Department of Computer Science, Cornell
University, January 1981.

Stepanov, McJones Elements of Programming March 14, 2008 486 / 880


rotate_0

template <typename I>


requires(Mutable(I) && ForwardIterator(I))
void rotate_0(I f, I m, I l)
{
I c = m;
while (true) {
pair<I, I> p = swap_ranges(f, m, m, l);
if (p.first == m && p.second == l)
return;
if (p.first == m) {
m = p.second;
f = p.first;
} else {
assert(p.second == l);
f = p.first;
}
}
}

Stepanov, McJones Elements of Programming March 14, 2008 487 / 880


Annotated rotate_0
template <typename I>
requires(Mutable(I) && ForwardIterator(I))
void rotate_0_annotated(I f, I m, I l)
{
DistanceType(I) u = m - f;
DistanceType(I) v = l - m;
I c = m;
while (true) {
pair<I, I> p = swap_ranges(f, m, m, l);
if (p.first == m && p.second == l) { assert(u == v);
return;
} else if (p.first == m) { assert(v > u);
m = p.second;
f = p.first; v = v - u;
} else {
assert(p.second == l); assert(u > v);
f = p.first; u = u - v;
}
}
}

u and v compute subtractive gcd of the initial lengths


Stepanov, McJones Elements of Programming March 14, 2008 488 / 880
Complexity of rotate for ForwardIterator

Lemma
The number of assignments is 3(n − gcd(n, k))

Stepanov, McJones Elements of Programming March 14, 2008 489 / 880


rotate_unguarded

Inlining swap_ranges and small optimizations give:

template <typename I>


requires(Mutable(I) && ForwardIterator(I))
void rotate_unguarded(I f, I m, I l)
{
// Precondition: f , m ∧ m , l
I c = m;
while (c != l) {
cycle_2(f, c);
++f;
++c;
if (f == m)
m = c;
if (c == l)
c = m;
}
}

The first time the second if clause is satisfied, f equals m 0

Stepanov, McJones Elements of Programming March 14, 2008 490 / 880


rotate for ForwardIterator
template <typename I>
requires(Mutable(I) && ForwardIterator(I))
I rotate(I f, I m, I l, forward_iterator_tag)
{
// Precondition: f , m ∧ m , l
I m_prime = l;
I c = m;
while (c != l) {
cycle_2(f, c);
++f;
++c;
if (f == m)
m = c;
if (c == l) {
if (m_prime == l) m_prime = f;
c = m;
}
}
return m_prime;
}

The guards in the category dispatch rotate guarantee the assertion


Stepanov, McJones Elements of Programming March 14, 2008 491 / 880
Conclusions

Rotation rearrangements allow interchanging the order of


adjacent ranges of different sizes
Applying elementary number theory allows us to determine the
cycle structure of rotations
Interesting algorithms exist for forward, bidirectional, and
indexed iterator algorithms
Complexity is often dominated by locality of reference concerns
rather than minimizing operation counts

Stepanov, McJones Elements of Programming March 14, 2008 492 / 880


Reading

Donald Knuth.
Section 1.3.3: Permutations and Factorials.
The Art of Computer Programming, Volume 1, Addison-Wesley, 1997,
pages 164-185.
In particular, see problems 34 and 35.

Stepanov, McJones Elements of Programming March 14, 2008 493 / 880


Project 1: industrial-strength rotate

Theoretically rotate for indexed iterators is much faster than the


other two algorithms because it performs approximately 31 as
many assignments, but its locality of reference is poor
Design a benchmark comparing performance of all three
algorithms for different array sizes, element sizes, and rotation
amounts
Based on the results of the benchmark, design a composite
algorithm that appropriately uses one of the three algorithms
depending on its inputs

Stepanov, McJones Elements of Programming March 14, 2008 494 / 880


Project 2: Research in position-based rearrangements

We have presented two kinds of position-based rearrangement


algorithms: reverse and rotate
There are, however, many other examples of such algorithms; you
can find some of them in the section of Knuth given in Reading as
well as in the Collected Algorithms of the ACM (see, for example,
numbers 302, 380, and 467)
Develop a taxonomy of position-based rearrangements, discover
additional algorithms, and produce a library

Stepanov, McJones Elements of Programming March 14, 2008 495 / 880


Contents I

1 Introduction

2 Foundations

3 Transformations and their orbits

4 Algorithms on algebraic structures

5 Orderings

6 Combining concepts

7 Refining concepts of iterators

Stepanov, McJones Elements of Programming March 14, 2008 496 / 880


Contents II
8 Permutations and rearrangements

9 Rotations

10 Algorithms on increasing ranges


Increasing ranges
Partition
Searching
Copying merging and sorting
In-place merging and sorting
Adaptive merging and sorting
Conclusions
Project

11 Coordinate structures

Stepanov, McJones Elements of Programming March 14, 2008 497 / 880


Contents III
12 Composite objects

13 Iterative algorithms for divide-and-conquer

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

17 Index

Stepanov, McJones Elements of Programming March 14, 2008 498 / 880


Increasing ranges

Definition
A range [f, l) is increasing if for every non-last iterator m ∈ [f, l),
¬(source(successor(m)) < source(m))
A range [f, l) is increasing with respect to a strict weak ordering r if
for every non-last iterator m ∈ [f, l),
¬r(source(successor(m)), source(m))
A range [f, l) is key-increasing a with respect to a regular function
key and a strict weak ordering r if for every non-last iterator
m ∈ [f, l), ¬r(key(source(successor(m))), key(source(k)))
a
In database parlance, a key is a part of a record on which sorting is done

Stepanov, McJones Elements of Programming March 14, 2008 499 / 880


Defaulting the key function

Note that an increasing range is a key-increasing range with the


key function being the identity function on the value type of the
iterator

Stepanov, McJones Elements of Programming March 14, 2008 500 / 880


find_out_of_order

template <typename I, typename F, typename R>


requires(Readable(I) && ForwardIterator(I) &&
RegularFunction(F) && Domain(F) == ValueType(I) &&
StrictWeakOrdering(R) && Domain(R) == Codomain(F))
I find_out_of_order(I f, I l, F key, R r, forward_iterator_tag)
{
if (f == l) return l;
I n(successor(f));
while (n != l) {
if (r(key(source(n)), key(source(f))))
return n;
f = n;
++n;
}
return l;
}

To adapt the algorithm for Iterator, we need to copy the value


before we advance

Stepanov, McJones Elements of Programming March 14, 2008 501 / 880


find_out_of_order

template <typename I, typename F, typename R>


requires(Readable(I) && Iterator(I) &&
RegularFunction(F) && Domain(F) == ValueType(I) &&
StrictWeakOrdering(R) && Domain(R) == Codomain(F))
I find_out_of_order(I f, I l, F key, R r, iterator_tag)
{
if (f == l) return l;
I n(successor(f));
Codomain(F) v = key(source(f));
while (n != l) {
Codomain(F) u = key(source(n));
if (r(u, v))
return n;
v = u;
++n;
}
return l;
}

Stepanov, McJones Elements of Programming March 14, 2008 502 / 880


is_increasing

template <typename I, typename F, typename R>


requires(Readable(I) && Iterator(I) &&
RegularFunction(F) && Domain(F) == ValueType(I) &&
StrictWeakOrdering(R) && Domain(R) == Codomain(F))
bool is_increasing(I f, I l, F key, R r)
{
return l == find_out_of_order(f, l, key, r);
}

Stepanov, McJones Elements of Programming March 14, 2008 503 / 880


Partition
Definition
UnaryPredicate(P) ⇒
Predicate(P)
Arity(P) = 1

Definition
A range is partitioned if it is key-increasing with respect to a key
function that is a unary predicate
More generally, a range is k-partitioned if it is key-increasing with
respect to a key function that returns one of k distinct values for
some (compile-time) constant integer k > 0

There exist faster specialized versions of many algorithms dealing


with key-increasing ranges when the range of the key function is a
bounded set of integers
Stepanov, McJones Elements of Programming March 14, 2008 504 / 880
Partition points

Definition
A partition point in a range [f, l) partitioned by a unary predicate p is an
iterator i ∈ [f, l] such that ¬p(source(j)) for all j ∈ [f, i) and
p(source(k)) for all k ∈ [i, l)

Informally, a partition point is the first position where the


predicate is satisfied, if such a position exists, or the limit of the
range otherwise

Stepanov, McJones Elements of Programming March 14, 2008 505 / 880


Lower bound and upper bound

Definition
In an increasing range [f, l), every value a of the value type of the
range determines two partition points, lower bound and upper bound,
defined by, respectively:

Pa (x) ≡ x < a
Pa0 (x) ≡ ¬(a < x)

It is trivial to provide similar definitions for key-increasing


sequences and sequences with respect to a strict weak order r

Lemma
lower bound  upper bound

Stepanov, McJones Elements of Programming March 14, 2008 506 / 880


Intuition for lower bound and upper bound

Informally, a lower bound is the first position where a value equal


to a could occur in the increasing sequence
Similarly, an upper bound is the successor of the last position
where a value equal to a could occur
Therefore, elements equal to a appear only in the semi-open range
from lower bound to upper bound

A sequence with lower bound i and upper bound j for the value a
(note any of the three regions may be empty):

x0 , x1 , . . . , xi−1 , xi , . . . , xj−1 , xj , xj+1 , . . . , xn−1


| {z } | {z } | {z }
<a =a a<

Stepanov, McJones Elements of Programming March 14, 2008 507 / 880


Partition algorithms

Mathematically a partitioned range is simply a key-increasing


range
Algorithmically, it is worth providing special-case algorithms for
dealing with partitions
Testing whether a range is partitioned dereferences half as many
iterators as testing whether it is increasing
Partitioning a range of size n is linear while sorting it is n log2 n
Efficient algorithms for dealing with increasing ranges can be
composed from algorithms dealing with partitioned ranges
Quicksort is implemented using a specialized partition algorithm
Finding a lower or upper bound uses the algorithm for finding a
partition point

Stepanov, McJones Elements of Programming March 14, 2008 508 / 880


Testing whether a range is partitioned

Lemma
Given any range and a unary predicate, some prefix of the range is
partitioned by the predicate

If we can determine the longest partitioned prefix of a range, then


by testing whether the prefix equals the range we can determine
whether the range is partitioned

Stepanov, McJones Elements of Programming March 14, 2008 509 / 880


find_if and find_if_not
We need to find the first element satisfying the predicate
template <typename I, typename P>
requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I find_if(I f, I l, P p) {
while (f != l && !p(source(f)))
++f;
return f;
}

Then finding the first element not satisfying the predicate gives
the longest partitioned prefix
template <typename I, typename P>
requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I find_if_not(I f, I l, P p) {
while (f != l && p(source(f)))
++f;
return f;
}

Stepanov, McJones Elements of Programming March 14, 2008 510 / 880


is_partitioned

template <typename I, typename P>


requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
pair<I, I> find_partitioned_prefix(I f, I l, P p)
{
I i = find_if(f, l, p);
I j = find_if_not(i, l, p);
return pair<I, I>(i, j);
}

The first value returned is the partition point of the partitioned


prefix

template <typename I, typename P>


requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
bool is_partitioned(I f, I l, P p)
{
return l == find_partitioned_prefix(f, l, p).second;
}

Stepanov, McJones Elements of Programming March 14, 2008 511 / 880


Quantifier functions

template <typename I, typename P>


requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
bool all(I f, I l, P p) { return l == find_if_not(f, l, p); }

template <typename I, typename P>


requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
bool none(I f, I l, P p) { return l == find_if(f, l, p); }

template <typename I, typename P>


requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
bool not_all(I f, I l, P p) { return !all(f, l, p); }

template <typename I, typename P>


requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
bool some(I f, I l, P p) { return !none(f, l, p); }

Stepanov, McJones Elements of Programming March 14, 2008 512 / 880


is_partition_point

template <typename I, typename P>


requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
bool is_partition_point(I f, I m, I l, P p)
{
return none(f, m, p) && all(m, l, p);
}

Stepanov, McJones Elements of Programming March 14, 2008 513 / 880


Importance of finding a partition point

Finding the partition point in an already-partitioned range by


itself is not that useful
On an increasing range, it can be combined with an appropriate
predicate to implement all the varieties of binary search –
allowing us to find a value without examining every position

Stepanov, McJones Elements of Programming March 14, 2008 514 / 880


Intuition for finding a partition point

Suppose we evaluate a predicate p at some point m in a


partitioned range [f, l)
If p(m) is false, we know the partition point of [f, l) is greater than
m, and in fact is equal to the partition point of [successor(m), l)
If p(m) is true, we know the partition point is not greater than m, so
it is equal to the partition point of [f, m)
Note the partition point of [f, m) will equal m if the predicate p is
false for every iterator in [f, m)

Lemma
Choosing m in the middle of the range assures optimal worst-case
performance

If we have prior knowledge of the distribution, biasing the choice


of m might give better average performance

Stepanov, McJones Elements of Programming March 14, 2008 515 / 880


partition_point_n

template <typename I, typename P>


requires(Readable(I) && ForwardIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I partition_point_n(I f, DistanceType(I) n, P p)
{
typedef DistanceType(I) N;
while (n != N(0)) {
N h = half_nonnegative(n);
I m = f + h;
if (!p(source(m))) {
n = n - successor(h); f = successor(m);
} else
n = h;
}
return f;
}

template <typename I, typename P>


requires(Readable(I) && ForwardIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I partition_point(I f, I l, P p) { return partition_point_n(f, l - f, p); }

Stepanov, McJones Elements of Programming March 14, 2008 516 / 880


Complexity of partition_point

partition_point_n applies the predicate blog2 nc + 1 times since


the length of the range is reduced by a factor of 2 at each step
It also performs a logarithmic number of iterator/integer
additions, which for an indexed or random access iterator are
constant-time but which for a forward or bidirectional iterator
(singly- or doubly-linked list) require n iterator increments
partition_point adds another n iterator increments for forward
and bidirectional iterators
It may still be worthwhile to use it on linked lists when the
predicate application is much more expensive than an iterator
increment

Stepanov, McJones Elements of Programming March 14, 2008 517 / 880


Origin of the bisection technique

The bisection technique underlying partition point dates back at


least as far as the proof of the Intermediate Value Theorem
discovered by Bernard Bolzano32 and, independently, by
Augustin-Louis Cauchy about 4 years later33
The proof of the Bolzano-Cauchy theorem leads us to the
following interpolation algorithm34

32
Bernard Bolzano. Rein analytischer Beweis des Lehrsatzes, daß zwischen je zwey
Werthen, die ein entgegengesetztes Resultat gewähren, wenigstens eine reelle Wurzel der
Gleichung liege. 1817
33
A.-L. Cauchy. Cours D’Analyse de L’Ecole Royale Polytechnique. 1821
34
This book explores the connections between some simple algebraic concepts and
algorithms; we strongly believe it is also possible to define algorithms on analytic
concepts
Stepanov, McJones Elements of Programming March 14, 2008 518 / 880
bisection_interpolation

template <typename F>


requires(Transformation(F) && ArchimedeanOrderedField(Domain(F)))
Domain(F) bisection_interpolation(
F f, Domain(F) x0, Domain(F) x1, Domain(F) y0, Domain(F) e)
{
// Precondition: f is continuous and increasing on [x0 , x1 ]
typedef Domain(F) R;
R x;
while (true) {
x = (x0 + x1) / R(2);
R y = f(x);
if (y <= y0) {
x0 = x; if (y0 < y + e) break;
} else {
x1 = x; if (y < y0 + e) break;
}
}
return x;
// Postcondition: x is in the original interval [x0 , x1 ] and |f(x) − y0 | < 
}

Stepanov, McJones Elements of Programming March 14, 2008 519 / 880


count_if and count_if_not
template <typename I, typename P>
requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I count_if(I f, I l, P p) {
DistanceType(I) n(0);
while (f != l) {
if (p(source(f))) n = successor(n);
++f;
}
return n;
}

template <typename I, typename P>


requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I count_if_not(I f, I l, P p) {
DistanceType(I) n(0);
while (f != l) {
if (!source(f)) n = successor(n);
++f;
}
return n;
}

Stepanov, McJones Elements of Programming March 14, 2008 520 / 880


potential_partition_point

template <typename I, typename P>


requires(Readable(I) && ForwardIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I potential_partition_point(I f, I l, P p)
{
I i = f;
while (f != l) {
if (!p(source(f))) ++i;
++f;
}
return i;
// Postcondition: i = f + count_if_not(f, l, p)
}

Stepanov, McJones Elements of Programming March 14, 2008 521 / 880


Multiplicity of partitioned rearrangements

Since partition requires only that false values of the predicate


precede true values, there are u!v! ways to partition a range with u
false values and v true values
A partition algorithm would satisfy all our needs if it had these
properties:
Requires only forward iterators
Stable
In place
Linear time, with minimal number of assignments and predicate
applications
We conjecture no such algorithm exists
We present some partition algorithms to demonstrate the kind of
tradeoffs that must be made

Stepanov, McJones Elements of Programming March 14, 2008 522 / 880


partition_copy35

template <typename I, typename O0, typename O1, typename P>


requires(Readable(I) && Iterator(I) &&
Writable(O0) && Iterator(O0) &&
Writable(O1) && Iterator(O1) &&
UnaryPredicate(P) && Domain(P) == ValueType(I) &&
ValueType(I) == ValueType(O0) &&
ValueType(I) == ValueType(O1))
pair<O0, O1> partition_copy(I f, I l, O0 o0, O1 o1, P p)
{
typedef DistanceType(I) N;
while (f != l) {
if (p(source(f))) {
sink(o1) = source(f); ++o1;
} else {
sink(o0) = source(f); ++o0;
}
++f;
}
return pair<O0, O1>(o0, o1);
}

35
T.K. Lakshman suggested the interface
Stepanov, McJones Elements of Programming March 14, 2008 523 / 880
Properties of partition_copy 37

It is stable: the values of each of the two output ranges are in the
same relative order as in the input range
It is a copying rearrangement36
It is optimal in terms both of number of predicate applications and
assignments (both numbers are n, the size of the range)

36
Note that if the input range is defined by mutable forward iterators, it may be
supplied as one of the output ranges
37
A variation we use later, partition_copy_n, is given in Appendix 2
Stepanov, McJones Elements of Programming March 14, 2008 524 / 880
Semistable partition rearrangements

Definition
A partition rearrangement is semistable if the relative order of elements
not satisfying the predicate is preserved

Semistable partition can be used for efficient removal from an


extent-based sequence (such as the array of Chapter 11 or the
C++ std::vector) of elements satisfying a given predicate
First partition the sequence by the predicate
Erase the elements from the partition point to the end of the
sequence
The sequence now contains the elements not satisfying the
predicate, in their original order 38

38
In Chapter 12 we will explain why this is the right thing to do
Stepanov, McJones Elements of Programming March 14, 2008 525 / 880
semistable_partition39

template <typename I, typename P>


requires(Mutable(I) && ForwardIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I semistable_partition(I f, I l, P p)
{
I i = find_if(f, l, p); // none(f, i, p)
I j = find_if_not(i, l, p); // none(f, i, p) ∧ all(i, j, p) ∧ (j = l ∨ ¬p(j))
while (j != l) { // none(f, i, p) ∧ all(i, j, p) ∧ p(i) ∧ ¬p(j)
cycle_2(i, j); // none(f, i, p) ∧ all(i + 1, j, p) ∧ ¬p(i) ∧ p(j)
// (none(f, i, p)∧¬p(i))∧(all(i+1, j, p)∧p(j))
++i; ++j; // none(f, i, p) ∧ all(i, j, p)
j = find_if_not(j, l, p); // j = l ∨ ¬p(j)
}
return i;
}

39
The algorithm is due to Nico Lomuto, according to: Jon Bentley. Programming
Pearls. Communications of the ACM, Volume 27, Number 4, April 1984, pages 287-291
Stepanov, McJones Elements of Programming March 14, 2008 526 / 880
Useful lemmas

The annotations on the previous slide follow from some trivial but
useful lemmas

Lemma
After executing i = find_if(f, l, p), none(f, i, p) ∧ (i = l ∨ p(i))
After executing j = find_if_not(f, l, p), all(f, j, p) ∧ (j = l ∨ ¬p(j))
If p(source(i)) ∧ q(source(j)) before executing cycle_2(i, j), then
afterward q(source(i)) ∧ p(source(j))
none(f, l, p) ∧ ¬p(l) ⇒ none(f, l + 1, p)
all(f + 1, l, p) ∧ p(f) ⇒ all(f, l, p)

Stepanov, McJones Elements of Programming March 14, 2008 527 / 880


Properties of semistable_partition

It is in place
Let n = l − f be the number of elements in the range and let w be
the number of elements not satisfying the predicate following the
partitioned prefix40
The predicate is applied n times
cycle_2 is performed w times
The number of iterator increments is n + w
It is totally inappropriate for use in quicksort since it does not
divide a range of equal values at the midpoint and that leads to
quadratic complexity in many practical situations

40
w = count_if_not(find_partitioned_prefix(f, l, p), l, p)
Stepanov, McJones Elements of Programming March 14, 2008 528 / 880
semistable_partition performs too many
assignments

Consider a range t, f, f, f, f, f, f, f
semistable_partition will perform seven calls of cycle_2, while
one suffices

Stepanov, McJones Elements of Programming March 14, 2008 529 / 880


assured_find_if and assured_find_if_not
We can eliminate one of the two tests in the loop for find_if if we
are assured there is an element in the range that satisfies the
predicate
template <typename I, typename P>
requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I assured_find_if(I f, P p) {
// Let l be the end of the implied range starting with f
// Precondition: some(f, l, p)
while (!p(source(f))) ++f;
return f;
}

template <typename I, typename P>


requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I assured_find_if_not(I f, P p) {
// Let l be the end of the implied range starting with f
// Precondition: not_all(f, l, p)
while (p(source(f))) ++f;
return f;
}
Stepanov, McJones Elements of Programming March 14, 2008 530 / 880
unstable_forward_partition
template <typename I, typename P>
requires(Mutable(I) && ForwardIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I unstable_forward_partition(I f, I l, P p)
{
I i = potential_partition_point(f, l, p);
I j = find_if_not(i, l, p);
while (j != l) {
f = assured_find_if(f, p);
cycle_2(f, j); ++f; ++j;
j = find_if_not(j, l, p);
}
return i;
}

We can use assured_find_if because the number of misplaced


elements before and after the partition point are equal
The code does one extra iterator comparison that could be
eliminated by inlining both calls to find_if_not (the same
optimization applies to many following algorithms)
Stepanov, McJones Elements of Programming March 14, 2008 531 / 880
Properties of unstable_forward_partition

It is in place
The number of times cycle_2 is performed, v, equals the number
of misplaced elements not satisfying the predicate, that is, the
number of elements not satisfying the predicate in the subrange
from the potential partition point to the end of the range41
The algorithm applies the predicate 2n times, therefore it is useful
when the cost of predicate application is insignificant compared to
the cost of cycle_2
This technique of computing the sizes of a set of buckets and then
moving the elements appears in radix sort42
With a bidirectional iterator, we can avoid precomputing the
bucket sizes
41
v = count_if_not(potential_partition_point(f, l, p))
42
See the use of this technique for in-place k-partition Exercise 5.2-13 in Donald
Knuth. The Art of Computer Programming. Volume 3, Addison-Wesley, 1998, page 80;
Solution, page 618
Stepanov, McJones Elements of Programming March 14, 2008 532 / 880
find_backward_if and find_backward_if_not
To indicate “not found”, we return f, which forces us to return
i + 1 if we find a satisfying element at position i
template <typename I, typename P>
requires(Readable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I find_backward_if(I f, I l, P p) {
while (true) {
if (f == l) return f;
--l;
if (p(source(l))) return successor(l);
}
}
template <typename I, typename P>
requires(Readable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I find_backward_if_not(I f, I l, P p) {
while (true) {
if (f == l) return f;
--l;
if (!p(source(l))) return successor(l);
}
}
Stepanov, McJones Elements of Programming March 14, 2008 533 / 880
unstable_bidirectional_partition

template <typename I, typename P>


requires(Mutable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I unstable_bidirectional_partition(I f, I l, P p)
{
while (true) {
f = find_if(f, l, p);
l = find_backward_if_not(f, l, p);
if (f == l) return f;
--l;
cycle_2(f, l);
++f;
}
return f;
}

Stepanov, McJones Elements of Programming March 14, 2008 534 / 880


Properties of unstable_bidirectional_partition

It is in place
The predicate is applied n times
cycle_2 is called v times (where v is as previously defined for
unstable_forward_partition)
The algorithm moves only those elements that need to be moved,
and puts two elements in their final destination with each call to
cycle_2
The total number of assignments, therefore, is 3v

Stepanov, McJones Elements of Programming March 14, 2008 535 / 880


Minimizing the number of assignments

It is possible to accomplish partition with a different


rearrangement that has only a single cycle, resulting in 2v + 1
assignments
The idea is to:
Save the first misplaced element, creating a “hole”
Repeatedly find a misplaced element on the opposite side of the
potential partition point and move it into the hole, creating a new
hole
Move the saved element into the final hole

Stepanov, McJones Elements of Programming March 14, 2008 536 / 880


assured_find_backward_if and
assured_find_backward_if_not

template <typename I, typename P>


requires(Readable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I assured_find_backward_if(I l, P p) {
do --l; while (!p(source(l)));
return l;
}

template <typename I, typename P>


requires(Readable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I assured_find_backward_if_not(I l, P p) {
do --l; while (p(source(l)));
return l;
}

We chose to return the iterator rather than its successor as in


find_backward_if since we do not need to indicate failure

Stepanov, McJones Elements of Programming March 14, 2008 537 / 880


single_cycle_partition

template <typename I, typename P>


requires(Readable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I single_cycle_partition(I f, I l, P p)
{
f = find_if(f, l, p);
l = find_backward_if_not(f, l, p);
if (f == l) return f;
--l;
ValueType(I) tmp = source(f);
while (true) {
sink(f) = source(l);
f = find_if(successor(f), l, p);
if (f == l) {
sink(l) = tmp;
return f;
}
sink(l) = source(f);
l = assured_find_backward_if_not(l, p);
}
}

Stepanov, McJones Elements of Programming March 14, 2008 538 / 880


Properties of single_cycle_partition

It remains to be observed whether using one large cycle instead of


multiple 2-cycles is faster on modern computers with
instruction-level parallelism
The technique, however, is worth careful study

Stepanov, McJones Elements of Programming March 14, 2008 539 / 880


assured_bidirectional_partition

template <typename I, typename P>


requires(Readable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I assured_bidirectional_partition(I f, I l, P p)
{
// Precondition:
// (¬all(f, l, p) ∧ some(f, l, p)) ∨ (¬p(source(f − 1)) ∧ p(source(l)))
while (true) {
f = assured_find_if(f, p);
l = assured_find_backward_if_not(l, p);
if (successor(l) == f) return f;
cycle_2(f, l);
++f; // ¬p(source(f − 1)) ∧ p(source(l))
}
}

Stepanov, McJones Elements of Programming March 14, 2008 540 / 880


Correctness of assured_bidirectional_partition

The precondition provides the guards for assured_find_if and


assured_find_backward_if_not
After these two calls, f , l, since they point to elements with
different predicate values
l − f decreases by at least two each iteration
f and l can cross over by only one position
Suppose they had crossed by more than one, and consider any
iterator m between l and f
If ¬p(source(m)), then f would have stopped there; otherwise, l
would have stopped there
Therefore, there is no such m
If the exit is not taken, f ≺ l, and f and l point, correspondingly, to
satisfying and non-satisfying elements
The cycle_2 call moves these elements to correct positions, and
the increment of f restores the invariant
Stepanov, McJones Elements of Programming March 14, 2008 541 / 880
sentinel_partition

With a little extra work we can establish the appropriate


precondition to use an assured function43

template <typename I, typename P>


requires(Readable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I sentinel_partition(I f, I l, P p)
{
f = find_if(f, l, p);
l = find_backward_if_not(f, l, p);
if (f == l) return f;
--l;
cycle_2(f, l);
++f;
return assured_bidirectional_partition(f, l, p);
}

43
It is possible to combine the sentinel and single-cycle techniques, producing
sentinel_single_cycle_partition
Stepanov, McJones Elements of Programming March 14, 2008 542 / 880
The origin of partition algorithms

Three previous algorithms are based on a paper44 by C.A.R.


Hoare:
unstable_bidirectional_partition
single_cycle_partition
sentinel_partition

44
C.A.R. Hoare. Quicksort. The Computer Journal, Volume 5, Number 1, 1962, pages
10-16. This paper, which is a serious contender for the best-ever computer science
paper, should not be confused with his earlier publication: C.A.R. Hoare. Algorithm
63, Partition; Algorithm 64, Quicksort; Algorithm 65, Find. Communications of the
ACM, Volume 4, Number 7, July 1961, pages 321-322. No one should attempt to
implement or teach quicksort without studying this paper.
Stepanov, McJones Elements of Programming March 14, 2008 543 / 880
Regularity of predicate for partition algorithms

The UnaryPredicate concept requires that the predicate is a regular


function; this assures the same boolean value is returned every
time the predicate is applied to a particular object
This in turn assures that after the partition algorithm returns, a
call to is_partitioned(f, l, p) would return true
However the algorithms using assured finds depend on regularity
in a more fundamental way: to assure they stay within bounds of
the range
Consider, for example, using a nonregular predicate to perform a
random shuffle

Stepanov, McJones Elements of Programming March 14, 2008 544 / 880


Exercise

Exercise
Design an algorithm implementing in-place uniform random
shuffle of a range of forward iterators (hint: recursively partition
the range with a non-regular coin-tossing predicate a )
Prove that the expected number of coin tosses is n log2 n
Prove that in-place uniform random shuffle of a range of forward
iterators cannot be done in linear time
a
The algorithm was discovered by Raymond Lo and Wilson Ho

Stepanov, McJones Elements of Programming March 14, 2008 545 / 880


Intuition for stable partition

An empty range is trivially partitioned


A singleton range can be partitioned with one predicate
application and possibly one iterator increment
An arbitrary range can be partitioned with a divide-and-conquer
algorithm:
Divide the range in the middle
Stably partition each half
Combine the results by rotating the range bounded by the first and
second partition points around the middle: **** simple picture *****
This easily extends to a memory adaptive version

Stepanov, McJones Elements of Programming March 14, 2008 546 / 880


Base cases for stable partition

template <typename I, typename P>


requires(Mutable(I) && ForwardIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
pair<I, I> stable_partition_0(I f, P)
{
return pair<I, I>(f, f);
}

template <typename I, typename P>


requires(Mutable(I) && ForwardIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
pair<I, I> stable_partition_1(I f, P p)
{
I l = successor(f);
if (p(source(f)))
return pair<I, I>(f, l);
else
return pair<I, I>(l, l);
}

Stepanov, McJones Elements of Programming March 14, 2008 547 / 880


Combining operation for stable partition

template <typename I>


requires(Mutable(I) && ForwardIterator(I))
pair<I, I> stable_partition_combine(pair<I, I> r0, pair<I, I> r1)
{
I m = rotate(r0.first, r0.second, r1.first);
return pair<I, I>(m, r1.second);
}

Stepanov, McJones Elements of Programming March 14, 2008 548 / 880


stable_partition_n

template <typename I, typename P>


requires(Mutable(I) && ForwardIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
pair<I, I> stable_partition_n(I f, DistanceType(I) n, P p)
{
typedef DistanceType(I) N;
if (n == N(0))
return stable_partition_0(f, p);
if (n == N(1))
return stable_partition_1(f, p);
N h = half_nonnegative(n);
pair<I, I> r0 = stable_partition_n(f, h, p);
pair<I, I> r1 = stable_partition_n(r0.second, n - h, p);
return stable_partition_combine(r0, r1);
}

Stepanov, McJones Elements of Programming March 14, 2008 549 / 880


stable_partition_n_with_buffer

template <typename I, typename B, typename P>


requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&
UnaryPredicate(P) && Domain(P) == ValueType(I) &&
ValueType(I) == ValueType(B))
pair<I, I> stable_partition_n_with_buffer(I f, DistanceType(I) n, B b, P p)
{
pair<I, B> r = partition_copy_n(f, n, f, b, p);
return pair<I, I>(r.first, copy(b, r.second, r.first));
}

Stepanov, McJones Elements of Programming March 14, 2008 550 / 880


stable_partition_n_adaptive

template <typename I, typename B, typename P>


requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&
UnaryPredicate(P) && Domain(P) == ValueType(I) &&
ValueType(I) == ValueType(B))
pair<I, I> stable_partition_n_adaptive(
I f, DistanceType(I) n, B b, DistanceType(I) n_b, P p)
{
typedef DistanceType(I) N;
if (n == N(0))
return stable_partition_0(f, p);
if (n == N(1))
return stable_partition_1(f, p);
if (n <= n_b)
return stable_partition_n_with_buffer(f, n, b, p);
N h = half_nonnegative(n);
pair<I, I> r0 = stable_partition_n_adaptive(f, h, b, n_b, p);
pair<I, I> r1 = stable_partition_n_adaptive(r0.second, n - h, b, n_b, p);
return stable_partition_combine(r0, r1);
}

Stepanov, McJones Elements of Programming March 14, 2008 551 / 880


Properties of stable partition

Exactly n predicate applications are performed at the bottom level


of recursion
The depth of the recursion for stable_partition_n is dlog2 ne
At every recursive level we rotate n/2 elements on the average,
requiring between n/2 and 3n/2 assignments depending on the
iterator category
The total number of assignments is n log2 n/2 for random access
iterators and 3n log2 n/2 for forward and bidirectional iterators

Exercise
Determine the number of assignments for the adaptive version

Stepanov, McJones Elements of Programming March 14, 2008 552 / 880


Interfaces for binary search

What is commonly referred to as binary search is really three


related algorithms for determining the lower bound, the upper
bound, or both
The advantage of these as compared to the way binary search is
usually specified in textbooks is that the lower and upper bounds
are always defined, so there is no issue of what to return when the
element is not present
Several algorithms in this chapter will demonstrate the utility of
these interfaces

Stepanov, McJones Elements of Programming March 14, 2008 553 / 880


Implementing lower bound and upper bound

Earlier in this chapter we defined the lower bound and upper


bound for a value a as the partition points of two predicates:

Pa (x) ≡ x < a
Pa0 (x) ≡ ¬(a < x)

They can be implemented by passing functions corresponding to


Pa and Pa0 to partition_point_n
See Appendix 2 for the C++ versions

Exercise
Implement a function that returns both lower and upper bounds and
does fewer comparisons than the sum of the comparisons that would
be done by calling both lower_bound_n and upper_bound_na
a
A similar STL function is called equal_range

Stepanov, McJones Elements of Programming March 14, 2008 554 / 880


Merging

Definition
Merge is an operation that combines two increasing ranges into a single
increasing range that contains all the elements of both ranges and no
other elements

Definition
A merge is stable if the output range preserves the relative order of
equivalent elements both within each input range, and between the
first and second input range

Lemma
Stability establishes a unique ordering of the elements in the output
range

Stepanov, McJones Elements of Programming March 14, 2008 555 / 880


Merge interfaces

A mutating45 merge operation takes two adjacent increasing


ranges and rearranges the elements in the combined range into
increasing order; a range [f, l) is mergeable if there is an iterator
m ∈ [f, l) such that the subranges [f, m) and [m, l) are increasing
A copying merge operation sets a writable range to the merge of
two readable increasing ranges
A melding merge operation combines two ranges defined by node
iterators by changing their relative order46

45
A mutating merge may or may not be in place depending on how much extra
memory it uses
46
We define node iterators in the next chapter
Stepanov, McJones Elements of Programming March 14, 2008 556 / 880
is_mergeable

template <typename I, typename R>


requires(Readable(I) && ForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
bool is_mergeable(I f, I m, I l, R r)
{
return is_increasing(f, m, r) && is_increasing(m, l, r);
}

Stepanov, McJones Elements of Programming March 14, 2008 557 / 880


is_merged

Exercise
Implement this function that checks whether the range starting at o is
equal to the merge of the ranges [f0, l0) and [f1, l1)
template <typename I, typename R>
requires(Readable(I) && ForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
bool is_merged(I f0, I l0, I f1, I l1, I o, R r);

Stepanov, McJones Elements of Programming March 14, 2008 558 / 880


Algebraic properties of merge

Merge is a binary operation on ranges


An empty range is its identity element (both left and right)
Merge is associative
Its complexity needs to be taken into consideration when
reassociating
Merge is commutative when stability is not taken into account

Stepanov, McJones Elements of Programming March 14, 2008 559 / 880


Complexity property of copying merge

If ◦ denotes the merge operation and C(x, y) denotes the cost (the
number of assignments) to merge ranges x and y, then C
satisfies47 :
C(x, y) + C(x, z) + C(y, z)
C(x ◦ y, z) =
2
A consequence of this is:

C(x, y) < C(y, z) ⇒ C(x, y) + C(x ◦ y, z) < C(y, z) + C(x, y ◦ z)

In other words, it is better to do the least-costly merge first

47
In the next chapter we will define a Huffman semigroup, where the cost of the
semigroup operation satisfies this axiom
Stepanov, McJones Elements of Programming March 14, 2008 560 / 880
merge_copy_n
template <typename I0, typename I1, typename O, typename R>
requires(Readable(I0) && Iterator(I0) && Readable(I1) && Iterator(I1) &&
Writable(O) && Iterator(O) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I0) &&
ValueType(I0) == ValueType(O) && ValueType(I1) == ValueType(O))
void merge_copy_n(I0 f0, DistanceType(I0) n0,
I1 f1, DistanceType(I1) n1, O o, R r) {
// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)
// Precondition: the input ranges do not overlap the output range
typedef DistanceType(I0) N0; typedef DistanceType(I1) N1;
while (true) {
if (n0 == N0(0)) { copy_n(f1, n1, o); return; }
if (n1 == N1(0)) { copy_n(f0, n0, o); return; }
if (r(source(f1), source(f0))) {
sink(o) = source(f1); ++f1; n1 = n1 - N1(1);
} else {
sink(o) = source(f0); ++f0; n0 = n0 - N0(1);
}
++o;
}
}

The overlap precondition is not the weakest possible


Stepanov, McJones Elements of Programming March 14, 2008 561 / 880
merge_copy_backward_n
template <typename I0, typename I1, typename O, typename R>
requires(Readable(I0) && BidirectionalIterator(I0) &&
Readable(I1) && BidirectionalIterator(I1) &&
Writable(O) && BidirectionalIterator(O) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I0) &&
ValueType(I0) == ValueType(O) && ValueType(I1) == ValueType(O))
void merge_copy_backward_n(I0 l0, DistanceType(I0) n0,
I1 l1, DistanceType(I1) n1, O o, R r) {
// Let f0 = l0 − n0 and f1 = l1 − n1
// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)
typedef DistanceType(I0) N0; typedef DistanceType(I1) N1;
while (true) {
if (n0 == N0(0)) { copy_backward_n(l1, n1, o); return; }
if (n1 == N1(0)) { copy_backward_n(l0, n0, o); return; }
--o;
if (r(source(predecessor(l1)), source(predecessor(l0)))) {
--l0; sink(o) = source(l0); n0 = n0 - N0(1);
} else {
--l1; sink(o) = source(l1); n1 = n1 - N1(1);
}
}
}

Stability requires the order of the if clauses is reversed from the one in merge_copy_n
Stepanov, McJones Elements of Programming March 14, 2008 562 / 880
Return value for copying merges

Exercise
Reimplement merge_copy_n and merge_copy_backward_n with the
correct return values

Stepanov, McJones Elements of Programming March 14, 2008 563 / 880


Complexity of copying merges

Lemma
The number of assignments is always n0 + n1 and the worst-case
number of comparisons is n0 + n1 − 1

Exercise
Determine the average number of comparisons

Stepanov, McJones Elements of Programming March 14, 2008 564 / 880


merge_n_with_buffer for ForwardIterator

template <typename I, typename B, typename R>


requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&
ValueType(I) == ValueType(B))
void merge_n_with_buffer(I f0, DistanceType(I) n0,
I f1, DistanceType(I) n1, B b, R r,
forward_iterator_tag)
{
// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)
// Precondition: the range beginning with b is of size at least min(n0, n1)
if (n0 <= n1) {
copy_n(f0, n0, b);
merge_copy_n(b, n0, f1, n1, f0, r);
} else {
f1 = rotate(f0, f1, f1 + n1);
copy_n(f0, n1, b);
merge_copy_n(f1, n0, b, n1, f0, r);
}
}

Stepanov, McJones Elements of Programming March 14, 2008 565 / 880


merge_n_with_buffer for BidirectionalIterator

template <typename I, typename B, typename R>


requires(Mutable(I) && BidirectionalIterator(I) &&
Mutable(B) && BidirectionalIterator(B) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&
ValueType(I) == ValueType(B))
void merge_n_with_buffer(I f0, DistanceType(I) n0,
I f1, DistanceType(I) n1, B b, R r,
bidirectional_iterator_tag)
{
// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)
// Precondition: the range beginning with b is of size at least min(n0, n1)
if (n0 <= n1) {
copy_n(f0, n0, b);
merge_copy_n(b, n0, f1, n1, f0, r);
} else {
pair<I, B> tmp = copy_n(f1, n1, b);
merge_copy_backward_n(f1, n0, tmp.second, n1, tmp.first, r);
}
}

Stepanov, McJones Elements of Programming March 14, 2008 566 / 880


Concept Merger
Definition
Merger(M) ⇒
The type function IteratorT ype(M) is defined
Procedure(M)
For all m ∈ M, I = IteratorT ype(M), and N = DistanceT ype(I),
m : I × N × I × N → void
There is an implicit function ordering such that for all m ∈ M,
ordering(m) is a strict weak ordering on the value type of the
iterator type of M
For all m ∈ M, a call m(f0, n0, f1, n1) satisfying the precondition

f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, ordering(m))

will establish the postcondition

is_increasing(f0, f0 + n0 + n1, ordering(m))


Stepanov, McJones Elements of Programming March 14, 2008 567 / 880
merger_with_buffer

template <typename I, typename B, typename R>


requires(Mutable(I) && BidirectionalIterator(I) &&
Mutable(B) && BidirectionalIterator(B) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&
ValueType(I) == ValueType(B))
struct merger_with_buffer
{
B b;
R r;
merger_with_buffer(B b, R r) : b(b), r(r) { }
void operator()(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1)
{
// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)
// Precondition: the range beginning with b is of size at least min(n0, n1)
merge_n_with_buffer(f0, n0, f1, n1, b, r);
}
};

Stepanov, McJones Elements of Programming March 14, 2008 568 / 880


Sorting algorithms

Definition
A key-sorting (or just sorting) algorithm is a rearrangement those
postcondition is that the range is key-increasing (or just
increasing) with respect to a given strict weak order
A key-sorting (or sorting) algorithm is stable if it does not change
the relative order of values that are equivalent under the
symmetric complement of the given strict weak order

While the key function is a useful feature, including it in the


sorting functions in this chapter is not instructive since
transforming a function dealing with an increasing sequence into
one with a key-increasing sequence is straightforward

Stepanov, McJones Elements of Programming March 14, 2008 569 / 880


merge_sort_n

template <typename I, typename M>


requires(Mutable(I) && ForwardIterator(I) &&
Merger(M) && IteratorType(M) == I)
I merge_sort_n(I f, DistanceType(I) n, M merger)
{
typedef DistanceType(I) N;
if (n < N(2)) return f + n;
N h = half_nonnegative(n);
I m = merge_sort_n(f, h, merger);
I l = merge_sort_n(m, n - h, merger);
merger(f, h, m, n - h);
return l;
}

Stepanov, McJones Elements of Programming March 14, 2008 570 / 880


stable_sort_n_with_buffer

template <typename I, typename B, typename R>


requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&
ValueType(I) == ValueType(B))
I stable_sort_n_with_buffer(I f, DistanceType(I) n, B b, R r)
{
// Precondition: the range beginning with b is of size at least n/2
return merge_sort_n(f, n, merger_with_buffer<I, B, R>(b, r));
}

Note that h 6 n − h always holds, so the call on rotate in the


ForwardIterator version of merge_n_with_buffer will never be
executed

Stepanov, McJones Elements of Programming March 14, 2008 571 / 880


Complexity of stable_sort_n_with_buffer

The algorithm has dlog2 ne recursive levels


Each level performs 3n/2 assignments, for a total number of
assignments of 32 ndlog2 ne
At ith level from the bottom, the worst-case number of
comparisons is n − 2ni ; summing over all levels gives

dlog2 ne
X n
ndlog2 ne − ≈ ndlog2 ne − n
2i
i=1

Stepanov, McJones Elements of Programming March 14, 2008 572 / 880


Intuition for stable in-place merge
Theorem
If is_mergeable(f, m, l, r), then for every iterator i ∈ [f, m), there is a
unique j ∈ [m, l) such that after x ← source(i) and
m 0 ← rotate(i, m, j),
[f, l) is stably partitioned, with [f, m 0 ) containing elements less
than or equal to x and [m 0 , l) containing elements greater than or
equal to x
is_mergeable(f, i, m 0 , r) ∧ is_mergeable(m 0 , j, l, r)

Proof.
Consider j = lower_bound(m, l, source(i), r) 

An analogous result holds for every j ∈ [m, l): let


i = upper_bound(f, m, source(j), r), and perform
m 0 ← rotate(i, m, successor(j))
Stepanov, McJones Elements of Programming March 14, 2008 573 / 880
A sketch of an algorithm for stable in-place merge

This suggests a divide-and-conquer algorithm: choose an i in


[f, m) and the corresponding j in [m, l), or a j in [m, l) and a
corresponding i in [f, m), rotate appropriately, and then merge
each of the mergeable subranges
Now since j − m 0 = m − i, choosing i halfway between f and m or
j halfway between m and l will cut the minimum of m − f and
l − m in half
These considerations lead to the following algorithm

Stepanov, McJones Elements of Programming March 14, 2008 574 / 880


merge_n_in_place
template <typename I, typename R>
requires(Mutable(I) && ForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
void merge_n_in_place(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1, R r) {
// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)
typedef DistanceType(I) N;
if (min(n0, n1) == N(0)) return;
I m0, m1;
N h0, h1;
if (n0 < n1) {
h0 = half_nonnegative(n0); m0 = f0 + h0;
m1 = lower_bound_n(f1, n1, source(m0), r); h1 = m1 - f1;
I m = rotate(m0, f1, m1);
merge_n_in_place(f0, h0, m0, h1, r);
merge_n_in_place(successor(m), n0 - successor(h0), m1, n1 - h1, r);
} else {
h1 = half_nonnegative(n1); m1 = f1 + h1;
m0 = upper_bound_n(f0, n0, source(m1), r); h0 = m0 - f0;
I m = rotate(m0, f1, successor(m1));
merge_n_in_place(f0, h0, m0, h1, r);
merge_n_in_place(m, n0 - h0, successor(m1), n1 - successor(h1), r);
}
}

Stepanov, McJones Elements of Programming March 14, 2008 575 / 880


Properties of merge_n_in_place

Lemma
1 Each call of rotate puts one element into its final position
2 The algorithm terminates with a sorted range
3 Each level of recursion reduces m = min(n0, n1) to no more than
bm/2c
4 There are at most blog2 (min(n0, n1))c + 1 recursive levels

The algorithm is due to Dudziński and Dydek, whose paper48


also contains a careful complexity analysis

48
Krzysztof Dudziński and Andrzej Dydek. On a Stable Minimum Storage Merging
Algorithm. Information Processing Letters, Volume 12, Number 1, February 1981, pages
5-8.
Stepanov, McJones Elements of Programming March 14, 2008 576 / 880
Complexity of merge_n_in_place: number of
assignments

Theorem
Let
kn be the maximum number of assignments required to rotate a
range of length n
a(n0 , n1 ) be the maximum number of assignments required for a
stable in-place merge of two ranges with lengths n0 and n1 , where
n0 6 n1
Then
a(n0 , n1 ) 6 k(n1 + dn0 /2e) log2 (2n0 ) 6 kn log2 n

k may be taken to be 3/2 for indexed iterators and 3 for forward


(and bidirectional) iterators

Stepanov, McJones Elements of Programming March 14, 2008 577 / 880


Number of assignments of merge_n_in_place49

Proof.
By induction on n0 .
n0 = 1 A single rotation of a range of length at most n1 + 1 accomplishes the merge, so
a(1, n1 ) = r(n1 + 1) 6 k(n1 + 1) = k(n1 + d1/2e) log2 (2).
n0 > 1 We have a(n0 , n1 ) 6 r(n1 + n0 − h0 ) + a(h0 , h1 ) + a(n0 − h0 − 1, n1 − h1 ). By
the induction hypothesis, a(h0 , h1 ) 6 k(h1 + dh0 /2e) log2 (2h0 ) and

a(n0 − h0 − 1, n1 − h1 ) 6 k(n1 − h1 + d(n0 − h0 − 1)/2e) log2 (2(n0 − h0 − 1)).

It is easily verified that dx/2e + dy/2e 6 d(x + y + 1)/2e for any x and y. Thus

dh0 /2e + d(n0 − h0 − 1)/2e 6 dn0 /2e

and

a(n0 , n1 ) 6 k(n1 + dn0 /2e)(log2 (n0 ) + 1) = k(n1 + dn0 /2e)(log2 (2n0 )).


49
We are indebted to John Wilkinson for this proof
Stepanov, McJones Elements of Programming March 14, 2008 578 / 880
Complexity of merge_n_in_place: number of
comparisons

Theorem
Let c(n0 , n1 ) be the maximum number of comparisons required for a
stable in-place merge of a range of length n0 with a range of length n1 ,
where n0 6 n1 ; then c(n0 , n1 ) 6 kn1

Exercise
Determine the bound k and prove the theorem

Stepanov, McJones Elements of Programming March 14, 2008 579 / 880


merger_in_place

template <typename I, typename R>


requires(Mutable(I) && BidirectionalIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
struct merger_in_place
{
R r;
merger_in_place(R r) : r(r) { }
void operator()(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1)
{
// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)
merge_n_in_place(f0, n0, f1, n1, r);
}
};

Stepanov, McJones Elements of Programming March 14, 2008 580 / 880


stable_sort_n_in_place

template <typename I, typename R>


requires(Mutable(I) && ForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
I stable_sort_n_in_place(I f, DistanceType(I) n, R r)
{
return merge_sort_n(f, n, merger_in_place<I, R>(r));
}

Stepanov, McJones Elements of Programming March 14, 2008 581 / 880


merge_n
template <typename I, typename B, typename R>
requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&
ValueType(I) == ValueType(B))
void merge_n(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1,
B b, DistanceType(I) n_b, R r) {
// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)
if (min(n0, n1) <= n_b) { merge_n_with_buffer(f0, n0, f1, n1, b, r); return; }
I m0, m1; DistanceType(I) h0, h1;
if (n0 < n1) {
h0 = half_nonnegative(n0); m0 = f0 + h0;
m1 = lower_bound_n(f1, n1, source(m0), r); h1 = m1 - f1;
I m = rotate(m0, f1, m1);
merge_n(f0, h0, m0, h1, b, n_b, r);
merge_n(successor(m), n0 - successor(h0), m1, n1 - h1, b, n_b, r);
} else {
h1 = half_nonnegative(n1); m1 = f1 + h1;
m0 = upper_bound_n(f0, n0, source(m1), r); h0 = m0 - f0;
I m = rotate(m0, f1, successor(m1));
merge_n(f0, h0, m0, h1, b, n_b, r);
merge_n(m, n0 - h0, successor(m1), n1 - successor(h1), b, n_b, r);
}
}
Stepanov, McJones Elements of Programming March 14, 2008 582 / 880
merger

template <typename I, typename B, typename R>


requires(Mutable(I) && BidirectionalIterator(I) &&
Mutable(B) && BidirectionalIterator(B) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&
ValueType(I) == ValueType(B))
struct merger_adaptive
{
typedef DistanceType(B) N;
B b;
N n_b;
R r;
merger_adaptive(B b, N n_b, R r) : b(b), n_b(n_b), r(r) { }
void operator()(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1)
{
// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)
// Precondition: the range beginning with b is of size at least min(n0, n1)
merge_n(f0, n0, f1, n1, b, n_b, r);
}
};

Stepanov, McJones Elements of Programming March 14, 2008 583 / 880


stable_sort_n

template <typename I, typename B, typename R>


requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&
ValueType(I) == ValueType(B))
I stable_sort_n(I f, DistanceType(I) n, B b, DistanceType(I) n_b, R r)
{
return merge_sort_n(f, n, merger_adaptive<I, B, R>(b, n_b, r));
}

Stepanov, McJones Elements of Programming March 14, 2008 584 / 880


Importance of stable in-place and adaptive merge and
sort

Stable in-place sort is of theoretical interest50 ; in practice, it should


never be used
Its simple extension to the memory-adaptive algorithm leads to a
good practical algorithm when a buffer of as low as 1% is available
and becomes extremely competitive with 10-25%

50
Donald Knuth ranked this problem as 47 out of 50 – just a notch below the Last
Fermat Theorem – in the first edition Volume 3 of his Art of Computer Programming
Stepanov, McJones Elements of Programming March 14, 2008 585 / 880
The origin of computer sorting and binary searching

The first description of binary searching, binary insertion sort, and


copying merge sort was John W. Mauchly’s 1946 lecture “Sorting
and Collating”51

51
Martin Campbell-Kelly and Michael R. Williams, editors. The Moore School
Lectures: Theory and Techniques for Design of Electronic Digital Computers. The
MIT Press and Tomash Publishers, 1985.
Stepanov, McJones Elements of Programming March 14, 2008 586 / 880
Conclusions
Mathematicians use definitions and theorems as building blocks
for ever more powerful results
This chapter shows how a difficult problem of stable sorting in
minimal space can be effectively solved using simple building
blocks
Copying (forward and backward)
Rotate (for various iterator categories)
Lower and upper bound
It provides specific examples of why our attention to interfaces
(arguments and return values) is important
It shows that the technique of memory adaptive algorithms,
which we first introduced in a rather artifical context (reverse for
forward iterators), leads to a simple, efficient version of an
important algorithm (stable sort)
Lower and upper bound, merge, and sort allow sequences to be
used efficiently as representations of sets and multisets
Stepanov, McJones Elements of Programming March 14, 2008 587 / 880
Project

Project
Design a benchmark comparing performance of
unstable_bidirectional_partition, single_cycle_partition,
sentinel_partition, and sentinel_single_cycle_partition for
different array sizes, element sizes, and distributions of elements
satisfying the predicate
Design a benchmark to analyze the performance of stable_sort_n
for different array sizes, element sizes, and buffer sizes

Stepanov, McJones Elements of Programming March 14, 2008 588 / 880


Contents I

1 Introduction

2 Foundations

3 Transformations and their orbits

4 Algorithms on algebraic structures

5 Orderings

6 Combining concepts

7 Refining concepts of iterators

Stepanov, McJones Elements of Programming March 14, 2008 589 / 880


Contents II
8 Permutations and rearrangements

9 Rotations

10 Algorithms on increasing ranges

11 Coordinate structures
Coordinate structures
Linkable iterators
Linking operations
Link rearrangement
Reversing a linkable range
Examples of linkable iterators
Generalized rearrangements
Partition

Stepanov, McJones Elements of Programming March 14, 2008 590 / 880


Contents III
Merging
Sorting
Temporarily breaking invariants
Bifurcate coordinates
Traversing trees
Similarity, equivalence, and ordering
Conclusion
Projects

12 Composite objects

13 Iterative algorithms for divide-and-conquer

14 Mathematical notation

Stepanov, McJones Elements of Programming March 14, 2008 591 / 880


Contents IV
15 C++ machinery

16 Acknowledgments

17 Index

Stepanov, McJones Elements of Programming March 14, 2008 592 / 880


Intuition for coordinate structures

In Chapter 6 we defined a family of iterator concepts that


generalize pointers and describe linear traversals within any
type-homogenous data structure
To allow traversal within arbitrary data structures, we generalize
iterators to a family of concepts called coordinate structures
While an iterator type has only linear traversals (e.g., i 7→ i + 1 or
i 7→ i + n), and a single static value type, a coordinate structure
may have several interrelated coordinate types, each with its own
value type and diverse traversal functions
Coordinate structures abstract the navigational aspects of data
structures, while containers (introduced in the next chapter)
abstract storage management and ownership
There could be multiple coordinate structures describing the same
set of objects

Stepanov, McJones Elements of Programming March 14, 2008 593 / 880


Definition of coordinate structure

Definition
A concept is a coordinate structure if it consists of:
One or more coordinate types
For each coordinate type Ti , a value type ValueT ype(Ti )
For each coordinate type, dereferencing functions source and/or
sink providing “fast” readable, writable, or mutable access
Access via a coordinate value is as fast as via any other mechanism,
so there is no need to “cache” a low-level pointer or reference
For each coordinate type Ti , one or more traversal functions:

fi,j : Ti × · · · → Tk

Stepanov, McJones Elements of Programming March 14, 2008 594 / 880


Reflections on coordinate structure

Note that “coordinate structure” is not a concept but a family of


related concepts
It is not a concept because it does not have specific coordinate types
or traversal functions
No algorithms can be defined on coordinate structure as such;
meta-algorithms are possible but are outside the scope of this book

Stepanov, McJones Elements of Programming March 14, 2008 595 / 880


Taxonomy of coordinate structures
Related types Most coordinate structures have a single
coordinate type; a few have two or more
Dereference regularity The dereferencing functions on a coordinate
type may be regular (repeatable) or may be
weak, meaning a protocol must be followed
between dereferences
Topology The traversal functions may allow linear,
bifurcating, multi-dimensional, or graph
traversal
Traversal regularity The traversal functions may be regular
(supporting multi-pass algorithms), or weak
(supporting only single-pass algorithms)
Complexity The traversal functions may have linear,
logarithmic, or amortized constant-time
complexity
Changeable topology The topology between coordinate values may
be fixed or mutable
Stepanov, McJones Elements of Programming March 14, 2008 596 / 880
Plan for the chapter

Iterators are the simplest and most useful coordinate structures;


they are concepts of linear fixed-topology coordinates
This chapter introduces coordinate structures illustrating more
dimensions in our taxonomy:
Linkable iterators: changeable topology, with different algorithms
for partition, merging, and sorting
Bifurcated coordinates: nonlinear topologies, with algorithms on
binary trees
Linkable bifurcated coordinates: algorithms for iterative,
constant-space binary tree traversal

Stepanov, McJones Elements of Programming March 14, 2008 597 / 880


Linkable iterators

A linkable iterator does not have a fixed successor or predecessor;


these relationships between linkable iterators are mutable
A linkable range is a range defined by a pair of linkable iterators
[f, l)
The mutability of the relationships between linkable iterators
requires care to preserve the invariants of ranges:
Iterators in a range are totally ordered by ≺
Multiple input or output ranges of an algorithm are disjoint
The iterators in the input to an algorithm are preserved in its output

Stepanov, McJones Elements of Programming March 14, 2008 598 / 880


Linking operations for linkable iterators

The iterator concepts require the unique traversal operations


successor and predecessor for a given type
We treat the linking operation differently, passing them as
parameters, because of different ways linking is performed on the
same type
Similarly, in chapter 3 the operation was an explicit parameter to
power
To define the requirements on the type of objects performing
linking operations, we define related concepts ForwardLinker,
BackwardLinker, and BidirectionalLinker

Stepanov, McJones Elements of Programming March 14, 2008 599 / 880


Concept ForwardLinker

Definition
ForwardLinker(S) ⇒
A type function IteratorT ype(S) is defined
ForwardIterator(IteratorT ype(S))
An object of type S is a proper mutator taking two iterators of the
affiliated type
For all set_link ∈ S, immediately after set_link(i, j) is performed,
successor(i) = j

Stepanov, McJones Elements of Programming March 14, 2008 600 / 880


Concept BackwardLinker

Definition
BackwardLinker(S) ⇒
A type function IteratorT ype(S) is defined
BidirectionalIterator(IteratorT ype(S))
An object of type S is a proper mutator taking two iterators of the
affiliated type
For all set_link ∈ S, immediately after set_link(i, j) is performed,
i = predecessor(j)

Stepanov, McJones Elements of Programming March 14, 2008 601 / 880


Concept BidirectionalLinker

Definition
BidirectionalLinker(S) ⇒
ForwardLinker(S)
BackwardLinker(S)

Stepanov, McJones Elements of Programming March 14, 2008 602 / 880


Link rearrangements

A link rearrangement52 is an algorithm taking a linked range and


producing a linkable range, with the properties:
Precondition (The input range is well-formed, i.e., doesn’t
contain a cycle)

Postcondition (The output range is well-formed)


Every iterator in the input range appears in the
output range
Every iterator in the output range appeared in the
input range
Every iterator in the output range designates the
same object as before the rearrangement, and this
object has the same value
successor and predecessor relationships that held
in the input range may or may not hold in the
output range
52
The rearrangements introduced in Chapter 7 rearrange data; link rearrangements
rearrange links
Stepanov, McJones Elements of Programming March 14, 2008 603 / 880
reverse_linkable

template <typename S>


requires(ForwardLinker(S))
IteratorType(S) reverse_linkable(IteratorType(S) f, IteratorType(S) l,
S set_link)
{
typedef IteratorType(S) I;
I r = l;
while (f != l) {
I tmp = successor(f);
set_link(f, r);
r = f;
f = tmp;
}
return r;
}

Stepanov, McJones Elements of Programming March 14, 2008 604 / 880


Concept LinkableForwardIterator

Definition
LinkableForwardIterator(I) ⇒
ForwardIterator(I)
For all i ∈ I,
source(i.p).forward_link = successor(i)
For all j ∈ I, immediately after sink(i.p).forward_link ← j,
successor(i) = j

Stepanov, McJones Elements of Programming March 14, 2008 605 / 880


Concept LinkableBidirectionalIterator

Definition
LinkableBidirectionalIterator(I) ⇒
BidirectionalIterator(I) ∧ LinkableForwardIterator(I)
For all j ∈ I,
source(j.p).backward_link = predecessor(j)
For all i ∈ I, immediately after sink(j.p).backward_link ← i,
i = predecessor(j)

Stepanov, McJones Elements of Programming March 14, 2008 606 / 880


forward_linker, backward_linker and
bidirectional_linker
template <typename I> requires(LinkableForwardIterator(I))
struct forward_linker
{
void operator()(I x, I y)
{ sink(x.p).forward_link = y.p; }
};

template <typename I> requires(LinkableBidirectionalIterator(I))


struct backward_linker
{
void operator()(I x, I y)
{ sink(y.p).backward_link = x.p; }
};

template <typename I> requires(LinkableBidirectionalIterator(I))


struct bidirectional_linker
{
void operator()(I x, I y)
{ sink(x.p).forward_link = y.p; sink(y.p).backward_link = x.p; }
};

Stepanov, McJones Elements of Programming March 14, 2008 607 / 880


reverse_forward_linkable and
reverse_bidirectional_linkable

template <typename I>


requires(LinkableForwardIterator(I))
I reverse_forward_linkable(I f, I l)
{
return reverse_linkable(f, l, l, forward_linker<I>());
}

template <typename I>


requires(LinkableBidirectionalIterator(I))
I reverse_bidirectional_linkable(I f, I l)
{
return reverse_linkable(f, l, l, bidirectional_linker<I>());
}

Stepanov, McJones Elements of Programming March 14, 2008 608 / 880


Generalized rearrangements

A generalized rearrangement is an algorithm taking k > 1 ranges


and producing l > 1 ranges
Copying merge takes two input ranges and produces one output
range
In-place partition may be viewed as taking one input range and
producing two output ranges
In-place rotate may be viewed as taking two input ranges ([f, m)
and [m, l)) and producing two output ranges ([f, m 0 ) and [m 0 , l))

Stepanov, McJones Elements of Programming March 14, 2008 609 / 880


Generalized link rearrangements

A generalized link rearrangement has additional properties:


Precondition The input ranges must be disjoint

Postcondition Every iterator in an input range appears in one of


the output ranges
Every iterator in an output range appeared in one
of the input ranges
Every iterator in the output range designates the
same object as before the rearrangement, and this
object has the same value
successor and predecessor relationships that held
in the input range may or may not hold in the
output range
The output ranges are disjoint

Stepanov, McJones Elements of Programming March 14, 2008 610 / 880


partition_linkable_unoptimized
template <typename S, typename P>
requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
UnaryPredicate(P) && Domain(P) == ValueType(IteratorType(S)))
pair<pair<IteratorType(S), IteratorType(S)>,
pair<IteratorType(S), IteratorType(S)> >
partition_linkable_unoptimized(IteratorType(S) f, IteratorType(S) l, P p,
S set_link)
{
typedef IteratorType(S) I;
I h0 = l; I t0 = l;
I h1 = l; I t1 = l;
while (f != l) {
if (p(source(f))) {
if (h1 == l) { h1 = f; t1 = f; } else { set_link(t1, f); t1 = f; }
} else {
if (h0 == l) { h0 = f; t1 = f; } else { set_link(t0, f); t0 = f; }
}
++f;
}
return pair<pair<I, I>, pair<I, I> >
(pair<I, I>(h0, t0), pair<I, I>(h1, t1));
}

Compare with partition_copy_n in Chapter 9


Stepanov, McJones Elements of Programming March 14, 2008 611 / 880
Improving on partition_linkable_unoptimized

By unrolling the loop, the inner tests on h0 and h1 can be avoided


after the not-satisfying and satisfying objects are found
By maintaining a state indicating whether the predicate was
satisfied for the previous element, set_link calls that do not
change the successor can be avoided
The state can be represented by the location in the code rather
than by an explicit state variable; using goto avoids setting and
testing a state variable53

53
Undisciplined use of goto leads to code that is harder to understand and less
likely to be correct; disciplined use frequently improves clarity and efficiency.
Benchmarking is required to verify that versions with goto are faster on processors
with instruction level parallelism.
Stepanov, McJones Elements of Programming March 14, 2008 612 / 880
partition_linkable signature

template <typename S, typename P>


requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
UnaryPredicate(P) && Domain(P) == ValueType(IteratorType(S)))
pair<pair<IteratorType(S), IteratorType(S)>,
pair<IteratorType(S), IteratorType(S)> >
partition_linkable(IteratorType(S) f, IteratorType(S) l, P p, S set_link)
{
typedef IteratorType(S) I;

Stepanov, McJones Elements of Programming March 14, 2008 613 / 880


partition_linkable body
entry: I h0 = l; I t0 = l; I h1 = l; I t1 = l;
if (f == l) { goto exit; }
if (p(source(f))) { h1 = f; goto s1; }
else { h0 = f; goto s0; }
s0: t0 = f; ++f;
if (f == l) { goto exit; }
if (p(source(f))) { h1 = f; goto s3; }
else { goto s0; }
s1: t1 = f; ++f;
if (f == l) { goto exit; }
if (p(source(f))) { goto s1; }
else { h0 = f; goto s2; }
s2: t0 = f; ++f;
if (f == l) { goto exit; }
if (p(source(f))) { set_link(t1, f); goto s3; }
else { goto s2; }
s3: t1 = f; ++f;
if (f == l) { goto exit; }
if (p(source(f))) { goto s3; }
else { set_link(t0, f); goto s2; }
exit: return pair<pair<I, I>, pair<I, I> >
(pair<I, I>(h0, t0), pair<I, I>(h1, t1));
}

Stepanov, McJones Elements of Programming March 14, 2008 614 / 880


merge_linkable_nonempty

template <typename S, typename R>


requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))
triple<IteratorType(S), IteratorType(S), IteratorType(S)>
merge_linkable_nonempty(IteratorType(S) f0, IteratorType(S) l0,
IteratorType(S) f1, IteratorType(S) l1, R r, S set_link)
{
typedef IteratorType(S) I;
typedef triple<I, I, I> Triple;
entry: I h, t;
if (r(source(f1), source(f0))) { h = f1; goto s1; }
else { h = f0; goto s0; }
s0: t = f0; ++f0;
if (f0 == l0) { set_link(t, f1); return Triple(h, t, l1); }
if (r(source(f1), source(f0))) { set_link(t, f1); goto s1; }
else { goto s0; }
s1: t = f1; ++f1;
if (f1 == l1) { set_link(t, f0); return Triple(h, t, l0); }
if (r(source(f1), source(f0))) { goto s1; }
else { set_link(t, f0); goto s0; }
}

Stepanov, McJones Elements of Programming March 14, 2008 615 / 880


Interface of merge_linkable_nonempty

merge_linkable_nonempty takes two linkable ranges and


returns the beginning and end of the merged linkable range as the
first and third elements of a triple
It returns the last-visited iterator in the output range as the second
element of the triple; this iterator could be used with the following
procedure to link the range to another linkable range

template <typename I>


requires(LinkableForwardIterator(I))
void link_at_end(I f0, I l0, I f1)
{
// Precondition: f0 , l0
I next = successor(f0);
while (next != l0) {
f0 = next;
++next;
}
set_successor(f0, f1);
}

Stepanov, McJones Elements of Programming March 14, 2008 616 / 880


Properties of partition_linkable and
merge_linkable_nonempty

partition_linkable and merge_linkable_nonempty are stable:


every set_link call links to an iterator occurring not earlier in the
original list(s)

Stepanov, McJones Elements of Programming March 14, 2008 617 / 880


sort_linkable_n
template <typename S, typename R>
requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))
pair<IteratorType(S), IteratorType(S)>
sort_linkable_n(IteratorType(S) f, DistanceType(IteratorType(S)) n,
R r, S set_link)
{
// Precondition: n > 0
typedef IteratorType(S) I;
typedef triple<I, I, I> Triple;
typedef pair<I, I> Pair;
if (n == 1) return Pair(f, successor(f));
Pair p1 = sort_linkable_n(f, half_nonnegative(n), r, set_link);
Pair p2 = sort_linkable_n(p1.second, n - half_nonnegative(n), r, set_link);
Triple t = merge_linkable_nonempty(p1.first, p1.second,
p2.first, p2.second, r, set_link);
link_at_end(t.second, t.third, p2.second);
return Pair(t.first, p2.second);
}

While the number of operations performed is close to optimal,


poor locality of reference limits its usefulness if the linked
structure does not fit into cache memory
Stepanov, McJones Elements of Programming March 14, 2008 618 / 880
Exercise

Exercise
sort_linkable_n and merge_sort_n from Chapter 9 are very similar;
figure out how to unify them

Stepanov, McJones Elements of Programming March 14, 2008 619 / 880


sort_linkable_recursive

template <typename S, typename R>


requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))
IteratorType(S) sort_linkable_recursive(IteratorType(S) f, IteratorType(S) l,
R r, S set_link)
{
if (f == l) return f;
else return sort_linkable_n(f, l - f, r, set_link).first;
}

Stepanov, McJones Elements of Programming March 14, 2008 620 / 880


sort_forward_linkable_recursive

template <typename I, typename R>


requires(Readable(I) && LinkableForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
I sort_forward_linkable_recursive(I f, I l, R r)
{
return sort_linkable_recursive(f, l, r, forward_linker<I>());
}

Stepanov, McJones Elements of Programming March 14, 2008 621 / 880


Temporarily breaking invariants

A nontrivial invariant involving more than one object will be


temporarily violated during an update of those variables
For example, there is a point during the execution of swap(x, y)
when both x and y are copies of the same value
Most linked rearrangements do not depend on predecessor links
Maintaining the invariant i = predecessor(successor(i)) requires
extra time; for sorting this requires n log2 n extra time
Parameterizing the ForwardLinker used by a linked rearrangement
allows the caller to control when to restore the predecessor
invariant

Stepanov, McJones Elements of Programming March 14, 2008 622 / 880


restore_backward_links

template <typename S>


requires(BackwardLinker(S))
void restore_backward_links(IteratorType(S) f, IteratorType(S) l, S set_link)
{
while (f != l) {
set_link(f, successor(f));
++f;
};
}

Stepanov, McJones Elements of Programming March 14, 2008 623 / 880


sort_bidirectional_linkable_recursive

template <int k, typename I, typename R>


requires(Readable(I) && LinkableBidirectionalIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
I sort_bidirectional_linkable_recursive(I f, I l, R r)
{
f = sort_linkable_recursive(f, l, r, forward_linker<I>());
restore_backward_links(f, l, backward_linker<I>());
return f;
}

Stepanov, McJones Elements of Programming March 14, 2008 624 / 880


Forward versus bidirectional linkable structures

We have demonstrated efficient algorithms for reverse, partition,


and merge (hence merge sort) of forward linkable structures
Adding backward links to a forward linkable structure does not
appear to improve algorithms on the structure as a whole
However, backward links allow constant-time removal and
insertion of elements at an arbitrary location
Since it is the efficiency of insertion and deletion that is often the
reason for choosing linkable structures in the first place,
bidirectional linkage should be seriously considered

Stepanov, McJones Elements of Programming March 14, 2008 625 / 880


Concept WeakBifurcateCoordinate

Definition
WeakBifurcateCoordinate(T ) ⇒
Regular(T )
The type function WeightT ype(T ) is defined
Integer(WeightT ype(T ))
The unary predicates is_empty, has_left_successor and
has_right_successor are defined on T
The actions step_left and step_right, and the corresponding
transformations left_successor and right_successor, are defined
on T
AmortizedConstantTime(step_left) ∧
AmortizedConstantTime(step_right)

Stepanov, McJones Elements of Programming March 14, 2008 626 / 880


Implication of weakness of WeakBifurcateCoordinate

Because of the lack of regularity of step_left and step_right, after


an algorithm takes one of the branches, the other branch can never
be taken

Stepanov, McJones Elements of Programming March 14, 2008 627 / 880


Concept BifurcateCoordinate

Definition
BifurcateCoordinate(T ) ⇒
WeakBifurcateCoordinate(T )
RegularAction(step_left) ∧ RegularAction(step_right)

Stepanov, McJones Elements of Programming March 14, 2008 628 / 880


Concept BidirectionalBifurcateCoordinate (1 of 3)

Definition (requirements, signatures)


BidirectionalBifurcateCoordinate(T ) ⇒
BifurcateCoordinate(T )
The unary predicate has_predecessor is defined on T
The action step_up and the corresponding transformation
predecessor are defined on T
RegularAction(step_up)
AmortizedConstantTime(step_up)

Stepanov, McJones Elements of Programming March 14, 2008 629 / 880


Concept BidirectionalBifurcateCoordinate (2 of 3)

Definition (signatures)
template <typename C>
requires(BidirectionalBifurcateCoordinate(C))
bool is_left_successor(C c)
{
// Precondition: has_predecessor(c)
C parent = predecessor(c);
return has_left_successor(parent) && left_successor(parent) == c;
}

template <typename C>


requires(BidirectionalBifurcateCoordinate(C))
bool is_right_successor(C c)
{
// Precondition: has_predecessor(c)
C parent = predecessor(c);
return has_right_successor(parent) && right_successor(parent) == c;
}

Stepanov, McJones Elements of Programming March 14, 2008 630 / 880


Concept BidirectionalBifurcateCoordinate (3 of 3)

Definition (axioms)
For any coordinate x ∈ T :

has_left_successor(x) ⇒ predecessor(left_successor(x)) = x
has_right_successor(x) ⇒ predecessor(right_successor(x)) = x
has_predecessor(x) ⇒
is_left_successor(x) ∨ is_right_successor(x)

Stepanov, McJones Elements of Programming March 14, 2008 631 / 880


Reachability of bifurcate coordinates
template <typename C>
requires(BifurcateCoordinate(C))
bool reachable(C x, C y)
{
if (x == y) return true;
if (has_left_successor(x)) return reachable(left_successor(x), y);
if (has_right_successor(x)) return reachable(right_successor(x), y);
return false;
}

template <typename C>


requires(BifurcateCoordinate(C))
bool left_reachable(C x, C y)
{
return has_left_successor(x) && reachable(left_successor(x), y);
}

template <typename C>


requires(BifurcateCoordinate(C))
bool right_reachable(C x, C y)
{
return has_right_successor(x) && reachable(right_successor(x), y);
}

Stepanov, McJones Elements of Programming March 14, 2008 632 / 880


Properties of bifurcate coordinates

Definition
The descendants of a bifurcate coordinate x are all the coordinates y
such that reachable(x, y)

Definition
The descendants of x are acyclic if for all y in the descendants of x,
¬left_reachable(y, y) ∧ ¬right_reachable(y, y)

Definition
The descendants of x are a tree if they are acyclic and for all y, z in the
descendants of x, ¬left_reachable(y, z) ∨ ¬right_reachable(y, z)

Stepanov, McJones Elements of Programming March 14, 2008 633 / 880


weight_recursive

template <typename C>


requires(BifurcateCoordinate(C))
WeightType(C) weight_recursive(C f)
{
WeightType(C) w(1);
if (has_left_successor(f)) w += weight_recursive(left_successor(f));
if (has_right_successor(f)) w += weight_recursive(right_successor(f));
return w;
}

Lemma
The descendants of x are a finite tree if and only if they are a tree and
weight_recursive(x) terminates

Stepanov, McJones Elements of Programming March 14, 2008 634 / 880


height_recursive

template <typename C>


requires(BifurcateCoordinate(C))
WeightType(C) height_recursive(C f)
{
WeightType(C) l(0), r(0);
if (has_left_successor(f)) l = height_recursive(left_successor(f));
if (has_right_successor(f)) r = height_recursive(right_successor(f));
return 1 + max(l, r);
}

Lemma
height(x) 6 weight(x)

Stepanov, McJones Elements of Programming March 14, 2008 635 / 880


Recursive tree traversal

template <typename C, typename Pre, typename In, typename Post>


requires(BifurcateCoordinate(I) &&
UnaryProcedure(Pre) && Domain(Pre) == ValueType(C) &&
UnaryProcedure(In) && Domain(Pre) == ValueType(C) &&
UnaryProcedure(Post) && Domain(Pre) == ValueType(C))
void traverse_nonempty_recursive(C r, Pre f, In g, Post h)
{
f(source(r));
if (has_left_successor(r))
traverse_nonempty_recursive(left_successor(r), v);
g(source(r));
if (has_right_successor(r))
traverse_nonempty_recursive(right_successor(r), v);
h(source(r));
}

Stepanov, McJones Elements of Programming March 14, 2008 636 / 880


Problems with recursive traversal

It requires stack space proportional to the height of the tree, which


can be as large as the weight, which is often unacceptable for
large, unbalanced trees
The interface is not flexible, for example it cannot be used to
compare two trees
We will now develop low-cost iterator-like traversal mechanisms

Stepanov, McJones Elements of Programming March 14, 2008 637 / 880


traverse_step

enum visit { pre, in, post };

template <typename C>


requires(BidirectionalBifurcateCoordinate(C))
void traverse_step(C& c, visit& v)
{
// Precondition: has_predecessor(c) ∨ v , post
switch (v) {
case pre:
if (has_left_successor(c)) step_left(c);
else v = in;
break;
case in:
if (has_right_successor(c)) { v = pre; step_right(c); }
else v = post;
break;
case post:
if (is_left_successor(c)) v = in;
step_up(c);
}
}

Stepanov, McJones Elements of Programming March 14, 2008 638 / 880


Iterative tree traversal

template <typename C, typename Pre, typename In, typename Post>


requires(BidirectionalBifurcateCoordinate(C) &&
UnaryProcedure(Pre) && Domain(Pre) == ValueType(C) &&
UnaryProcedure(In) && Domain(Pre) == ValueType(C) &&
UnaryProcedure(Post) && Domain(Pre) == ValueType(C))
void traverse(C c, Pre f_pre, In f_in, Post f_post)
{
if (is_empty(c)) return;
C root = c;
visit v = pre;
while (true) {
switch (v) {
case pre: f_pre (source(c)); break;
case in: f_in (source(c)); break;
case post: f_post(source(c)); if (c == root) return;
}
traverse_step(c, v);
}
}

Stepanov, McJones Elements of Programming March 14, 2008 639 / 880


Binary trees and iterators

Traversal through binary trees is fundamentally different than


iterating through a range because of the three directions of
movement
traverse_step and traverse provide full access to the structure of
a binary tree
It is possible to write iterator adaptors to traverse_step that
provide sequential access to the nodes of a binary tree in preorder,
inorder, or postorder, but such an iterator loses structural
information
An iterator adapter whose value type pairs a visit with a
coordinate preserves the structural information
(See the Projects section at the end of this chapter for more details)

Stepanov, McJones Elements of Programming March 14, 2008 640 / 880


Advantages of bidirectional binary trees

We recommended bidirectional linkage for constant-time insertion


and deletion of elements
With binary trees the situation is different: without predecessor
links even the simplest traversal algorithms require linear space
Adding predecessor links allows constant-space traversal, and
also makes possible various rebalancing algorithms
The overhead for the extra link, while 50% in the worst case, is
typically much less in practice

Stepanov, McJones Elements of Programming March 14, 2008 641 / 880


Linkable bifurcate coordinates

For linkable iterators, we passed the linking operation as a


parameter because of the need to use different linking operations,
such as restoring back links after sort
For linkable bifurcate coordinates there does not appear to be a
need for alternate versions of the linking operations, so we
overload them on the concept

Stepanov, McJones Elements of Programming March 14, 2008 642 / 880


Concept LinkableBifurcateCoordinate

Definition
LinkableBifurcateCoordinate(T ) ⇒
BifurcateCoordinate(T )
Proper mutators set_left_successor and set_right_successor
taking two iterators of type T are defined
Immediately after set_left_successor(i, j) is performed,
left_succesor(i) = j
Immediately after set_right_successor(i, j) is performed,
right_succesor(i) = j

Stepanov, McJones Elements of Programming March 14, 2008 643 / 880


Concept LinkableBidirectionalBifurcateCoordinate

Definition
LinkableBidirectionalBifurcateCoordinate(T ) ⇒
LinkableBifurcateCoordinate(T )
Immediately after set_left_successor(i, j) is performed,
has_left_successor(i) ⇒ predecessor(j) = i
Immediately after set_right_successor(i, j) is performed,
has_right_successor(i) ⇒ predecessor(j) = i

Stepanov, McJones Elements of Programming March 14, 2008 644 / 880


Reversing links
traverse_step is an efficient way to traverse via bidirectional
bifurcating coordinates, but requires the predecessor function
When the predecessor function is not available and recursive
(stack-based) traversal is unacceptable because of unbalanced
trees, there are interesting algorithms that depend on temporarily
storing the link to the predecessor in a link normally containing a
successor; this assures there is a path back to the root 54
54
Link reversal was introduced in: H. Schorr and W.M. Waite. An Efficient and
Machine-Independent Procedure for Garbage Collection in Various List Structures.
Communications of the ACM, Volume 10, Number 8, August 1967, pages 501-506 (it was
independetly discovered by L.P. Deutsch). A version without tag bits was published
in: J.M. Robson. An Improved Algorithm for Traversing Binary Trees Without
Auxiliary Stack. Information Processing Letters, Volume 2, 1973, pages 12-14; and Joseph
M. Morris. Traversing Binary Trees Simply and Cheaply, Information Processing Letters,
Volume 9, Number 5, 1979, pages 197-200. An ingenious technique of “rotating” the
links was published in: Gary Lindstrom. Scanning List Structures Without Stack or
Tag Bits. Information Processing Letters, Volume 2, 1973, pages 47-51; and
independently in: Barry Dwyer. Simple Algorithms for Traversing a Tree Without an
Auxiliary Stack. Information Processing Letters, Volume 2, 1974, pages 143-145.
Stepanov, McJones Elements of Programming March 14, 2008 645 / 880
Exclusive-or’ing links

An interesting technique for iterative traversal without extra


storage for predecessor links stores in each successor link the
exclusive-or of the predecessor and the corresponding successor55
By additionally requiring that the address of a node’s left
successor is less than the address of its right successor, it is
possible to design a data structure that supports a non-recursive
traversal algorithm that does not modify the tree and uses only a
constant amount of additional storage

55
The exclusive-or technique was first applied to binary trees in: Laurent Siklóssy.
Fast and Read-only Algorithms for Traversing Trees Without an Auxiliary Stack.
Information Processing Letters, Volume 1, 1972, pages 149-152.
Stepanov, McJones Elements of Programming March 14, 2008 646 / 880
Similarity

Definition
Two sets S and S 0 of coordinates from the same coordinate type T of a
coordinate structure are similar under a given one-to-one
correspondence if any traversal function maps corresponding
coordinates to corresponding coordinates

This definition can be extended in various ways


To coordinate structures with more than one coordinate type
To sets of coordinates from different but isomorphic coordinate
structures
Similarity does not depend on the values of the objects pointed to
by the coordinates
Algorithms for testing similarity use only traversal functions but
not dereferencing functions

Stepanov, McJones Elements of Programming March 14, 2008 647 / 880


Examples of similarity

Examples
Two ranges of iterators [f, l) and [f 0 , l 0 ) are similar if l − f = l 0 − f 0
Two trees are similar if both are empty or they have similar left
and right subtrees

Stepanov, McJones Elements of Programming March 14, 2008 648 / 880


Equivalence

Definition
Two sets S and S 0 of coordinates from the same coordinate type T of a
coordinate structure represent equivalent value sets under a given
one-to-one correspondence and a given equivalence relation if they are
similar and dereferencing corresponding coordinates gives equivalent
objects

Stepanov, McJones Elements of Programming March 14, 2008 649 / 880


lexicographic_equivalence

template <typename I0, typename I1, typename R>


requires(Readable(I0) && Iterator(I0) &&
Readable(I1) && Iterator(I1) &&
ValueType(I0) == ValueType(I1) &&
EquivalenceRelation(R) && Domain(R) == ValueType(I0))
bool lexicographic_equivalence(I0 f0, I0 l0, I1 f1, I1 l1, R r)
{
while (true) {
if (f0 == l0 && f1 == l1) return true;
if (f0 == l0 || f1 == l1) return false;
if (!r(source(f0), source(f1))) return false;
++f0; ++f1;
}
}

Stepanov, McJones Elements of Programming March 14, 2008 650 / 880


lexicographic_ordering

template <typename I0, typename I1, typename R>


requires(Readable(I0) && Iterator(I0) &&
Readable(I1) && Iterator(I1) &&
ValueType(I0) == ValueType(I1) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I0))
bool lexicographic_ordering(I0 f0, I0 l0, I1 f1, I1 l1, R r)
{
while (true) {
if (f1 == l1) return false;
if (f0 == l0) return true;
if (r(source(f0), source(f1))) return true;
if (r(source(f0), source(f1))) return false;
++f0; ++f1;
}
}

This would benefit from having a three-way comparison instead


of an ordering

Stepanov, McJones Elements of Programming March 14, 2008 651 / 880


bifurcate_similarity

template <typename C, typename R>


requires(BidirectionalBifurcateCoordinate(C))
bool bifurcate_similarity(C c0, C c1)
{
if (is_empty(c0) || is_empty(c1)) return is_empty(c0) && is_empty(c1);
C root0 = c0; C root1 = c1;
visit v0 = pre; visit v1 = pre;
while (true) {
traverse_step(c0, v0); traverse_step(c1, v1);
if (d0 != d1) return false;
if (c0 == root0 && v0 == post) return true;
}
}

Stepanov, McJones Elements of Programming March 14, 2008 652 / 880


bifurcate_equivalence

template <typename C, typename R>


requires(BidirectionalBifurcateCoordinate(C) &&
EquivalenceRelation(R) && Domain(R) == ValueType(C))
bool bifurcate_equivalence(C c0, C c1, R r)
{
if (is_empty(c0) || is_empty(c1)) return is_empty(c0) && is_empty(c1);
C root0 = c0; C root1 = c1;
visit v0 = pre; visit v1 = pre;
while (true) {
if (v0 == pre && !r(source(c0), source(c1))) return false;
traverse_step(c0, v0); traverse_step(c1, v1);
if (v0 != v1) return false;
if (c0 == root0 && v0 == post) return true;
}
}

Stepanov, McJones Elements of Programming March 14, 2008 653 / 880


bifurcate_ordering

template <typename C, typename R>


requires(BidirectionalBifurcateCoordinate(C) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(C))
bool bifurcate_ordering(C c0, C c1, R r)
{
if (is_empty(c0) || is_empty(c1)) return is_empty(c0) && !is_empty(c1);
C root0 = c0; C root1 = c1;
visit v0 = pre; visit v1 = pre;
while (true) {
if (v0 == pre) {
if (r(source(c0), source(c1))) return true;
if (r(source(c1), source(c0))) return false;
}
traverse_step(c0, v0); traverse_step(c1, v1);
if (v0 != v1) return v0 > v1;
if (c0 == root0 && v0 == post) return false;
}
}

Stepanov, McJones Elements of Programming March 14, 2008 654 / 880


Conclusions

While iterators are very flexible, we introduced several


generalizations:
Linkable iterators allow the topology between iterators to be
changed rather than changing the objects to which the iterators
point
Bifurcate coordinates extend from linear to binary tree topology
There is, of course, a wide variety of different coordinate structures
No computational device (e.g., goto) is harmful per se; it could and
should be used when appropriate
An invariant is associated with a scope, such as a loop, an
algorithm, or a datatype
Outside the scope, the invariant must hold
Inside the scope, it is permissible to violate the invariant
Proper maintenance of the invariant requires attention to
concurrency and exceptions

Stepanov, McJones Elements of Programming March 14, 2008 655 / 880


Projects

Project 1
Design an adapter that converts traverse_step into an iterator, taking
into consideration the following issues:
What should the value type be?
What iterator concept should be modeled?
What should the limit iterator be to allow traversal of a subtree
rooted at an arbitrary coordinate?

Project 2
Extend ordering to an arbitrary coordinate structure by defining an
appropriate meta-algorithm

Stepanov, McJones Elements of Programming March 14, 2008 656 / 880


Contents I

1 Introduction

2 Foundations

3 Transformations and their orbits

4 Algorithms on algebraic structures

5 Orderings

6 Combining concepts

7 Refining concepts of iterators

Stepanov, McJones Elements of Programming March 14, 2008 657 / 880


Contents II
8 Permutations and rearrangements

9 Rotations

10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects
Composite objects, regular types, and containers
Equality on composite objects
Collocated types
Distributed types
Construction

Stepanov, McJones Elements of Programming March 14, 2008 658 / 880


Contents III
Memory allocation and deallocation
Explicit construction and destruction
Lists
Arrays
Containers as elements
Relational concepts
Compatibility
Generalized copy algorithms
Underlying type
Generalized permutation algorithms
More generalized permutation algorithms
Generalized partition algorithms
Underlying iterator
More generalized partition algorithms
underlying_compare

Stepanov, McJones Elements of Programming March 14, 2008 659 / 880


Contents IV
More generalized merge and sort algorithms
Composite objects

13 Iterative algorithms for divide-and-conquer

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

17 Index

Stepanov, McJones Elements of Programming March 14, 2008 660 / 880


Definition of composite object (1 of 4)

Definition
A type T belongs to a concept that is a composite object concept if:
Parts Objects of type T are made up of other objects called
parts
Subparts Subsidiary definition: x is a subpart of y if x is a part of y
or x is a subpart of a part of y

Stepanov, McJones Elements of Programming March 14, 2008 661 / 880


Definition of composite object (2 of 4)

Definition
Connected T has an affiliated coordinate structure
Every object of type T has a distinguished starting
address
It is possible to reach every part of an object of type T
using traversals in the coordinate structure, beginning
at the starting address

Stepanov, McJones Elements of Programming March 14, 2008 662 / 880


Definition of composite object (3 of 4)

Definition
Noncircular No object is a subpart of itself
Disjoint Subsidiary definition: Two objects are disjoint if they
have no subpart in common
If x and y are objects, either one is a subpart of the
other or they are disjoint

Stepanov, McJones Elements of Programming March 14, 2008 663 / 880


Definition of composite object (4 of 4)

Definition
Complete destruction Destroying an object of type T causes the
destruction of every part of the object (and,
therefore, every subpart)

Stepanov, McJones Elements of Programming March 14, 2008 664 / 880


Reflections on composite objects

Like coordinate structure, “composite object” is not a concept but


a family of related concepts
No algorithms can be defined on composite objects as such;
meta-algorithms are possible but are outside the scope of this book

Stepanov, McJones Elements of Programming March 14, 2008 665 / 880


Examples of composite objects

Example
A tuple: traversal via member access
An array: traversal via random access iterator
A binary tree: traversal via bifurcate coordinate
A complex number: traversal via member access

Stepanov, McJones Elements of Programming March 14, 2008 666 / 880


Regularity and composite objects

In Chapter 1 we introduced the concept Regular and noted


The requirements on the element type allowing objects to be
sorted should be satisfied by arrays themselves, so we can sort
an array of arrays

In this chapter we show how to implement a variety of composite


objects in such a way that they satisfy the requirements for regular
types
Equality
Assignment (respecting equality)
Default construction (making an object assignable/destructable)
Copy construction (equivalent to default construction followed by
assignment)
Total ordering (natural or default)
Destruction

Stepanov, McJones Elements of Programming March 14, 2008 667 / 880


Container

Definition
A container is a composite object whose interpretation is a collection of
the interpretations of the parts of the container

Example
A tuple
An array
A binary tree

Stepanov, McJones Elements of Programming March 14, 2008 668 / 880


The role of containers

Mathematicians construct all of mathematics on a foundation


consisting only of sets
The computer science analog of sets is containers, and they come
in many varieties

Stepanov, McJones Elements of Programming March 14, 2008 669 / 880


Coordinate structure of a container

***** This is subsumed by our definition of composite object

Definition
Each container type defines a corresponding coordinate structure and
functions giving one or more specific coordinates to initiate traversals
or access

Example
Tuple: member access
Array: random-access iterators and begin and end
Binary tree: bifurcate coordinates and root

Stepanov, McJones Elements of Programming March 14, 2008 670 / 880


Equality on containers

Definition
Two containers are equal if corresponding parts are equal

Example
Array equality: use lexicographic_equivalence with equality on
the element type
Binary tree equality: use bifircate_equivalence with equality on
the element type

As with attributes, there may be nonessential parts that are not


taken into account by equality; we will see examples later in this
chapter

Stepanov, McJones Elements of Programming March 14, 2008 671 / 880


Structural equality

Recall from Chapter 1 that we follow Leibniz in defining equality


in terms of consistent observed behavior
There is a related notion of structural equality

Definition
Two composite objects are structurally equal if their states are equal

For a container, structural equality is the same as Leibniz equality


Objects that are not structurally equal could still be equal in the
Leibniz sense (such objects do not have unique representations)

Stepanov, McJones Elements of Programming March 14, 2008 672 / 880


Structural equality versus equality

Examples
Complex numbers x + iy and x 0 + iy 0 are equal if x = x 0 ∧ y = y 0
0
Complex numbers reiϕ and r 0 eiϕ are equal if

r = r0 ∧ ϕ ≡ ϕ0 (mod 2π)

Rational numbers a/b and a 0 /b 0 are equal if ab 0 = ba 0

For two objects of the same type, these implications hold

They are identical (same object) ⇒


They are structurally equal (same state) ⇒
They are equal (same behavior) ⇒
They are equivalent (same under some equivalence relation)

Stepanov, McJones Elements of Programming March 14, 2008 673 / 880


Leibniz equality is difficult to implement for certain
types

Example
Implementing Leibniz equality for priority queues requires comparing
successive values from both queues
This consumes the elements compared, requiring copies to be
made of the two queues
It requires n log n time

Stepanov, McJones Elements of Programming March 14, 2008 674 / 880


Collocated types

Definition
A collocated type is one all of whose parts are adjacent in memory

Example
Given types T0 , . . . , Tk−1 , we can construct the heterogeneous type
structT0 ,...,Tk−1 with useful special cases
pairT0 ,T1
tripleT0 ,T1 ,T2
If all the Ti are the same, we can define the homogeneous type
arrayk,T

Stepanov, McJones Elements of Programming March 14, 2008 675 / 880


Properties of collocated types

The correct semantics for assignment, default and copy


construction, and destruction of a collocated type is elementwise;
for equality and ordering it is the lexicographic techniques
defined in the previous chapter56
Collocated types have constant area, so they can be fully allocated
at compile time
There are several situations when a collocated type is not optimal
When the area of an object is not constant, that is, its area is fixed or
dynamic
When the object might be frequently moved (e.g., an element of an
array being sorted)

56
In some situations C++ automatically generates elementwise constructors,
destructor, and assignment for struct types; equality and ordering are never
automatically generated. Constant-size arrays in C++ are not first-class citizens and
one must define a type such as the array_k in Appendix 2 to make them be regular.
Stepanov, McJones Elements of Programming March 14, 2008 676 / 880
Distributed types

Definition
A distributed type has parts that are not adjacent:
Data representation of the externally-visible behavior of the
type
Connectors traversal through all the data
Header the origin for all traversal
Padding memory locations included to maintain hardware
alignment or optimize cache behavior
Reserve memory locations reserved for future growth
Other locks, software caches, handles for system resources, etc.

Stepanov, McJones Elements of Programming March 14, 2008 677 / 880


Ownership: containers versus iterators

We distinguish pointers representing ownership within a


container (connectors) from pointers representing intermediate
positions within an algorithm (iterators or coordinates)

Example: list
The functions begin and end, when applied to a list, each return a list
iterator
The function successor, when applied to a list iterator, returns another
list iterator; there is no function like successor defined on list
The destructor of the listT type is responsible for iterating through all
the elements, destroying each object of type T , and freeing the storage of
its containing list node
Destroying a list iterator has no impact on any list object

Stepanov, McJones Elements of Programming March 14, 2008 678 / 880


list: example of container versus iterator

template <typename T> requires(Regular(T))


struct list_node
{
T value;
list_node* forward_link;
// Destructor does nothing
};

template <typename T> requires(Regular(T))


struct list_iterator
{
list_node<T>* p;
// Destructor does nothing
};

template <typename T> requires(Regular(T))


struct list
{
list_iterator<T> first;
// Destructor erases all elements
};

Stepanov, McJones Elements of Programming March 14, 2008 679 / 880


Primary objects

Definition
A primary object is one that is not a subpart of another object; an object
that is not primary is secondary

A primary object may have static storage duration


A primary object may have automatic storage duration
A distributed primary object may have parts with dynamic
storage duration

Stepanov, McJones Elements of Programming March 14, 2008 680 / 880


Construction

Construction of a primary object takes place in static or automatic


storage
Construction of a secondary object takes place in a subpart of a
primary object

Example
***** Show code with global object, local object, and insert constructor
?????

Stepanov, McJones Elements of Programming March 14, 2008 681 / 880


(Languages without ownership)

Many languages follow the example of Lisp57 use the same type
for both a list as a container and a list iterator
This design decision means it is not possible for the programmer
to explicitly free a list since there might be other variables
containing list objects with overlapping storage

57
John McCarthy. Recursive Functions of Symbolic Expressions and Their
Computation by Machine, Part I. Communications of the ACM, Volume 3, Number 4,
1960, pages 184-195
Stepanov, McJones Elements of Programming March 14, 2008 682 / 880
remote

template <typename T>


requires(Regular(T))
struct remote
{
T* p;
remote() : p(0) {}
remote(const T& x) : p(new T(x)) { }
remote(const remote& x) : p(new T(source(x.p))) { }
~remote() { delete p; }
friend bool operator==(const remote& x, const remote& y)
{ return source(x.p) == source(y.p); }
friend void swap(remote& x, remote& y) { swap(x.p, y.p); }
void operator=(remote& x) { swap(sink(this), x); }
friend bool operator<(const remote& x, const remote& y)
{ return source(x.p) < source(y.p); }
};

Stepanov, McJones Elements of Programming March 14, 2008 683 / 880


Allocation and deallocation

void* raw_allocate(size_t n)
{
return ::operator new(n);
}

template <typename T>


requires(Regular(T))
T* allocate(size_t n)
{
return (T*)raw_allocate(n * sizeof(T));
}

void deallocate(void* p)
{
::operator delete(p);
}

Stepanov, McJones Elements of Programming March 14, 2008 684 / 880


construct_copy and destroy

template <typename T>


requires(Regular(T))
struct construct_copy
{
const T* p;
construct_copy(const T& x) : p(&x) {}
void operator()(T& q)
{
new (&q) T(source(p));
}
};

template <typename T>


requires(Regular(T))
void destroy(T& p)
{
sink(&p).~T();
}

Stepanov, McJones Elements of Programming March 14, 2008 685 / 880


list_node and list_iterator

template <typename T>


requires(Regular(T))
struct list_node
{
T value;
list_node* forward_link;
list_node(const T& v, list_node* f) : value(v), forward_link(f) {}
};

template <typename T>


requires(Regular(T))
struct list_iterator
{
list_node<T>* p;
list_iterator() : p(0) {}
list_iterator(list_node<T>* p) : p(p) {}
};

Stepanov, McJones Elements of Programming March 14, 2008 686 / 880


Functions for list_iterator

template <typename T> requires(Regular(T))


void operator++(list_iterator<T>& i) { i.p = source(i.p).forward_link; }

template <typename T> requires(Regular(T))


void set_successor(list_iterator<T> i, list_iterator<T> j)
{ sink(i.p).forward_link = j.p; }

template <typename T> requires(Regular(T))


bool operator==(list_iterator<T> i, list_iterator<T> j)
{ return i.p == j.p; }

template <typename T> requires(Regular(T))


const T& source(list_iterator<T> i) { return source(i.p).value; }

template <typename T> requires(Regular(T))


T& sink(list_iterator<T> i) { return sink(i.p).value; }

Stepanov, McJones Elements of Programming March 14, 2008 687 / 880


Type functions for list_iterator

template <typename T>


requires(Regular(T))
struct value_type< list_iterator<T> >
{
typedef T type;
};

template <typename T>


requires(Regular(T))
struct distance_type< list_iterator<T> >
{
typedef DistanceType(list_node<T>*) type;
};

template <typename T>


requires(Regular(T))
struct iterator_category< list_iterator<T> >
{
typedef forward_iterator_tag category;
};

Stepanov, McJones Elements of Programming March 14, 2008 688 / 880


Insert functions for list_node

template <typename T, typename F>


requires(Regular(T) && ConstructorObject(F) && Domain(F) == T)
list_iterator<T> insert_construct(list_iterator<T> j, F f)
{
list_iterator<T> i(allocate< list_node<T> >(1));
f(sink(i));
set_successor(i, j);
return i;
}

template <typename T>


requires(Regular(T))
list_iterator<T> insert(list_iterator<T> i, const T& x)
{
return insert_construct(i, construct_copy<T>(x));
}

Stepanov, McJones Elements of Programming March 14, 2008 689 / 880


More insert functions for list_node

template <typename T, typename F>


requires(Regular(T) && ConstructorObject(F) && Domain(F) == T)
list_iterator<T> insert_after_construct(list_iterator<T> i, F f)
{
list_iterator<T> j = insert_construct(successor(i), f);
set_successor(i, j);
return j;
}

template <typename T>


requires(Regular(T))
list_iterator<T> insert_after(list_iterator<T> i, const T& x)
{
return insert_after_construct(i, construct_copy<T>(x));
}

Stepanov, McJones Elements of Programming March 14, 2008 690 / 880


list_insert_after_iterator

template <typename T>


requires(Regular(T))
struct list_insert_after_iterator
{
list_iterator<T> h;
list_iterator<T> t;
list_insert_after_iterator() { }
list_insert_after_iterator(list_iterator<T> h, list_iterator<T> t)
: h(h), t(t) { }
void operator=(const T& x)
{
if (t == list_iterator<T>()) {
h = insert(h, x);
t = h;
} else
t = insert_after(t, x);
}
void operator++() { }
};

Stepanov, McJones Elements of Programming March 14, 2008 691 / 880


(Equality for list_insert_after_iterator)

template <typename T>


requires(Regular(T))
bool operator==(const list_insert_after_iterator<T>& i,
const list_insert_after_iterator<T>& j)
{
return i.h == j.h && i.t == j.t;
}

Stepanov, McJones Elements of Programming March 14, 2008 692 / 880


Type functions for list_insert_after_iterator

template <typename T>


requires(Regular(T))
struct value_type< list_insert_after_iterator<T> >
{
typedef T type;
};

template <typename T>


requires(Regular(T))
struct distance_type< list_insert_after_iterator<T> >
{
typedef DistanceType(list_iterator<T>) type;
};

template <typename T>


requires(Regular(T))
struct iterator_category< list_insert_after_iterator<T> >
{
typedef iterator_tag category;
};

Stepanov, McJones Elements of Programming March 14, 2008 693 / 880


Erase functions for list_node

template <typename T>


requires(Regular(T))
list_iterator<T> erase_first(list_iterator<T> i)
{
list_iterator<T> j = successor(i);
destroy(sink(i));
deallocate(i.p);
return j;
}

template <typename T>


requires(Regular(T))
void erase_after(list_iterator<T> i)
{
set_successor(i, erase_first(successor(i)));
}

Stepanov, McJones Elements of Programming March 14, 2008 694 / 880


list container

template <typename T>


requires(Regular(T))
struct list
{
list_iterator<T> first;
list() : first(0) {}
template <typename I>
requires(Readable(I) && Iterator(I) && ValueType(I) == T)
list(I f, I l);
list(const list& x);
~list();
friend void swap(list& x, list& y) { swap(x.first, y.first); }
void operator=(list x) { swap(sink(this), x); }
};

Stepanov, McJones Elements of Programming March 14, 2008 695 / 880


Type functions for list

***** Really need a concept Container ?????

Definition
The type function IteratorT ype(C) is defined on any container type C;
it returns a type such that Iterator(IteratorT ype(C))

Stepanov, McJones Elements of Programming March 14, 2008 696 / 880


Type functions for list

template <typename T>


requires(Regular(T))
struct iterator_type< list<T> >
{
typedef list_iterator<T> type;
};

Stepanov, McJones Elements of Programming March 14, 2008 697 / 880


Functions for list
template <typename T> requires(Regular(T))
IteratorType(list<T>) begin(const list<T>& x)
{
return x.first;
}

template <typename T> requires(Regular(T))


IteratorType(list<T>) end(const list<T>& x)
{
return list_iterator<T>();
}

template <typename C> requires(Container(C))


DistanceType(IteratorType(C)) size(const C& x)
{
return end(x) - begin(x);
}

template <typename C>


requires(Container(C))
bool is_empty(const C& x)
{
return begin(x) == end(x);
}
Stepanov, McJones Elements of Programming March 14, 2008 698 / 880
Constructors for list
template <typename T, typename I>
requires(Regular(T) && Iterator(I) &&
ValueType(I) == T)
void insert(list<T>& x, IteratorType(list<T>) i, I f, I l)
{
x.first = copy(f, l, list_insert_after_iterator<T>(begin(x), i)).h;
}

template <typename T>


template <typename I>
requires(Regular(T) && Readable(I) && Iterator(I) &&
ValueType(I) == T)
list<T>::list(I f, I l) : first(0)
{
insert(sink(this), end(sink(this)), f, l);
}

template <typename T>


requires(Regular(T))
list<T>::list(const list& x) : first(0)
{
insert(sink(this), end(sink(this)), begin(x), end(x));
}

Stepanov, McJones Elements of Programming March 14, 2008 699 / 880


Destructor for list

template <typename T>


requires(Regular(T))
void pop_all(list<T>& x)
{
while (!is_empty(x))
x.first = erase_first(begin(x));
}

template <typename T>


requires(Regular(T))
list<T>::~list()
{
pop_all(sink(this));
}

Stepanov, McJones Elements of Programming March 14, 2008 700 / 880


Equality and less than for list
template <typename T>
requires(Regular(T))
struct equality
{
bool operator()(const T& x, const T& y) { return x == y; }
};

template <typename T>


requires(Regular(T))
bool operator==(const list<T>& x, const list<T>& y)
{
return lexicographic_equivalence(begin(x), end(x), begin(y), end(y),
equality<T>());
}

template <typename T>


requires(Regular(T))
bool operator<(const list<T>& x, const list<T>& y)
{
return lexicographic_ordering(begin(x), end(x), begin(y), end(y), less<T>());
}

These definitions work for any container *****


Stepanov, McJones Elements of Programming March 14, 2008 701 / 880
partition for list

template <typename T, typename P>


requires(Regular(T) && UnaryPredicate(P) && Domain(P) == T)
void partition_list(list<T>& x, list<T>& y, P p)
{
typedef IteratorType(list<T>) I;
pair< pair<I, I>, pair<I, I> > pp =
partition_linkable(begin(x), end(x), p, forward_linker<I>());
x.first = pp.m0.m0;
if (pp.m1.m0 != end(x)) {
set_successor(pp.m1.m1, begin(y));
y.first = pp.m1.first;
}
}

Stepanov, McJones Elements of Programming March 14, 2008 702 / 880


merge for list

template <typename T, typename R>


requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)
void merge(list<T>& x, list<T>& y, R r)
{
if (is_empty(y)) return;
if (is_empty(x)) x = y;
else
x.first = merge_linkable_nonempty(
begin(x), end(x), begin(y), end(y),
r, forward_linker<IteratorType(list<T>)>()).m0;
y.first = end(y);
}

Stepanov, McJones Elements of Programming March 14, 2008 703 / 880


sort for list

template <typename T, typename R>


requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)
void sort_recursive(list<T>& x, R r)
{
x.first = sort_forward_linkable_recursive(begin(x), end(x), r);
}

template <typename T, typename R>


requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)
void sort(list<T>& x, R r)
{
x.first = sort_forward_linkable<32>(begin(x), end(x), r);
}

Stepanov, McJones Elements of Programming March 14, 2008 704 / 880


array_header

template <typename T>


requires(Regular(T))
struct array_header
{
T* m;
T* l;
T a;
};

Invariants, where f = &a:


[f, m) are constructed elements
[m, l) are unconstructed (reserved) elements

Stepanov, McJones Elements of Programming March 14, 2008 705 / 880


allocate_array_header

template <typename T>


requires(Regular(T))
array_header<T>* allocate_array(DistanceType(T*) n)
{
typedef array_header<T>* P;
size_t bsize = size_t(predecessor(n)) * sizeof(T);
P p = P(raw_allocate(sizeof(array_header<T>) + bsize));
T* f = &sink(p).a;
sink(p).m = f;
sink(p).l = f + n;
return p;
}

Stepanov, McJones Elements of Programming March 14, 2008 706 / 880


deallocate_array_header

template <typename T>


requires(Regular(T))
void deallocate_array(array_header<T>* p)
{
deallocate(p);
}

Stepanov, McJones Elements of Programming March 14, 2008 707 / 880


array container

template <typename T>


requires(Regular(T))
struct array
{
typedef DistanceType(IteratorType(array<T>)) N;
array_header<T>* p;
array() : p(0) {}
array(N c); // size is 0 and capacity is c
array(N s, N c, const T& x); // size is s, capacity is c, all elements equal to x
template <typename I>
requires(Readable(I) && Iterator(I) && ValueType(I) == T)
array(I f, I l);
array(const array& x);
~array();
friend void swap(array& x, array& y) { swap(x.p, y.p); }
void operator=(array x) { swap(sink(this), x); }
};

template <typename T>


requires(Regular(T))
void reserve(array<T>& x, DistanceType(IteratorType(array<T>)) n);

Stepanov, McJones Elements of Programming March 14, 2008 708 / 880


Type functions for array

template <typename T>


requires(Regular(T))
struct iterator_type< array<T> >
{
typedef T* type;
};

Stepanov, McJones Elements of Programming March 14, 2008 709 / 880


Functions for array
template <typename T>
requires(Regular(T))
IteratorType(array<T>) begin(const array<T>& x)
{
if (x.p == 0) return IteratorType(array<T>)(0);
return IteratorType(array<T>)(&source(x.p).a);
}

template <typename T>


requires(Regular(T))
IteratorType(array<T>) end(const array<T>& x)
{
if (x.p == 0) return IteratorType(array<T>)(0);
return IteratorType(array<T>)(source(x.p).m);
}

template <typename T>


requires(Regular(T))
IteratorType(array<T>) end_of_storage(const array<T>& x)
{
if (x.p == 0) return IteratorType(array<T>)(0);
return IteratorType(array<T>)(source(x.p).l);
}

Stepanov, McJones Elements of Programming March 14, 2008 710 / 880


More functions for array

template <typename T>


requires(Regular(T))
DistanceType(IteratorType(array<T>)) capacity(const array<T>& x)
{
return end_of_storage(x) - begin(x);
}

template <typename T>


requires(Regular(T))
bool is_full(const array<T>& x)
{
return end(x) == end_of_storage(x);
}

Stepanov, McJones Elements of Programming March 14, 2008 711 / 880


push_construct

template <typename T, typename F>


requires(Regular(T) && ConstructorObject(F) && Domain(F) == T)
T& push_construct(array<T>& x, F f)
{
typedef DistanceType(IteratorType(array<T>)) N;
N n = size(x);
if (n == capacity(x))
reserve(x, max(N(1), n + n));
f(sink(source(x.p).m));
++sink(x.p).m;
return sink(end(x) - 1);
}

Stepanov, McJones Elements of Programming March 14, 2008 712 / 880


push

template <typename T>


requires(Regular(T))
T& push(array<T>& x, const T& y)
{
return push_construct(x, construct_copy<T>(y));
}

Stepanov, McJones Elements of Programming March 14, 2008 713 / 880


array_push_iterator

template <typename T>


requires(Regular(T))
struct array_push_iterator
{
array<T>* p;
array_push_iterator() {}
array_push_iterator(array<T>& x) : p(&x) {}
friend bool operator==(array_push_iterator x, array_push_iterator y)
{
return x.p == y.p;
}
friend void operator++(array_push_iterator& x) {}
void operator=(const T& x)
{
push(sink(p), x);
}
};

Stepanov, McJones Elements of Programming March 14, 2008 714 / 880


Type functions for array_push_iterator

template <typename T>


requires(Regular(T))
struct value_type< array_push_iterator<T> >
{
typedef T type;
};

template <typename T>


requires(Regular(T))
struct distance_type< array_push_iterator<T> >
{
typedef DistanceType(IteratorType(array<T>)) type;
};

template <typename T>


requires(Regular(T))
struct iterator_category< array_push_iterator<T> >
{
typedef iterator_tag category;
};

Stepanov, McJones Elements of Programming March 14, 2008 715 / 880


push for a range

template <typename T, typename I>


requires(Regular(T) &&
Readable(I) && Iterator(I) && ValueType(I) == T)
void push(array<T>& x, I f, I l)
{
copy(f, l, array_push_iterator<T>(x));
}

Stepanov, McJones Elements of Programming March 14, 2008 716 / 880


Pop functions for array

template <typename T>


requires(Regular(T))
void pop(array<T>& x)
{
--sink(x.p).m;
destroy(sink(source(x.p).m));
if (is_empty(x)) {
deallocate_array(x.p);
x.p = 0;
}
}

template <typename T>


requires(Regular(T))
void pop_all(array<T>& x)
{
while (!is_empty(x)) pop(x);
}

Stepanov, McJones Elements of Programming March 14, 2008 717 / 880


Constructors for array

template <typename T>


requires(Regular(T))
array<T>::array(DistanceType(IteratorType(array<T>)) c) :
p(allocate_array<T>(c))
{
}

template <typename T>


requires(Regular(T))
array<T>::array(DistanceType(IteratorType(array<T>)) s,
DistanceType(IteratorType(array<T>)) c,
const T& x) :
p(allocate_array<T>(c))
{
while (s != N(0)) { push(sink(this), x); --s; }
}

Stepanov, McJones Elements of Programming March 14, 2008 718 / 880


More constructors for array

template <typename T>


template <typename I>
requires(Regular(T) && Readable(I) && Iterator(I) && ValueType(I) == T)
array<T>::array(I f, I l) : p(0)
{
push<T, I>(sink(this), f, l);
}

template <typename T>


requires(Regular(T))
array<T>::array(const array& x) : p(allocate_array<T>(size(x)))
{
push<T>(sink(this), begin(x), end(x));
}

Stepanov, McJones Elements of Programming March 14, 2008 719 / 880


Destructor for array

template <typename T>


requires(Regular(T))
array<T>::~array()
{
pop_all(sink(this));
}

Stepanov, McJones Elements of Programming March 14, 2008 720 / 880


reserve

template <typename T>


requires(Regular(T))
void reserve(array<T>& x, DistanceType(IteratorType(array<T>)) n)
{
if (n < size(x) || n == capacity(x)) return;
array<T> tmp(n);
copy(begin(x), end(x), array_push_iterator<T>(tmp));
swap(tmp, x);
}

Stepanov, McJones Elements of Programming March 14, 2008 721 / 880


Equality and less than for array

template <typename T>


requires(Regular(T))
bool operator==(const array<T>& x, const array<T>& y)
{
return lexicographic_equivalence(begin(x), end(x), begin(y), end(y),
equality<T>());
}

template <typename T>


requires(Regular(T))
bool operator<(const array<T>& x, const array<T>& y)
{
return lexicographic_ordering(begin(x), end(x), begin(y), end(y), less<T>());
}

Stepanov, McJones Elements of Programming March 14, 2008 722 / 880


insert for array

template <typename T, typename I>


requires(Regular(T) &&
Readable(I) && Iterator(I) && ValueType(I) == T)
void insert(array<T>& x, IteratorType(array<T>) i, I f, I l)
{
DistanceType(IteratorType(array<T>)) o_f = i - begin(x);
DistanceType(IteratorType(array<T>)) o_m = end(x) - begin(x);
push(x, f, l);
rotate(begin(x) + o_f, begin(x) + o_m, end(x));
}

Stepanov, McJones Elements of Programming March 14, 2008 723 / 880


sort for array

template <typename T, typename R>


requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)
void sort(array<T>& x, R r)
{
typedef DistanceType(IteratorType(array<T>)) N;
N n = size(x) / N(10);
array<T> buffer(n, n, T());
stable_sort_n(begin(x), size(x), begin(buffer), n, r);
}

Stepanov, McJones Elements of Programming March 14, 2008 724 / 880


Introduction

We have presented algorithms and containers that work well


together when the elements of the containers are built-in types or
C-style structs
However performance suffers when the elements are themselves
larger containers
Sorting an array of arrays
In this section we present a solution to this problem using the
notions of underlying type and type compatibility
We rewrite versions of various algorithms from chapters 6-10
using this approach and define the underlying type for the
containers in chapter 11

Stepanov, McJones Elements of Programming March 14, 2008 725 / 880


Relational concepts (from Chapter 1)

Definition
A relational concept is a concept defined on two types

Stepanov, McJones Elements of Programming March 14, 2008 726 / 880


Concept SameGenus (from Chapter 1)

Definition
SameGenus(T , T 0 ) ⇒
The genus of T and the genus of T 0 are equal

Example
SameGenus(int, long)

Stepanov, McJones Elements of Programming March 14, 2008 727 / 880


Concept PartiallyCompatible (1 of 3) (from Chapter 6)

Definition (requirements, signatures)


PartiallyCompatible(T , U) ⇒
SameGenus(T , U)
The partial equality predicates (=) on T × U and U × T are defined
The (potentially implicit) unary predicates is_representableU (T )
and is_representableT (U) are defined
The partial assignments (←) on T × U and U × T are defined
The morphisms U(T ) and T (U) are defined

Stepanov, McJones Elements of Programming March 14, 2008 728 / 880


Concept PartiallyCompatible (2 of 3) (from Chapter 6)

Definition (definition spaces)


PartiallyCompatible(T , U) ⇒
For all t ∈ T and for all u ∈ U,
is_representableU (t) ∧ is_representableT (u) ⇒
is_defined(=, t, u) ∧ is_defined(=, u, t)
For all t ∈ T and for all u ∈ U,
is_representableT (u) ⇒ is_defined(←, t, u)
For all t ∈ T and for all u ∈ U,
is_representableU (t) ∧ is_representableT (u) ⇒
is_defined(U(·), t) ∧ is_defined(T (·), u)

Stepanov, McJones Elements of Programming March 14, 2008 729 / 880


Concept PartiallyCompatible (3 of 3) (from Chapter 6)

Definition (axioms)
PartiallyCompatible(T , U) ⇒
Where = is defined, it is true if and only if its arguments have the
same interpretation
← depends on the type but not the value of its first argument;
where it is defined, it makes the first argument equal to the second
without modifying the second
Where the morphisms U(T ) and T (U) are defined, each constructs
a new element of the target type that has the same interpretation
as its argument

Stepanov, McJones Elements of Programming March 14, 2008 730 / 880


Properties of PartiallyCompatible types (from Chapter 6)

Lemma
PartiallyCompatible(T , U) ⇒ PartiallyCompatible(U, T )

Lemma
If PartiallyCompatible(T , U), PartiallyCompatible(U, V),
PartiallyCompatible(T , V), then (∀t ∈ T )(∀u ∈ U)(∀v ∈ V):
t=u⇒u=t
(t = u ∧ u = v) ⇒ t = v
t←u⇒t=u
T t(u) ⇒ t = u

(Implication between a statement and a predicate means the


predicate holds immediately after execution of the statement)

Stepanov, McJones Elements of Programming March 14, 2008 731 / 880


Concept Compatible (from Chapter 6)

Definition
Compatible(T , U) ⇒
PartiallyCompatible(T , U)
For all t ∈ T and for all u ∈ U,
is_representableU (t) ∧ is_representableT (u)

Stepanov, McJones Elements of Programming March 14, 2008 732 / 880


Type requirements of copy (from Chapter 6)

#if 0 // *****

template <typename I, typename O>


requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))
O copy(I f, I l, O r);

Readable(I) ensures source is defined


Iterator(I) ensures ++ is defined
Writable(O) ensures sink is defined
Iterator(O) ensures ++ is defined
PartiallyCompatible(ValueT ype(I), ValueT ype(O)) ensures
assignment is defined

Stepanov, McJones Elements of Programming March 14, 2008 733 / 880


Preconditions and postconditions of copy (from
Chapter 6)
template <typename I, typename O>
requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))
O copy(I f, I l, O r);

Preconditions The input range must be readable


The output range must be writable and of at least
the size of the input range
Each value of the input range must be
representable in the output value type
If the ith iterator in the input range is aliased to the
jth iterator of the output, then i 6 j (informally,
every value is used before it is overwritten)
Postconditions The sequence of values in the output range is
equal to the sequence of original values in the
input range
Stepanov, McJones Elements of Programming March 14, 2008 734 / 880
Implementation of copy (from Chapter 6)

template <typename I, typename O>


requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))
O copy(I f, I l, O r)
{
while (f != l) {
sink(r) = source(f);
++f;
++r;
}
return r;
}

We return the end of the output range because it might not be


known to the caller, who might find it useful
It is worth the small constant time to return it

Stepanov, McJones Elements of Programming March 14, 2008 735 / 880


Example using copy (from Chapter 6)

template <typename N, typename O>


requires(Integer(N) && Writable(O) && Iterator(O) &&
PartiallyCompatible(N, ValueType(O)))
O iota(N n, O r) // like APL ι
{
return copy(N(0), n, r);
}

Stepanov, McJones Elements of Programming March 14, 2008 736 / 880


copy_bounded (from Chapter 6)

template <typename I, typename O>


requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))
pair<I, O> copy_bounded(I f, I l, O r_f, O r_l)
{
while (f != l && r_f != r_l) {
sink(r_f) = source(f);
++f;
++r_f;
}
return pair<I, O>(f, r_f);
}

While the ends of both ranges are known to the caller, returning
the pair allows determining which range is smaller and where in
the larger range copying stopped

Stepanov, McJones Elements of Programming March 14, 2008 737 / 880


copy_n (from Chapter 6)

template <typename I, typename O>


requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))
pair<I, O> copy_n(I f, DistanceType(I) n, O r)
{
typedef DistanceType(I) N;
while (n != N(0)) {
sink(r) = source(f);
++f;
++r;
n = n - N(1);
}
return pair<I, O>(f, r);
}

Stepanov, McJones Elements of Programming March 14, 2008 738 / 880


copy_k (from Chapter 6)

template <typename I, typename O, int k>


requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))
void copy_k(I& f, O& r)
{
copy_k<k - 1>(f, r);
sink(r) = source(f);
++f; ++r;
}

template <typename I, typename O>


void copy_k<0>(I&, O&) { }

Stepanov, McJones Elements of Programming March 14, 2008 739 / 880


copy_n_unrolled (from Chapter 6)

template <typename I, typename O>


requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))
pair<I, O> copy_n_unrolled(I f, DistanceType(I) n, I r)
{
typedef DistanceType(I) N;
const int k = 4; // unroll factor
while (n >= N(k)) {
copy_k<k>(f, r);
n = n - N(k);
}
return copy_n(f, n, r);
}

Stepanov, McJones Elements of Programming March 14, 2008 740 / 880


copy_backward (from Chapter 6)

The main difference from copy is the aliasing precondition: if the


ith iterator in the input range is aliased to the jth iterator of the
output, then j 6 i

template <typename I, typename O>


requires(Readable(I) && BidirectionalIterator(I) &&
Writable(O) && BidirectionalIterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))
O copy_backward(I f, I l, O r)
{
while (f != l) {
--l;
--r;
sink(r) = source(l);
}
return r;
}

Stepanov, McJones Elements of Programming March 14, 2008 741 / 880


copy_k_indexed (from Chapter 6)

template <typename I, typename O, int k>


requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))
void basic_copy_k_indexed(I f, O r)
{
basic_copy_k<k - 1>()(f, r);
sink(r + (k - 1)) = source(f + (k - 1));
}

template <typename I, typename O>


void basic_copy_k_indexed<0>(I, O) { }

template <typename I, typename O, int k>


requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) && ValueType(I) == ValueType(O))
void copy_k_indexed(I& f, O& r)
{
basic_copy_k<k>(f, r);
f += k; r += k;
}

Stepanov, McJones Elements of Programming March 14, 2008 742 / 880


copy_n_unrolled_indexed (from Chapter 6)

template <typename I, typename O>


requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))
pair<I, O> copy_n_unrolled_indexed(I f, DistanceType(I) n, O r)
{
typedef DistanceType(I) N;
const int k = 4; // unroll factor
while (n >= N(k)) {
copy_k_indexed<k>(f, r);
n = n - N(k);
}
return copy_n(f, n, r);
}

Stepanov, McJones Elements of Programming March 14, 2008 743 / 880


copy_parallel for disjoint ranges (from Chapter 6)

The main difference from copy is the aliasing precondition: the


input and output ranges do not alias
A special forall executes its body for all values of its iteration
variable in arbitrary order, possibly concurrently
template <typename I, typename O>
requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))
void copy_parallel(I f, DistanceType(I) n, O r)
{
typedef DistanceType(I) N;
forall(N i, N(0), n) {
sink(r + i) = source(f + i);
}
}

Stepanov, McJones Elements of Programming March 14, 2008 744 / 880


copy_backward_n (from Chapter 6)

copy_backward is not realizable for indexed iterators


copy_backward_n, which is often just as useful, is realizable

template <typename I, typename O>


requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))
O copy_backward_n(I f, DistanceType(I) n, O r)
{
typedef DistanceType(I) N;
while (n != N(0)) {
n = n - N(1);
sink(r + n) = source(f + n);
}
return r;
}

Stepanov, McJones Elements of Programming March 14, 2008 745 / 880


copy_from_segmented (from Chapter 6)

template <typename I, typename O>


requires(Readable(I) && SegmentedIterator(I) &&
Writable(O) && Iterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))
O copy_from_segmented(I f, I l, O r)
{
while (f + end(f) != l + end(l)) {
// f and l are in different segments
r = copy(begin(f), end(f), r);
f = f + end(f);
}
// f and l are in the same segment
return copy(begin(f), begin(l), r);
}

Stepanov, McJones Elements of Programming March 14, 2008 746 / 880


Underlying type (from Chapter 7)

Definition
For any type T , the underlying type U is an affiliated type satisfying:
T and U are compatible
References with value types T and U may be reinterpretively cast
into each other
Construction of type U and assignment to type U never throw an
exception
An object of type U may only be used to hold temporary values
while implementing a rearrangement of a range of T objects
A reference to an object of type U may be reinterpretively cast to a
reference to T and passed as a const T& parameter
We denote the underlying type of T as UnderlyingT ype(T )

Stepanov, McJones Elements of Programming March 14, 2008 747 / 880


Motivation for underlying type (from Chapter 7)

The implementation of types T and U could exploit this restriction


to save time
As we shall see when implementing containers, underlying type
allows multiple headers to point to the same data while
performing rearrangements

Stepanov, McJones Elements of Programming March 14, 2008 748 / 880


UnderlyingT ype type function (from Chapter 7)
template <typename T> requires(Regular(T))
struct underlying_type
{
typedef T type; // default
};

#define UnderlyingType(T) typename underlying_type<T>::type

template <typename I> requires(Readable(I) && Iterator(I))


const UnderlyingType(ValueType(I))&
underlying_source(I x)
{
return reinterpret_cast<
const UnderlyingType(ValueType(I))&>(source(x));
}

template <typename I> requires(Writable(I) && Iterator(I))


UnderlyingType(ValueType(I))&
underlying_sink(I x)
{
return reinterpret_cast<
UnderlyingType(ValueType(I))&>(sink(x));
}

Stepanov, McJones Elements of Programming March 14, 2008 749 / 880


cycle_2 (from Chapter 7)

template <typename I>


requires(Mutable(I) && Iterator(I))
void cycle_2(I x, I y)
{
UnderlyingType(ValueType(I)) t = underlying_source(x);
underlying_sink(x) = underlying_source(y);
underlying_sink(y) = t;
}

Stepanov, McJones Elements of Programming March 14, 2008 750 / 880


Underlying type and invariants (from Chapter 7)

To be correct, it is sufficient for a sequence of code using


UnderlyingT ype(T ) to:
Be within a critical section: that is, avoid concurrent access
Avoid any exceptions being thrown
Restore the class invariants of T by the end of the sequence

Reflection
Disciplined violation of invariants to enhance performance is perfectly
legitimate

Stepanov, McJones Elements of Programming March 14, 2008 751 / 880


cycle_left_3 (from Chapter 7)

template <typename I>


requires(Mutable(I) && Iterator(I))
void cycle_left_3(I x, I y, I z)
{
UnderlyingType(ValueType(I)) t = underlying_source(x);
underlying_sink(x) = underlying_source(y);
underlying_sink(y) = underlying_source(z);
underlying_sink(z) = t;
}

Stepanov, McJones Elements of Programming March 14, 2008 752 / 880


swap (from Chapter 7)

We can interchange the contents of two variables using


underlying type

#define UnderlyingRef(T) reinterpret_cast<UnderlyingType(T)&>

template <typename T>


requires(Regular(T))
void swap(T& x, T& y)
{
UnderlyingType(T) tmp;
tmp = UnderlyingRef(T)(x);
UnderlyingRef(T)(x) = UnderlyingRef(T)(y);
UnderlyingRef(T)(y) = tmp;
}

Stepanov, McJones Elements of Programming March 14, 2008 753 / 880


do_cycle_from (from Chapter 7)

template <typename I, typename P>


requires(Mutable(I) && ForwardIterator(I) &&
Transformation(P) && Domain(P) == I)
void do_cycle_from(I i, P p)
{
// Precondition: p is a from-permutation of the range containing i
UnderlyingType(ValueType(I)) t = underlying_source(i);
I f = i;
I n = p(i);
while (n != i) {
underlying_sink(f) = underlying_source(n);
f = n;
n = p(n);
}
underlying_sink(f) = t;
}

Stepanov, McJones Elements of Programming March 14, 2008 754 / 880


swap_ranges_n (from Chapter 7)

template <typename I1, typename I2, typename N>


requires(Mutable(I1) && Iterator(I1) &&
Mutable(I2) && Iterator(I2) &&
PartiallyCompatible(ValueType(I1), ValueType(I2)) &&
Integer(N))
pair<I1, I2> swap_ranges_n(I1 f1, I2 f2, N n)
{
while (n != N(0)) {
cycle_2(f1, f2);
++f1;
++f2;
n = n - N(1);
};
return pair<I1, I2>(f1, f2);
}

Stepanov, McJones Elements of Programming March 14, 2008 755 / 880


reverse_n_with_buffer (from Chapter 7)

template <typename I, typename B>


requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && BidirectionalIterator(B) &&
UnderlyingType(ValueType(I)) == ValueType(B))
I reverse_n_with_buffer(I f, DistanceType(I) n, B b)
{
return reverse_copy_n(copy_n(f, n, b), n, f);
}

Stepanov, McJones Elements of Programming March 14, 2008 756 / 880


reverse_n_adaptive (from Chapter 7)

template <typename I, typename B>


requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && BidirectionalIterator(B) &&
UnderlyingType(ValueType(I)) == ValueType(B))
I reverse_n_adaptive(I f, DistanceType(I) n, B b, DistanceType(I) n_b)
{
typedef DistanceType(I) N;
const N h = n / N(2);
const N r = n - N(2) * h;
if (h == N(0))
return f + n;
if (n <= n_b)
return reverse_n_with_buffer(f, n, b);
I m = reverse_n_adaptive(f, h, b, n_b);
m += r;
I l = reverse_n_adaptive(m, h, b, n_b);
swap_ranges_n(f, m, h);
return l;
}

Stepanov, McJones Elements of Programming March 14, 2008 757 / 880


do_fused_cycles_from_with_buffer (from Chapter
8)

template <typename I, typename P, typename B>


requires(Mutable(I) && ForwardIterator(I) &&
Transformation(P) && Domain(P) == I &&
Mutable(B) && ForwardIterator(B) &&
UnderlyingType(ValueType(I)) == ValueType(B))
void do_fused_cycles_from_with_buffer(I i, P p, B b, DistanceType(I) n)
{
// Precondition: p is a from-permutation on the range containing i
copy_n(i, b, n);
I f = i;
I next = p(i);
while (next != i) {
copy_n(next, f, n);
f = next;
next = p(next);
}
copy_n(b, f, n);
}

Stepanov, McJones Elements of Programming March 14, 2008 758 / 880


rotate_indexed_helper_fused (from Chapter 8)

template <typename I, typename P>


requires(Mutable(I) && IndexedIterator(I) &&
Transformation(P) && Domain(P) == I)
I rotate_indexed_helper_fused(I f, I m, I l, P p)
{
// Precondition: p is a from-permutation on [f, l)
typedef DistanceType(I) N;
const N fusion_factor(16);
UnderlyingType(ValueType(I)) buffer[fusion_factor];
N d = gcd(m - f, l - m);
N i(0);
while (i + fusion_factor < d) {
do_fused_cycles_from_with_buffer(f + i, p, buffer, fusion_factor);
i = i + fusion_factor;
}
do_fused_cycles_from_with_buffer(f + i, p, buffer, d - i);
return f + (l - m);
};

Stepanov, McJones Elements of Programming March 14, 2008 759 / 880


swap_ranges (from Chapter 8)

template <typename I1, typename I2>


requires(Mutable(I1) && Iterator(I1) &&
Mutable(I2) && Iterator(I2) &&
PartiallyCompatible(ValueType(I1), ValueType(I2)))
pair<I1, I2> swap_ranges(I1 f1, I1 l1, I2 f2, I2 l2)
{
while (f1 != l1 && f2 != l2) {
cycle_2(f1, f2);
++f1;
++f2;
};
return pair<I1, I2>(f1, f2);
}

For m ∈ [f, l) such that l − m = m − f, swap_ranges(f, m, m, l)


rotates [f, l) about m

Stepanov, McJones Elements of Programming March 14, 2008 760 / 880


partition_copy_n (from Chapter 9)

template <typename I, typename O0, typename O1, typename P>


requires(Readable(I) && Iterator(I) &&
Writable(O0) && Iterator(O0) &&
Writable(O1) && Iterator(O1) &&
UnaryPredicate(P) && Domain(P) == ValueType(I) &&
PartiallyCompatible(ValueType(I), ValueType(O0)) &&
PartiallyCompatible(ValueType(I), ValueType(O1)))
pair<O0, O1> partition_copy_n(I f, DistanceType(I) n, O0 r0, O1 r1, P p)
{
typedef DistanceType(I) N;
while (n != N(0)) {
if (p(source(f))) {
sink(r1) = source(f); ++r1;
} else {
sink(r0) = source(f); ++r0;
}
++f;
n = n - N(1);
}
return pair<O0, O1>(r0, r1);
}

Stepanov, McJones Elements of Programming March 14, 2008 761 / 880


single_cycle_partition (from Chapter 9)

template <typename I, typename P>


requires(Readable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I single_cycle_partition(I f, I l, P p)
{
f = find_if(f, l, p);
l = find_backward_if_not(f, l, p);
if (f == l) return f;
UnderlyingType(ValueType(I)) tmp = source(f);
while (true) {
--l;
sink(f) = source(l);
f = find_if(successor(f), l, p);
if (f == l) {
sink(l) = tmp;
return f;
}
sink(l) = source(f);
l = assured_find_backward_if_not(l, p);
}
}

Stepanov, McJones Elements of Programming March 14, 2008 762 / 880


underlying_forward_iterator (from Chapter 9)

template <typename I>


requires(ForwardIterator(I))
struct underlying_forward_iterator
{
typedef UnderlyingType(ValueType(I)) U;
typedef DistanceType(I) N;
typedef underlying_forward_iterator UFI;
I i;
underlying_forward_iterator() {}
underlying_forward_iterator(const I& x) : i(x) {}
operator I() { return i; }
void operator++() { ++i; }
UFI operator+(N n) { return i + n; }
friend N operator-(UFI x, UFI y) { return x.i - y.i; }
friend bool operator==(const UFI& x, const UFI& y) { return x.i == y.i; }
friend const U& source(const UFI& x) { return underlying_source(x.i); }
friend U& sink(UFI& x) { return underlying_sink(x.i); }
};
#define UFI(I) underlying_forward_iterator<I>

Stepanov, McJones Elements of Programming March 14, 2008 763 / 880


underlying_bidirectional_iterator (from Chapter 9)

template <typename I>


requires(BidirectionalIterator(I))
struct underlying_bidirectional_iterator
{
typedef UnderlyingType(ValueType(I)) U;
typedef DistanceType(I) N;
typedef underlying_bidirectional_iterator UBI;
I i;
underlying_bidirectional_iterator() {}
underlying_bidirectional_iterator(const I& x) : i(x) {}
operator I() { return i; }
void operator++() { ++i; }
void operator--() { --i; }
UBI operator+(N n) { return i + n; }
friend N operator-(UBI x, UBI y) { return x.i - y.i; }
friend bool operator==(const UBI& x, const UBI& y) { return x.i == y.i; }
friend const U& source(const UBI& x) { return underlying_source(x.i); }
friend U& sink(UBI& x) { return underlying_sink(x.i); }
};
#define UBI(I) underlying_bidirectional_iterator<I>

Stepanov, McJones Elements of Programming March 14, 2008 764 / 880


Type functions for underlying_forward_iterator
(from Chapter 9)
template <typename I>
requires(ForwardIterator(I))
struct value_type<underlying_forward_iterator<I> >
{
typedef UnderlyingType(ValueType(I)) type;
};

template <typename I>


requires(ForwardIterator(I))
struct distance_type<underlying_forward_iterator<I> >
{
typedef DistanceType(I) type;
};

template <typename I>


requires(ForwardIterator(I))
struct iterator_category<underlying_forward_iterator<I> >
{
typedef forward_iterator_tag category;
};

Stepanov, McJones Elements of Programming March 14, 2008 765 / 880


stable_partition_n_with_buffer (from Chapter 9)

template <typename I, typename B, typename P>


requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&
UnaryPredicate(P) && Domain(P) == ValueType(I) &&
UnderlyingType(ValueType(I)) == ValueType(B))
pair<I, I> stable_partition_n_with_buffer(I f, DistanceType(I) n, B b, P p)
{
pair<UFI(I), B> r = partition_copy_n(UFI(I)(f), n, UFI(I)(f), b, p);
return pair<I, I>(I(r.first), I(copy(b, r.m1, r.first)));
}

Stepanov, McJones Elements of Programming March 14, 2008 766 / 880


stable_partition_n_adaptive (from Chapter 9)

template <typename I, typename B, typename P>


requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&
UnaryPredicate(P) && Domain(P) == ValueType(I) &&
UnderlyingType(ValueType(I)) == ValueType(B))
pair<I, I> stable_partition_n_adaptive(
I f, DistanceType(I) n, B b, DistanceType(I) n_b, P p)
{
typedef DistanceType(I) N;
if (n == N(0))
return stable_partition_0(f, p);
if (n == N(1))
return stable_partition_1(f, p);
if (n <= n_b)
return stable_partition_n_with_buffer(f, n, b, p);
N h = half_nonnegative(n);
pair<I, I> r0 = stable_partition_n_adaptive(f, h, b, n_b, p);
pair<I, I> r1 = stable_partition_n_adaptive(r0.m1, n - h, b, n_b, p);
return stable_partition_combine(r0, r1);
}

Stepanov, McJones Elements of Programming March 14, 2008 767 / 880


merge_copy_n (from Chapter 9)

template <typename I0, typename I1, typename O, typename R>


requires(Readable(I0) && Iterator(I0) && Readable(I1) && Iterator(I1) &&
Writable(O) && Iterator(O) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&
PartiallyCompatible(ValueType(I0), ValueType(O)) &&
PartiallyCompatible(ValueType(I1), ValueType(O)))
void merge_copy_n(I0 f0, DistanceType(I0) n0,
I1 f1, DistanceType(I1) n1, O o, R r)
{
typedef DistanceType(I0) N0; typedef DistanceType(I1) N1;
while (true) {
if (n0 == N0(0)) { copy_n(f1, n1, o); return; }
if (n1 == N1(0)) { copy_n(f0, n0, o); return; }
if (r(source(f1), source(f0))) {
sink(o) = source(f1); ++f1; n1 = n1 - N1(1);
} else {
sink(o) = source(f0); ++f0; n0 = n0 - N0(1);
}
++o;
}
}

Stepanov, McJones Elements of Programming March 14, 2008 768 / 880


underlying_compare (from Chapter 9)

template <typename T, typename R>


requires(Regular(T) && Relation(R) && Domain(R) == T)
struct underlying_compare
{
typedef UnderlyingType(T) U;
R r;
underlying_compare(R r) : r(r) {}
bool operator()(const U& x, const U& y)
{
return r(reinterpret_cast<const T&>(x), reinterpret_cast<const T&>(y));
}
};

Stepanov, McJones Elements of Programming March 14, 2008 769 / 880


merge_n_with_buffer (from Chapter 9)

template <typename I, typename B, typename R>


requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&
UnderlyingType(ValueType(I)) == ValueType(B))
void merge_n_with_buffer(
I f0, DistanceType(I) n0, I f1, DistanceType(I) n1, B b, R r)
{
typedef underlying_compare<ValueType(I), R> UR;
copy_n(UFI(I)(f0), n0, b);
merge_copy_n(b, n0, UFI(I)(f1), n1, UFI(I)(f0), UR(r));
}

Stepanov, McJones Elements of Programming March 14, 2008 770 / 880


merge_n_adaptive (from Chapter 9)
template <typename I, typename B, typename R>
requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&
UnderlyingType(ValueType(I)) == ValueType(B))
void merge_n_adaptive(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1,
B b, DistanceType(I) n_b, R r)
{
typedef DistanceType(I) N;
if (n0 + n1 < N(8)) // ***** MEASURE AND TUNE
return insertion_merge_n(f0, n0, f1, n1, r);
if (n0 <= n_b)
return merge_n_with_buffer(f0, n0, f1, n1, b, r);
I i, j;
if (n0 > n1) {
i = f0 + half_nonnegative(n0);
j = lower_bound_n(f1, n1, source(i), r);
} else {
j = f1 + half_nonnegative(n1);
i = upper_bound_n(f0, n0, source(j), r);
}
I m = rotate(i, f1, j);
merge_n_adaptive(f0, i - f0, i, m - i, b, n_b, r);
merge_n_adaptive(m, j - m, j, n1 - (j - f1), b, n_b, r);
} Stepanov, McJones Elements of Programming March 14, 2008 771 / 880
stable_sort_n_adaptive (from Chapter 9)

template <typename I, typename B, typename R>


requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&
UnderlyingType(ValueType(I)) == ValueType(B))
I stable_sort_n_adaptive(
I f, DistanceType(I) n, B b, DistanceType(I) n_b, R r)
{
typedef DistanceType(I) N;
if (n < N(16))
return binary_insertion_sort_n(f, n, r);
N h = half_nonnegative(n);
I m = stable_sort_n_adaptive(f, h, b, n_b, r);
I l = stable_sort_n_adaptive(m, n - h, b, n_b, r);
merge_n_adaptive(f, n / 2, m, n - h, b, n_b, r);
return l;
}

Stepanov, McJones Elements of Programming March 14, 2008 772 / 880


Type functions for list (from Chapter 11)

template <typename T> requires(Regular(T))


struct underlying_type< list<T> >
{
typedef list_iterator<T> type; // or IteratorType(list<T>)
};

template <typename C>


requires(Container(C))
struct iterator_type
{
typedef C type;
};
#define IteratorType(C) typename iterator_type<C>::type

template <typename T> requires(Regular(T))


struct iterator_type< list<T> >
{
typedef list_iterator<T> type;
};

Stepanov, McJones Elements of Programming March 14, 2008 773 / 880


Type functions for array (from Chapter 11)

template <typename T> requires(Regular(T))


struct underlying_type< array<T> >
{
typedef struct { array_header<T>* p; } type;
};

template <typename T> requires(Regular(T))


struct iterator_type< array<T> >
{
typedef T* type;
};

Stepanov, McJones Elements of Programming March 14, 2008 774 / 880


reserve (from Chapter 11)

template <typename T>


requires(Regular(T))
void reserve(array<T>& a, DistanceType(IteratorType(array<T>)) n)
{
if (n < size(a) || n == capacity(a)) return;
typedef UnderlyingType<T> U;
typedef UFI(IteratorType(array<T>)) UI;
array<U> tmp(n)
copy(UI(begin(a)), UI(end(a)), array_push_iterator<U>); // never throws
swap(tmp, a);
}

Stepanov, McJones Elements of Programming March 14, 2008 775 / 880


sort for array (from Chapter 11)

template <typename T, typename R>


requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)
void sort(array<T>& x, R r)
{
typedef DistanceType(IteratorType(array<T>)) N;
typedef UnderlyingType(T) U;
N n = size(x) / N(10);
array<U> buffer(n, n, U());
stable_sort_n_adaptive(begin(x), size(x), begin(buffer), n, r);
}

#endif // *****

Stepanov, McJones Elements of Programming March 14, 2008 776 / 880


Conclusions

...

Stepanov, McJones Elements of Programming March 14, 2008 777 / 880


Contents I

1 Introduction

2 Foundations

3 Transformations and their orbits

4 Algorithms on algebraic structures

5 Orderings

6 Combining concepts

7 Refining concepts of iterators

Stepanov, McJones Elements of Programming March 14, 2008 778 / 880


Contents II
8 Permutations and rearrangements

9 Rotations

10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects

13 Iterative algorithms for divide-and-conquer


Reduction
Sorting
Stable partition redux

Stepanov, McJones Elements of Programming March 14, 2008 779 / 880


Contents III
Conclusions
Reading
Project

14 Mathematical notation

15 C++ machinery

16 Acknowledgments

17 Index

Stepanov, McJones Elements of Programming March 14, 2008 780 / 880


Concept PartialSemigroupOperation

Definition
PartialSemigroupOperation(Op) ⇒
BinaryOperation(Op)
For all op ∈ Op and for all a, b, c, ∈ Domain(Op), if

is_defined(op, a, b) ∧ is_defined(op, b, c)

then
is_defined(op, op(a, b), c) ∧ is_defined(op, a, op(b, c)) ∧
op(op(a, b), c) = op(a, op(b, c))

Stepanov, McJones Elements of Programming March 14, 2008 781 / 880


reduce_nonempty

template <typename I, typename Op>


requires(Readable(I) && Iterator(I) &&
SemigroupOperation(Op) && Domain(Op) == ValueType(I))
Domain(Op) reduce_nonempty(I f, I l, Op op)
{
// Precondition: f , l
Domain(Op) r = source(f);
while (true) {
++f;
if (f == l) return r;
r = op(r, source(f));
}
}

Stepanov, McJones Elements of Programming March 14, 2008 782 / 880


reduce

template <typename I, typename Op>


requires(Readable(I) && Iterator(I) &&
MonoidOperation(Op) && Domain(Op) == ValueType(I))
Domain(Op) reduce(I f, I l, Op op, const Domain(Op)& z)
{
if (f == l) return z;
return reduce_nonempty(f, l, op);
}

In some situations the caller may need to supply a value for z


other than identity_element(op)
***** Explain that there can be many monoids within Domain(Op)
?????

Stepanov, McJones Elements of Programming March 14, 2008 783 / 880


find_not

template <typename I>


requires(Readable(I) && Iterator(I))
I find_not(I f, I l, const ValueType(I)& x)
{
while (f != l && source(f) == x) ++f;
return f;
}

Stepanov, McJones Elements of Programming March 14, 2008 784 / 880


reduce_nonzeroes

template <typename I, typename Op>


requires(Readable(I) && Iterator(I) &&
MonoidOperation(Op) && Domain(Op) == ValueType(I))
Domain(Op) reduce_nonzeroes(I f, I l, Op op, const Domain(Op)& z)
{
f = find_not(f, l, z);
if (f == l) return z;
Domain(Op) r = source(f);
while (true) {
f = find_not(successor(f), l, z);
if (f == l) return r;
r = op(r, source(f));
}
}

Stepanov, McJones Elements of Programming March 14, 2008 785 / 880


add_to_counter
template <typename I, typename Op>
requires(Mutable(I) && ForwardIterator(I) &&
BinaryOperation(Op) && Domain(Op) == ValueType(I))
Domain(Op) add_to_counter(I f, I l, Op op, Domain(Op) x, const Domain(Op)& z)
{
if (x == z) return z;
while (f != l) {
if (source(f) != z) {
x = op(source(f), x);
sink(f) = z;
} else {
sink(f) = x;
return z;
}
++f;
}
return x;
}

Formally, this procedure requires an Iterator type


The intent is to call it many times with the same mutable counter
Thus we specify the requirement as ForwardIterator
Stepanov, McJones Elements of Programming March 14, 2008 786 / 880
tranpose_operation

template <typename Op>


requires(BinaryOperation(Op))
struct transpose_operation
{
Op op;
transpose_operation(Op op) : op(op) { }
typedef Domain(Op) T;
T operator()(const T& x, const T& y)
{
return op(y, x);
}
};

A more general version would allow functions with different


codomain and domain

Stepanov, McJones Elements of Programming March 14, 2008 787 / 880


reduce_balanced

template <int k, typename I, typename Op>


requires(Readable(I) && ForwardIterator(I) &&
MonoidOperation(Op) && Domain(Op) == ValueType(I))
Domain(Op) reduce_balanced(I f, I l, Op op, const Domain(Op)& z)
{
// Precondition: 2k > l − f
typedef array_k<k, Domain(Op)> C;
C counter;
IteratorType(C) c_f = begin(counter), c_l = c_f;
while (f != l) {
Domain(Op) carry = add_to_counter(c_f, c_l, op, source(f), z);
if (carry != z) {
sink(c_l) = carry;
++c_l;
};
++f;
}
return reduce_nonzeroes(c_f, c_l, transpose_operation<Op>(op), z);
}

Stepanov, McJones Elements of Programming March 14, 2008 788 / 880


merger_linkable

template <typename S, typename R>


requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))
struct merger_linkable
{
typedef IteratorType(S) I;
I l;
R r;
S set_link;
merger_linkable(I l, R r, S set_link) : l(l), r(r), set_link(set_link) {}
I operator()(I x, I y)
{
return merge_linkable_nonempty(x, l, y, l, r, set_link).first;
}
};

Stepanov, McJones Elements of Programming March 14, 2008 789 / 880


sort_linkable
template <int k, typename S, typename R>
requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))
IteratorType(S) sort_linkable(IteratorType(S) f, IteratorType(S) l, R r, S set_link
{
// Precondition: 2k > l − f
typedef merger_linkable<S, R> Op;
Op op(l, r, set_link);
typedef IteratorType(S) I;
typedef array_k<k, I> C;
C counter;
IteratorType(C) c_f = begin(counter), c_l = c_f;
while (f != l) {
I old_f = f;
++f;
set_link(old_f, l);
I carry = add_to_counter(c_f, c_l, op, old_f, l);
if (carry != l) {
sink(c_l) = carry;
++c_l;
}
}
return reduce_nonzeroes(c_f, c_l, transpose_operation<Op>(op), l);
}
Stepanov, McJones Elements of Programming March 14, 2008 790 / 880
sort_forward_linkable

template <int k, typename I, typename R>


requires(Readable(I) && LinkableForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
I sort_forward_linkable(I f, I l, R r)
{
return sort_linkable<32>(f, l, r, forward_linker<I>());
}

Stepanov, McJones Elements of Programming March 14, 2008 791 / 880


sort_bidirectional_linkable

template <int k, typename I, typename R>


requires(Readable(I) && LinkableBidirectionalIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
I sort_bidirectional_linkable(I f, I l, R r)
{
f = sort_linkable<32>(f, l, r, forward_linker<I>());
restore_backward_links(f, l, backward_linker<I>());
return f;
}

Stepanov, McJones Elements of Programming March 14, 2008 792 / 880


combine_ranges

template <typename I>


requires(ForwardIterator(I)) // but not Readable(I)
struct combine_ranges
{
typedef pair<I, I> Pair;
Pair operator()(const Pair& x, const Pair& y) const
{
return Pair(
rotate(x.first, x.second, y.first),
y.second);
}
};

Stepanov, McJones Elements of Programming March 14, 2008 793 / 880


partition_trivial

template <typename I, typename P>


requires(ForwardIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
struct partition_trivial
{
P p;
partition_trivial(const P & p) : p(p) {}
pair<I, I> operator()(I i) const {
// return stable_partition_1<I, P>(i, p);
if (p(source(i)))
return pair<I, I>(i, i);
else
return pair<I, I>(i, successor(i));
}
};

Stepanov, McJones Elements of Programming March 14, 2008 794 / 880


value_iterator
template <typename I, typename F>
requires(Incrementable(I) && Transformation(F) && Domain(F) == I)
struct value_iterator
{
I i;
F f;
value_iterator() {}
value_iterator(const I& i, const F& f) : i(i), f(f) {}
void operator++() {
++i;
}
friend Codomain(F) source(const value_iterator& x) {
return x.f(x.i);
}
friend bool operator==(const value_iterator& x, const value_iterator& y) {
// Precondition: x.f = y.f
return x.i == y.i;
}
friend bool operator!=(const value_iterator& x, const value_iterator& y) {
return !(x == y);
}
};

Stepanov, McJones Elements of Programming March 14, 2008 795 / 880


stable_partition_iterative

template <typename I, typename P>


requires(ForwardIterator(I) && UnaryPredicate(P) && ValueType(I) == Domain(P))
I stable_partition_inplace_iterative(I f, I l, P p)
{
typedef partition_trivial<I, P> Fun;
typedef value_iterator<I, Fun> RangeIterator;
typedef combine_ranges<I> Op;
Fun fun(p);
RangeIterator f1(f, fun);
RangeIterator l1(l, fun);
combine_ranges<I> op;
pair<I, I> z(l, l);
return reduce_balanced<32, RangeIterator, Op>(f1, l1, op, z).first;
}

***** There is a bug in this code

Stepanov, McJones Elements of Programming March 14, 2008 796 / 880


Conclusions

***** reduce_balanced and Huffman monoid *****

Stepanov, McJones Elements of Programming March 14, 2008 797 / 880


Reading

Stepanov, McJones Elements of Programming March 14, 2008 798 / 880


Project

Project
***** Write an iterative version of stable_partition_n and
stable_partition_n_adaptive from Chapter 9 using
reduce_balanced
***** Give hint that many z’s per op will be required ?????
***** Compare the performance of the corresponding recursive
and iterative versions

Stepanov, McJones Elements of Programming March 14, 2008 799 / 880


Contents I

1 Introduction

2 Foundations

3 Transformations and their orbits

4 Algorithms on algebraic structures

5 Orderings

6 Combining concepts

7 Refining concepts of iterators

Stepanov, McJones Elements of Programming March 14, 2008 800 / 880


Contents II
8 Permutations and rearrangements

9 Rotations

10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects

13 Iterative algorithms for divide-and-conquer

14 Mathematical notation

Stepanov, McJones Elements of Programming March 14, 2008 801 / 880


Contents III
15 C++ machinery

16 Acknowledgments

17 Index

Stepanov, McJones Elements of Programming March 14, 2008 802 / 880


Connectives

Definition
If P and Q are propositions, then so are ¬P (read as “not P”), P ∨ Q (“P
or Q”), P ∧ Q (“P and Q”), P ⇒ Q (“P implies Q”), and P ⇔ Q (“P is
equivalent to Q”), with these truth tables:

P Q ¬P P∨Q P∧Q P⇒Q P⇔Q


F F T F F T T
F T T T F T F
T F F T F F F
T T F T T T T

For equivalence, we often write P if and only if Q, abbreviated


P iff Q

Stepanov, McJones Elements of Programming March 14, 2008 803 / 880


Existential and universal quantifiers

Definition
If P is a proposition and x is a variable, then (∃x)P is a proposition
(read as "there exists x such that P")

Definition
If P is a proposition and x is a variable, then (∀x)P is a proposition
(read as “for all x, P”); (∀x)P ⇔ (¬(∃x)¬P)

Stepanov, McJones Elements of Programming March 14, 2008 804 / 880


Sets and functions

We use the common vocabulary from naive set theory:


a ∈ X (“a is an element of X”)
X ⊂ Y (“X is a subset of Y”)
{a0 , . . . , an } (“the finite set with elements a0 , . . . , and an ”)
{a ∈ X|P(a)} (“the subset of X for which the predicate P holds”)
X ∪ Y (“the union of X and Y”)
X ∩ Y (“the intersection of X and Y”)
X − Y (“the complement of Y in X”)
X × Y (“the direct product of X and Y”)
f : X → Y (“f is a function from X to Y”)
X is the domain of f
Y is the codomain of f

Stepanov, McJones Elements of Programming March 14, 2008 805 / 880


Contents I

1 Introduction

2 Foundations

3 Transformations and their orbits

4 Algorithms on algebraic structures

5 Orderings

6 Combining concepts

7 Refining concepts of iterators

Stepanov, McJones Elements of Programming March 14, 2008 806 / 880


Contents II
8 Permutations and rearrangements

9 Rotations

10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects

13 Iterative algorithms for divide-and-conquer

14 Mathematical notation

Stepanov, McJones Elements of Programming March 14, 2008 807 / 880


Contents III
15 C++ machinery
Foundations
DistanceT ype type function
Special cases of Integer procedures
Default implementations for additive and multiplicative types
Orderings
Iterators
Category dispatch, with reverse and reverse_n examples
Increasing sequences
Linkable ranges
Bifurcate coordinates
Collocated types
fill_iterator

16 Acknowledgments

Stepanov, McJones Elements of Programming March 14, 2008 808 / 880


Contents IV
17 Index

Stepanov, McJones Elements of Programming March 14, 2008 809 / 880


Syntax for requires and models

#define requires(...)

#define models(...)

Stepanov, McJones Elements of Programming March 14, 2008 810 / 880


ArgumentT ype, Domain, and Codomain type
functions

A C++ technique called a trait class can be used to simulate a type


function:
template <typename T, int i>
requires(Procedure(T))
struct argument_type;

#define ArgumentType(T, i) typename argument_type< T, i >::type


#define Domain(T) ArgumentType(T, 0)

template <typename T>


requires(Procedure(T))
struct codomain_type;

#define Codomain(T) typename codomain_type< T >::type

// The macros work only inside a template definition


// because of the use of the keyword typename

Stepanov, McJones Elements of Programming March 14, 2008 811 / 880


DistanceT ype type function

template <typename T>


requires(Countable(T)) /∗ ????? ∗/
struct distance_type
{
typedef size_t type;
};

template <typename T>


requires(Regular(T))
struct distance_type<T*>
{
typedef ptrdiff_t type;
};
#define DistanceType(T) typename distance_type< T >::type

template <>
struct distance_type<short>
{
typedef unsigned short type;
};

Stepanov, McJones Elements of Programming March 14, 2008 812 / 880


Default implementations of special cases of Integer
procedures
// The successor defined in Chapter 6 for Iterator suffices
template <typename I> requires(Integer(I))
I predecessor(I a) { --a; return a; }
template <typename I> requires(Integer(I))
I half_nonnegative(const I& a) { return a >> I(1); }
template <typename I> requires(Integer(I))
void halve_nonnegative(I& a) { a >>= I(1); }
template <typename I> requires(Integer(I))
I binary_scale_down_nonnegative(const I& a, const I& k) { return a >> k; }
template <typename I> requires(Integer(I))
I binary_scale_up_nonnegative(const I& a, const I& k) { return a << k; }
template <typename I> requires(Integer(I))
bool is_positive(const I& a) { return I(0) < a; }
template <typename I> requires(Integer(I))
bool is_negative(const I& a) { return a < I(0); }
template <typename I> requires(Integer(I))
bool is_zero(const I& a) { return a == I(0); }
template <typename I> requires(Integer(I))
bool is_even(const I& a) { return (a & I(1)) == I(0); }
template <typename I> requires(Integer(I))
bool is_odd(const I& a) { return (a & I(1)) != I(0); }

Stepanov, McJones Elements of Programming March 14, 2008 813 / 880


Default identity and inverse for additive types

template <typename T>


requires(AdditiveSemigroup(T))
struct plus {
T operator()(const T& x, const T& y) const { return x + y; }
};

template <typename T>


requires(AdditiveMonoid(T))
T identity_element(const plus<T>&) { return T(0); }

template <typename T>


requires(AdditiveGroup(T))
struct negate {
T operator()(const T& x) const { return -x; }
};

template <typename T>


requires(AdditiveGroup(T))
negate<T> inverse_operation(const plus<T>&) {
return negate<T>();
}

Stepanov, McJones Elements of Programming March 14, 2008 814 / 880


Define ArgumentT ype(plusT , 0)

template <typename T>


requires(AdditiveSemigroup(T))
struct argument_type<plus<T>, 0>
{
typedef T type;
};

Stepanov, McJones Elements of Programming March 14, 2008 815 / 880


Default identity and inverse for multiplicative types

template <typename T>


requires(MultiplicativeSemigroup(T))
struct multiplies {
T operator()(const T& x, const T& y) const { return x * y; }
};

template <typename T>


requires(MultiplicativeMonoid(T))
T identity_element(const multiplies<T>&) { return T(1); }

template <typename T>


requires(MultiplicativeGroup(T))
struct reciprocal {
T operator()(const T& x) const { return T(1) / x; }
};

template <typename T>


requires(MultiplicativeGroup(T))
reciprocal<T> inverse_operation(const multiplies<T>&) {
return reciprocal<T>();
}

Stepanov, McJones Elements of Programming March 14, 2008 816 / 880


Default ordering for StrictTotallyOrdered

template <typename T>


requires(StrictTotallyOrdered(T))
struct less
{
bool operator()(const T& x, const T& y) const { return x < y; }
};

template <typename T>


requires(StrictTotallyOrdered(T))
struct argument_type<less<T>, 0>
{
typedef T type;
};

Stepanov, McJones Elements of Programming March 14, 2008 817 / 880


Default order selection for StrictTotallyOrdered

template <typename T>


requires(StrictTotallyOrdered(T))
T& min(T& a, T& b)
{
return min(a, b, less<T>());
}

Stepanov, McJones Elements of Programming March 14, 2008 818 / 880


Dereferenceable functions for T ∗

// We will define this later


template <typename T>
requires(Regular(T))
struct value_type;

template <typename T>


requires(Regular(T))
struct value_type<T*> { typedef T type; };

#define ValueType(T) typename value_type< T >::type

template <typename T>


requires(Regular(T))
const T& source(T* x) { return *x; }

template <typename T>


requires(Regular(T))
T& sink(T* x) { return *x; }

Stepanov, McJones Elements of Programming March 14, 2008 819 / 880


More aliasing tests
template <typename T0, typename T1>
requires(Writable(T0) && Writable(T1) && ValueType(T0) == ValueType(T1))
bool is_aliased_sinks(const T0& x0, const T1& x1)
{
// Return true ⇔
// For all U with Readable(U) and for all y ∈ U,
// is_aliased(x0, y) == is_aliased(x1, y)
// For many types, this works:
typedef const ValueType(T0)* P;
return P(&sink(x0)) == P(&sink(x1));
}

template <typename T0, typename T1>


requires(Readable(T0) && Readable(T1) && ValueType(T0) == ValueType(T1))
bool is_aliased_sources(const T0& x0, const T1& x1)
{
// Return true ⇔
// For all U with Writable(U) and for all y ∈ U,
// is_aliased(y, x0) == is_aliased(y, x1)
// For many types, this works:
typedef const ValueType(T0)* P;
return P(&source(x0)) == P(&source(x1));
}

Stepanov, McJones Elements of Programming March 14, 2008 820 / 880


Default Dereferenceable functions for T

template <typename T>


requires(Regular(T))
struct value_type
{
typedef T type;
};

template <typename T>


requires(Regular(T))
const T& source(const T& x)
{
return x;
}

template <typename T>


requires(Regular(T))
T& sink(T& x)
{
return x;
}

Stepanov, McJones Elements of Programming March 14, 2008 821 / 880


Mechanics of category dispatch

A tag type (containing no data) is defined for each category


A type function is defined to obtain a type tag from a type
Overloading is used to select from multiple signatures differing
only by a tag type

Stepanov, McJones Elements of Programming March 14, 2008 822 / 880


Iterator tag types

BI
It FI RI
II

struct iterator_tag {};


struct forward_iterator_tag {};
struct bidirectional_iterator_tag {};
struct indexed_iterator_tag {};
struct random_access_iterator_tag {};

Stepanov, McJones Elements of Programming March 14, 2008 823 / 880


IteratorCategory type function

template <typename T>


requires(Iterator(T))
struct iterator_category
{
typedef iterator_tag category;
};

#define IteratorCategory(T) typename iterator_category< T >::category

template <typename T>


requires(Regular(T))
struct iterator_category<T*>
{
typedef random_access_iterator_tag category;
};

Stepanov, McJones Elements of Programming March 14, 2008 824 / 880


Category dispatch for reverse_n and reverse

template <typename I>


requires(Mutable(I) && ForwardIterator(I))
void reverse_n(I f, DistanceType(I) n)
{
reverse_n(f, n, IteratorCategory(I)());
}

template <typename I>


requires(Mutable(I) && ForwardIterator(I))
void reverse(I f, I l)
{
reverse(f, l, IteratorCategory(I)());
}

Stepanov, McJones Elements of Programming March 14, 2008 825 / 880


Category dispatch cases for reverse_n
template <typename I> requires(Mutable(I) && ForwardIterator(I))
void reverse_n(I f, DistanceType(I) n, forward_iterator_tag)
{
reverse_n_forward(f, n);
}

template <typename I> requires(Mutable(I) && BidirectionalIterator(I))


void reverse_n(I f, DistanceType(I) n, bidirectional_iterator_tag)
{
reverse_n_bidirectional(f, n);
}

template <typename I> requires(Mutable(I) && IndexedIterator(I))


void reverse_n(I f, DistanceType(I) n, indexed_iterator_tag)
{
reverse_n_indexed(f, n);
}

template <typename I> requires(Mutable(I) && RandomAccessIterator(I))


void reverse_n(I f, DistanceType(I) n, random_access_iterator_tag)
{
reverse_n_indexed(f, n);
}

Stepanov, McJones Elements of Programming March 14, 2008 826 / 880


Category dispatch cases for reverse
template <typename I> requires(Mutable(I) && ForwardIterator(I))
void reverse(I f, I l, forward_iterator_tag)
{
reverse_forward(f, l);
}

template <typename I> requires(Mutable(I) && BidirectionalIterator(I))


void reverse(I f, I l, bidirectional_iterator_tag)
{
reverse_bidirectional(f, l);
}

template <typename I> requires(Mutable(I) && IndexedIterator(I))


void reverse(I f, I l, indexed_iterator_tag)
{
reverse_indexed(f, l);
}

template <typename I> requires(Mutable(I) && RandomAccessIterator(I))


void reverse(I f, I l, random_access_iterator_tag)
{
reverse_indexed(f, l);
}

Stepanov, McJones Elements of Programming March 14, 2008 827 / 880


find_out_of_order

template <typename I, typename R>


requires(Readable(I) && ForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
I find_out_of_order(I f, I l, R r, forward_iterator_tag)
{
if (f == l) return l;
I n(successor(f));
while (n != l) {
if (r(source(n), source(f)))
return n;
f = n;
++n;
}
return l;
}

Stepanov, McJones Elements of Programming March 14, 2008 828 / 880


find_out_of_order

template <typename I, typename R>


requires(Readable(I) && Iterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
I find_out_of_order(I f, I l, R r, iterator_tag)
{
if (f == l) return l;
I n(successor(f));
ValueType(I) v = source(f);
while (n != l) {
if (r(source(n), v))
return n;
v = source(n);
++n;
}
return l;
}

Stepanov, McJones Elements of Programming March 14, 2008 829 / 880


Category dispatch for find_out_of_order
template <typename I, typename R>
requires(Readable(I) && RandomAccessIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
I find_out_of_order(I f, I l, R r, random_access_iterator_tag)
{
return find_out_of_order(f, l, r, forward_iterator_tag());
}

template <typename I, typename R>


requires(Readable(I) && BidirectionalIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
I find_out_of_order(I f, I l, R r, bidirectional_iterator_tag)
{
return find_out_of_order(f, l, r, forward_iterator_tag());
}

template <typename I, typename R>


requires(Readable(I) && Iterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
I find_out_of_order(I f, I l, R r)
{
return find_out_of_order(f, l, r, IteratorCategory(I)());
}

Stepanov, McJones Elements of Programming March 14, 2008 830 / 880


is_increasing

template <typename I, typename R>


requires(Readable(I) && Iterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
bool is_increasing(I f, I l, R r)
{
return l == find_out_of_order(f, l, r);
}

Stepanov, McJones Elements of Programming March 14, 2008 831 / 880


partition_copy_n

template <typename I, typename O0, typename O1, typename P>


requires(Readable(I) && Iterator(I) &&
Writable(O0) && Iterator(O0) &&
Writable(O1) && Iterator(O1) &&
UnaryPredicate(P) && Domain(P) == ValueType(I) &&
ValueType(I) == ValueType(O0) &&
ValueType(I) == ValueType(O1))
pair<O0, O1> partition_copy_n(I f, DistanceType(I) n, O0 o0, O1 o1, P p)
{
typedef DistanceType(I) N;
while (n != N(0)) {
if (p(source(f))) {
sink(o1) = source(f); ++o1;
} else {
sink(o0) = source(f); ++o0;
}
++f;
n = n - N(1);
}
return pair<O0, O1>(o0, o1);
}

Stepanov, McJones Elements of Programming March 14, 2008 832 / 880


unstable_indexed_partition
template <typename I, typename P>
requires(Mutable(I) && IndexedIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I unstable_indexed_partition(I f, I l, P p)
{
DistanceType(I) i = 0;
DistanceType(I) j = l - f;
while (true) {
while (true) {
if (i == j) return f + i;
if (p(source(f + i))) break;
i = i + 1;
}
while (true) {
j = j - 1;
if (i == j) return f + j + 1;
if (!p(source(f + j))) break;
}
cycle_2(f + i, f + j);
i = i + 1;
}
}

Stepanov, McJones Elements of Programming March 14, 2008 833 / 880


lower_bound_n

template <typename T, typename R>


requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)
struct greater_than_or_equal_to_a
{
const T& a;
R r;
greater_than_or_equal_to_a(const T& a, R r) : a(a), r(r) { }
bool operator()(const T& x) const { return !r(x, a); }
};

template <typename I, typename R>


requires(Mutable(I) && ForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
I lower_bound_n(I f, DistanceType(I) n, const ValueType(I)& a, R r)
{
greater_than_or_equal_to_a<ValueType(I), R> predicate(a, r);
return partition_point_n(f, n, predicate);
}

Stepanov, McJones Elements of Programming March 14, 2008 834 / 880


upper_bound_n

template <typename T, typename R>


requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)
struct greater_than_a
{
const T& a;
R r;
greater_than_a(const T& a, R r) : a(a), r(r) { }
bool operator()(const T& x) const { return r(a, x); }
};

template <typename I, typename R>


requires(Mutable(I) && ForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
I upper_bound_n(I f, DistanceType(I) n, const ValueType(I)& a, R r)
{
greater_than_a<ValueType(I), R> predicate(a, r);
return partition_point_n(f, n, predicate);
}

Stepanov, McJones Elements of Programming March 14, 2008 835 / 880


Category dispatch for merge_n_with_buffer
template <typename I, typename B, typename R>
requires(Mutable(I) && RandomAccessIterator(I) &&
Mutable(B) && RandomAccessIterator(B) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&
ValueType(I) == ValueType(B))
void merge_n_with_buffer(I f0, DistanceType(I) n0,
I f1, DistanceType(I) n1, B b, R r,
random_access_iterator_tag)
{
merge_n_with_buffer(f0, n0, f1, n1, b, r, bidirectional_iterator_tag());
}

template <typename I, typename B, typename R>


requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&
ValueType(I) == ValueType(B))
void merge_n_with_buffer(I f0, DistanceType(I) n0,
I f1, DistanceType(I) n1, B b, R r)
{
merge_n_with_buffer(f0, n0, f1, n1, b, r, IteratorCategory(I)());
}

Stepanov, McJones Elements of Programming March 14, 2008 836 / 880


IteratorT ype type function

template <typename T>


requires(ImplementsIteratorType(T))
struct iterator_type;

#define IteratorType(T) typename iterator_type< T >::type

Stepanov, McJones Elements of Programming March 14, 2008 837 / 880


IteratorT ype for linkers

template <typename I> requires(LinkableForwardIterator(I))


struct forward_linker;

template <typename I> requires(LinkableForwardIterator(I))


struct iterator_type< forward_linker<I> > { typedef I type; };

template <typename I> requires(LinkableBidirectionalIterator(I))


struct backward_linker;

template <typename I> requires(LinkableBidirectionalIterator(I))


struct iterator_type< backward_linker<I> > { typedef I type; };

template <typename I> requires(LinkableBidirectionalIterator(I))


struct bidirectional_linker;

template <typename I> requires(LinkableBidirectionalIterator(I))


struct iterator_type< bidirectional_linker<I> > { typedef I type; };

Stepanov, McJones Elements of Programming March 14, 2008 838 / 880


Domain for transpose_operation

template <typename Op>


requires(BinaryOperation(Op))
struct transpose_operation;

template <typename Op>


requires(BinaryOperation(Op))
struct argument_type< transpose_operation<Op>, 0 >
{
typedef Domain(Op) type;
};

Stepanov, McJones Elements of Programming March 14, 2008 839 / 880


Domain for merger_linkable

template <typename S, typename R>


requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))
struct merger_linkable;

template <typename S, typename R>


requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))
struct argument_type<merger_linkable<S, R>, 0>
{
typedef IteratorType(S) type;
};

Stepanov, McJones Elements of Programming March 14, 2008 840 / 880


Domain for combine_ranges

template <typename I>


requires(ForwardIterator(I)) // but not Readable(I)
struct combine_ranges;

template <typename I>


requires(ForwardIterator(I)) // but not Readable(I)
struct argument_type<combine_ranges<I>, 0>
{
typedef pair<I, I> type;
};

Stepanov, McJones Elements of Programming March 14, 2008 841 / 880


Codomain for partition_trivial

template <typename I, typename P>

requires(ForwardIterator(I) && UnaryPredicate(P) && ValueType(I) == Domain(P))


struct partition_trivial;

template <typename I, typename P>

requires(ForwardIterator(I) && UnaryPredicate(P) && ValueType(I) == Domain(P))


struct codomain_type< partition_trivial<I, P> >
{
typedef pair<I, I> type;
};

Stepanov, McJones Elements of Programming March 14, 2008 842 / 880


WeightT ype type function

template <typename T>


requires(WeakForwardBifurcateCoordinate(T))
struct weight_type
{
typedef size_t type; // ***** What should the default be ?????
};

template <typename T>


requires(WeakForwardBifurcateCoordinate(T))
struct weight_type<T*>
{
typedef ptrdiff_t type;
};
#define WeightType(T) typename weight_type< T >::type // ***** ?????

Stepanov, McJones Elements of Programming March 14, 2008 843 / 880


left_successor and right_successor for
WeakForwardBifurcateCoordinate

template <typename C>


requires(WeakForwardBifurcateCoordinate(C))
C left_successor(C f)
{
move_left(f);
return f;
}

template <typename C>


requires(WeakForwardBifurcateCoordinate(C))
C right_successor(C f)
{
move_right(f);
return f;
}

Stepanov, McJones Elements of Programming March 14, 2008 844 / 880


pair

template <typename T0, typename T1>


requires(Regular(T0) && Regular(T1))
struct pair
{
T0 m0;
T1 m1;
pair() {}
pair(T0 m0, T1 m1) : m0(m0), m1(m1) { }
pair(const pair& x) : m0(x.m0), m1(x.m1) { }
void operator=(const pair& x) { m0 = x.m0; m1 = x.m1; }
friend void bool operator==(const pair& x, const pair& y)
{ return x.m0 == y.m0 && x.m1 == y.m1; }
friend void bool operator<(const pair& x, const pair& y)
{ return x.m0 < y.m0 || (!y.m0 < x.m0 && x.m1 < y.m1); }
};

Stepanov, McJones Elements of Programming March 14, 2008 845 / 880


triple

template <typename T0, typename T1, typename T2>


requires(Regular(T0) && Regular(T1) && Regular(T2))
struct triple
{
T0 m0;
T1 m1;
T2 m2;
triple() {}
triple(T0 m0, T1 m1, T2 m2) :
m0(m0), m1(m1), m2(m2) {}
triple(const triple& x) : m0(x.m0), m1(x.m1), m2(x.m2) { }
void operator=(const triple& x)
{ m0 = x.m0; m1 = x.m1; m2 = x.m2}
friend void bool operator==(const triple& x, const triple& y)
{ return x.m0 == y.m0 && x.m1 == y.m1 && x.m2 == y.m2; }
friend void bool operator<(const triple& x, const triple<T0, T1, T2>& y)
{ return x.m0 < y.m0 || (!y.m0 < x.m0 && x.m1 < y.m1 ||
(!y.m1 < x.m1 && x.m2 == y.m2)); }
};

Stepanov, McJones Elements of Programming March 14, 2008 846 / 880


lexicographic_equalityk

template <int k, typename I0, typename I1>


requires(Readable(I0) && ForwardIterator(I0) &&
Readable(I1) && ForwardIterator(I1) &&
ValueType(I0) == ValueType(I1))
struct lexicographic_equality_k
{
bool operator()(I0 f0, I1 f1)
{
if (source(f0) != source(f1)) return false;
return lexicographic_equality_k<k - 1, I0, I1>()(f0 + 1, f1 + 1);
}
};

template <typename I0, typename I1>


struct lexicographic_equality_k<0, I0, I1>
{
bool operator()(I0, I1)
{
return true;
}
};

Stepanov, McJones Elements of Programming March 14, 2008 847 / 880


lexicographic_orderingk
template <int k, typename I0, typename I1>
requires(Readable(I0) && ForwardIterator(I0) &&
Readable(I1) && ForwardIterator(I1) &&
ValueType(I0) == ValueType(I1))
struct lexicographic_ordering_k
{
bool operator()(I0 f0, I1 f1)
{
if (source(f0) < source(f1)) return true;
if (source(f0) > source(f1)) return false;
return lexicographic_equality_k<k - 1, I0, I1>()(f0 + 1, f1 + 1);
}
};

template <typename I0, typename I1>


struct lexicographic_ordering_k<0, I0, I1>
{
bool operator()(I0, I1)
{
return false;
}
};

Stepanov, McJones Elements of Programming March 14, 2008 848 / 880


arrayk

***** Need const and non-const versions of begin and end ?????
***** What about subscript: a[i] ?????
template <int k, typename T>
requires(Regular(T))
struct array_k
{
typedef T* I; // IteratorT ype(arrayk,T )
T a[k];
array_k() { }
array_k(const array_k& x) { basic_copy_k_indexed<k, I, O>(&x.a, &a); }
friend void operator=(const array_k& x)
{ basic_copy_k_indexed<k, I, O>(&x.a, &a); }
friend size_t size(const array_k& x) { return size_t(k); }
friend I begin(array_k& x) { return &x.a[0]; }
friend I end(array_k& x) { return &x.a[k]; }
friend bool operator==(const array_k& x, const array_k& y)
{ return lexicographic_equality_k<k, I, I>(begin(x), begin(y)); }
friend bool operator<(const array_k& x, const array_k& y)
{ return lexicographic_ordering_k<k, I, I>(begin(x), begin(y)); }
};

Stepanov, McJones Elements of Programming March 14, 2008 849 / 880


Forward declaration for array_k data structure

template <int k, typename T>


requires(Regular(T))
struct array_k;

array_k is defined in Chapter 11

Stepanov, McJones Elements of Programming March 14, 2008 850 / 880


IteratorT ype for array_k

template <int k, typename T>


requires(Regular(T))
struct iterator_type< array_k<k, T> >
{
typedef T* type;
};

Stepanov, McJones Elements of Programming March 14, 2008 851 / 880


fill_iterator

template <typename T, typename N>


requires(Regular(T) && Integer(N))
struct fill_iterator
{
const T* v;
N n;
fill_iterator() {}
fill_iterator(const T& v, N n) : v(&v), n(n) {}
friend void operator++(fill_iterator& x) { ++x.n; }
friend const T& source(fill_iterator& x) { return source(x.v); }
friend void operator+=(fill_iterator& x, N n) { x.n += n; }
friend N operator-(fill_iterator x, fill_iterator y) { return x.n - y.n; }
friend bool operator==(fill_iterator x, fill_iterator y) {
return x.n == y.n;
}
};

Stepanov, McJones Elements of Programming March 14, 2008 852 / 880


Type functions for fill_iterator

template <typename T, typename N>


requires(Regular(T) && Integer(N))
struct value_type< fill_iterator<T, N> >
{
typedef T type;
};

template <typename T, typename N>


requires(Regular(T) && Integer(N))
struct distance_type< fill_iterator<T, N> >
{
typedef N type;
};

template <typename T, typename N>


requires(Regular(T) && Integer(N))
struct iterator_category< fill_iterator<T, N> >
{
typedef indexed_iterator_tag category;
};

Stepanov, McJones Elements of Programming March 14, 2008 853 / 880


Acknowledgments

We are grateful for comments and suggestions from


Andrei Alexandrescu,Matt Austern, Dave Abrahams, Jon Brandt,
Boris Fomitchev, Kevlin Henney, Jussi Ketonen, Karl Malbrain,
Larry Masinter, David Musser, Dave Parent, Sean Parent,
Dmitry Polukhin, Jon Reid, Mark Ruzon, Geoff Scott,
David Simons, Tony Van Eerd, Walter Vannini, John Wilkinson,
and Oleg Zabluda

Stepanov, McJones Elements of Programming March 14, 2008 854 / 880


Index I
++
definition space on range, 361
6
standard definition, 242
>
standard definition, 242
>
standard definition, 242

absolute value, 272


absorption
for min, max, 251
abstraction, 19
action
duality with transformation, 343
Action concept, 342
activity on entities, 19
acyclic
descendants of bifurcate coordinate, 633
AdditiveGroup concept, 200
AdditiveMonoid concept, 198
AdditiveSemigroup concept, 196
algorithm
memory-adaptive, 426
with buffer, 426
algorithmically-induced structure, 276
aliasing, 338, 385
amortized complexity, 347
AmortizedConstantTime concept, 347

Stepanov, McJones Elements of Programming March 14, 2008 855 / 880


Index II
and (∧), 803
Archimedean monoid
discrete, 287
ArchimedeanMonoid concept, 276
ArchimedeanOrderedField concept, 519
area
constant, 82
fixed, 82
of an object, 82
variable, 82
area function, 82
argument, 40
ArgumentT ype type function, 55
Arity type attribute, 54
assignment, 49
for Regular, 65
associativity, 130
exploited by power, 145
of min, max, 251
of permutation composition, 412
asymmetric relation, 218
attribute of entity, 19
average cost, 84
Axiom of Archimedes, 276, 277

backward movement in range, 533


BackwardLinker concept, 601
Banning, John, 358
bidirectional binary trees, advantages, 641

Stepanov, McJones Elements of Programming March 14, 2008 856 / 880


Index III
BidirectionalBifurcateCoordinate concept, 629
BidirectionalIterator concept, 374
BidirectionalLinker concept, 602
BifurcateCoordinate concept, 628
bignum, 139
bin-based rearrangement, 422
binary gcd, 327
binary search, 553
BinaryOperation concept, 129
bisection technique, 518
Bolzano, Bernard, 518
bounded integer type, 322
buffer
with buffer algorithm, 426

CancellableOperation concept, 195


category dispatch
mechanism, 822
Cauchy, Augustin Louis, 518
closed bounded range, 357
closed counted range, 357
closed interval, 243
clusters
of derived procedures, 238
code transformations
enabled by regular types, 81, 181
codomain of function, 805
codomain of procedure, 42
Codomain type function, 55

Stepanov, McJones Elements of Programming March 14, 2008 857 / 880


Index IV
CommutativeOperation concept, 194
CommutativeSemiringconcept, 298
commutativity
of min, max, 251
Compatible concept, 732
complement (−), 805
complement of converse of relation, 239
complement of relation, 239
complexity
algorithm selection, 175
amortized, 347
counting operations, 176
in specifications, 175
of generalized algorithm, 175
slow_power versus power, 177, 178
to determine feasibility, 175
composition
of permutations, 412
concept, 56
ArchimedeanMonoid, 276
Action, 342
AdditiveGroup, 200
AdditiveMonoid, 198
AdditiveSemigroup, 196
AmortizedConstantTime, 347
ArchimedeanOrderedField, 519
BackwardLinker, 601
BidirectionalBifurcateCoordinate, 629
BidirectionalIterator, 374

Stepanov, McJones Elements of Programming March 14, 2008 858 / 880


Index V
BidirectionalLinker, 602
BifurcateCoordinate, 628
BinaryOperation, 129
CancellableOperation, 195
CommutativeOperation, 194
CommutativeSemiring, 298
Compatible, 732
Dereferenceable, 340
DiscreteArchimedeanMonoid, 287
DiscreteArchimedeanRing, 319
DiscreteArchimedeanSemiring, 317
EquivalenceRelation, 219
EuclideanMonoid, 293
EuclideanSemimodule, 304
EuclideanSemiring, 306
examples from C++, 59
ForwardIterator, 371
ForwardLinker, 600
Function, 60
GroupOperation, 137
HomogeneousFunction, 61
IndexedIterator, 380
Integer, 139
Iterator, 348
LinkableBidirectionalBifurcateCoordinate, 644
LinkableBidirectionalIterator, 606
LinkableBifurcateCoordinate, 643
LinkableForwardIterator, 605
Merger, 567

Stepanov, McJones Elements of Programming March 14, 2008 859 / 880


Index VI
MonoidOperation, 136
MultiplicativeGroup, 201
MultiplicativeMonoid, 199
MultiplicativeSemigroup, 197
Mutable, 339
NonnegativeDiscreteArchimedeanSemiring, 318
Operation, 88
OrderedAdditiveGroup, 271
OrderedAdditiveMonoid, 271
OrderedAdditiveSemigroup, 271
OrderedCancellableMonoid, 273
Ordering, 215
PartiallyCompatible, 728
PartialSemigroupOperation, 781
Predicate, 214
Procedure, 60
ProperProcedure, 60
QRSemimodule, 302
RandomAccessIterator, 387
Readable, 336
refinement, 57
Regular, 63
RegularAction, 346
RegularFunction, 79
Relation, 214
relational concept, 726
SameGenus, 727
SegmentedIterator, 399
SemigroupOperation, 130

Stepanov, McJones Elements of Programming March 14, 2008 860 / 880


Index VII
Semimodule, 300
StrictTotallyOrdered, 237
StrictTotalOrdering, 223
StrictWeakOrdering, 226
Transformation, 89
type concept, 58
UnaryPredicate, 504
univalent, 320
WeakBifurcateCoordinate, 626
weakening, 57
WeaklyHalvableMonoid, 283
Writable, 337
constant area, 82
constructor, 52
type constructor, 53
container, 668
converse of relation, 239
coordinate structure, 594
copy constructor
for Regular, 68
copying rearrangement, 425
cost
average, 84
fixed, 84
uniform, 84
worst-case, 84
cost function, 84
cost of a procedure call, 84
cycle in a permutation, 415

Stepanov, McJones Elements of Programming March 14, 2008 861 / 880


Index VIII
cyclic permutation, 417

default constructor
for Regular, 67
default ordering, 236
definition space of procedure, 41
Dereferenceable concept, 340
derived relations, 239
descendants
of bifurcate coordinate, 633
description, 18
destructor, 50
for Regular, 66
DifferenceT ype type function, 387
direct product (×), 805
discrete Archimedean monoid, 287
DiscreteArchimedeanMonoid concept, 287
DiscreteArchimedeanRing concept, 319
DiscreteArchimedeanSemiring concept, 317
discreteness, 317
distance type, 110
DistanceT ype type function, 348
distributivity
of semiring, 298
divisibility lattice of integers, 466
divisibility on an Archimedean monoid, 285
division
Egyptian, 279
division on multiplicative group, 201

Stepanov, McJones Elements of Programming March 14, 2008 862 / 880


Index IX
domain of function, 805
Domain type function, 61
Dudziński, Krzysztof, 576
Dydek, Andrzej, 576

Egyptian division, 279


element (∈), 805
empty range, 360
entity
activity on, 19
attribute, 19
part, 19
particulars, 20
relationship, 19
equality
for Regular, 64
in the real world, 33
of objects, 38
on containers, 670, 671
properties of, 31
unique on a type, 234
equality (=), 30
equational reasoning, 32
in mathematics, 32
EquivalenceRelation concept, 219
equivalent (⇔), 803
essential object, 39
Euclidean domain, 297
Euclidean function, 306

Stepanov, McJones Elements of Programming March 14, 2008 863 / 880


Index X
EuclideanMonoid concept, 293
EuclideanSemimodule concept, 304
EuclideanSemiring concept, 306
exclusive-or of links, 646
exists (∃), 804

Fibonacci
matrices, 185
numbers, 182
Fiduccia, Charles M., 206
finite set, 418
finite tree
descendents of bifurcate coordinate, 634
first in a range, 358
fixed area, 82
fixed cost, 84
fixed point, 411
for all (∀), 804
ForwardIterator concept, 371
ForwardLinker concept, 600
from-permutation, 431
function
one-to-one, 408
onto, 407
function (→), 805
Function concept, 60
function procedure, 47

Stepanov, McJones Elements of Programming March 14, 2008 864 / 880


Index XI
Gaussian integers
defined, 166
example of Euclidean domain, 297
Stein’s algorithm, 328
gcd
and lcm, 465
binary, 327
greatest common divisor, 288
subtractive, 290
generalized link rearrangement, 610
generalized procedure, 71
generalized rearrangement, 609
genus, 35
global state, 40
goto
considered not harmful, 655
greatest common divisor, 288
group
of permutations, 414
GroupOperation concept, 137

half-open interval, 243


Ho, Wilson, 482, 545
Hoare, C.A.R., 543
HomogeneousFunction concept, 61

idea, 18
idempotency

Stepanov, McJones Elements of Programming March 14, 2008 865 / 880


Index XII
of min, max, 251
identity
in the real world, 29
of objects, 29
identity permutation, 411
implies (⇒), 803
improper procedure, 46
in place, 426
in situ, 426
increasing order, 248
increasing range, 499
index permutation, 419
indexed iterator
equivalent to random access iterator, 391
IndexedIterator concept, 380
inequality
standard definition, 241
Integer concept, 139
interface
for rotate, 469
useful variations, 180
interoperability, 62
interpretatation of object, 25
intersection (∩), 805
interval, 243
invariant, 655
invariants
breaking, 622
inverse

Stepanov, McJones Elements of Programming March 14, 2008 866 / 880


Index XIII
of permutation, 413, 417
is_defined predicate, 43
isomorphic types, 320
iterator
linkable, 598
Iterator concept, 348

k-partition, 504
key function, 499, 569
key-increasing range, 499
key-sorting, 569

Lakshman, T.K., 523


last in a range, 358
lattice, 466
lcm
and gcd, 465
least common multiple, 464
least common multiple, 464
Leibniz’s Law, 30
lexicographic ordering, 225, 229
limit in a range, 358
linear ordering
importance of, 213
linear recurrences, 206
link rearrangement, 603
linkable iterator, 598
linkable range, 598

Stepanov, McJones Elements of Programming March 14, 2008 867 / 880


Index XIV
linkable structures
forward versus bidirectional, 625
LinkableBidirectionalBifurcateCoordinate concept, 644
LinkableBidirectionalIterator concept, 606
LinkableBifurcateCoordinate concept, 643
LinkableForwardIterator concept, 605
linking operation, 599
links
exclusive-or of, 646
reversing, 645
Lo, Raymond, 482, 545
local state, 40
locality of reference, 476, 618
located sequence, 356
Lomuto, Nico, 526
loop fusion, 476
lower bound, 506
implementation, 554

Mauchly, John W., 586


memory, 335
memory-adaptive algorithm, 426
merge, 555
algebraic properties, 559
complexity property, 560
copying, 556
melding, 556
mutating, 556
stability, 555

Stepanov, McJones Elements of Programming March 14, 2008 868 / 880


Index XV
mergeable range, 556
Merger concept, 567
models
models (|=), 56
type models concept, 56
MonoidOperation concept, 136
MultiplicativeGroup concept, 201
MultiplicativeMonoid concept, 199
MultiplicativeSemigroup concept, 197
Mutable concept, 339
mutable range, 362
mutative rearrangement, 425
mutator procedure, 47

natural total ordering, 235


< reserved for, 237
Noether, Emmy, 307
non-terminating procedure, 44
nonessential object, 39
NonnegativeDiscreteArchimedeanSemiring concept, 318
not (¬), 803
not last in a range, 358

object, 24, 25
interpretation, 25
state, 25
one-to-one function, 408
onto function, 407

Stepanov, McJones Elements of Programming March 14, 2008 869 / 880


Index XVI
open interval, 243
Operation concept, 88
optimization
enabled by regular types, 81
loop fusion, 476
using segmented iterator, 398
or (∨), 803
orbit, 92
OrderedAdditiveGroup concept, 271
OrderedAdditiveMonoid concept, 271
OrderedAdditiveSemigroup concept, 271
OrderedCancellableMonoid concept, 273
ordering
reflexive, 217
strict, 217
weakly reflexive, 217
Ordering concept, 215
ordering-based rearrangement, 422
out-of-order execution, 385
overlapping ranges, 377
overloading, 190, 599, 642
own state, 40

partial
procedure, 43
representation, 27
partial ordering
example, 231
topological sort defined on, 233

Stepanov, McJones Elements of Programming March 14, 2008 870 / 880


Index XVII
partially-formed, 51
PartiallyCompatible concept, 728
PartialSemigroupOperation concept, 781
particulars of entity, 20
partition, 504
k-partition, 504
partition algorithm, 508
partition algorithms, origin of, 543
partition point, 505
finding, 515
lower bound, 506
upper bound, 506
partition rearrangement
semistable, 525
partitioned prefix of a range, 509
parts of an entity, 19
permutation, 409
composition, 412
cycle, 415
cyclic, 417
from, 431
identity, 411
index, 419
inverse, 413, 417
product of its cycles, 417
reverse, 436
rotation, 463
to, 431
permutation group, 414

Stepanov, McJones Elements of Programming March 14, 2008 871 / 880


Index XVIII
pointers
exclusive-or of, 646
polynomials
example of Euclidean domain, 297
position-based rearrangement, 422
powers
non-positive exponent, 138
of same element commute, 132
positive exponent, 131
Predicate concept, 214
procedure, 40
function, 47
generalized, 71
improper, 46
mutator, 47
non-terminating, 44
partial, 43
proper, 46
semi-terminating, 44
terminating, 44
total, 43
Procedure concept, 60
proper procedure, 46
ProperProcedure concept, 60

QRSemimodule concept, 302


quicksort, 528, 543

Stepanov, McJones Elements of Programming March 14, 2008 872 / 880


Index XIX
random access iterator
equivalent to indexed iterator, 391
RandomAccessIterator concept, 387
range, 356
backward movement, 533
closed bounded, 357
closed counted, 357
empty, 360
first, 358
increasing, 499
k-partition, 504
key-increasing, 499
last, 358
limit, 358
linkable, 598
lower bound, 506
mergeable, 556
mutable, 362
not last, 358
overlapping, 377
partition, 504
partition point, 505
partitioned prefix, 509
readable, 362
semi-open bounded, 357
semi-open counted, 357
size, 359
uniform random shuffle, 545
upper bound, 506

Stepanov, McJones Elements of Programming March 14, 2008 873 / 880


Index XX
writable, 362
reachability of bifurcate coordinates, 632
reachable, 90
Readable concept, 336
readable range, 362
rearrangement, 420
bin-based, 422
copying, 425
generalized, 609
generalized link, 610
key-sorting, 569
link, 603
mutative, 425
ordering-based, 422
partition, 508
position-based, 422
reverse, 437
rotation, 469
sorting, 569
time complexity, 427, 434
refinement of concept, 57
reflexive ordering, 217
reflexivity of equality, 31
regular
allows code transformations, 81, 181
required for predicate with assured find, 544
Regular concept, 63
RegularAction concept, 346
RegularFunction concept, 79

Stepanov, McJones Elements of Programming March 14, 2008 874 / 880


Index XXI
relation
complement, 239
complement of converse, 239
converse, 239
symmetric, 218
symmetric complement of, 222
Relation concept, 214
relational concept, 726
relations
derived, 239
relationship between entities, 19
representation, 27
partial, 27
total, 27
unique, 28
requires clause, 71, 72
result space of procedure, 42
returning useful information, 365, 367, 394, 439, 469, 482, 523, 533, 537, 553, 563, 735, 737
reverse
permutation, 436
rearrangement, 437
reversing links, 645
rotation
permutation, 463
rearrangement, 469
Russian peasant algorithm, 144

same genus, 35
SameGenus concept, 727

Stepanov, McJones Elements of Programming March 14, 2008 875 / 880


Index XXII
Scalar type function, 300
segmented iterator
intuition, 398
SegmentedIterator concept, 399
SegmentIterator type function, 399
semi
usage convention, 232
semi-open bounded range, 357
semi-open counted range, 357
semi-terminating procedure, 44
SemigroupOperation concept, 130
Semimodule concept, 300
semistable partition rearrangement, 525
sequence
located, 356
sequence of iterators, 355
sets, 805
sink, 337
size
of range, 359
sort (type) of value, 21
sorting
stability, 569
sorting algorithm, 569
space complexity
forward iterator reverse conjecture, 455
in place, 426
memory adaptive, 426
with buffer, 426

Stepanov, McJones Elements of Programming March 14, 2008 876 / 880


Index XXIII
stability, 247
of min, max, 250
of rearrangement, 423
of sorting algorithm, 569
stable adaptive merge and sort, importance of, 585
stable in-place merge and sort, importance of, 585
stable merge, 555
state of object, 25
Stein, Josef, 327
storage efficiency, 83
strict lower bound, 434
strict ordering, 217
strictly increasing order, 248
StrictTotallyOrdered concept, 237
StrictTotalOrdering concept, 223
StrictWeakOrdering concept, 226
subset (⊂), 805
subtraction on additive group, 200
subtractive gcd, 290
symmetric complement of a relation, 222
symmetric relation, 218
symmetry of equality, 31

terminating procedure, 44
thing, 18
Tighe, Joseph, 469
time complexity
of rearrangement, 427, 434
to-permutation, 431

Stepanov, McJones Elements of Programming March 14, 2008 877 / 880


Index XXIV
total
procedure, 43
representation, 27
trait class, 811
transformation
duality with action, 343
fixed point of, 411
self-composable, 90
Transformation concept, 89
transitivity, 215
transitivity of equality, 31
traversal
of tree, recursive, 637
tree
descendants of bifurcate coordinate, 633
trichotomy law
for total ordering, 224
for weak ordering, 228
trivial cycle, 415
type, 36
constant area, 82
fixed area, 82
intuition, 37
isomorphism, 320
variable area, 82
type attribute, 54
Arity, 54
type concept, 58
type constructor, 53

Stepanov, McJones Elements of Programming March 14, 2008 878 / 880


Index XXV
type expression, 72
type function, 55
ArgumentT ype, 55
Codomain, 55
DifferenceT ype, 387
DistanceT ype, 348
Domain, 61
example, 109
Scalar, 300
SegmentIterator, 399
simulated in C++ via trait class, 811
UBI, 764
UFI, 763
UnderlyingT ype, 747
ValueT ype, 336, 337
WeightT ype, 626

UBI type function, 764


UFI type function, 763
UnaryPredicate concept, 504
underlying type, 429, 747
UnderlyingT ype type function, 747
uniform cost, 84
uniform random shuffle, 545
union (∪), 805
unique representation, 28
univalent concept, 320
upper bound, 506
implementation, 554

Stepanov, McJones Elements of Programming March 14, 2008 879 / 880


Index XXVI
value, 19
sort (type), 21
value type, 336
ValueT ype type function, 336, 337
van der Waerden, Bartel Leendert, 307
variable area, 82

weak
usage convention, 232
WeakBifurcateCoordinate concept, 626
weakening of concept, 57
weakly reflexive ordering, 217
WeaklyHalvableMonoid concept, 283
WeightT ype type function, 626
well-formed object state, 26
with buffer algorithm, 426
worst-case cost, 84
Writable concept, 337
writable range, 362

Zabluda, Oleg, 205

Stepanov, McJones Elements of Programming March 14, 2008 880 / 880