- STUDENT'S COPY MODULE 1-FUNCTIONS
- Bit-Level Binaries and Generalized Comprehensions in Erlang
- Lab View
- Sec1 Mod5 Features Tn 71713
- Comparing Models of Computation
- Madhava Mc Paper 12
- Lab1 Solutions
- Unit 4. Andalusia Loves You
- Study Guide & Intervention Alg22
- An interval neutrosophic linguistic multi-criteria group decision-making method and its application in selecting medical treatment options
- Function Past Year Spm Questions
- What is Programming
- What I Wish I Knew When Learning Haskell ( Stephen Diehl )
- A Theory of Probabilistic Functional Testing
- complexity theory
- 1151957_E_20180304
- chap03
- ch02
- Usine Scripting Tutorial
- SAMPLE_DIPUM_PROJECT_SOLUTIONS
- Injection and Surjection
- Crib Sheet - Final Exam
- Functions
- EXPERIMENT 2
- IntroC DataTypesCtrlFlow
- Abstract Data Type
- Towards a General Definition of Biometric Systems,, IJCSI, Volume 1, August 2009.
- ECM6Hw1_2012
- Chapter 1 -Part 1 Real Functions
- guttagADT77
- SOAPUI
- Subqueries to Solve Queries Questions
- Test Mobile Testing
- View Look Up
- functional testing

Maarten M. Fokkinga

Version of

December 5, 2008, 16:29

ABSTRACT

Map and Reduce are generic, useful notions for computing science; together they are equally expressive as simple

inductive definitions over trees/lists/bags/sets.

1. Datatypes

Let A be a set. Consider the datatype of finite binary trees

over A; it consists of a set TA and two constructors tip

and join:

join

TA

: set

¡@join

tip

: A → TA

¡

tip tip¡@tip

join : TA × TA → TA

a

b

c

Function tip makes an A-element into a “singleton” tree,

and operation join joins two trees into one. By definition,

set TA contains precisely the elements that are generated

by the constructors. We’ll use the notational convention

that x join y stands for join(x , y). So, the example tree

above is denoted: tip a join (tip b join tip c).

Some other useful datatypes can be obtained by further

postulations:

• Postulating associativity of join (that is, x join (y join

z ) = (x join y) join z ) makes the datatype into that

of lists; the “tree structure” doesn’t make sense but

ordering of the tips remains significant, e.g.:

=

**tip a join (tip b join (. . . join tip c)...)
**

(...(tip a join tip b) join . . .) join tip c

Hence the parentheses can be left out without causing semantic ambiguity. So, the list containing precisely a, b, . . . , c, in that order, is denoted by the

expression tip a join tip b join . . . join tip c.

• In addition postulating commutativity of join (that

is, x join y = y join x ) makes the datatype into that

of bags; ordering of the tips doesn’t make sense but

multiplicity of tips remains significant:

=

**tip a join tip b join tip a
**

tip a join tip a join tip b

**Thus these two expressions denote the bag containing
**

precisely a twice and b once.

• In addition postulating absorptivity of join (that is,

x join x = x ) makes the datatype into that of sets;

multiplicity of the tips doesn’t make sense but existence (or membership) remains significant:

tip a join tip a join tip b = tip a join tip b

Thus these two expressions denote the set containing

precisely a and b.

**• Postulating the existence in TA of a neutral element
**

nil for join (that is, nil join x = x = x join nil ), the

set TA has an element that plays the role of “empty

tree/list/bag/set”, namely nil .

These datatypes occur frequently in computing science.

Many more datatypes can be formalized along these lines;

§5 discusses an alternative, “right biassed,” definition of

lists.

2. Catamorphism

In order to do computations over datatypes like those

above, the very first thing that comes to mind is: functions defined by induction on the structure of the argument. Here is a typical example; we define a function h :

TA → B , for given set B , function f : A → B and operation

⊗ : B × B → B , by:

h (tip a)

=

h (x join y) =

f a,

(h x ) ⊗ (h y),

for all a : A

for all x ,y:TA

The effect of applying h boils down to the systematic replacement of the constructors tip and join by the items

f and ⊗; schematically:

x =

h x =

**. . . tip a join (tip b join tip c) . . .
**

. . . f a ⊗ ( f b ⊗ f c) . . .

**Let us denote the h thus defined by ([tip, join → f , ⊗]) or
**

simply ([f , ⊗]), and call it the catamorphism or fold determined by f , ⊗. In order that h is well defined, operation ⊗

should at least satisfy the properties of join; for example,

if join is commutative, then ⊗ must be commutative for

otherwise the definition of h might lead to a contradiction;

see §4. And, if join has a neutral element, then so must ⊗,

and in that case we define: h niljoin = nil⊗ .

There exist computable total functions on these datatypes

that cannot be expressed by the above simple form of induction, but nevertheless many forms of inductive definitions and indeed many algorithms can be expressed by

catamorphisms in combination with other well-known constructs that are not specific for our datatypes. Some examples will be given in §4. So, this kind of inductively

defined functions, which we have called catamorphism or

fold, suffices for a great deal of practical algorithms.

3. MapReduce

The systematic replacement of the two items tip and join

by the two items f and ⊗, respectively, has been denoted

by one single expression: ([f , ⊗]). However, it makes sense

to do the replacements separately. So, let us tear them

Closing remarks (1) Maps. barbed wire. In M. id x = x . pages 124–144. Sc.W. The Netherlands. category theory is the field where the notions themselves are studied (datatype. he had many followers. +]) = +/ ◦ f ∗ where f a =1 Since + is not absorptive. IFIP WG 2. (2) For all inductively defined datatypes the notions of catamorphism and map make sense. 1986.M. de Bakker and J. Alternative definition of lists Suppose we define the datatype of lists in the following “right biassed” way: LA nil : set : LA cons : A × LA → LA An example list is: a cons (b cons (c cons nil )). for sets. after making each element into a singleton: sort = ([tip. size is not defined when join is absorptive. Proc. 1991. (f ◦g)x = f (g x ). using ⊗ at every join-node: ⊗/ ( tip a join (tip b join tip c) ) = ⊗ ( b a ⊗ c ) Here are the definitions.M.. ⊗]) = ⊗/ ◦ f ∗ These three equations show that maps and reduces together are equally expressive as catamorphisms.. and the presence of equations (laws) [5]. envelopes. 1996. 1992. maps and reduces (together with other operations not specific for our datatypes) suffice for a great deal of practical algorithms on our datatypes. CWI Symp. by systematic replacements): ([f . (c ⊗ e )) This catamorphism h is sometimes called the right-reduce or foldr determined by ⊗ and e. 1996. [3] H.. In J. map. (Applying size to absorptive join gives the contradiction: 1+1 = size(tip a join tip a) = size(tip a) = 1. The Netherlands. pages 289–334. pages 3–42. catamorphism. Working paper ELC-9. North-Holland. ⊗]) : TB → B f∗ ⊗/ One can prove (e. with corresponding symbols: lenses. but for some the notion of reduce is problematic. Meertens [9] showed the equivalence of catamorphisms with maps-and-reduces. and to a lesser extent reduces. 1987.apart and define an “f -map” (denoted f ∗) that applies f inside every tip.g. Meijer. in this case) these numbers at each node: size = ([f . 6(1):1–32. join]) : TA → TB = ([id .M.) Merge sort.g. [8] G. merge]) = merge/ ◦ tip∗ 5. it is a function h defined by induction on the structure of its argument. An Introduction to the Theory of Lists.. Algebraic Data Types and Program Transformation. and culminated in the master piece [2]. using ◦ for function composition. Examples The number of tips (“size”) of a tree/list/bag is computed by taking 1 at each tip and aggregating (summing. eds. [4] J. [10] E. and R. In FPCA91: Functional Programming Languages and Computer Architecture. Logic of Programming and Calculi of Discrete Design. Fokkinga. MapReduce: Simplified Data Processing on Large Clusters. Our paper [10] describes further concepts analogous to catamorphisms. are suitable for distributed evaluation. Ghemawat.C. M. 1990. Law and Order in Algorithmics. [2] R. Malcolm [8] invented the banana brackets ([ ]). . Here too. Sc. sorting is just “aggregation by merge”. . Cambridge University Press. [7] Graham Hutton. Springer Verlag. Springer Verlag. lenses. on Math. I myself have studied the categorical approach to data types [6]. Bird and O. Fokkinga. (3) Algorithmics is the field where these ideas are put into practice to derive efficient algorithms. volume 523 of Lect. Structures in Computer Science. Algorithmics — towards programming as a mathematical activity. if a≤b if a≥b Now. Broy. (c cons nil )) h x = a ⊗ (b ⊗ . Functional Programming. Functional programming with bananas. This led Bird to exploit these notions for a Theory of Lists [1]. the notion of catamorphism makes sense. PhD thesis. Bird. 6. Taking a ⊗ x = f a cons x and e = nil it turns out that the h thus defined is “f -map” in the sense that it applies f to every tip value in the list. = tip b join ((tip a join x ) merge y). [5] M. In OSDI’04. REFERENCES [1] R. J. and is completely determined by a given set B and element e : B and operation ⊗ : A × B → B : x = a cons (b cons . Notes in Comp.J. Hence. van Vliet. . i. . University of Groningen. PhD thesis. Dean and S. . Let us first define an aggregation operation merge that merges two sorted trees/lists/bags/sets into a single sorted one: x merge nil nil merge x = = x x (tip a join x ) merge (tip b join y) = tip a join (x merge (tip b join y)). Paterson.1. University of Twente. pages 137–150. Further thoughts on Abstracto. 4. for example [7]. Sources The tree/list/bag/set approach has been proposed in 1981 by Boom [3]. Meertens. Recently. 1999. and id for identity. 9(4):355–372. Google [4] has applied maps and reduces to execute programs on a cluster of machines. A tutorial on the universality and expressiveness of fold.. 1981. Prentice-Hall International. Fokkinga. and Comp. Algebra of Programming. Malcolm.: f ∗ ( tip a join (tip b join tip c ) ) = tip fa join (tip fb join tip fc) and a “⊗-reduce” (denoted ⊗/) that aggregates all tip values. de Moor. e. assuming f : A→B and ⊗ : B ×B →B : = ([tip ◦ f . [6] M. 2004.S. Datatype laws without signatures. Boom. 7.e. Math. and reduce). envelopes and barbed wire. USENIX. [9] L. editor.S.

- STUDENT'S COPY MODULE 1-FUNCTIONSUploaded byJamaliah Daud
- Bit-Level Binaries and Generalized Comprehensions in ErlangUploaded bymra_ssuet
- Lab ViewUploaded bySundarRajan
- Sec1 Mod5 Features Tn 71713Uploaded byRob
- Comparing Models of ComputationUploaded byVarun Nagpal
- Madhava Mc Paper 12Uploaded byAman-Sharma
- Lab1 SolutionsUploaded bygreatfaber
- Unit 4. Andalusia Loves YouUploaded byplciesmiguelcrespo
- Study Guide & Intervention Alg22Uploaded byOrlando Guevara
- An interval neutrosophic linguistic multi-criteria group decision-making method and its application in selecting medical treatment optionsUploaded byMia Amalia
- Function Past Year Spm QuestionsUploaded byjesunathan44@yahoo.com
- What is ProgrammingUploaded bybiodunofyankees2000
- What I Wish I Knew When Learning Haskell ( Stephen Diehl )Uploaded bywelovescribd41
- A Theory of Probabilistic Functional TestingUploaded byArash Mazandarani
- complexity theoryUploaded byArslan Munir
- 1151957_E_20180304Uploaded byRavi Ranjan
- chap03Uploaded byBhanu Pratap Rathore
- ch02Uploaded byViji Ram
- Usine Scripting TutorialUploaded bycrisstian01
- SAMPLE_DIPUM_PROJECT_SOLUTIONSUploaded byapi-19584744
- Injection and SurjectionUploaded byVarshant Dhar
- Crib Sheet - Final ExamUploaded byBen Perez
- FunctionsUploaded byRamKrishna108
- EXPERIMENT 2Uploaded byshubham20061204
- IntroC DataTypesCtrlFlowUploaded byRhad
- Abstract Data TypeUploaded bylynnecious
- Towards a General Definition of Biometric Systems,, IJCSI, Volume 1, August 2009.Uploaded byIJCSI Editor
- ECM6Hw1_2012Uploaded byAthonioMourinho
- Chapter 1 -Part 1 Real FunctionsUploaded byMohd Rashdan
- guttagADT77Uploaded bySneha_Mittal