Specialising Generators For High-Performance Monte-Carlo Simulation ... in Haskell

Specialising Simulator Generators
for High-Performance Monte-Carlo Methods
Gabriele Keller
Hugh Chaffey-Millar
Manuel M. T. Chakravarty
Don Stewart
Christopher Barner-Kowollik
University of New South Wales

Programming Languages and Systems &
Centre for Advanced Macromolecular Design
Don Stewart | Galois Inc Specialising Simulator Generators

Imagine. . .
You are not a computer scientist, but a chemist!
• You study new polymers and their efficient production
• The search space is too big to try all possible approaches
in the lab

Imagine. . .
in the lab
Kinetic models (here, polymerisation of bulk styrene)

Imagine. . .
in the lab
Kinetic models (here, polymerisation of bulk styrene)
Exploration by way of computational chemistry

• Turn kinetic models into PDEs
• Solve with a deterministic solver, esp. h-p-Galerkin method

Solving PDEs gives molecular weight distribution (MWD)
14000
concentration 12000
-1
/mol L
10000
14000 8000
6000
4000
7000 2000
0
10000
0
7500
5000
800 simulated time
700 / seconds
600 2500
500
400
300
degree 200 0
of 100
0
polymerisation

Solving PDEs gives molecular weight distribution (MWD)
14000
concentration 12000
-1
/mol L
10000
14000 8000
6000
4000
7000 2000
0
10000
0
7500
5000
800 simulated time
700 / seconds
600 2500
500
400
300
degree 200 0
of 100
0
polymerisation
Problems encountered by chemists

• Scalability: slow for complex systems (star polymers)
• Generality: missing microscopic information:
I multiple chain lengths (i.e., star polymers)
I cross-linking densities and branching
I copolymer composition
• Improving both speed and information content seems hard

Back To Being a Programming Languages
Researcher!
Attack the problem from a different angle
• Replace deterministic by stochastic solver
• Don’t solve PDEs, use microscopic simulation

Researcher!
Turn the problem into a programming languages problem
Regard kinetic models as

probabilistic rewrite systems

Researcher!
Turn the problem into a programming languages problem
Regard kinetic models as

probabilistic rewrite systems
Potential to gain both speed and information content

• Microscopic simulation gives microscopic information
• Stochastic solvers (Monte-Carlo) are easier to parallelise
• We know much about speeding up the execution of RSes

Microscopic simulation of polymerisation kinetics
The soup—aka system state

• Multiset of molecules
• Subscript is chain length of a polymer
• Monomers have no subscript
• Star polymers have multiple subscripts

Microscopic simulation of polymerisation kinetics
The soup—aka system state

• Multiset of molecules
• Subscript is chain length of a polymer
• Monomers have no subscript
• Star polymers have multiple subscripts
Reactions with rate coefficients—aka probabilistic rewrite rules

Reactions:
• Consume one or two molecules
I 7→ I• + I• [kd ]
• Produce one or two molecules
I• + M 7→ P1 [ki ]
Rate coefficients:
Pn + M 7→ Pn+1 [kp ]
• Reaction probability relative to
molecule concentration Pn + Pm 7→ Dn+m [kt ]

Structure of the simulation
À Compute reaction probabilities, known as rates:

I Product of the reaction’s rate coefficient and current
concentration of the reactants
I Stored in reaction rate tree
I Determines the probability distribution function
I Rate coefficients are experimentally determined

Á Randomly pick a reaction

I Reaction rate tree enables fast reaction selection given a
uniformly distributed random number
I E.g., we might pick Pn + Pm 7→ Pn+m
[NB: we haven’t fixed n and m yet]

Â Randomly pick the molecules

I For monomers, there is nothing to do
I For polymers, we need to pick one or more chain lengths
I E.g., For Pn + Pm 7→ Pn+m , pick n and m randomly (from
available lengths)
I In some systems, different chain lengths react with different
probabilities
Ã Compute reaction products

I Update the concentration of molecules
I E.g., remove Pn and Pm and add Pn+m
I Advance system clock (non-trivial)

Specialisation
Reaction kinetics as probabilistic multiset rewriting

• Direct implementation of the four simulator steps
=⇒ interpreter for the rewrite system

Specialisation

• Performance is an issue: Let’s compile the rewrite system!
Which part of the simulator is worth compiling?

Specialisation

• Performance is an issue: Let’s compile the rewrite system!
Which part of the simulator is worth compiling?

• Only code that depends on the kinetics model (our
“program”)
• The code that updates the system state according to a
selected reaction and reactants:
1 Update molecule count—i.e., reactant concentrations
2 Adapt reaction probabilities
3 Modify reaction rate tree

Runtime system as template (using CPP to instantiate)
#include "genpolymer.h" // generated by simulator generator
void oneReaction () {
// . . . variable declarations omitted. . .
À updateProbabilities ();
Á reactIndex = pickRndReact ();
Â mol1Len = pickRndMol (reactToSpecInd1 (reactIndex));
if (consumesTwoMols (reactIndex))
mol2Len = pickRndMol (reactToSpecInd2 (reactIndex));
Ã switch (reactIndex) // compute reaction products
DO_REACT_BODY // defined in genpolymer.h
incrementMolCnt (resMol1Spec, resMol1Len);
if (resMolCnt == 2)
incrementMolCnt (resMol2Spec, resMol2Len);
advanceSystemTime (); // compute ∆t of this reaction
}

Simulator generator in Haskell
• Compiles the kinetics model (probabilistic rewrite system)
• For example,
I 7→ I• + I•
⇓
#define I_Star 2
...
#define DECOMPOSITION 0
...
#define DO_REACT_BODY \
{case DECOMPOSITION:\
resMolCnt = 2;\
resMol1Spec = I_Star;\
resMol2Spec = I_Star;\
break;\
...
genpolymer.h

Haskell
Programming in Haskell
• Popular functional language: fast, all functions, no OO!
• Pure – no side effects by default
• Richly and strongly typed – no runtime type errors
• Easy to parallelise, implicit and explicit cheap threads
• Algebraic data types – good for symbolic manipulation
Example: data parallel dot product
dotp :: [: Double :] → [: Double :] → Double

dotp v w = sumP (zipWithP (∗) v w)

Specialising Simulator Generator
• Input is reaction specification
• Custom C runtime for reaction generated from Haskell
• All inputs become static constants
• C compiler then aggressively optimises the reaction
implementation
• Run compiled reaction and inspect results
Portability
• Simulator generation easy to retarget: single core,
multicore, clusters
• Just port the simulator runtime

Parallelisation
Vanilla Monte-Carlo
• Probability distribution function is static
• All stochastic events are independent
• Embarrassingly parallel

Parallelisation
Vanilla Monte-Carlo
Markov-chain Monte-Carlo
• Probability distribution function is dynamic
• It is dependent on previous stochastic events
• Fully sequential (no parallelism)

Parallelisation
Vanilla Monte-Carlo
Markov-chain Monte-Carlo
• Probability distribution function is dynamic
• It is dependent on previous stochastic events
• Fully sequential (no parallelism)
Our solution: stirring

• Physical reality: only molecules in close proximity can react
• Stirring: regularly, exchange and average system state
• Optimal stirring frequency dependant on Brownian motion

Benchmarks
Specialised versus unspecialised
70
generic simulator (gcc)
generic simulator (icc)
specialised simulator (gcc)
specialised simulator (icc)
60
50
40
Time (sec)
30
20
10
0
107 2*107 3*107 4*107 5*107
Simulator steps
[icc = Intel C Compiler; gcc = GNU C Compiler]

Parallel speedup
8 8
10
P4 cluster (1010 particles)
Opteron SMP (10 9 particles)
P4 cluster (109 particles)
7 Opteron SMP (10 particles) 7
6 6
Relative Speedup
5 5
4 4
3 3
2 2
1 1
1 2 3 4 5 6 7 8
PE
[P4 cluster = Intel P4, 3.2GHz, with GigaBit Ethernet;

Opteron SMP = AMD Athlon 64 3200+, 2.2GHz, with HyperTransport 1.0]

Deterministic versus Monte-Carlo
40000
35000
logarithmic depiction
100000
30000
simulation time / seconds
10000
25000
1000
20000
100
15000
10
10000 (1) (2) (3) (4) (5) (6) (7) (8) (9)
5000
0
(1) (2) (3) (4) (5) (6) (7) (8) (9)
PREDICI PREDICI PREDICI PREDICI MC 1010 MC 109 MC 1010 / 8 MC 1010 / 16 MC 109 / 8

original optimised optimised optimised
0.02 0.01 0.02 0.05
[PREDICI = commercial coarse-grained h-p-Galerkin simulator;

MC with 4 PEs matches PREDICI with 109 particles and accuracy 0.02]

Lessons Learnt
Make it a PL problem
• It can be worthwhile to turn seemingly non-PL problems
(shortcomings with deterministic PDE solvers) into PL
problems (multiset rewriting & code specialisation)

Lessons Learnt
Declarative languages for computationally-intensive code

• Generative approaches can outperform hand-coded
low-level code
• Generators/compilers are a core domain of functional
languages
• Why not generate code for your next hard simulator
problem?

Lessons Learnt
Declarative languages for computationally-intensive code

• Generative approaches can outperform hand-coded
low-level code
• Generators/compilers are a core domain of functional
languages
• Why not generate code for your next hard simulator
problem?
Prototyping
• Simulator development by prototyping in Haskell (→ paper)

Specialisation of inner loops in Monte-Carlo solvers
• MC always execute an inner loop very many times
• Specialisation is worthwhile whenever significant
parameters are fixed over all or many iterations

Specialisation of inner loops in Monte-Carlo solvers
• MC always execute an inner loop very many times
• Specialisation is worthwhile whenever significant
parameters are fixed over all or many iterations
Parallelisation of Markov-chain Monte-Carlo

• Averaging of parallel system states (i.e., stirring) is
generally applicable in MCMC

Conclusions
Specialising Monte-Carlo simulator generators

• Kinetics models as probabilistic rewrite systems
• Highly optimised low-level code
• Parallelisation of Markov-chain Monte-Carlo
• First competitive Monte-Carlo simulator for polymerisation
kinetics, produces microscopic information
Future work
• Other polymer structures (meshes)
• Application to financial mathematics

Thanks!
Make sure you understand compilers

Specialising Generators For High-Performance Monte-Carlo Simulation ... in Haskell

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Specialising Generators For High-Performance Monte-Carlo Simulation ... in Haskell

Uploaded by

Copyright:

Available Formats

Specialising Simulator Generators

for High-Performance Monte-Carlo Methods

University of New South Wales

Don Stewart | Galois Inc Specialising Simulator Generators

Don Stewart | Galois Inc Specialising Simulator Generators

Kinetic models (here, polymerisation of bulk styrene)

Don Stewart | Galois Inc Specialising Simulator Generators

Kinetic models (here, polymerisation of bulk styrene)

Exploration by way of computational chemistry

Don Stewart | Galois Inc Specialising Simulator Generators

Don Stewart | Galois Inc Specialising Simulator Generators

Problems encountered by chemists

• Improving both speed and information content seems hard

Don Stewart | Galois Inc Specialising Simulator Generators

Don Stewart | Galois Inc Specialising Simulator Generators

Turn the problem into a programming languages problem

Regard kinetic models as

Don Stewart | Galois Inc Specialising Simulator Generators

Turn the problem into a programming languages problem

Regard kinetic models as

Potential to gain both speed and information content

Don Stewart | Galois Inc Specialising Simulator Generators

The soup—aka system state

Don Stewart | Galois Inc Specialising Simulator Generators

The soup—aka system state

Reactions with rate coefficients—aka probabilistic rewrite rules

Don Stewart | Galois Inc Specialising Simulator Generators

À Compute reaction probabilities, known as rates:

Don Stewart | Galois Inc Specialising Simulator Generators

Á Randomly pick a reaction

Don Stewart | Galois Inc Specialising Simulator Generators

Â Randomly pick the molecules

Ã Compute reaction products

Don Stewart | Galois Inc Specialising Simulator Generators

Reaction kinetics as probabilistic multiset rewriting

Don Stewart | Galois Inc Specialising Simulator Generators

Reaction kinetics as probabilistic multiset rewriting

Which part of the simulator is worth compiling?

Don Stewart | Galois Inc Specialising Simulator Generators

Reaction kinetics as probabilistic multiset rewriting

Which part of the simulator is worth compiling?

Don Stewart | Galois Inc Specialising Simulator Generators

Don Stewart | Galois Inc Specialising Simulator Generators

Don Stewart | Galois Inc Specialising Simulator Generators

Example: data parallel dot product

dotp :: [: Double :] → [: Double :] → Double

Don Stewart | Galois Inc Specialising Simulator Generators

Don Stewart | Galois Inc Specialising Simulator Generators

Don Stewart | Galois Inc Specialising Simulator Generators

Don Stewart | Galois Inc Specialising Simulator Generators

Our solution: stirring

Don Stewart | Galois Inc Specialising Simulator Generators

[icc = Intel C Compiler; gcc = GNU C Compiler]

[P4 cluster = Intel P4, 3.2GHz, with GigaBit Ethernet;

Don Stewart | Galois Inc Specialising Simulator Generators

PREDICI PREDICI PREDICI PREDICI MC 1010 MC 109 MC 1010 / 8 MC 1010 / 16 MC 109 / 8

[PREDICI = commercial coarse-grained h-p-Galerkin simulator;

Don Stewart | Galois Inc Specialising Simulator Generators

Don Stewart | Galois Inc Specialising Simulator Generators

Declarative languages for computationally-intensive code

Don Stewart | Galois Inc Specialising Simulator Generators

Declarative languages for computationally-intensive code

Don Stewart | Galois Inc Specialising Simulator Generators