This action might not be possible to undo. Are you sure you want to continue?

BooksAudiobooksComicsSheet Music### Categories

### Categories

### Categories

### Publishers

Scribd Selects Books

Hand-picked favorites from

our editors

our editors

Scribd Selects Audiobooks

Hand-picked favorites from

our editors

our editors

Scribd Selects Comics

Hand-picked favorites from

our editors

our editors

Scribd Selects Sheet Music

Hand-picked favorites from

our editors

our editors

Top Books

What's trending, bestsellers,

award-winners & more

award-winners & more

Top Audiobooks

What's trending, bestsellers,

award-winners & more

award-winners & more

Top Comics

What's trending, bestsellers,

award-winners & more

award-winners & more

Top Sheet Music

What's trending, bestsellers,

award-winners & more

award-winners & more

P. 1

Concurrent Orchestration in Haskell|Views: 419|Likes: 5

Published by Don Stewart

See more

See less

https://www.scribd.com/doc/38549998/Concurrent-Orchestration-in-Haskell

03/26/2012

text

original

John Launchbury Trevor Elliott

Motivating Example

Helper

Server

Client1 Xen hypervisor

Client2

Tester

• Tester talks with each of the VMs concurrently • Many possible behaviors are “correct” / “incorrect” • Timeouts, VMs dying, etc. • Subtle concurrency bugs in test framework

Concurrent Scripting

• Management of multiple resources

– Multiple paths – Timeouts and defaults – Processing of data and results

**• “Scripting” rather than “programming”
**

– Minimal concurrency paraphernalia

• E.g. thread identifiers, locks, etc. etc.

– Easy to get right

Orc Intuitions

l

O iftI

lift Lis t

IO

putMVar

Orc

<|>

List

readTVar

MonadIO

MonadPlus

Basic Examples

fplang :: Orc String fplang = return “Haskell” <|> return “ML” <|> return “Scheme”

fplang

“Haskell”

“ML”

“Scheme”

Basic Examples

metronome :: Orc () metronome = return () <|> (delay 2.5 >> metronome)

metronome

delay 2.5 ()

Basic Examples

metro :: Float -> Orc () metro j = return () <|> (delay j >> metro j) rsiReminder :: Orc () rsiReminder = do metro 3600 email “john at galois.com” “Get up and stretch!!”

Search

queens = fmap show (extend []) <|> return ("Computing 8-queens...") extend :: [Int] -> Orc [Int] extend xs = if length xs == 8 then return xs else do j <- liftList [1..8] guard $ not (conflict xs j) extend (j:xs) conflict :: [Int] -> Int conflict = ...

Concurrency point

MonadPlus operation

Search

*Main> printOrc (queens) Ans = "Computing 8-queens..." Ans = "[5,7,1,3,8,6,4,2]" Ans = "[6,4,2,8,5,7,1,3]" Ans = "[5,3,8,4,7,1,6,2]" Ans = "[4,2,7,3,6,8,5,1]" : *Main> printOrc (queens) Ans = "Computing 8-queens..." Ans = "[4,2,7,3,6,8,5,1]" Ans = "[6,4,7,1,8,2,5,3]" Ans = "[3,6,4,2,8,5,7,1]" Ans = "[2,7,3,6,8,5,1,4]" :

Orc API

newtype return (>>=) stop (<|>) eagerly (<+>) liftIO runOrc Orc a :: a -> Orc a :: Orc a -> (a -> Orc b) -> Orc b :: Orc a :: Orc a -> Orc a -> Orc a :: Orc a -> Orc (Orc a) :: Orc a -> Orc a -> Orc a :: IO a -> Orc a :: Orc a -> IO () Classes: Functor, Monad, Applicative, Alternative, MonadPlus, MonadIO

Using Eagerly

sync :: (a->b->c) -> Orc a -> Orc b -> Orc c sync f p q = do po <- eagerly p qo <- eagerly q

Monadic function application

• Entering the handle waits for the result • Synchronization

return f <*> po <*> qo notBefore:: Orc a -> Float -> Orc a p `notBefore` w = sync const p (delay w)

cut:: Orc a -> Orc a cut p = do po <- eagerly p po

cut:: Orc a -> Orc a cut = join . eagerly

Web Query

quotes :: Query -> Query -> Orc Quote quotes srcA srcB = do quoteA <- eagerly $ getQuote srcA quoteB <- eagerly $ getQuote srcB cut ( (return least <*> quoteA <*> quoteB) <|> (quoteA >>= threshold) <|> (quoteB >>= threshold) <|> (delay 25 >> (quoteA <|> quoteB)) <|> (delay 30 >> return noQuote))

A B

Need to book a ticket, under $300 if possible…

least x y = if price x < price y then x else y threshold x = guard (price x < 300) >> return x

quote

Counting the number of answers

Using <+>

accum

count :: Orc a -> Orc (Either a Int) count p = do accum <- newTVar 0 do x <- p modifyTVar accum (+1) return $ Left x <+> fmap Right (readTVar accum) newTVar readTVar

p

readTVar

:: MonadIO io => a -> io (TVar a) :: MonadIO io => TVar a -> io a

modifyTVar :: MonadIO io => TVar a -> (a -> a) -> io (a,a)

Control.Concurrent.STM.MonadIO

Orc Laws

Left-Return: Right-Return:

(return x >>= k) = k x (p >>= return) = p

**Bind-Associativity: ((p >>= k) >>= h) = (p >>= (k >=> h)) Stop-Identity: Par-Associativity: Left-Zero: Par-Bind:
**

p <|> stop = p

Par-Commutativity: p <|> q = q <|> p

p <|> (q <|> r) = (p <|> q) <|> r

(stop >>= k) = stop ((p <|> q) >>= k) = ((p >>= k) <|> (q >>= k))

Non-Laws

Bind-Par?:

p >>= (\x -> h x <|> k x) = (p >>= h) <|> (p >>= k)

Right-Zero?: p >> stop = stop

p `onlyUntil` done = cut (silent p <|> done) silent p = p >> stop

Repetition

Re-invoke the Orc argument after each success

repeating :: Orc a -> Orc a repeating p = do x <- p return x <|> repeating p

p

x

p

The computation bifurcates if p returns multiple values

x’

Permitting

An Orc term controls the execution of another

p q

(<#>) :: Orc a -> Orc b -> Orc a p <#> q = do end <- newEmptyMVar (p <+> silent (putMVar end ())) <|> silent (q `onlyUntil` takeMVar end)

q is run for its effects q is killed when p finishes

Controlling file downloads

Worked Example

downloads :: [(URL,Path)] -> IO () downloads fileList = do let downloads = map getFile fileList runOrc $ msum downloads getFile :: (URL,Path) -> Orc () getFile (url,path) = liftIO $ do res <- openURIString url case res of Left err -> putStrLn (url++": "++err) Right content -> writeFile ("Files/"++path) content

Runs all the downloads concurrently

IO code lifted into Orc

Bounded Parallelism

Unbounded parallelism

msum :: [Orc a] -> Orc a msum ps = foldr (<|>) stop ps

Add a bound

msum’ :: Orc Int[Orc[Orc-> Orc Orc a Int -> -> a] a] -> a msum’ bound ps = ...

Using Orc for the bound allows it to be dynamic

msum’ :: Orc Int -> [Orc a] -> Orc a msum’ bound ps = do t <- newTVar 0 msum (map (wrap t) ps) <#> setBound t bound wrap :: TVar Int -> Orc a -> Orc a wrap t p = do checkModifyTVar t (>0) (\x->x-1) p <+> silent (modifyTVar_ t (+1)) setBound :: TVar Int -> Orc Int -> Orc () setBound t bound = do n <- bound modifyTVar_ t (+n)

Bounded Parallelism

Launch p only if a resource is available

Update resource bound for each new bound

**Example: file downloads
**

downloads :: [(URL,Path)] -> IO () downloads fileList = do let downloads = map getFile fileList runOrc $ msum’ bounds downloads bounds = return 4

Runs all the downloads concurrently (bounded)

**Example: file downloads
**

downloads :: [(URL,Path)] -> IO () downloads fileList = do let downloads = map getFile fileList runOrc $ msum’ bounds downloads bounds = return 4 <|> promptUser promptUser :: Orc Int promptUser = do xs <- repeating $ liftIO (putStrLn "Enter bounds changes (+/-) " >> getLine) return (read xs)

Runs all the downloads concurrently (bounded)

Dynamic bound updating

Layered Implementation

• Layered implementation — layered semantics – Properties at one level depend on properties at the level below

multiple results thread control external effects

Orc Scripts Orc Monad HIO Monad IO Monad Transition Semantics

• What properties should Orc terms satisfy? – Hence, what properties should be built into HIO? • Unresolved question: what laws should the basic operations of the IO monad satisfy?

type Orc a = (a -> HIO ()) -> HIO () return x = \k -> k x p >>= h stop p <|> q = \k -> p (\x -> h x k) = \k -> return () = \k -> fork (p k) >> q k

Key Definitions

Discard the continuation

Duplicate the continuation

runOrc p = p (\x -> return ()) liftIO m = \k -> (liftIO m >>= k)

printOrc = runOrc $ do a <- p liftIO $ putStrLn ("Ans = " ++ show a)

HIO Monad

~~~~ ~ ~~ ~~~~ ~~~ ~~ ~~~~ ~~~~ ~~~ ~~ ~~ ~ ~~ ~~~~

• Don’t want the programmer to have to do explicit thread management – Nested groups of threads • Want richer equational theory than IO – e.g. by managing asynchronous exceptions

:: Group -> IO a}

~~~~ ~~~ ~~ ~~~ ~ ~~ ~~~~ a =

newtype HIO

HIO

~~~~ ~ = ~~

~~~~~ ~~ ~~ ~~~~~ ~~~ {inGroup ~~~~

type Group = (TVar Int, TVar Inhabitants) data Inhabitants Closed | Open [Entry]

data Entry = Thread ThreadId | Group Group

newPrimGroup :: IO Group register :: Entry -> Group -> IO ()

killGroup :: Group -> IO () increment, decrement, isZero :: Group -> IO ()

Concluding

• Orc in Haskell

– Ready for use today for concurrent scripting

• cabal install orc

**– Flexibility of embedded DSLs: discover new idioms – Design principles emerge
**

• Two kinds of communications • Many kinds of state elements

**• Ongoing exploration in concurrency
**

– What are the right abstractions?

**• Misra and Cook
**

– Thanks for their pioneering work on the Orc calculus

Backup

Eagerly Laws

Eagerly-Par: eagerly p >>= (\x -> k x <|> h) = (eagerly p >>= k) <|> h Eagerly-Swap:

do y <- eagerly p x <- eagerly q return (x,y) = do x <- eagerly q y <- eagerly p return (x,y)

Eagerly-IO: eagerly (liftIO m) >> p = (liftIO m >> stop) <|> p

eagerly :: Orc a -> Orc (Orc a) eagerly p = \k -> do r <- newEmptyMVar forkM (p (\x -> putMVar r x)) k (liftIO $ readMVarM r)

Eagerly

read r ?

eagerly p p

eagerly p

p =

a

k

put r

k

• Give p a continuation that will store its result • Return the “value” that accesses that result for the then current continuation

eagerly :: Orc a -> Orc (Orc a) eagerly p = \k -> do r <- newEmptyMVar forkM (p `saveOnce` r)) k (liftIO $ readMVar r)

Eagerly

saveOnce :: Orc a -> MVar a -> HIO () p `saveOnce` r = do p (\x -> putMVar r x)

• Give p a continuation that will store its result (but once only even if duplicated) • Return the “value” that accesses that result for the then current continuation

eagerly :: Orc a -> Orc (Orc a) eagerly p = \k -> do r <- newEmptyMVar forkM (p `saveOnce` r)) k (liftIO $ readMVar r)

Eagerly

saveOnce :: Orc a -> MVar a -> HIO () p `saveOnce` r = do ticket <- newMVar () p (\x -> takeMVar ticket >> putMVar r x)

• Give p a continuation that will store its result (but once only even if duplicated) • Return the “value” that accesses that result for the then current continuation

eagerly :: Orc a -> Orc (Orc a) eagerly p = \k -> do r <- newEmptyMVar g <- newGroup local e $ forkM (p `saveOnce` (r,g)) k (liftIO $ readMVar r) saveOnce :: Orc a -> (MVar a,Group) -> HIO () p `saveOnce` (r,g) = do ticket <- newMVar () p (\x -> takeMVar ticket >> putMVar r x >> close g)

Eagerly

• Give p a continuation that will store its result (but once only even if duplicated) • Return the “value” that accesses that result for the then current continuation • Thread management can be carried over too

val val p

:: Orc a -> Orc a = \k -> do r <- newEmptyMVar g <- newGroup local e $ forkM (p `saveOnce` (r,g)) k (unsafePerformIO $ readMVar r)

Val

val p p

a

saveOnce :: Orc a -> (MVar a,Group) -> HIO () p `saveOnce` (r,g) = do ticket <- newMVar () p (\x -> takeMVar ticket >> putMVar r x >> close g)

• The implementation of val (the alternative that uses lazy thunks) is almost identical

quotesVal :: Query -> Query -> Orc Quote quotesVal srcA srcB = do quoteA <- val $ getQuote srcA quoteB <- val $ getQuote srcB cut ( publish (least quoteA quoteB) <|> (threshold quoteA) <|> (threshold quoteB)

Web Query

<|> (delay 25 >> (publish quoteA <|> publish quoteB)) <|> (delay 30 >> return noQuote)) publish :: NFData a => a -> Orc a publish x = deepseq x $ return x

• Good: use the lazy values directly • Bad: have to be careful about evaluation

Building a business with Haskell

Hackage, Cabal and the Haskell Platform

Practical Haskell Programming

Engineering Large Projects in a Functional Language

Multicore Haskell Now!

An Introduction to Communicating Haskell Processes

Multicore programming in Haskell

The Semantics of Asynchronous Exceptions

Evaluation strategies and synchronization

Modern Benchmarking in Haskell

Galois Tech Talk

Loop Fusion in Haskell

Multicore Haskell Now!

Domain Specific Languages for Domain Specific Problems

Specialising Generators for High-Performance Monte-Carlo Simulation ... in Haskell

Haskell

Improving Data Structures

Haskell Arrays Accelerated with GPUs

The Birth of the Industrial Haskell Group

A Wander Through GHC's New IO Library

The Design and Implementation of xmonad

Stream Fusion for Haskell Arrays

Engineering Large Projects in Haskell

Supercompilation for Haskell

- Read and print without ads
- Download to keep your version
- Edit, email or read offline

Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

CANCEL

OK

You've been reading!

NO, THANKS

OK

scribd

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->