You are on page 1of 8

Building Scientific Workflows for Earth System

Modelling with Windows Workflow Foundation
Matthew J. Fairman1, Andrew R. Price1, Gang Xue1, Marc Molinari1,
Denis A. Nicole2, Timothy M. Lenton3, Robert Marsh4,
Kenji Takeda1, and Simon J. Cox1
1

Microsoft Institute for High Performance Computing,
School of Engineering Sciences, University of Southampton, Southampton, UK
{mjf, andrew, gx, mm, ktakeda, sjc}@soton.ac.uk
2
School of Electronics and Computer Science, University of Southampton, UK
dan@ecs.soton.ac.uk
3
School of Environmental Sciences, University of East Anglia, Norwich, UK
t.lenton@uea.ac.uk
4
National Oceanography Centre, University of Southampton, Southampton, UK
rma@noc.soton.ac.uk

Abstract. The GENIE project has built a Grid enabled framework that
facilitates the integration, execution and management of component models for
the study of the Earth system over millennial timescales. The existing
framework supports collaborative study of GENIE models across
heterogeneous compute grids through scripted workflows in the Matlab
environment. In this paper, Windows Workflow Foundation technologies are
applied to demonstrate the benefits of an environment that provides rapid
composition, event driven logic and reliable hosting of scientific workflows.
These improvements are demonstrated through a parametric study of bi-stability
of the oceanic thermohaline circulation in a GENIE model.
Keywords: Workflow, GENIE, Windows Workflow Foundation.

1 Introduction
E-Science applications are usually composite systems that involve collaborations
among various computational resources and software components in a distributed,
heterogeneous environment. Workflow technologies have become a key element to eScience systems, which help to orchestrate interactions with services so that they are
seamlessly knitted together to implement the desired behaviour of the system.
A typical example of such an e-Science system can be found in GENIE [1]. The
GENIE project has developed a framework for the composition, execution and
management of integrated Earth system models. Component codes (e.g. ocean,
atmosphere, land surface, sea-ice, ice-sheets, biogeochemistry, etc.) of varying
resolution and complexity can be flexibly coupled together to form a suite of efficient
climate models capable of simulation over millennial timescales. The project brings
Y. Shi et al. (Eds.): ICCS 2007, Part III, LNCS 4489, pp. 273–280, 2007.
© Springer-Verlag Berlin Heidelberg 2007

Submitting the work units to the compute resources specified by the user. a set of tools that facilitate workflow design. It has therefore put rigorous requirements on the selection of workflow technologies for GENIE. Grid computing technology supports the virtual organization to collaboratively study the models from the framework. As an integral part of the . Results from a scientific study of bi-stability in the thermohaline circulation (THC) in a GENIE model. The recently released Microsoft Windows Workflow Foundation (WF) [2] is a general. Earth system simulations are both computationally and data intensive processes. extensible framework for developing workflow solutions. . undertaken using our WF based system.0 [3]. In contrast to system likes GridSolve [4] and Nimrod/G [5].NET Framework 3.J.NET technologies to provide support for distributed communication and rich user experiences. together a distributed group of environmental scientists with a common interest in developing and using GENIE models to understand the Earth system. In this paper we discuss and demonstrate how the WF technologies are applied to build efficient and flexible workflows for running Earth system simulations based on the GENIE framework. the GENIE framework devolves responsibility for the management of work units to the client side. are presented in section 5. and pools compute resources contributed by members of the project to carry out large parametric studies. often consisting of a discrete series of steps. 2 Workflow in the GENIE Framework The GENIE framework has been designed to facilitate collaboration between a distributed team of environmental scientists to perform large ensemble studies of Earth system models that would be impossible for a single user. A GENIE workflow running at the client side is therefore responsible for the following tasks when contributing to a particular study: - Querying the central database for available work units. We have therefore applied WF to the GENIE framework to provide a solid base for the design and running of Earth system simulations. A central shared database accessible from all clients has been set up to provide the means to upload model configurations and experiment definitions. and a robust workflow hosting environment. Section 3 gives a detailed summary of the features of WF. In section 2 we describe the functions of the GENIE workflows and explain why the attempt has been made to adopt WF technologies. Section 4 describes the design and running of our WF based GENIE workflow. and be queried against to find available work units. The GENIE framework has been designed to support running of such simulations across multiple distributed data and computing resources over a lengthy period of time. WF provides a common development model for creating workflows. Fairman et al.274 M. We draw our conclusions in section 6. in which a central manager controls all available resources and makes decisions about where to assign work units. It also allows seamless integration with other .

However. and post-process data from completed work items. Finally. The GENIE system is robust to such failures since the experiment status is always defined in the shared repository but resumption of a study can only be achieved by restarting the workflow. Another problem with hosting workflows in the scripting environment is the lack of support for persisting state of the workflow. Post-processing the results and saving them in the database. or it is terminated by the user.1 Workflow Design with WF WF supports two built-in workflow styles. the GENIE workflow is implemented using the Matlab scripting language and is hosted in the Matlab environment. As an attempt to address these problems. WF can be seamlessly integrated with other . a few problems remain. Presently.Building Scientific Workflows for Earth System Modelling - 275 Monitoring the status of the submitted jobs.0. submit and monitor jobs. implementing the GENIE workflow in the Matlab script ties the system to a single user environment. the user needs to keep running the script all the time and query the central database periodically to find work items. While it is possible for the users to manually save some of the state information. Callback mechanisms or publish/subscribe mechanisms would be an optimal choice for improvement. and state machine workflows. and managing workflows.NET technologies to provide a solid base for the design and running of Earth system simulations. 3. It is very difficult to migrate existing workflows to other systems. When running a GENIE simulation. 3 Windows Workflow Foundation Windows Workflow Foundation (WF) [2] is a Microsoft technology for quickly building workflow enabled applications. we have exploited the WF technologies to develop a dynamic and robust workflow implementation which actively manages the operations of the GENIE simulations with improved efficiency and stability. The workflow runs until there is no work available. This makes it hard to maintain or to try to reuse the workflow activities. it is difficult to make such modifications in the current scripting environment. restoring the running of a workflow in the cases of a system or network failure is always difficult. Time intervals between two scheduled tasks can only be set in a speculative mode. which . which carry out a series of operations in a pre-defined pattern. While simplicity and flexibility are achieved through scripting. WF significantly enhances a developer’s ability to model and support sophisticated science and engineering processes. executing. a host engine and tools for defining. retrieving and checking the results once jobs are finished. For example. Workflows in scripting languages are also less well-structured and might require individual human knowledge to interpret or modify. The scripting approach for workflow tends to be less efficient when dealing with external events.NET Framework 3. sequential workflows.e. Being part of the recently released . the database could publish new work as it becomes available and the clients would subscribe for these events. It consists of a programming model. Events such as new work units being added and compute tasks completing are unlikely to be handled efficiently. i.

It also works as a proxy for all interactions between the workflow and external software. which provides important services.J. Workflows can be created using any programming language that is compliant with the . including Web services. sophisticated enterprise services. 3. it is possible for WF workflows to take advantage of other Microsoft technologies including Windows Communication Foundation (WCF) [7]. Figure 1 shows the structure of the WF based Genie system. WF is a general framework not limited to a single language or a single tool. and supporting transactions.276 M. 4 A Composite Workflow for GENIE Simulations A number of key components have been developed for the GENIE project to facilitate the running of Earth system simulations. WF also provides its own graphical designer. and a shared file store managed by using the Geodise Toolkit [13]. WF supports defining workflows in XAML (eXtensible Application Markup Language) [6].NET Framework’s Common Language Specification. a number of gateway systems to integrate compute resources from different HPC platforms including Microsoft Cluster Computing Server [10]. such as C#. . are driven by external events. an XML based object initialization language. A base activity library is supplied which contains a group of pre-defined activities most commonly used in both sequential and state machine workflows. Windows CardSpace [8] and MS SQL Server 2005 [9] to address problems such as communication in a distributed environment.NET Framework. Condor [11] and Globus [12]. and reuse activities in different workflows. build new activities by aggregating existing ones. where each step of a workflow can be implemented by a specific piece of software called ‘activity’. WF adopts a component like approach. federated security control and data access. tracking the execution. usually hosted in Visual Studio 2005. Being an integrated part of the . This enables WF to accommodate both the well structured. a composite workflow has been implemented using the WF technologies. such as persisting workflow state. In addition. so that different workflow designers can be used to generate or parse WF workflows. Visual Basic and Visual C++. flexible human workflows. These include a shared database hosted in Oracle 10g which contains the definitions. to support visual design of workflows. and load them back when necessary. The WF runtime can make decisions to unload long-running workflows that have been inactive for a period of time. In order to bring these components together in a single system that manages the Earth system simulations. Application of some of these technologies together with WF can be found in our work on the Earth system modelling workflows. states and results of the experiments. automated system workflows and the loosely defined.2 Workflow Hosting with WF WF workflows can be hosted in any Windows process ranging from simple console applications to large. Users are also allowed to build their own customised activities. Fairman et al. in an asynchronous style. which is discussed in the following section. The workflow execution is carried out within the WF runtime engine.

1.Building Scientific Workflows for Earth System Modelling Fig. 2. Design of the GENIE workflow in the WF Designer with Visual Studio 277 . The Complete Structure of the WF based GENIE Simulation System Fig.

principally controls meridional moisture transport between the equator and poles. The Job Service notifies the workflow instance of changes to the job status by firing a batched ‘jobStatusChanged’ event. which consists of a 3D frictional geostrophic ocean coupled to a simple 2D energy-moisture balance model and a 2D sea-ice component. the workflow registers their identifiers with the Job Service which is responsible for monitoring the jobs’ running statuses. the Resource Service. Consequently. The Genie workflow employs a set of external data exchange services that run within the workflow runtime. Fairman et al. It is a sequential workflow that consists of a while loop activity which contains a single parallel asynchronous event listening activity that waits for the following changes in the state of an experiment: newly specified resources. The individual model runs each required approximately 2 hours compute time on a typical desktop. When compute jobs are submitted. we repeat a systematic 2-parameter study of bi-stability in the Atlantic thermohaline circulation [14] using a higher resolution version of the GENIE model. The study investigates the CGOLDSTEINcomposition referred to as genie_ea_go_gs in the GENIE framework nomenclature. The initial parameter sweep therefore required approximately 242 hours of compute time. This not only provides a more responsive solution than the scripting based approach of polling. These services provide the means by which events occurring within the experiment system are passed into the workflow. allows the users to instruct the workflow to create new experiment work units by registering additional compute resources with it. Figure 2 shows the main part of the design of the GENIE workflow. Another external data exchange service. In addition. but also achieves better efficiency and robustness of the system by allowing the runtime to persist workflow instances in a database and unload them during periods of idleness. changes of job states and completion of work item post-processing. Each simulation was integrated over 4000 years to ensure an equilibrium state was achieved. The Genie WF workflow implementation automates the operations and interactions on the system components. A systematic sweep over a range of values for these parameters was undertaken. All of the asynchronous events are indications that either compute resources have become available or subsequent work items are now available. With the benefit of knowledge of the behaviour of the model at a lower resolution we chose to perform a coarser sampling of the parameter space (11x11 as opposed to 31x31) than the original study.J. the Job Service also performs asynchronous post-possessing of results from completed jobs when the workflow receives a ‘jobStatusChanged’ event of either ‘Completed’ or ‘Failed’.278 M. . Kq: Atmospheric moisture diffusivity . 5 Earth System Simulation with WF To demonstrate the WF based GENIE system. The study seeks to investigate the properties of the Atlantic Thermohaline Circulation as a function of two of the key parameters that dominate atmospheric moisture transport in the model: - FWx: Zonal moisture transport from the Atlantic to the Pacific. a find work activity is performed after any one of these events has occurred.

3. Having provided a coarse mapping of the behaviour of the Atlantic THC in the model. a) Summary of distribution of compute jobs across the computational grid. nine end-states were taken from the extremes of the parameter space to search for bi-stability. Fig. using a 16-level seasonal version of the genie_ea_go_gs model. a. the study proceeded by performing two further identical parameter sweeps using models with different boundary conditions. Such runs are designed to investigate whether there are points in the parameter space of the atmospheric water transport properties for which the equilibrium THC state of the model is dependent upon its initial conditions. In this study we perform just two secondary ensembles using the end-states from the “top-left” and “bottom-right” points (Figure 3 (c)) which represent the extreme “off” and “on” states in the THC respectively. b) distribution of compute jobs in the parameter space. c. b. Study of bi-stability in the Atlantic Thermohaline Circulation. . c) Strength of maximum Atlantic Overturning Circulation in the initial parameter sweep and d) narrow region of bi-stability in the parameter space. Following the coarse initial mapping the parameter space was further refined through the addition of another 100 (10x10) members to each ensemble. d. These secondary ensembles performed identical model runs initializing the models using particular end-states from the initial ensemble. In Marsh et al.Building Scientific Workflows for Earth System Modelling 279 The WF based GENIE system has been used to repeat the study of Marsh et al. An interactive plot of the results was then exploited to add further ensemble members in the emerging regions of bi-stability in the THC to focus further work in the most appropriate locations.

com/en-gb/netframework/ 4. YarKhan. Recent Developments in Gridsolve.280 M. Buyya. Shukla. A more detailed study of the parameter space and analysis of the results will be the subject of future work. Microsoft.aspx 7. 131-141 5. It is evident that further runs will be required to fully resolve the extent of the bi-stable region. 23 (2004) 761-777 .. M.. The Globus Project.. D. GENIE: Delivering e-Science to the environmental scientist. Addison Wesley Professional.com/technet/ 11. 2. J. Figure 3 (a) summarises the resource usage in performing the initial ensemble.J. Microsoft. http://www. et al. 145-152..geodise. 20 (2006). http://msdn2. of High Performance Computing Applications. Microsoft. UK.microsoft. The distribution of simulations across the resource pool is plotted in the parameter space in Figure 3 (b). HPC Asia 2000. pp 283 289.. UK eScience All Hands Meeting. 13.microsoft.0. Windows Compute Cluster Server 2003 Product Overview.com/windowsserver2003/ccs/overview. The Condor Project. J. ISBN 1-904425-11-9. We reviewed features of WF technologies and explained how they have been deployed in the GENIE framework to facilitate coordinated operations among various distributed components. B. A.edu/condor/ 15. http://www.com/library/enus/dnlong/html/introinfocard. D. References 1. Z. Microsoft . http://www. 6 Summary In this paper we discussed and demonstrated how the Windows Workflow Foundation technologies can be exploited to implement scientific workflows for e-Science systems such as GENIE. http://www. and Giddy.. Marsh. Chappell. Abramson. D. prodtechnol/sql/2005/downloads/books. Nottingham. Bistability of the thermohaline circulation identified through comprehensive 2-parameter sweeps of an efficient climate model. The distribution of compute jobs broadly reflects the relative numbers of single concurrent compute tasks that the schedulers on each system allow to a single user. First indications are that the newly tuned 16-level model has a narrower region of bi-stability than the original 8-level model [14].org 16. Introducing Windows CardSpace. Beijing.asp 10.org 17.mspx 14.microsoft. pp. SQL Sever 2005 Books Online. 8. 2003.cs. ISBN-10: 0-321-39983-8 3. China. Nimrod/G: An Architecture of a Resource Management and Scheduling System in a Global Computational Grid.microsoft. Gulamali. Dyn.globus. May 14-17. http://msdn2. http://msdn.wisc. Essential Windows Workflow Foundation. Int. XAML Overview... A collaborative study of bistability in the thermohaline circulation of an efficient climate model has been performed to demonstrate the running of the WF based GENIE system. and Dongarra. http://www. http://msdn. et al.asp 9.microsoft. Fairman et al.NET Framework 3.. 6. R..microsoft. Shi. R. Sagi.mspx 12.Y. The Geodise project. Schmidt. Windows Communication Foundation Architecture Overview. The results of the bi-stability study are presented in Figure 3.com/library/en-us/dnlong/html/wcfarch. 2000.. K. Microsoft.. The user additions of model runs are focused along the “cliff-edge” of THC collapse (Figure 3 (c)) where a narrow region of bi-stability emerges (Figure 3 (d)). Clim.com/en-us/library/ms752059. Seymour. K. J.