Data Services in your Spreadsheet!
Régis Saint-Paul, Boualem Benatallah
School of Computer Science & Engineering
University of New South Wales
Sydney NSW 2052, Australia
{regiss, boualem}@cse.unsw.edu.au
ABSTRACT
Service-oriented architecture offers a high-level and inte-
grated access to data and applications across the company.
Using Data Services, together with Service Data Objects,
developers can now refer to business entities rather than
storage structures. They are relieved from the burden of
repetitive and low-level tasks such as joining tables.
Sadly, a vast community of developers don’t as yet enjoy
this relief. They are end-user programmers—and their pro-
gramming environment of choice is the spreadsheet. In this
paper, we identify what are the criteria that define—from
an end-user point of view—a good integration of spread-
sheet with a service oriented architecture. Based on these
criteria, we propose SpreadATOR, a generic bridge between
data services and spreadsheets.
Categories and Subject Descriptors
H.4.1 [Office Automation]: Spreadsheets; E.2 [Data Stor-
age Representations]: Object representation
General Terms
Languages, Design
Keywords
Spreadsheet, Service-oriented, architecture, integration
1. INTRODUCTION
End-user programmers—the 45 million of them, as esti-
mated for 2001 in US alone [18]—routinely use spreadsheet
to visualize, manipulate, and analyze data. Thanks to this
environment, they can build more or less complex applica-
tions that solve their daily problems. Even building a report
can be seen as programming an application that takes corpo-
rate data as input and outputs a presentation. To build this
application, spreadsheet users have to import data and place
them in spreadsheet cells, highlight the important pieces,
compute maybe some aggregates, add a chart or two. If
well done, this application can be used again later on a new
set of data to effortlessly produce a new report.
Service oriented architecture (SOA) emerged as a response
to the general problems of enterprise application integration
(EAI) and enterprise information integration (EII) [11, 20,
Copyright is held by the author/owner(s).
WWW2007, May 8–12, 2007, Banff, Canada.
.
Julien Vayssière
SAP Research, RC Brisbane
Level 12, 133 Mary Street
Brisbane QLD 4000, Australia
julien.vayssiere@sap.com
5]. This architecture offers a unified and high-level view of
the company resources.
In particular, the advent of data services [8] and standards
such as Service Data Objects (SDO)[19], professional devel-
opers now benefit from high-level and integrated data access.
Integrated because developers can access transparently from
a single end-point to information that may be managed by
various systems and stored in various location. High-level
because data services rely on a conceptual modeling of the
information—namely the Entity-Relationship (ER) model.
For example, developers can retrieve from a data service an
entity customer. This entity presents information retrieved
for some parts from a relational database and for the rest
from a supply chain management software. From this en-
tity, developers can also access to related entities such as the
purchase orders or the invoices of this particular customer.
Undoubtedly, spreadsheets need to be integrated with SOA.
Indeed, there already exist some efforts to this end. For in-
stance, Microsoft Excel Services [1] allows to incorporate
Excel computations as part of a larger process. Another
example is given by Visual Studio Tool for Office (VSTO)
[14] which allows to isolate the presentation elements of a
spreadsheet from the data it contains. Data are stored as
separate XML documents and can be consumed by other
applications.
Those initiatives, however, are meant for professional de-
velopers. A solid background in object-oriented program-
ming is needed to use VSTO. In this article, we are con-
cerned with the large majority of spreadsheet users—the
non-professional programmer ones. Manipulating SDO en-
tities can already be done by using the macro language that
accompanies spreadsheet environments. But few end-users
are ready to invest time in learning a macro language, which
resort to learning object-oriented programming.
For the majority of end-users, spreadsheet programming
means formulas and cells manipulations only, sometimes as-
sisted by wizards or visual assistants. An integration solu-
tion has to preserve this programming model and it has to
accommodate existing spreadsheet environment. With this
in mind, we want to investigate how integration of spread-
sheet with SOA can impact the daily tasks of end-users pos-
itively; what would characterize—to them—a good integra-
tion.
The integration of spreadsheet with data services bears
several facets. In this article, we focus on the problem of
importation and manipulation of data delivered by data ser-
vices. Other aspects such as the exportation of data from
spreadsheet is left for future research.
 This paper is organized as follows. In section 2 we define
more precisely the model of spreadsheet programming and
present the framework of service-oriented architecture and
data services.
In Section 3.1, our first contribution is to identify the
dimensions of the importation and manipulation of data ob-
jects in spreadsheet—that is, we define what are the char-
acteristics of a good integration solution with data services.
In particular, we identify how the richness of the ER model
can benefit to end-users when they program with the flat
tabular representation of data proposed by a spreadsheet.
On this basis, we review in Section 3.2 the existing ap-
proaches to data importation. As we will see, those ap-
proaches are not fully satisfactory because (i) they do not
embrace the spreadsheet programming model and, thus, have
limited programmability and, (ii) while they offer some pre-
defined manipulations that take advantage of the underlying
structure of imported data, users can not benefit from this
structure in their programming.
In Section 4 we present our approach, called SpreadA-
TOR, for data importation. Our other contributions are:
• In order to give more programming possibilities to end-
users and to leverage their expertise in spreadsheet
programming, we propose to import data by using the
traditional spreadsheet programming model. To this
end, SpreadATOR proposes to blend the specification
of importation by using formulas. But contrary to
other approaches, those formulas are not limited to
primitive data types and can be used to access collec-
tions of data or composite data structures.
• We propose a method to allow users to exploit rela-
tionships that exist between entities to access related
information. This method also lets users program com-
putations that apply to a business entity (e.g. a cus-
tomer) and can be reused when the entity appears in
other context (e.g. in a list of customers).
• We propose to enhance the traceability of imported
data by offering a generic facility for the display of
meta-data and allow users to use meta-data in formula.
In Section 5, we show how our prototype implementation
of SpreadATOR—as an add-in to MS Excel—can be used
with publicly available data sources and how it simplifies the
spreadsheet developer task when working with composite
data. We present in section 6 additional related work and
conclude in Section 7.
2. BACKGROUND
2.1 Spreadsheet Programming Model
Spreadsheet applications come in a variety of implemen-
tation and bear an even more varied set of features. In ad-
dition, several research proposals have been made to extend
the spreadsheet model in various directions.
In order to facilitate our exposition, we will introduce in
this section what we call the traditional model of spread-
sheet programming. We intend here to capture the essence
of spreadsheet common to the vast majority of commercial
implementations. We will reserve the discussion of the vari-
ous extensions to this model—mainly research prototypes—
to section 6.
Grid model
Spreadsheet layer
Data Data Data Data
Conceptual model
Entity-Relationship (SDO)
Service Service Service Service
Service layer
Logical model
(e.g. relational)
Supply chain
application
CRM
application
DBMS OLAP Resource layer
Figure 1: Spreadsheets in Service Oriented Archi-
tecture
The traditional spreadsheet model is based on a grid, also
called a worksheet, where cells are identified by their coor-
dinates, denoted using letters for the columns and numbers
for the rows. For example C4 is the cell located in the third
column of the fourth row. Each cell contains either a single
atomic value or no value at all. This value is obtained ei-
ther from a direct user input (e.g. user may input C4=6) or
from a formula expression (e.g. user may set C4=SIN(C3)+5).
Formulas are expressions that combine i) functions, such as
SIN(x) or ‘+’ in this last example, ii) constant expressions
and iii) variable references denoted as cell coordinates—C3
in this example. A spreadsheet application may contain sev-
eral worksheets, together referred to as the workbook.
Spreadsheets are often tagged as the most successful end-
user development environment [15, 12]. An explanation to
this well deserved reputation can be sought in the cognitive
dimension (CD) framework [6]. CDs are criteria that help
to understand the difficulties faced by developers in using a
system. An example of cognitive dimension is Progressive
evaluation. It measures how far developers need to go into
their programming before being able to check if what they
did is correct. Spreadsheets evaluate continuously and thus,
they do very well along this dimension.
Some tasks, such as adding a chart, can not be done by for-
mula programming. A wizard dialog can in this case be used
to guide users through the sequence of operations needed to
accomplish this task. Indeed, formula themselves can be
built with the help of such a visual assistant.
The spreadsheet programming model is thus a combination
of functional programming, grid manipulations and visual
programming. All these elements concur to deliver an in-
tuitive experience—with little or no learning barriers—and
great pays-off. Extensions to spreadsheet have to preserve
as much as possible the qualities of this programming model.
We refer the reader to [12] for a more complete exposition
of the spreadsheet programming model. In the following,
we will introduce when needed the characteristics of this
model that we believe are important when importing and
manipulating data.
2.2 Service Oriented Architecture
Our framework is the service oriented architecture illus-
trated Figure 1. It consists of three layers.
The resource layer is the domain of data management sys-
tems, e.g. relational databases or data warehouses, as well
as applications, e.g. customer relationship or supply chain
management softwares. At this layer, data are structured
according to a logical model, concerned with issues such as
retrieval performances or storage costs. For example, data
can be organized in relational tables with various degree of
normalization in order to reduce the computation time of
queries. For their part, applications are accessed in this
layer using specific—often heterogeneous—programming in-
terfaces (e.g. C++ for the CRM and java for the supply
chain software).
Programming at this level often implies tedious and repet-
itive manipulations. For example, in order to retrieve, from
a relational database, all the desired information regarding
a customer, programmers may have to join several tables in
rather complex SQL expression.
The service layer helps in providing a higher level expe-
rience to developers. Application interfaces are made ho-
mogeneous thanks to web services and information can be
accessed through data services. For instance, ALDSP from
BEA [8] proposes data access through the Service Data Ob-
ject (SDO) standard [19, 21]. Service data objects—or their
.Net cousin [2]— rely on the Entity-Relationship model and
allow developers to use this model for accessing and updat-
ing data.
This approach relieves developers from low-level manipu-
lations akin to the resource layer. When an entity such as
customer is accessed, all the related information can also be
accessed easily. The customer entity has explicit relation-
ships with other entities such as purchase orders. Developers
can use these relationships to gain access to related infor-
mation.
The third layer in Figure 1 represents the spreadsheet ap-
plication. We have seen in section 2.1 that in the spreadsheet
environment, variables are cells and the only available data
structures are cell matrices.
When going from the bottom layer of resources to the
superior layer of services, developers gained in terms of ab-
straction. By contrast, programming in spreadsheet looks
like a huge step backward. Here lies the main difficulty. A
good integration will allow spreadsheet developers to ben-
efit from the high-level abstractions of data services while
letting them work with the simple structures offered by a
worksheet. In the next section, we discuss what this im-
plies.
3. DATA IMPORTATION AND MANIPULA-
TION
This section presents the dimensions of the problem of
data importation from data services to spreadsheet and their
subsequent manipulations. We then review the existing ap-
proaches to data importation and identify their strengths
and weaknesses.
3.1 Dimensions of the problem
3.1.1 Incremental construction of grid representa-
tion
We have seen in section 2.1 that one of the characteristics
of the spreadsheet programming model is the possibility to
incrementally build applications, starting from simple and
gradually adding more and more formulas.
Importation consists in locating a data resource and build-
ing its representation on the worksheet grid. This should be
accomplished in a way that preserves the incrementality of
spreadsheet programming. Users should be able to modify
any part of a grid representation, adding or removing im-
ported information, and this with a minimum of side-effects
on the rest of the application. And since the modification
unit in a spreadsheet is the cell, building the grid represen-
tation ought to be possible cell by cell. In terms of cognitive
dimensions, this notion is referred to as viscosity.
3.1.2 Traceability
It is important to know where the information displayed
in a spreadsheet comes from and what it means; not only
at the time the application is built, but throughout all its
life-cycle. For example, if a cell displays the value ‘Prefect’,
does it stand for the last name of a customer, a mis-spelling
for ‘Perfect’ or the model name of a car.
Clearly, users should be able at any given time to know
which external information is represented in any given cell.
This information is twofold. First, the cell value has to be
precisely bound with an external resource. It is needed to
perform the retrieval of this value and its subsequent ac-
tualization. This identification is thus at a system level.
Second, the information need to be identified at a user level
with what is called meta-data. Meta-data include for exam-
ple the semantic of the value, its precision or, if its a dis-
tance, in which unit it is expressed (e.g. miles, kilometers or
light-years). Finally, users may need to refer to meta-data
in formula expressions.
3.1.3 Parametric data access
A parametric data access is needed in order to compose
importation operations. For example, a user may first im-
port a list of customers from the Customer Relationship
Management software. Then, additional data regarding the
current processing of customer orders may be retrieved from
the supply chain management software. In this scenario, the
second importation depends on data that come from the first
importation.
We have seen in section 2.1 that in spreadsheet program-
ming, cells references are used as variables. In this example,
it means that the stock name would be stored in a cell, and
that the importation of its quote would use that cell refer-
ence as parameter.
3.1.4 Data access efficiency
Importation may impact both the data service used to de-
livers the information and the spreadsheet application that
retrieves it. To illustrate this problem, consider the portfo-
lio application mentioned above. A table consisting of the
names and quotations of a list of stocks have to be imported
from some relational database. It is more efficient to retrieve
this list with a single selection and projection query rather
than querying each individual cell values separately.
3.1.5 Relationship-based manipulations
As mentioned in section 2.2, data services offer a high
level access to corporate information based on the Entity-
Relationship model. We argued that a good integration of
spreadsheet has to give to users access to this model. But
what does that mean concretely? To illustrate this, consider
a cell that displays, as in Figure 2, the last name of a cus-
 Customer 003
Beeblebrox
A
Spreadsheet application
B C D
details from a given aggregate. But the structure is only ac-
Customer 002
Zaphod
Customer Dent 1 Last name First name cessible through the set of pre-defined manipulations offered
lastName Arthur Prefect Ford
firstName
Customer
Prefect
001
Importation of a
2
3 Dent Arthur by this tool. It is possible to perform a drill-down operation
1
Ford list of customers 4 Beeblebrox Zaphod
by using the pivot table API from the macro language of
Navigation through
Order relationship
Excel but not to build spreadsheet-based computation that
order order for customer 001
refer to the drilled-down data.
*
Template: PODetailsAndAverage
This hinders the interoperability between importation so-
PO 001-040801
PO 05/10/2005
PO 001-051001
Betellgeuse 7
A B C D lutions. The spreadsheet becomes integrated with a col-
poDate 05/10/2005 1 Date Total PO
poShipAddress 07/10/2005
Betellgeuse
70.00
7
2 05/10/2005 70.00
lection of separate systems, but no interaction is possible
poTotal 70.00 Importation of a
list of orders for
3
4
08/08/2004

Average Total PO
95.00

82.50
between them in spreadsheet programming. However, we
a given customer
observe that this situation is not due to a defect in the pivot
table itself. It has to do with the approach that consists in
integrating the spreadsheet directly with the resource layer
Figure 2: Relationship navigation
(see Section 2.2 and Figure 1), where resources are hetero-
geneous. Integrating with the service layer allows to build a
unique solution to access a variety of systems, and closeness
tomer, say ‘Prefect’. Accessing the underlying ER model,
in this case would hopefully become irrelevant.
represented on the left of the figure, means here that we
We also want to emphasize that a relationship navigation
can display additional details about this customer, say his
is different from a parametric importation. The difference
first name and address, or list of his recent purchases. That
is that relationships are pre-built in the conceptual model
is, the value is not seen as isolated, but as an element of a
and users don’t need to express them—they are ready to
larger composite entity, here the customer, in relationship
use. Relationships are precious when the parameters are
with other entities, e.g. purchase orders.
not trivial (for example when obtaining the purchase orders
The Pivot Table, a feature found in MS Excel, implements
of a customer involves to join several tables with composite
such a mechanism and, incidentally, offers an illustration
foreign keys). Put it roughly, relationship navigations are
of the shortcomings that integration solution should avoid.
to parametric data access what SDOs are to SQL.
The pivot table allows to compute an aggregate, e.g. a sum
Now that we have a clearer picture of what to expect from
or an average, of a collection of values grouped along some
an integration solution with SOA, we propose to examine
dimensions. It is the spreadsheet representation of an OLAP
existing approaches to data importation and see how well
cube. It takes the form of a table, i.e. a collection of cells,
they do along these five dimensions.
where horizontal and vertical headers represent the chosen
dimensions and where each non-header cell is an aggregate. 3.2 Review of existing approaches
A right-click on an aggregated value pops up a context menu
We observed that approaches for data importation and
that offers to display the details of this aggregate; that is
manipulation could helpfully be classified into two categories:
the list of individual values that were summed or averaged
formula-based importation and external mapping definition.
to produce this cell content. This is a form of relationship
For each, we picture below its main traits, give some ex-
navigation. The aggregated value is in relation with the
ample of commercial products, and we discuss their relative
individual values used to compute the aggregate. However,
merits according to the criteria identified in the previous
the navigation experience offered by this method poses two
section.
problems: it is fixed and closed.
The navigation is fixed since it is not possible to specify 3.2.1 Formula-based importation
how the details are displayed. In the pivot table case, they
In this model, the grid representation of external data is
are displayed in a new worksheet as a table where each col-
obtained from formula evaluation as illustrated in Figure 3.
umn is a dimension and each row an individual value. But
Examples of this method for MS Excel include the built-in
suppose that you modify this new worksheet to compute
functions for database (e.g. DGET, DSUM, DAVG, etc.) and the
some custom aggregations, say a sum of all values greater
Real Time Data (RTD) provider. The function DGET(x,y,z),
than 100. If you happen to need a similar computation for
when used in a cell formula, retrieves from a database x the
some other aggregate values, you unfortunately will have to
value corresponding to attribute y of a tuple identified by
do the work again, as the details of each aggregate is going to
z. RTD is an extension mechanism that can be used by a
be displayed in its own newly created worksheet.There are
professional developer to provide access to dynamic values.
workarounds. For example, the computation can be pro-
They correspond respectively to a push and a pull model of
grammed in a separate worksheet that refers the the work-
data importation. In addition to those features, a profes-
sheet automatically produced by this tool. But this method
sional developer can easily extend the library of functions
is more complex. What would be needed here is the possi-
available in the formula language by User Defined Func-
bility to customize the navigation.
tions (UDF). Figure 3 illustrates this approach with a UDF
The navigation is also closed since the details of an aggre-
Customer that takes the customer number and an attribute
gate are accessible only from within the pivot table, through
as parameter.
this particular context menu. It is not possible, for exam-
The main advantage that derives from using formula for
ple, to access the details of an aggregate from another tool
data importation is that it is perfectly in line with the
or from a formula expression, nor is it possible to refer to the
spreadsheet programming model and, thus, share its good
origin of a cell value from a formula or another tool. Only
properties:
the resulting aggregate value is accessible, the structure to
which it belongs, a lattice in this case, is known to the pivot • The grid representation can be built incrementally and
table since it allows users to drill-down, roll-up or display the each cell can individually be modified;
 Customer 001
Prefect
Ford Spreadsheet application
A B
Customer 002
Customer 1 Customer(>001, {lastName, firstName})
lastName Dent
Arthur 2 =Customer(001, ‘lastName’)
firstName
3 lastName
4 firstName
Customer 003
Beeblebrox
Zaphod
Figure 3: Formula-based importation
• The traceability is immediate since the formula reflects
the origin of the information—provided its syntax is
not too abstruse. However, only the binding of a value
with an external data is expressed in a formula. Ad-
ditional meta-data support is needed;
• Data access is naturally parametric, and only limited
by the way functions are defined. For instance, import-
ing data with the function Customer(id, attributeN ame)
gives more flexibility than with CustomerLastName(id)
but less than GetData(entityT ype, id, attributeN ame).
Note that the three versions could be provided.
The efficiency of the data access is however more problem-
atic. First, a straightforward implementation of the func-
tions used for data retrieval implies an individual query to
the data resource. This can have a significant impact on the
data provider, the spreadsheet application and even on the
network. It can be mitigated by implementing some cache
mechanism local to the spreadsheet, but i) this implementa-
tion is not trivial and 2) even so, the evaluation of separate
functions for each of the cells will have an impact.
The major concern is thus that formula are not suited
to import collections of data. They can return only values
compatible with the cell content, i.e. a single primitive value
such as a text, a date, or a number. In addition to the
performance impact, this limitation also makes it impossible
to import collections of data with varying—or unknown—
size. This, of course, is not acceptable as most of the data
served by data services are in one or both of these categories.
Finally, although relationship navigation can be imple-
mented, very few approaches actually propose this feature.
For example, Excel add-in for MS Analysis server [3] im-
plements this feature by parsing the formula to identify the
component of the structure to which it refers and offers in
a context menu some navigation choices. As mentioned in
section 3.1 this navigation is closed and fixed.
3.2.2 External mapping definition
An alternative to import data in spreadsheets is to specify
which data to import and where on the spreadsheet to place
them, as shown in Figure 4. We refer to this specification
as the mapping definition. It is programmed by spreadsheet
users through wizards or visual assistant. This definition
doesn’t involve formula, and in that sense, it is external to
the spreadsheet application.
The XML mapping tool [17] that ships with MS Excel is
an example of this approach. It relies on drag-and-drop op-
erations for the mapping definition. Users can select XML
elements from the tree representation of an XML schema
and drop them over the cells where they want the informa-
External Mapping Definition
Customer(001, firstName) C2
Customer 001 Customer(001, lastName) B2
C D Prefect
Ford
B3:C4
Customer Customer 002
Dent
Arthur Spreadsheet application
A B C D
Customer 003 1 Last name First name
Beeblebrox 2 Prefect Ford
Zaphod 3 Dent Arthur
4 Beeblebrox Zaphod
Figure 4: External mapping definition
tion to appear. This results in a mapping between cells and
XML schema elements not much different from that repre-
sented in Figure 4. It is not a complete mapping definition
yet—as the data to import also need to be specified. This
is done in a separate operation in which user actually select
an XML document of that schema.
Another example is given by the pivot table1 already in-
troduced in section 3.1. When used with external data, a
series of wizard dialogs helps users to formulate the which
part of their mapping, i.e. to select a data source and build
a query. Then, users can specify the where part which, in
this case, means choosing a single cell: the upper-left corner
of the pivot table. The number of cells used by the pivot
table will depend on the size of the data that need to be
displayed.
This approach—a mix of wizard dialogs to select the data
and of visual assistant to build the grid representation—has
been adopted by the vast majority of software vendors to
propose the integration of spreadsheet with their applica-
tions; far too many indeed to start citing them. The ad-
vantages of the approach are clear: (i) the data access can
be made very efficient. Since users first build a query to re-
trieve in one shot all the related data, only one query need
to be sent to the data resource. (ii) It is possible to specify
a grid representation of collections of data of unknown or
varying size.
There are drawbacks however; mainly because the map-
ping definition is not programmed in a spreadsheet style,
with formulas.
• Traceability is problematic. The cell content is usually
identified by headers; only the cell location determines
its content, meaning that it can not be moved. Some
approaches resort to using the comment zone of cells
to store information regarding the origin of the data.
Using the comment zone in fact highlights that spread-
sheets don’t provide any facilities to store, display and
use meta-data. This problem is recognized as very sig-
nificant when spreadsheets are used in context such as
Business Intelligence or for reporting [10]. This has
led us to propose in Section 4.3 a systematic support
of meta-data informations.
• A parametric data access supposes that some spread-
sheet cells can be referenced in the external mapping
1
Arguably, the pivot table is not a “pure” importation solu-
tion in the sense it can also transform the imported data to
compute aggregates; but (i) it is an importation tool when
used with an OLAP data source and (ii) being standard in
Excel makes it a good example.
 Formula-based External
Efficiency Problematic Good
Traceability Good Fragile
Incremental Good Pre-commitment
Parametric Good Inconsistency, Hidden dep.
Relationship Closed and fixed Closed and fixed
Table 1: A comparison of importation models
definition. If we take the example of Figure 4, it means
that rather than defining the mapping for customer
001, we would define it for customer A1. This is not
a technical difficulty, but it introduces an important
problem and, probably for that reason, we haven’t re-
viewed any approach allowing it. The problem is that
of hidden dependency; one of the cognitive dimensions
we introduced in section 2.1. It means that if we define
the mapping this way, the cells C2 and B2 of our ex-
ample now become dependent on the value of cell A1.
When this happen in spreadsheet programming, the
dependency between C2 and A1 would be clear from
the fact the formula in C2 refers to A1. Here however,
cell C2 itself doesn’t show this dependency, it is hid-
den unless the appropriate dialog used to define the
external mapping is displayed.
• The grid representation is not anymore built incre-
mentally: all the data are imported at the same time.
More often than not, in order to add an attribute to
an imported table, it is necessary to go through all the
steps of the importation process again. The previous
importation—and possibly all the modifications made
by the user—is simply erased and replaced by a new
one.
Note that to mitigate this last problem, or the fact, for
instance, that the way mappings are defined imposes to im-
port collections of values as contiguous collections of cells,
some products allow to transform an external mapping into
a formula-based importation (e.g. SAP BEx Analyzer[4],
Oracle BI [13] or Microsoft Analysis server [3]). A set of
cells which obtains its values by an external mapping can
be refactored into formula expressions to retrieve cell val-
ues. It becomes then possible to relocate any of these cells
and, thus, to insert an empty row or column in a table with-
out preventing future data refresh. However, this method
can’t be called a formula-based. Data are still queried as
defined during the mapping definition and the formulas refer
to the result of this query. Thus, both traceability—since
the formula doesn’t refer to the external data—and para-
metric access—since the query isn’t in the formula—remain
a problem.
Regarding relationship-based manipulation, the situation
is the same as in formula-based approach. They are possible
and some form of navigation is usually supported by prod-
ucts in this category. But as discussed about the pivot table
in section 3.1, they are closed and fixed.
4. A NOVEL APPROACH: SPREADATOR
A formula-based importation—since it conforms to the
spreadsheet programming model—is more satisfying than
an external mapping definition on the dimensions concerned
with the programming aspects (see Table 1). However, formula-
based importation is not suited for importing collections
External Mapping Definition
SpreadATOR External mapping definition
Customer 001 A B C D
Prefect 1
Ford
2 =Customers[“001”].LastName
3
Customer 002
=\\Customers\002@lastName
Customer 4
lastName Dent
Arthur
firstName Spreadsheet application
A B C D
Customer 003 1
Beeblebrox
2 Prefect
Zaphod
3 Dent
4
Figure 5: Formula-based External mapping defini-
tion
of data or composite data: formulas return only primitive
types. This weakness has a huge practical impact as most of
the data users need to import are indeed collection or com-
posite. As a result, existing systems for data importation
almost solely rely on an external definition of the mapping.
It appears then that in order to satisfy all the criteria
listed in Section 3.1, we need to blend the qualities of both
approaches. This is what we attempt to do with SpreadA-
TOR2 . In a nutshell, SpreadATOR is essentially an exter-
nal mapping approach but it also offers a spreadsheet-like
programming experience based on formulas. Section 4.1
presents the formula language that we use to construct the
external mapping definition.
In Section 3.1.5, we discussed why we believe that both
formula and external mapping approaches are not entirely
satisfying when it comes to relationship-based manipula-
tions. We present in Section 4.2 how those manipulations
are eased in SpreadATOR and what are the resulting bene-
fits for the end-user programmer.
Finally, we saw that conveying meta-data information is
necessary to achieve good traceability. We propose in Sec-
tion 4.3 an innovative method to (i) convey these informa-
tion and, (ii) allow end-users to incorporate them in their
computation.
4.1 Formula-based external mapping
We define SpreadATOR as a middleware for spreadsheet
integration. It adopts an external mapping approach to im-
port data retrieved from data services. The innovation of
SpreadATOR is to make this mapping definition explicit and
to blend it with the rest of the formula-based programming
of the spreadsheet application.
SpreadATOR mapping definition is thus based on formula
expressions that are very similar to spreadsheet formula.
They are stored in cells and can use other cell references.
Figure 5 shows such a mapping definition in cells B2 and B3.
The exact syntax chosen for the language is not important
here. It is merely a matter of preference or implementation.
To emphasize this idea, we used a formula with an object-
oriented syntax in cell B2 and one in the style of XPath in
B3.
Our implementation of SpreadATOR relies on JScript.Net
for formula evaluation. JScript is an implementation of
javascript for the .Net framework. Therefore, the formula
2
SpreadATOR stands for “Spreadsheets and dATA Objects
Reconciled”.
syntax we adopted corresponds to that of cell B2.
Although not strictly speaking an object-oriented language,
javascript has the advantage of being easily interfaced with
pure object libraries. In particular, it is possible to use
any assembly compliant with .Net from JScript expressions.
Thus, our implementation can benefit from ADO.Net [2],
the .Net equivalent of SDO, and can interact with other
data access libraries such as Application Programming In-
terfaces (API). Section 5 presents such a scenario. To avoid
unnecessary confusion, we will adopt for the rest of this pa-
per an object-oriented terminology and, for example, speak
of object instances rather than entities.
Figure 5 shows the mapping definition in one spreadsheet
grid and its evaluation in another. In reality, the spread-
sheet user only sees one grid and can choose to display ei-
ther the formulas or their evaluations as it is already the
case with traditional formulas. Traditional formulas and
SpreadATOR formulas are merged in a single interface, mak-
ing the programming very intuitive to spreadsheet develop-
ers. They are oblivious of the fact the two types of formulas
are maintained by different systems.
Formula expressions in (current implementation of) Spread-
ATOR are essentially javascript statements. However, we
needed to extend slightly the language with few key-words
and syntactic sugars. First, spreadsheet formulas need to
reference cells. Because cell coordinates could collide with
the name of object members or variables, we enclose them
with angle brackets (e.g. =customers[<A1>].lastName im-
ports the last name of the customer whose number corre-
sponds to cell A1). Second, we needed some mechanism to
allow the mapping of collections of values to collections of
cells. This is achieved by using the character * instead of
the identifier of an element of the list. For example, a col-
umn containing the last names of a list of customers is ob-
tained by =Customers[*].lastName3 . Finally, three other
key-words—obj, template, and metadata—take a special
meaning in SpreadATOR formula and are presented in the
following sections.
Statements that return an object reference are also valid.
For example, =Customers[’001’] returns a reference to an
instance of customer and =Customers returns a reference to
the complete list of customers. The reference returned is
managed by SpreadATOR; for Excel, the cell simply con-
tains a string representation of these objects (obtained by
the default transtyping given by toString()). The advan-
tages of storing a reference to a composite entity in a cell
are:
• It is now possible to refer directly to these composite
objects to build their representation on the spread-
sheet. For example, if B2=Customers[’001’], we can
have a formula B3=<B2>.lastName. Thus, it suffices
that the content of B2 changes (e.g. if it is replaced by
a reference to customer 002), for all related formula
to change accordingly. This makes formula shorter,
easier to read and more efficient to compute;
• The content of cell B2 now has a type. It is possible to
display additional information corresponding to that
particular entity type and permit navigation to related
entities through the template mechanism, described in
the next Section.
3
Note that a only one * is allowed per formula
We want to emphasize that, despite the object-like syn-
tax of the formula language used in SpreadATOR, we don’t
assume any familiarity of end-users with object-oriented pro-
gramming. First, users are only accessing pre-built objects
available from data services, they are not actually “creating”
these objects. Second, the use of formula doesn’t exclude
the complementary usage of wizards and visual assistants
to generate the formulas. Traditional spreadsheet formulas
are themselves often built by using a wizard dialog. The vi-
sual assistant we propose in SpreadATOR—called the object
explorer and presented in Section 5—makes the grid repre-
sentation construction an experience very close to that of
using the XML mapping tool in Excel.
4.2 Relation-ship based manipulations
We saw in section Section 3.1.5 that existing approaches
offer limited support for relationship-based manipulation. In
particular, these approaches do not allow spreadsheet pro-
grammers to actually benefit from relationships in their pro-
gramming.
To address this problem, we propose to introduce a tem-
plate mechanism. The idea of template is not new to spread-
sheet and end-users are already familiar with it. The innova-
tion of SpreadATOR is to associate templates with the type
of composite objects—each type may have several templates—
and allow to define a generic grid representation for in-
stances of that type. Templates are given names and are
proposed in a drop-down menu (see Figure 6(a)); its con-
tent depends on the current cell selection.
It is the fact that SpreadATOR formula can return ref-
erences to composite objects that permits to associate tem-
plates with types. To illustrate this mechanism, suppose a
worksheet with a formula A1=Customers[001]; that is, cell
A1 contains, from SpreadATOR point of view, a reference to
the instance of type Customer that represents the customer
001. When A1 is selected, users can open a template asso-
ciated to the type Customer (or create a new template for
that type). An internal object named obj is associated to
the instance referenced in A1. Users can use this reference
to build a representation of this object.
So the difference between a template and a worksheet is
that instead of referencing an external object, such as in
=Customers[001].lastName, a template references an inter-
nal object denoted obj. For example, in a template suited
for objects of type Customer, we can have formula such as
=obj.lastName. The approach used to build the template
is exactly the same as for a worksheet and rely on the same
visual assistant, only the formula generated by the assistant
are different.
The template defines how to represent an object obj. Ac-
cessing a template is equivalent to a relationship navigation
since the template can display any information related to the
instance selected—for example, the list of purchase orders of
the selected customer. It is indeed more powerful since users
are not limited to displaying only the destination entity(ies)
of a relationship.
Furthermore, SpreadATOR allows to access the customized
grid representation of an object type from a worksheet that
contains instances of that type. For example, suppose that
a customer template called “PO details” is used to compute
some custom aggregate—say an average of the PO which to-
tal exceeds 100$—which result is in cell G4 of the template.
From our worksheet example above, where cell A1 contains a
reference to customer 001, we can access the custom aggre-
gate of the template by using the formula =template(A1,’PO
details’,G4).
This formula can easily be duplicated for all the customers
present on a worksheet, simply changing the reference A1.
In object-oriented terminology, it is as if the type Customer
was extended with a new method that computes the custom
aggregate. When this formula is evaluated, obj is associ-
ated with the reference contained in A1. It can be seen as
the SpreadATOR equivalent to keyword this in object ori-
ented programming. However, obj stands for “current cell
composite content”, rather than “current instance of that
class”.
By comparison, computing this custom aggregate on a
list of customers in traditional spreadsheet programming
approach—that is, without resorting to another program-
ming paradigm such as a macro language—is more com-
plex. For example, you could import the list of customers
in a worksheet and either (i) import as a large table all the
purchase orders of all customers in a second worksheet or
(ii) import the list of POs in a separate worksheet for each
customer. In either case, you’ll have to rebuild the join be-
tween the list of customers and that of POs; that is, you’ll
have to search the starting and ending row corresponding
to a customer (case (i)) or you’ll have to search the work-
sheet corresponding to a customer (case (ii)). The formula
language of spreadsheet includes search functions, precisely
for such situations. But why should users have to build this
join when it is readily available in the ER model of data?
4.3 Meta-data management
Meta-data are not very different in nature from other in-
formation related to a given entity. They mainly differ by
their semantic and usage. A meta-data is typically not a
value that users want to lay out on the spreadsheet because
they are not an essential attribute of the information ac-
cessed. They represent a complement of information, kind
of a documentation; they speak about data.
For example, a Business Intelligence (BI) software often
provides access to Key Performance Indicator (KPI) such as
“Order processing delay” which expresses in days the aver-
age time needed to process a customer purchase order. This
value is a high level aggregate and users need to know what
they exactly mean, how they are evaluated, when the value
was computed for the last time, what its precision is, etc.
We need meta-data (i) to always be accessible whenever we
examine a KPI and (ii) not to occupy cells of their own on
the worksheet—unless, of course, the specific application we
build calls for it.
Thus, we propose in SpreadATOR to display meta-data
separately from the worksheet in some reserved space of the
user interface (see the bottom-right area in Figure 6(a)).
We define meta-data as a collection of hname, valuei pairs
obtained from a collection of hname, f ormulai tuples that
depend—in the same way as the template mechanism—on
the type of the composite object contained in the cell. The
former is used for display in a list when a cell containing a
composite object is selected, while the later corresponds to
a collection defined by the user where each formula refers
to the selected object through the keyword obj introduced
in the previous section. For example, the end user can de-
fine a meta-data for type Customer with hLast contacted,
obj.lastContactDatei. When a cell containing a customer
is selected, the evaluation of this formula is displayed in
the bottom-right section of the screen, e.g. hLast contacted,
21/06/2006i.
Though this value is not needed for display, it could be
needed for computation. For example, users may want to
highlight customers that have not been contacted for a while.
Since meta-data are visible at the same time as the work-
sheet, they can be used in drag-and-drop operations. The
keyword metadata is used to refer to their value. For exam-
ple =metadata(A1, ’Last contacted’) returns the meta-
data named Last contacted for the composite object—here of
type Customer —contained in cell A1. Meta-data are hence
at the same time very similar but complementary to tem-
plates. While template can be used to display a large quan-
tity of related information, the purchase orders of a cus-
tomer for instance, meta-data offer a simple and intuitive
mechanism to display complementary information about an
imported entity.
5. CONSUMING REAL-WORLD DATA
To validate our approach, we demonstrate in this section
how RSS feeds can be easily accessed and manipulated in
Excel using SpreadATOR. We show how (i) access to a RSS
feed can be performed using a generic library—that is, a li-
brary which has not been developed specifically for use from
within a spreadsheet—and (ii) that the composite structure
of RSS feeds can be laid out on the worksheet and ma-
nipulated in formulas using pure visual programming, i.e.
through a combination of point-and-click and drag-and-drop
operations.
RSS is a popular XML format for news syndication. RSS
documents are XML documents with several levels of nested
parent/child relationships. A typical RSS document in-
cludes one or more channels where each channel contains
a collection of news items. They are published by news-
oriented web sites or blogs and are widely available on the
Internet.
We use here the library called RSS.Net 4 . This library
exposes a class called RssFeed which is an object-oriented
representation of an RSS document. An instance of RssFeed
object can be built using the static method Read(url ) of this
class.
To be available in SpreadATOR, a library or a service has
first to be referenced. This is done using a dialog where
users have to provide the service URL (from which a library
is produced using .Net utilities), or, as in our example, to
select the compiled file of a library (called an assembly in
.Net terminology).
Once this is done, all the public resources of the library,
including the RssFeed class, can be used in spreadsheet for-
mulas. Figure 6(a) presents the SpreadATOR add-in as it
appears in Excel. In the top right corner, SpreadATOR
offers a zone where users can input formulas (though formu-
las can also be input using the Object Explorer as explained
hereafter). In this Figure, we can see that cell A8 contains
the formula RssFeed.Read(URL). The URL used in this
example is that of the IEEE Transactions on Computers5 .
This formula evaluation returns an object of type RssFeed
whose default transtyping as a string is displayed in the cell
(in this case, it corresponds to the URL of the feed).
4
available at http://www.rssdotnet.com
5
located at: http://csdl.computer.org/rss/tc.xml
 (a) A worksheet accessing several RSS feeds
Figure 6: The SpreadATOR add-in user interface in MS Excel
In this example, three different feeds have been accessed in
three different rows of the spreadsheet. Details of those feeds
are provided in other columns. For row 8, the last refresh
date and the most recent news item are accessed using, for-
mulas <A8>.LastModified and <A8>.Channels[0].Items[0]
respectively. Again, these formulas can be entered by us-
ing the Object Explorer, as described below. Other rows
use similar formulas, with only the cell reference changed.
This illustrates that any level in the nested structure of the
RSS document can be freely laid out on the worksheet. It
also shows that the individual components of the compos-
ite value contained in a cell can be accessed directly within
formula expressions using only cell references, hence avoid-
ing the impedance mismatch problem faced in traditional
spreadsheet programming.
The composite value contained in cell A8 can also be dis-
played in details using a template corresponding to its type
(see section 4.2). Figure 6(b) shows a detailed display of the
RssFeed instance of cell A8. In this template, all formulas
use the keyword obj to refer to the current instance to be
displayed. For example, the formula used in cell C6 Figure
6(b) is obj.Channels[0]. This formula returns an object of
type RssChannel, whose default string representation is the
title of the channel.
The Object Explorer can be seen on the right of Figure
6(b). In this mode, it displays the details of the type con-
tained in the selected cell which, in our case, corresponds to
the type RssChannel since cell C6 is selected. The Object
Explorer can be used to build a layout by simply selecting
one of the properties of the object contained in the selected
cell, and dragging the corresponding node over to an empty
cell. This results in the formula corresponding to that node
being copied into the cell. For instance, one can see that the
node description is selected, and the corresponding formula
is displayed in the bottom part of the panel.
The detailed layout of Figure 6(b) can be reused for any
instance of RssFeed class. Hence, similar details can be ob-
tained for any of the three RssFeed objects on Figure 6(a)
through a simple click. Moreover, it is possible to create
several templates, corresponding to as many views of com-
plex data of a given type. Templates can also be nested. For
instance, it is possible to create a template for the type Rss-
(b) Template view of one RSS feed
Channel and access it from any cell containing a composite
value of that type (e.g. cell C6 Figure 6(b)).
6. RELATED WORK
We already reviewed in Section 3.2 the existing approaches
to data importation in spreadsheet and our discussion was
focused on mainstream spreadsheets. But spreadsheets have
received from the research community a sustained attention.
Several proposals have been made to extend spreadsheets in
order to introduce features found in conventional program-
ming languages and make these features easy to exploit for
end-users.
An early work in that area is the Analytic Spreadsheet
Package (ASP) [16] where the language Smalltalk 80 is used
to build a spreadsheet where cells can contain instances of
objects. Object visualization within cells is provided either
by the default transtyping mechanism offered by smalltalk
with the printString protocol or by instanciating objects
that derive from DisplayObject to build custom visualiza-
tion.
A more advanced integration of object-oriented features,
as well as functional programming, into the spreadsheet en-
vironment was proposed in [9]. This work extends the tra-
ditional spreadsheet to support programming abstractions
such as encapsulation, reuse, recursive functions, higher or-
der functions or polymorphisms. It defines a full spreadsheet-
based language where worksheets are seen as methods and,
when grouped in a workbook collection, collectively define
a class.
In [12], an extension to Excel is proposed to allow end-
users to build custom functions. In this approach, the type
system of the spreadsheet is extended so that whole matrices
can be stored in a single cell. A cell that contains a matrix
is displayed in a different way so that end-users clearly know
that their content is composite. Forms/3 [7] is a prototype
that implements several extensions to spreadsheet program-
ming. It allows for instance recursive computations, or ex-
ception handling. Cells in Forms/3 can contain any type of
data.
All these approaches are very interesting; they explore
how to redefine the spreadsheet programming model in order
to bring into it the powerful abstractions found in other
languages. The integration of spreadsheets in SOA would
be much facilitated if mainstream spreadsheet applications
(and their user-base) decided to adopt some of the ideas
proposed in those works. For example, if Excel actually
handled matrix types as cell values, we could benefit from
this type system and propose a richer mapping of composite
external objects.
Our concern in this article is almost opposite since we
precisely try to leave the spreadsheet programming model
untouched. SpreadATOR acts as a middleware; its formula
language could as-well be hidden to users who could choose
to rely solely on the visual assistant (the object explorer).
SpreadATOR for instance do not provide any mechanism to
actually build those object abstractions. Thanks to this, we
were able to implement our prototype as an add-in to an
existing spreadsheet application. We try to bring to end-
users this small part of the benefits of programming at the
conceptual level that we think does not imply any major
change in the way they already work with spreadsheets.
7. CONCLUSION
In this article, we have defined the problem of spread-
sheet integration with data services from the viewpoint of
spreadsheet users. We tried to answer how developers could
benefit from the higher level and integrated view of IT re-
sources offered by service-oriented architecture. More specif-
ically, we discussed how to leverage, in the importation and
manipulation of data, the conceptual modeling of informa-
tion as provided by data services and API such as SDO or
ADO.Net. We identified the shortcomings of existing solu-
tions and proposed a novel approach to spreadsheet integra-
tion called SpreadATOR.
An important aspect of SpreadATOR is that it can be in-
tegrated with existing spreadsheet applications such as MS
Excel. It does not suppose an extension of the spreadsheet
language and can act as a middleware. In the same time, its
interface blends with MS Excel. This allows users who have
the need to easily introduce programmatic aspects in their
importation, for example using a cell reference to make it
parametric. An additional benefit is an improved readabil-
ity of the mapping. Finally, the object explorer offers the
necessary support to avoid formula input and is very similar
to the schema mapping tool already proposed in Excel.
The superiority of specialized importation tools over a
generic approach followed by SpreadATOR is their capacity
to provide very specific wizard dialogs or visual metaphors
to assist users (e.g. they can refer to dimensions when ac-
cessing an OLAP server and to tables with accessing a re-
lational database). However, we believe that SpreadATOR
is in fact compatible with these high-level features. We ar-
gue that these specific assistants should output a mapping
definition in a common formula-based importation language
such as the one introduced in this article. It can easily be
done since, as demonstrated, SpreadATOR is able to work
with any (.Net) API. For end-users, the benefit is a spread-
sheet application over which they have a complete control
as well as the possibility to combine various importation
tools in a same application. By using a common mapping
definition, importation systems would also leverage the com-
mon facilities offered by SpreadATOR such as the template
mechanism or the meta-data management and save signifi-
cant development time.
8. REFERENCES
[1] Excel Services Overview. Technical report, Microsoft
Corp., 2006.
[2] ADO.Net Tech Preview Entity Data Model. Technical
report, Microsoft Corp., June 2006.
[3] Designing Reports with the Microsoft Excel Add-in for
SQL Server analysis services. Microsoft Corp., 2004.
[4] SAP NetWeaver: A Complete Platform for
Large-Scale Business Intelligence. Technical report,
Winter Corp., 2005.
[5] G. Alonso et al. Web Services - Concepts,
Architectures and Application. Springer-Verlag, 2004.
[6] A. Blackwell and T. Green. HCI Models, Theories,
and Frameworks: Toward an Interdisciplinary Science.
J.M. Carroll Editor, chapter Notational systems – the
cognitive dimensions of notations framework. Morgan
Kaufmann, 2003.
[7] M. Burnett et al. Forms/3: A first-order visual
language to explore the boundaries of the spreadsheet
paradigm. Journal of Functional Programming,
11(2):155–206, 2001.
[8] M. Carey. Data delivery in a service-oriented world:
the BEA AquaLogic data services platform. In
SIGMOD’06, pages 695–705, New York, USA, 2006.
[9] C. Clack and L. Braine. Object-oriented functional
spreadsheets. In GlaFP’97, september 1997.
[10] K. Gile. Keeping IT sane in a crazy BI world of Excel.
Technical Report 36353, Forrester, 2005.
[11] A. Halevy et al. Enterprise information integration:
successes, challenges and controversies. In
SIGMOD’05, pages 778–787, New York, USA, 2005.
[12] S. P. Jones, A. Blackwell, and M. Burnett. A
user-centred approach to functions in excel. In ICFP
’03, pages 165–176, New York, NY, USA, 2003.
[13] K. Laker. Exploiting the power of oracle using
microsoft excel. Technical report, Oracle Corp., 2004.
[14] E. Lippert and E. Carter. .Net programming for office:
using C# with Excel, Word, Outlook and Infopath.
Addison Wesley, 2005.
[15] B. A. Nardi and J. R. Miller. The spreadsheet
interface: A basis for end user programming. In
INTERACT’90, pages 977–983. North-Holland, 1990.
[16] K. W. Piersol. Object-oriented spreadsheets: the
analytic spreadsheet package. In OOPLSA’86, pages
385–390, New York, USA, 1986. ACM Press.
[17] F. Rice. Creating XML mappings in excel 2003.
Technical report, Microsoft Corp., 2005.
[18] C. Scaffidi, M. Shaw, and B. Myers. Estimating the
numbers of end users and end user programmers. In
VL/HCC’05, pages 207–214, 2005.
[19] Next-generation data programming: Service data
objects. Technical report, IBM, BEA, 2003.
[20] http://www.service-architecture.com.
[21] K. Williams and B. Daniel. An introduction to service
data objects. Java Developer’s Journal, October 2004.