Front. Comput. Sci.

DOI 10.1007/s11704-014-4025-6

Efficient query processing framework for big data warehouse:

an almost join-free approach

1,2,3
Huiju WANG , Xiongpai QIN1,2, Xuan ZHOU1, Furong LI1,2 , Zuoyan QIN1,
Qing ZHU1,2 , Shan WANG1,2

1 DEKE Lab (Renmin University of China), Beijing 100872, China

2 School of Information, Renmin University of China, Beijing 100872, China
3 School of Computing, National University of Singapore, Singapore 117417, Singapore


c Higher Education Press and Springer-Verlag Berlin Heidelberg 2014

Abstract The rapidly increasing scale of data warehouses
is challenging today’s data analytical technologies. A con-
ventional data analytical platform processes data warehouse
queries using a star schema — it normalizes the data into
a fact table and a number of dimension tables, and during
query processing it selectively joins the tables according to
users’ demands. This model is space economical. However,
it faces two problems when applied to big data. First, join is
an expensive operation, which prohibits a parallel database
or a MapReduce-based system from achieving efficiency and
scalability simultaneously. Second, join operations have to be
executed repeatedly, while numerous join results can actually
be reused by different queries.
In this paper, we propose a new query processing frame-
work for data warehouses. It pushes the join operations par-
tially to the pre-processing phase and partially to the post-
processing phase, so that data warehouse queries can be
transformed into massive parallelized filter-aggregation oper-
ations on the fact table. In contrast to the conventional query
processing models, our approach is efficient, scalable and sta-
ble despite of the large number of tables involved in the join.
It is especially suitable for a large-scale parallel data ware-
house. Our empirical evaluation on Hadoop shows that our
framework exhibits linear scalability and outperforms some
existing approaches by an order of magnitude.
Received January 20, 2014; accepted August 20, 2014
E-mail: wanghj@comp.nus.edu.sg
Keywords data warehouse, large scale, TAMP, join-free,
multi-version schema
1 Introduction
A data warehouse is usually organized in one or multiple
star (snowflake) schemas [1]. In a typical star (snowflake)
schema, a large central fact table is linked to multiple dimen-
sion tables through primary-foreign key relationships. Pro-
cessing an OLAP query over a star (snowflake) schema is
expensive. It needs to first perform a multi-table join to con-
nect the fact table to the dimension tables, and then imposes
restrictions on the dimension tables to select the desired data
from the fact table. The selected data are finally grouped and
aggregated according to the user’s demands. The major bot-
tleneck in star query processing is the execution of multi-
table joins, which are also known as star join.
The data in today’s enterprise information systems con-
tinue to explode. Large computer clusters, especially the ones
composed of low-cost machines, have become a popular plat-
form for processing big data. To architect a data warehouse
over a computer cluster, we can choose from the following
architectures, each of which has its own strengths:
• Parallel DBMS (PDBMS) A parallel database is usu-
ally deployed on hundreds of high-end servers, where queries
take no more than a few hours to finish. Failures are rela-
tively rare in such an environment. Once a failure happens, a
parallel database can simply re-execute the query. When de-
2 Front. Comput. Sci.
ployed on a large cluster consisting of thousands of low-cost
machines, PDBMS encounters a scalability wall. As the ma-
chines are less reliable, system failures become more com-
mon and PDBMS tends to be less tolerable than MapReduce.
• MapReduce Most big data analysis systems employ the
MapReduce [2] framework to achieve scalability and fault
tolerance. A typical example is Hadoop [3]. MapReduce par-
titions a data processing task into a large number of inde-
pendent subtasks, such that a failure of a single subtask does
not affect the others. Usually, these subtasks are very sim-
ple, such as filtering and aggregation. While MapReduce-
based system is superior to PDBMS in scalability and fault
tolerance, it is inferior to PDBMS in efficiency [4]. In partic-
ular, MapReduce-based systems are very inefficient in pro-
cessing join operations, let alone star (snowflake) joins that
involve a significant number of tables. Although a number of
approaches have been proposed to improve the efficiency of
join operations [4–9], most of them have to incur some ex-
tra cost such as network transmission cost [4–6], I/O cost [9],
storage cost [8, 10] etc., or assume strong hardware config-
urations [4]. These approaches did not solve the efficiency
problem fundamentally.
• Hybrid systems of MapReduce and PDBMS In recent
years, researchers have proposed a number of solutions to
combine the advantages of MapReduce and those of PDBMS.
The examples include Hive1) and Pig Latin [7], Teradata [11],
Vertica2) , Greenplum3) , Aster Data4) , etc. Some approaches
try to import DBMS interface into MapReduce platforms, or
import MapReduce interfaces in DBMSs. However, as these
approaches do not touch the internal execution engines, they
do not take the full advantages of either PDBMS or MapRe-
duce.
Some pioneering work, such as HadoopDB [12] (whose
commercialized version is called Hadapt5) ), attempts to
tightly integrate PDBMS and MapReduce. Some improve-
ments have been made on both scalability and efficiency.
However, the results are still not satisfactory for data ware-
house applications. Especially when a join operation involves
multiple join attributes, such as a star join, HadoopDB would
lose its performance advantage. If we replicate each dimen-
sion table to all the database nodes, it will incur high space
cost. If we adopt a partition method, we can only partition a
dimension table and the fact table based on one join key. In
this case, the access on other dimension tables will be very
1) Apache hive. http://hive.apache.org/
2) Vertica. http://www.vertica.com/the-analytics-platform/native-bi-etl-and-hadoop-mapreduce-integration/
3)
Greenplum. http://www.greenplum.com/technology/mapreduce/
4) Aster. http://www.asterdata.com/product/mapreduce.php
5) Hadapt Inc. http://www.hadapt.com
expensive.
Most of the existing data warehouse approaches do not
perform satisfactorily when confronted with big data. This
is mainly due to the common processing model they adopt
— it first extracts, transforms and loads data into a fact ta-
ble and a number of dimension tables first, and joins them
together again during the query processing. This model faces
two problems: 1) Join is an expensive and complex opera-
tion, which prohibits a parallel database or a MapReduce-
based system from achieving efficiency and scalability simul-
taneously; 2) Join operations have to be executed repeatedly,
though numerous join results can actually be reused by dif-
ferent data warehouse queries.
In this paper, we proposed a novel query processing frame-
work for big data warehouses, which aims to achieve high
scalability and high performance simultaneously. The frame-
work, in principle, can be applied to any data analytical plat-
forms mentioned above (as detailed in Section 3). Under
this framework, a data warehouse query can be easily par-
titioned into massive independent sub-tasks and handled by
a large scale computer cluster. Our framework consists of a
new storage model and a query processing model customized
for data warehouse queries. The storage model employs hi-
erarchy encoding to compress the hierarchical information
of the dimension tables into the fact table, to minimize the
space consumption. Our query processing model is called
TAMP, standing for transform, aggregation, merge, and post-
processing, which are the crucial steps for processing data
warehouse queries. TAMP translates a data warehouse query
into a unified plan which scans, filters and aggregates the fact
table in a pipeline, and then merges all aggregated data re-
turned from each data node and performs a residual join to
complement the final aggregation results with additional in-
formation from the dimension tables. To handle the dimen-
sion update problem, we design a multi-version schema up-
date protocol. We also propose a scan-index algorithm to en-
sure the efficiency of our approach.
We implemented our query processing framework and de-
ployed it on Hadoop. Extensive experiments were conducted
to evaluate the scalability and performance. Our experiment
results show that our approach can scale almost linearly with
the size of the data warehouse. The Hadoop platform adopt-
ing our framework can outperform Hadoop and HadoopDB
significantly in star query processing.
 Huiju WANG et al. Efficient query processing framework for big data warehouse: an almost join-free approach
The rest of the paper is organized as follows. We first
compare our approach with some related work in Section 2.
Section 3 introduces the storage model we designed for star
schema. Section 4 presents our query processing framework.
Section 5 discuss our dimension update protocol and opti-
mization issues. In Section 6, we provide the experiment re-
sults. Finally, Section 7 concludes the paper and discusses the
future research directions.
2 Related work
• Processing framework Our framework shares some idea
with MapReduce, such as scan-based processing model, new
abstraction of processing, etc. However our framework is a
domain-specific abstraction for data warehouse queries pro-
cessing, which answers all queries in one scan job, despite
of the number of tables involved in the query. In contrast,
MapReduce is a universal framework and usually has to
launch multiple jobs to process a data warehouse query.
Our processing framework is dramatically different from
conventional RDBMSs. Typically, RDBMSs translate differ-
ent queries into different plans, and find one optimal plan to
execute the query; while our approach uses one unified plan
to answer all queries. Different processing schema is applica-
ble to different scenarios. RDBMSs may benefit from sophis-
ticated optimization techniques to achieve high performance.
However, it is difficult for them to handle data with very large
size because of scalability issue (see discussions in introduc-
tion section); and if the join involves a little more relations
i.e., eight relations, the optimization search space is too large
to be affordable [13]. However, our approach depends on scan
to process queries which is more scalable and stable despite
of the large number of tables involved in the join. Many tradi-
tional optimization techniques, especially index, can also be
applied in our processing framework to improve the perfor-
mance further.
• Pre-join Pre-join has been utilized to speed up database
query processing. In [8, 14], the universal relation is used to
put all information into the fact table, so that queries can be
processed by sequential scan. However, this approach incurs
high storage space cost. Different from the universal relation,
our approach compresses only the key hierarchy information
of the dimension tables into the surrogate keys of the fact ta-
ble, and pushes the residual joins to the post-processing of
the aggregation results. This is supposed to achieve a good
tradeoff between storage cost and join cost. Join-index [15] is
another form of pre-join, which materializes the addresses of
3
the join keys in the foreign key table. If we apply join-index
to star query processing, the dimension tables have to be ac-
cessed repeatedly to obtain the join results, which will intro-
duce expensive random accesses. In contrast, our approach
materializes the values of the join keys and its parent hierar-
chies in the fact tables, so that a single scan is sufficient to
generate the join results.
• Hierarchy encoding Optimizing star join queries us-
ing hierarchy encoding was first introduced in [16, 17]. In
[16, 17], the surrogate keys based on the dimension hierar-
chies are used to link dimension tables and fact tables. In
their approach, a fact table is hierarchically clustered and star
joins can be transformed to multidimensional range queries
using UB-Tree [18]. In [19], the authors proposed a CSB star
schema based on composite surrogate key, which is similar
to our dimension surrogate key. Compared to the above ap-
proaches, our hierarchy encoding method is more space ef-
ficient. As we encode hierarchy members based on the local
domains rather than the global domains, our method needs
less bits to represent a hierarchy member.
3 Join-free storage format
In a nutshell, we compress the important information from
the dimension tables into a set of surrogate keys, and put them
into the fact table, so that the queries on the dimension tables
can mostly be answered by accessing the surrogate keys. The
compression is achieved by hierarchy encoding [16, 17]. In
this section, we will discuss how to use hierarchy encoding
to design efficient storage strategy for processing data ware-
house queries.
3.1 Dimension storage
• Local Hierarchy Domain Both the star schema and the
snowflake schema are used to describe multidimensional
data. In general cases, the values of each dimension in a
schema can be organized into a hierarchy tree (For a value
domain without hierarchical structure, it can be regarded as a
special hierarchy consists of a single level), such as the City
dimension in Fig. 1. Let L be the hierarchy tree of a dimen-
sion. For each node in L, we call its children a local hierarchy
domain (LHD). As shown in Fig. 1, China, Japan, etc. share
the same parent Asia. Then China, Japan, . . ., is a LHD of the
Region level. The highest level of a hierarchy tree is also a
LHD, denoted by All.
• Local hierarchy surrogate For each local hierarchy do-
main, its values can always be ordered. Let LHD x be a local
4 Front. Comput. Sci.

dimension tables in the server’s memory and finish the trans-

formation efficiently. For some big hierarchies which cannot
be held in memory, we can store them on disk and build in-
dexes on each field to accelerate the lookup operation on it.
Table 1 Encoding of the dimension tables for city
Dimension region Dimension nation
dsk Region dsk Nation
01 Asia 01.01 China
10 Europe 01.10 Japan
... ... ... ...
Fig. 1 An example of hierarchy encoding

hierarchy domain with C values, and let member(LHD x ) be
the possible values of LHD x . We can define a one-to-one
function S : member(LHDx) → [0, C −1], such that, for every
u, u ∈ member(LHD x ), u < u implies S (u) < S (u ). S (u) is
called the local hierarchy surrogate of u, which is denoted by
hierarghy surrogate key(hsk). (Note that if < is not defined
for the domain of LHD x , S is simply defined as an arbitrary
one-to-one function from the value domain to [0, C −1].) Fig-
ure 1 shows some examples of hsk. We can see that the num-
ber of bits that are reserved to represent the hsks of the level L
is [log2 Cmax ], in which Cmax represents the size of the LHD
with the maximum candidates in L. For example, in the City
level, Beijing, Hongkong and Shanghai share the common
parent China, which has the maximum number of children.
Thus we use [log2 3] = 2 bits to encode the City level.
• Global hierarchy surrogate The global hierarchy sur-
rogate of a node v, denoted by dsk(v), is the concatenation
of all the hsks on the pre-ordered transversal path of v. In
other words, each dsk contains all the hierarchical informa-
tion of the corresponding dimension. As shown in Fig. 1, the
dsk of Osaka is the concatenation of the hsks of Asia, Japan
and Osaka, which is 01.10.01. We denote the global hierar-
chy surrogate of the lowest level of a dimension as dimension
surrogate key(dsk).
In some special cases, a dimension may correspond to mul-
tiple hierarchies. In such a case, we treat each hierarchy as a
separate dimension and encode them separately.
Using the hierarchy encoding, we encode each hierarchy
to form a new dimension table. Therefore, a dimension ta-
ble consists of: 1) global hierarchy surrogate key dsk, and 2)
other attributes of the original dimension table. For example,
the dimension tables of the hierarchy in Fig. 1 are shown in
Table 1.
• Physical storage In most cases, as a modern server is
usually equipped with a large main memory (normally from
hundreds of GBs to several TBs), we can actually keep the
Dimension city
dsk City dsk City
01.01.01 Beijing 01.10.01 Osaka
01.01.10 Henan 01.10.10 Tokyo
... ... ... ...
3.2 Fact table storage
• Compress dimension surrogate key into fact table To inte-
grate the dimension information into the fact table, we add a
surrogate attribute named mdsk to the fact table. We first re-
place each foreign key of the fact table with its corresponding
dsk in the dimension table. Then we combine these dsks into
a single surrogate key by interleaving their hsks. We call the
result as mdsk(multi dimensional surrogate key) of the fact
table. After obtaining the mdsk, we may drop former foreign
keys from the fact table to save space.
For instance, let dski be the hierarchy surrogate of the
ith dimension of a fact table tuple. Let hski.1 , hski.2 , . . . ,
hski.n be the elementary surrogates of dski . Then the mdsk of
the tuple is hsk1.1 . . . hskn.1 .hsk1.2 . . . hskn.2 . . . hsk1.d . . . hskn.d ,
which interleaves all the dsks. Suppose we have two dimen-
sions, that is, the date dimension (Year–Month–Day) and the
City dimension as shown in Fig. 1. The mdsk will be like
hskYear .hskRegion .hskMonth . hskNation .hskDay .hskCity . If the date
dimension has only two levels, i.e., Year–Month, the mdsk
will be hskYear .hskRegion .hskMonth . hskNation .hskCity .
By combining all the dsks into a single multidimensional
surrogate key, we can transform all the operations on the di-
mension tables to bit string operations on the mdsk, which
can be executed highly efficiently [14]. This interleaving en-
coding method ensures that if we pre-sorted data on mdsk,
the higher the hierarchy, the better the clustering effect. As
most data warehouse-style queries operate on the relatively
high hierarchical levels, they can benefit from the pre-sort
significantly. Utilizing pre-sorting is our future work.
• Key-value pair storage model For analytical workload,
 Huiju WANG et al. Efficient query processing framework for big data warehouse: an almost join-free approach
column-store is commonly regarded as s superior solution to
row-store. Thus we adopt a key, value storage model to dis-
tribute the fact table on the HDFS, where each key attribute
stores a mdsk and each value attribute stores a single attribute
value of the fact table. In this model, the key-value pairs shar-
ing the same key constitute a tuple of the fact table, and the
key-value pairs corresponding to the same attribute constitute
a column of the fact table. Such a key, value model shares
the advantages of the column-store, as it allows us to avoid
redundant data accesses. The advantage of our key, value
model over the column-store is that it does not need to per-
form the tuple reconstruction during the query processing, as
the keys already contain all the needed information for query
processing.
4 TAMP execution model
In this section, we introduce the design of our framework. We
discuss the key design idea first, and then introduce our data
warehouse query processing model TAMP.
4.1 Key idea
The dependence of join operation makes current query pro-
cessing frameworks difficult to get performance and scala-
bility at the same time. Although universal relation [20] can
avoid join, it will consume too much storage space, thus it
is not applicable. We aim to avoid expensive join operations
without high storage cost. To achieve this, we encode the di-
mension tables’ hierarchical information (not all information)
into the fact table at the data loading phase(as discussed in
last section), so that the fact table can be processed without
referring to the dimension tables. In this way, we can get the
best trade-off: eliminate the expensive join operation with-
out the space overhead of universal relation. When a query
arrives, we first transform the operations on the dimension
Fig. 2 TAMP execution model
5
tables into the operations on hierarchical information in the
fact table. Then we conduct selection and aggregation on the
fact table. Finally, we decode the aggregation results by join-
ing them with the dimension tables.
4.2 TAMP execution model
The high level architecture of our framework is depicted in
Fig. 2. In a typical data warehouse, the dimension tables only
occupy a small proportion of the storage space compared to
the fact table. Therefore, we store the dimension tables on the
master node and distribute the fact table over the cluster.
4.2.1 Transform
When a query is issued, the transform module interprets the
SQL query, extracts the operations (predicates, group-by, etc)
on the dimension tables and translates them into the opera-
tions on the fact table. The predicates on the fact table, on the
other hand, remain unchanged. After the transform phase, a
new query for processing the fact table is produced.
We restrict ourselves to those queries whose selection
predicate can be expressed as a conjunction of predicates of
the form attrib op const, where attrib is an attribute of the
relation, op is a comparison operator, and const is a con-
stant value. Queries containing disjunctions can always be
expressed as a union of such queries, so this is not a limita-
tion. The transformation rules are as follows:
• Predicates on Hierarchy
Equation conjunctive predicates transformation. Predi-
cates in an equation conjunctive are all equation predicates.
To transform this type of predicates, we produce two bit
strings. One is a mask, extr_mask_equal, to exact the needed
hierarchies by a bitwise AND operation on the mdsk. An-
other one is the expected result which contains correspond-
ing hierarchies’ dsk at appropriate offset, we denote this
bit string as mdsk_e. One conjunctive produces a pair of
6 Front. Comput. Sci.
extr_mask_equal and mdsk_e. Then, on each tuple t, we test
this: extr_mask_equal & t = mdsk_e, which only need one
single comparison to evaluate all predicates.
Range conjunctive predicates transformation. Range con-
junctive predicates transformation is more complex because
there may be both range predicates and equation predicates
in the conjunctive. For this type of transformation, we pro-
duce a new conjunctive predicate containing two predicates.
One is equation predicate which contains all equation predi-
cates in this conjunctive, we get this equation predicate in the
same way as equation conjunctive transformation described
above. Another one is range predicate which evaluates all
range predicates in this conjunctive. We transform all range
predicates in a conjunctive into two bit strings which rep-
resent the expected bound if all predicates pass: one rep-
resents the lower bound (denotes mdsk_l), and another one
upper bound (denotes mdsk_u). The mdsk_l is combined by
dsk of all hierarchies’ lower bound in the conjunctive. If a
hierarchy has no lower bound, its corresponding bits in the
mdsk are left to be false (represent the minimal value). In
the same way, the upper bound mdsk_u are combined by dsk
key of all hierarchies’ upper bound in the conjunctive. If a
hierarchy has no upper bound, its corresponding bits in the
mdsk are left to be true (represent the maximum value). Bits
that does not correspond to a range predicates are set false
in both mdsk_l and mdsk_u. Two masks are also produced:
1) extr_mask_rng, which is to extract the appropriate sur-
rogates for range predicate from the mdsk; 2) group_mask,
which is to extract the code of the hierarchies which appear
in the group-by clause for aggregation. When testing a tu-
ple t, we execute such a conjunctive: extr_mask_equal &
t = mdsk_e and (extr_mask_rng & t) between mdsk_l and
mdsk_u. If a range predicate is the form of attrib < const (or
attrib > const), we translate it into a form of attrib  const
(or attrib  const ) where const is the biggest sibling that is
less than it (or the smallest sibling that is greater than it) in
the hierarchy tree. For example, year < 1995 can be trans-
lated into year  1994.
Like and IN conjunctive predicates transformation. We
handle these types of transformation by transforming pred-
icate into an IN predicate. The value list is a set of mdsk keys
which are produced by evaluating the predicates on dimen-
sion tables.
• Predicates on other fields
There are two ways to transform this type of predicates.
One way is to transform the predicates into an IN predicate,
just discussed above. For some hot fields, we can use another
way: encode these fields into dsk and mdsk key which the
predicates on these fields can be evaluated on the fact table in
batch, just like predicates on hierarchies.
• Removal of Join clauses from SQL query
In our design, join operation is replaced with table scan,
thus join operation is unnecessary and can be removed safely.
To help readers to understand the transformation rules, we
give a simple example. Before transformation, most opera-
tions perform on dimension tables:
SELECT d.year, c.nation, sum(revenue)
FROM fact, d_date, d_customer
WHERE d.year  1994 AND d.year  1996
AND c.region= Asia
GROUP BY d.year, c_nation
After transformation, all operations are translated into new
ones on the fact table(the binary code are new operations, see
Section 3 for details):
SELECT mdsk &  1111001100000 ,
sum(revenue)
FROM fact
WHERE mdsk &  0011000000000 =

0001000000000 AND mdsk &

1100000000000 BETWEEN

0100000000000 AND

1100000000000
GROUP BY mdsk &  1111001100000
4.2.2 Aggregate
In this phase, each data node executes the following opera-
tions in a pipeline: (1) scan the corresponding columns of the
fact table; (2) aggregate the data that satisfies the predicates;
(3) sort the results.
4.2.3 Merge
After the aggregation phase, a merger starts to merge the re-
sults produced by each data node. As the aggregation results
are usually small in size, the network transmission cost is lim-
ited and the merge operation can be finished quickly.
4.2.4 Post-process
As the results returned by the slave nodes consist of encoded
dimensional information, residuary joins are needed to turn
the codes into the values of the dimension tables. Following
that, we execute the having-clause and sort the final results if
necessary.
As we can see, after transformation, each data node fil-
ters and aggregates its own data without the need of moving
 Huiju WANG et al. Efficient query processing framework for big data warehouse: an almost join-free approach
dimension tables or fact table across remote nodes. The net-
work transmission and the memory overhead is minimized.
By using hierarchy encoding (see Section 3), the storage
overhead is far lower than universal relation. Besides, given a
set of queries over the same star schema, existing query mod-
els will conduct one star join for each query. However, for
our schema, each query may be processed on the same Join-
Free schema without join operation, which means such set of
queries may reuse the join results. The pre-processing cost is
also amortized by such set of queries.
4.3 Application scenarios
In this section, we show how the TAMP model can be applied
to a class of data-analytical platforms.
• RDBMS The TAMP model performs query process-
ing in a common execution plan. Thus it can be natually ap-
plied to traditional RDBMS. Especially for PDBMS, it can
improve its scalability greatly. Using this framework, we can
divide a query into massive independent sub-tasks. Each sub-
task can be accomplished through a scan operation. This al-
lows a PDBMS to achieve the same level of fault tolerance as
MapReduce — when a sub-task fails, we only need to re-
execute that particular sub-task and avoid re-executing the
whole query. A custom relational database can also benefit
from our model, especially when processing multi-way star
join.
• MapReduce-based systems As the TAMP model de-
pends on scan operations to answer queries, it fits well with
the filter-aggregation processing model of MapReduce. Thus,
it is easy to integrate it into the MapReduce framework. In
experiment section, we will provide more details.
• HadoopDB In our framework, the fact table is inde-
pendent of the dimension tables. Therefore, we can store the
dimension tables on the master node and partition the fact ta-
ble over the local databases on the HadoopDB nodes. Then
we can push all the star queries into database layer and ob-
tain better performance. This deployment can also be applied
to database cluster with a middleware.
5 Other optimization issues
In this section, we focus on the most crucial parts of our
framework, the tuple reconstruction and dimension update
problems.
5.1 Scan-index
The logical data units in OLAP query are tuples. Thus, we
7
usually need to perform tuple reconstruction when conduct-
ing queries over our storage format. To speedup the process,
our tuple reconstruction algorithm utilizes the following prin-
ciple: as the selective conditions in a query are normally com-
posed of conjunctions of predicates, if one field of a tuple
failed the predicate evaluation, the complete tuple will fail
the evaluation. Therefore, if some tuples have been elimi-
nated during the scan of one column, they can be skipped
when we scan the successive columns. Applying this princi-
ple, our system scans the columns in the descending order of
the selectivity of their corresponding predicates. During the
scan, it uses a scan-index to record the offsets of the tuples
that have passed the predicate evaluation. We implement the
scan-index as a bitmap, in which the ith bit indicates whether
the ith value in the preceding columns satisfies the predicates.
By using the bitmap, we can skip the records which have been
eliminated from the scan-index.
5.2 Multi-version schema update protocol
Update on dimension is not rare. For example, a company
often adds a new product into their product list. And once it
changes, we need to update the existing data’s mdsks. The
most straightforward implementation is to update dimension
surrogate codes first, then update the fact table. However,
this implementation may incur expensive data synchroniza-
tion cost.
To handle this problem, we designed a update protocol
(called multi-version schema update protocol). We annotated
the star schema with version information. Whenever the di-
mension table is updated, we generate a new version that can
be stored, accessed and managed separately from the old ver-
sion. In our protocol, we can keep both the old and the new
versions.
• Insert When insertion occurs, the corresponding hsk
needs to be extended. If the hsk has extra space, we can sim-
ply use a new code to represent the new item. If the hsk is
already fully utilized, we have to increase its length. In these
cases, we will generate a new version of the star schema.
When new queries arrive, each query will be transformed into
two queries, one for the old version, another for the new ver-
sion. And we need a final merge operation after finishing the
two queries.
• Update After update, new value should not overwrite old
value in most cases. For instance, Hungary was a developing
country before 2008, and now it is counted as a developed
country. In this circumstance, we cannot erase its historical
data. In other words, even though the dimension data was up-
8 Front. Comput. Sci.
dated, its history information should be preserved. Therefore,
we keep the data after update in a new version. For a query, if
it accesses data before 2008, we use its old mdsk. Otherwise,
we use the new mdsk. If new value needs to overwrite old
value, there will be a mapping between new mdsks and old
ones while accessing old version of star schema.
To reduce storage cost, we can combine multiple versions
of data into just one by unifying their surrogate key.
6 Experiments
We conducted extensive experiments to analyze the perfor-
mance of our framework. We also showed the various over-
heads of our framework, such as the overheads of data load-
ing, storage, etc. We implemented the modules of TAMP as
well as all the optimization algorithms using C and deployed
them on Hadoop. To guarantee that all the fields of fact ta-
ble in the same record are co-located in the same node in a
cluster, we implemented a column-store similar to [21] on
Hadoop distributed file system.
• Platform All experiments were conducted on a cluster of
14 nodes, one serving as the NameNode and the others as the
DataNodes. Each node has a 2 GB memory, an Intel Core2
Duo 1.87 GHZ processor and 140 GB disk space. Each of
them runs the 32-bit Ubuntu 10.10 OS. We installed Hadoop
0.21. To achieve the best performance, we changed its config-
uration settings: (1) we set the data block size of the HDFS to
64 MB, (2) we enabled each task tracker to run with a max-
imum heap size of 500 MB, (3) we increased the sort buffer
to 200 MB, (4) we set the concurrency value of each node to
two map slots and one reduce slot.
• SSB test data Our experiments used the Star Schema
Benchmark6) which includes one fact table(lineorders) and
four dimension tables. With scale factor 500, we used SSB
to generate 500 GB of data: a fact table with 498 GB and
dimension tables totaling 1.3 GB.
• Query workload The full set of SSB queries con-
sists of four “query flights”, of three to four queries each,
for a total of 13 queries. According to the number of mea-
sures the query uses, we classify SSB queries into two
categories: single-measure query and multi-measure query.
Single-measure queries includes Q2.1–Q3.4 which operate
on only one measure, and multi-measure includes the first
query group (Q1.1–Q1.3) and the last query group (Q4.1–
Q4.2) which use three measures and two measures respec-
tively. Multi-measure queries may be used to evaluate the
6) SSB benchmark. http://www.cs.umb.edu/ xuedchen/research/publications/StarSchemaB.PDF
performance of scan-index. Below are two sample queries of
varying complexity from the SSB query set.
Query 1.1:
SELECT SUM(lo_extendedprice*lo_discount)
AS revenue
FROM lineorder, date
WHERE lo_orderdate = d_datekey
AND d_year = 1993
AND lo_discount BETWEEN 1 AND 3
AND lo_quantity < 25;
Query 1.1 is the first query in the set and returns a single row,
it performs an aggregate over the entire table, filtering with
selectivity 0.019. Query 4.3, on the other hand, is the last
query in the set, returns the fewest rows of all test queries. Its
filters have a selectivity of 0.000 091.
Query 4.3:
SELECT d_year, s_city, p_brand1,
sum(lo_revenue - lo_supplycost)
AS profit
FROM date, customer, supplier,
part, lineorder
WHERE lo_custkey = c_custkey
AND lo_suppkey = s_suppkey
AND lo_partkey = p_partkey
AND lo_orderdate = d_datekey
AND s_nation = ’UNITED STATES’
AND (d_year = 1997
OR d_year = 1998)
AND p_category = ’MFGR#14’
6.1 Performance
To study the performance of our framework, we compared the
performance of Hadoop, HadoopDB and Hadoop with our
framework (denoted as HadoopTAMP). For HadoopTAMP,
we deployed the transform module and the post-processing
module on the master node, and used mapper tasks to con-
duct aggregation operations based on Hadoop streaming and
a reducer task to conduct merge operations.
To conduct joins on Hadoop, a small difference in im-
plementation will result in big difference in performance.
Therefore, we conducted a set of initial experiments on
Hive and Hadoop to choose the fastest implementation. For
Hive, we tested the hash-based broadcast join and the left-
deep tree join. For Hadoop, we tested our own hash-based
broadcast join. It turned out that the hash-based broad-
cast join is the best [4] scheme for most queries. We used
 Huiju WANG et al. Efficient query processing framework for big data warehouse: an almost join-free approach 9

HadoopDB for comparison. This can show us the perfor- scan-index algorithm will be very useful. In the second set of
mance difference between our module and a custom RDBMS experiments, we compared HadoopTAMP against Hadoop,
(HadoopDB uses PostgreSQL as its execution engine). We HadoopDB and HadoopTAMP without scan-index (denoted
did not compare with Hadoop++, because HadoopDB out- by HadoopTAMP-NoSI). To perform tuple reconstruction,
performed Hadoop++ in data loading time and performance HadoopTAMP without scan-index directly accesses the data
in general cases [8]. To get the best possible performance files to retrieve the values belonging to the same tuple. Figure
of HadoopDB , we re-engineered its processor. Basically, 3 shows the experiment results.
we co-partitioned the customer table (the biggest dimension
table) and the fact table based on their join key and repli-
cated all the other dimension tables on each slave node. This
method is supposed to be highly optimized for HadoopDB5),
as all joins are pushed to the database layer. We used Post-
greSQL 9.0.2 as local databases of HadoopDB and increased
the memory for shared buffers to 256MB and the working
memory to 128 MB. We did not use PostgreSQL’s data com-
pression feature, as the compression ratio of SSB is very Fig. 3 Scan-index (SF=500)
small. Each node in HadoopDB was installed with Hive 0.6.0.

6.1.1 Single measure with full scan

To evaluate the performance of our approach on single-

measure queries, we conducted a set of experiments us-
ing SSB Q2.1–Q3.4. In the experiments, we turned off the
scan-index optimization module, as it is not helpful in these
queries. Our experiment results are shown in Fig. 4. We can
see that even though we tuned HadoopDB and Hadoop to
their optimal status, the performance of Hadoop with our
framework (denoted by HadoopTAMP) is eight times as fast
as that of HadoopDB, and 13 times as fast as that of Hadoop.
HadoopTAMP also exhibits better stability than HadoopDB
and Hadoop. Firstly, Hadoop needs to materialize interme-
diate results to the local disk, which incurs high I/O cost.
HadoopDB relies on costly join operations to process data
warehouse queries. In contrast, HadoopTAMP depends on
scan operation and performs join only on aggregated data.
Moreover, it only needs to materialize aggregation results
to the HDFS, whose size is usually very small (in SSB, all
queries produce less than 1 000 tuples). Secondly, as Hadoop-
TAMP adopts the PAX storage model, it can avoid access-
ing unnecessary columns. By contrast, HadoopDB uses Post-
greSQL which builds upon row-store and has to consume
more I/O operations. Hadoop performs poorly for the same
reasons.
6.1.2 Scan-index performance
The first query group (Q1.1–Q1.3) and the last query group
(Q4.1–Q4.2) of SSB use more than one measures (three mea-
sures and two measures respectively). In this circumstance,
Fig. 4 SSB Query performance (SF=500)
As we can see, HadoopTAMP is still the best. In average,
its execution time is about 13 times faster than Hadoop and
six times faster than HadoopDB. The performance advantage
of HadoopTAMP over HadoopDB is reduced a little, for there
is no long-running query, such as Q3.1. The query execu-
tion time of HadoopTAMP is only about 40% of the time
consumed by HadoopTAMP without scan-index. We attribute
this to the scan-index algorithm. Scan-index allows the sys-
tem to avoid a significant amount of redundant data accesses.
6.2 Scalability
As described in Section 3, we divide the total cost of TAMP
execution model into four parts: the transformation cost
(TC), the scan and aggregation cost (AC), the merge cost
(MC) and post-processing cost (PC). The total cost expres-
sion is: T otalCost = TC + AC + MC + PC.
A significant part of TC is consumed by by the lookup op-
eration to get the surrogate keys corresponding to the predi-
cates in the query. As discussed in section IV.A, this operation
can be facilitated by an index and thus can be very efficient.
The AC should be proportional to the size of the fact table,
10 Front. Comput. Sci.
as the operations on each node are limited to table scans The
MC is determined by the result size, as it is mostly network
transmission cost and I/O cost. Since there is only a small
amount of aggregated data, MC is usually very low. As post-
processing is the inverse operation of transformation, thus its
cost (PC) is close to TC.
Based on the analysis above, we carried out an experiment
to study the time distribution of TAMP execution model. The
experiment was performed on the master node and a data
node. The latter performs the scan, aggregation and sort op-
eration on fact table, other operations in TAMP model were
executed on the master node. We ran SSB Query 3.2 over a
dataset of about 32 GB. This query is supposed to incur big
TC + MC + PC, because it will produce large results (six hun-
dred records) from small data (just one column). Without loss
of generality, instead of keeping dimension tables in memory,
we stored all dimension on disk and used B+ tree index-based
access method. Nevertheless, our experiments show that the
proportion of TC+MC+PC to T otalCost is less than 0.12%
for Query 3.2. It is predictable that the bigger the data vol-
ume, the lower the ratio between results and data. Hence, the
query processing overhead is dominated by the scan and ag-
gregation cost. Suppose the size of the fact table is F and
there are N nodes of the same configuration in the cluster.
Then the fragment of each data node is F/N. Thus, the to-
tal cost is proportional to F/N. This indicates that the sys-
tem scales almost linearly with the data size (as illustrated in
Fig. 5).
Fig. 5 Processing time vs data size
6.3 Data loading
As the loading time for dimension tables are relatively small
and stable, we only report the experiment results on the fact
table.
To load the fact data, Hadoop only needs to copy the files
from the local hard disk to HDFS. In contrast, HadoopDB
needs three steps to load data: first, it distributes data based
on a key; second, it partitions data locally in each node; fi-
nally, it loads locally partitioned data into PostgreSQL. Our
framework needs two steps to load data: it generates surro-
gate keys and inserts them into the fact table first, then it
writes the data to the slave nodes in the format of PAX model.
Figure 6 shows the loading time of the 500GB dataset using
the free different approaches.
As we can see from Fig. 6, the loading time of Hadoop-
TAMP is 3.4 times of Hadoop and 29% of HadoopDB. By
analyzing our framework’s execution process in detail, we
found that the major overhead is incurred by the hash-based
broadcast join to get dimension surrogates. The generation
cost of surrogate keys occupies 80% of the total cost. In fact,
the data pre-processing cost can be reduced greatly if we im-
plement a dedicated ETL module, as Hadoop’s performance
is not as good as that of RDBMS.
Fig. 6 Data loading (SF=500)
Although our system incurs higher loading cost than
Hadoop, it will be rewarded at runtime, which can be seen
in the performance results.
6.4 Storage
In this experiment, we compared the storage overhead of our
framework against that of HadoopDB and Hadoop. We used a
32 GB SSB dataset. As shown in Fig. 7, HadoopTAMP con-
sumes 12% more space than Hadoop and 35% more space
than HadoopDB. The extra cost, though not much, is mainly
due to the key, value storage model, which replicates mdsk
attributes for each column. In fact, our key, value storage
model is similar to column-store, which indicates that we can
design efficient compression algorithm to further reduce the
storage overhead. This is our future work.
 Huiju WANG et al. Efficient query processing framework for big data warehouse: an almost join-free approach
Fig. 7 Storage (SF=32)
6.5 Dimension update
Based on our multi-version schema update protocol, one di-
mension update (either insert or update) operation will either
generate a new version of schema or insert a new code into
current version of schema. As dimension update operations
usually occur periodically which are far less frequent than
queries and can be done at some pre-defined time, we mainly
focus on the evaluation of query performance. To evaluate the
query performance, we use the same dataset and queries; to
see the effect of the number of schema versions on query per-
formance, we generate several derived datasets with different
number of versions of schema data based on the same dataset.
For simplicity, we choose one dimension such as c_nation as
target dimension to be updated. #N versions of data may be
generated by updating 100/N% fact table tuples and corre-
sponding values in c_nation dimension. For example, if there
are three versions, each version contains about 33% percent
of data which means each version has 167 GB of 500 GB data
in average.
We ran the second query in each set of SSB queries over
each derived dataset. From Fig. 8, we can see that for each
query, as the number of versions increases, execution time
increases accordingly; and more versions, more time it takes
to execute the query. This is reasonable because with the
same size of data, one more version will introduce one more
transformation operation and one more job launch operation,
which results in the increasing of execution time. It is pre-
dictable that increased time caused by increasing of versions
will take the most time possibly when the number of versions
reaches some point. However, this can be avoided by simply
combining multiple versions of schema data into one period-
ically as discussed in Section 2. As increased time is intro-
duced mainly by job launch operation which is bounded by
Hadoop platform, the situation should be able to be alleviated
11
greatly on other platforms such as PDBMSs which can start
one query more efficiently.
Fig. 8 Performance vs #Schema Versions
7 Conclusion
In this paper, we propose a novel framework for efficient pro-
cessing of data warehouse queries over big data. Our frame-
work abstracts data warehouse processing in a single proce-
dure: transform, aggregate, merge and post- processing. We
achieve this by compressing dimension hierarchies informa-
tion into the fact table using hierarchy encoding which elim-
inates the star (snowflake) joins in query processing. There-
fore, the fact table can be filtered and aggregated without re-
ferring to the dimension tables and the query execution time
is not affected by the number of joins.
Much work remains in the future. First, our key, value
store is similar to column-store, thus it is possible to achieve
high compression ratio if we compress the data. Second, as
all queries in our framework are answered by scans, we can
devise inter-parallel mechanism to allow multiple queries to
share I/Os and the network bandwidth. Finally, as our frame-
work turns multidimensional data to single-dimension surro-
gate key, it is interesting to investigate how to utilize the tra-
ditional one-dimensional index, such as hash, to further opti-
mize the query processing.
References
1. Chaudhuri S, Dayal U. An overview of data warehousing and olap tech-
nology. SIGMOD Record, 1997, 26(1): 65–74
2. Dean J, Ghemawat S. Mapreduce: Simplified data processing on large
clusters. In: Proceedings of the 6th Symposium on Operating Systems
Design and Implementation. 2004, 137–150
3. Apache hadoop. http://hadoop.apache.org
4. Pavlo A, Paulson E, Rasin A, Abadi D J, DeWitt D J, Madden S, Stone-
12 Front. Comput. Sci.

braker M. A comparison of approaches to large-scale data analysis. In: ACM Transactions on Database Systems, 1984, 9(3): 331–347
Proceedings of the 35th SIGMOD International Conference on Man- 21. Floratou A, Patel J M, Shekita E J, Tata S. Column-oriented storage
agement of Data. 2009, 165–178 techniques for mapreduce. Proceedings of the VLDB Endowent, 2011,
5. Afrati F N, Ullman J D. Optimizing joins in a map-reduce environment. 4(7): 419–429
In: Proceedings of the 2010 International Conference on Extending
Databas Technology. 2010, 99–110
6. Dawei Jiang G CA. K. H. Map-join-reduce: Towards scalable and effi- Huiju Wang graduated from Renmin
cient data analysis on large clusters. IEEE Transactions on Knowledge University of China in 2012 and works
and Data Engineering, 2011, 23(9): 1299–1311 as postdoctoral research fellow at the
7. Olston C, Reed B, Srivastava U, Kumar R, Tomkins A. Pig latin: a not-
school of computing of National Uni-
so-foreign language for data processing. In: Proceedings of the 2008
SIGMOD International Conference on Management of Data. 2008, versity of Singapore. His research
1099–1110 spans the areas of big data, cloud-
8. Dittrich J, Quiané-Ruiz J A, Jindal A, Kargin Y, Setty V, Schad J. ing computing, databases and data
Hadoop++: Making a yellow elephant run like a cheetah (without
management, with emphasis on graph
it even noticing). Proceedings of the VLDB Endowment, 2010, 3(1):
518–529 database, graph index, graph data ex-
9. Floratou A, Patel J M, Shekita E J, Tata S. Column-oriented storage ploration.
techniques for mapreduce. Proceedings of the VLDB Endowent, 2011,
4(7): 419–429
Xiongpai Qin received his MS and PhD
10. Lin Y, Agrawal D, Chen C, Ooi B C, Wu S. LLAMA: leveraging
degree in computer science from Ren-
columnar storage for scalable join processing in the mapreduce frame-
work. In: Proceedings of the 2011 SIGMOD International Conference min University of China in 1998 and
on Management of Data. 2011, 961–972 2009 respectively, and works as a lec-
11. Xu Y, Kostamaa P, Gao L. Integrating hadoop and parallel DBMS. turer at Information School of Ren-
In: Proceedings of the 2010 SIGMOD Conference on Management
min University of China. His research
of Data. 2010, 969–974
12. Abouzeid A, Bajda-Pawlikowski K, Abadi D J, Rasin A, Silberschatz interests include semantic based in-
A. Hadoopdb: An architectural hybrid of mapreduce and DBMS tech- formation retrieval, high performance
nologies for analytical workloads. Proceedings of the VLDB Endow- database and big data.
ment, 2009, 2(1): 922–933
13. Swami A, Gupta A. Optimization of large join queries. SIGMOD
Record, 1988, 17(3): 8–17 Xuan Zhou is an associate professor at
14. Raman V, Swart G, Qiao L, Reiss F, Dialani V, Kossmann D, Narang I, the Renmin University of China. He
Sidle R. Constant-time query processing. In: Proceedings of the 2008 obtained his BS in computer science
International Conference of Data Engineering. 2008, 60–69
from Fudan University, China in 2001,
15. Valduriez P. Join indices. ACM Transactions on Database Systems,
1987, 12: 218–246 and his PhD from the National Univer-
16. Markl V, Ramsak F, Bayer R. Improving OLAP performance by multi- sity of Singapore in 2005. His research
dimensional hierarchical clustering. In: Proceedings of the 1999 Inter- interests include database and informa-
national Symposium on Database Engineering and Applications. 1999,
tion management. He has published his
165–177
work in the top conferences and jour-
17. Karayannidis N, Tsois A, Sellis T K, Pieringer R, Markl V, Ramsak F,
Fenk R, Elhardt K, Bayer R. Processing star queries on hierarchically- nals on data management.
clustered fact tables. In: Proceedings of the 28th VLDB Conference.
2002, 730–741
Furong Li is a PhD candidate at Na-
18. Bayer R. The universal b-tree for multidimensional indexing: gen-
tional University of Singapore. She ob-
eral concepts. In: Proceedings of the 1997 International Conference
on Worldwide Computing and Its Applications. 1997, 198–209 tained her BS from Renmin University
19. Theodoratos D, Tsois A. Heuristic optimization of olap queries in mul- of China in 2012. Her research inter-
tidimensionally hierarchically clustered databases. In: Proceedings of ests include data integration, social net-
ACM 4th International Workshop on Data Warehousing and OLAP.
works and big data management.
2001, 48–55
20. Korth H F, Kuper G M, Feigenbaum J, Gelder A V, Ullman J D. Sys-
tem/u: A database system based on the universal relation assumption.
Huiju WANG et al. Efficient query processing framework for big data warehouse: an almost join-free approach 13

Zuoyan Qin received his BS and MS
from Renmin University of China in
2008 and 2011 respectively. He is one
senior engineer in Baidu company one.
Before joining Baidu, he worked in
Tencent. His main focus is big data pro-
cessing and cloud computing.
Professor Shan Wang finished her un-
dergraduate studies at the Peking Uni-
versity, China in 1968, and completed
her Master study at Renmin University
of China in 1981. Her research interests
include high performance database,
data warehouse and knowledge engi-
neering, information retrieval, etc.

Qing Zhu is an associate professor of

School of Information, Renmin Univer-
sity of China. She completed her Phd in
2005 in Renmin University and MS in
1991 in Beijing University of Technol-
ogy, China. Her research interests in-
clude Grid computing, distributed algo-
rithms, Semantic Web service and high
performance Database.