See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/316285760

Comparison of Different Solutions for Solving the Optimization Problem of

Large Join Queries

Conference Paper · April 2010

DOI: 10.1109/DBKDA.2010.1 · Source: doi.ieeecomputersociety.org

CITATIONS READS
6 352

1 author:

Dusan Petkovic
Technische Hochschule Rosenheim
43 PUBLICATIONS 153 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

SQL/JSON: Implementing JSON in RDBMSs View project

Performance of Large Join Queries View project

All content following this page was uploaded by Dusan Petkovic on 16 January 2018.

The user has requested enhancement of the downloaded file.

 Comparison of Different Solutions for Solving the Optimization Problem of Large
Join Queries

Dušan Petković
University of Applied Sciences
Hochschulstrasse 1
Rosenheim, Germany
petkovic@fh-rosenheim.de

Abstract—The article explores the optimization of queries
using genetic algorithms and compares it with the conventional
query optimization component. Genetic algorithms (GAs), as a
data mining technique, have been shown to be a promising
technique in solving the ordering of join operations in large
join queries. In practice, a genetic algorithm has been
implemented in the PostgreSQL database system. Using this
implementation, we compare the conventional component for
an exhaustive search with the corresponding module based on
a genetic algorithm. Our results show that the use of a genetic
algorithm is a viable solution for optimization of large join
queries, i.e., that the use of such a module outperforms the
conventional query optimization component for queries with
more than 12 join operations.
Keywords-Database systems, Query optimization, Large join
queries, Genetic algorithms, PostgreSQL.
I. INTRODUCTION
The question that generally arises when a relational
database system executes a query is how the data that is
necessary for the query can be accessed and processed in the
most efficient manner. The component of a database system
that is responsible for the processing is called query
optimizer. The task of query optimizer is to consider a
variety of possible execution strategies for querying the data
concerning a given query and to select the most efficient
strategy. The selected strategy is called query execution plan
(QEP). The optimizer makes its decisions using
considerations such as how big the tables are that are
involved in the query, what indices exist, and which
Boolean operators are used for existing conditions.
The query optimization process comprises several
phases [14]. In the first query optimization phase, the query
is transformed in a query tree, where the leaves of a tree are
database relations and its non-leaf nodes represent
operations of relational algebra, such as projection, selection
and join. After that, the query tree is compiled. In the third
step, query optimizer takes as input the compiled query tree
and investigates several access strategies before it decides
how to process the given query. To find the most efficient
strategy, the query optimizer first makes the query analysis,
during which it searches for arguments and join operations.
The optimizer then selects which indices to use. Finally, if
join operations exist, the optimizer selects the order of join
operations and chooses one of the join processing
techniques. In the last step, QEP is permanently stored and
executed.
Genetic algorithms belong to data mining techniques
that are based on an analogy to biological processes [15].
Generally, they apply natural selection to find the globally
optimal solution to the given problem. A genetic algorithm
is an iterative process, which maintains a constant
population size of feasible solutions. During each step, the
fitness of current population is evaluated and population is
selected based on the values of the function. Chromosomes,
i.e., members of population with higher fitness are selected,
while ones with lower fitness are eliminated. The new
population is evaluated and selected for the next generation
(iteration). The iterative process is finished, when an
optimal solution is reached.
The population is modified in each step using the
following operators: selection, crossover and mutation.
Selection is similar to natural selection, i.e., to the principle
of “survival of the fittest”. Unlike nature, the size of the
population remains constant from one generation to the
next. The chance of a chromosome to be selected for the
next generation is proportional to its fitness. In other words,
chromosomes with higher fitness are selected, while from
poor or lower fitness chromosomes only a few (or no one)
are selected. For this reason, genetic algorithms use the
fitness function, which quantifies the optimality of
chromosomes so that particular chromosome may be ranked
against all the other ones.
The next operator applied to the surviving
chromosomes is crossover. Crossover creates two new
chromosomes from two existing ones by mixing together
pieces of each of them. This operator starts with two
chromosomes and a random position. The first part of one
chromosome swaps places with the corresponding part of
the second one. The intuition behind the crossover operator
is a creation of new solutions by exploiting the old ones.
Genetic algorithms construct a better solution by mixing
good characteristic of chromosomes. Hence, the probability
that chromosomes with better fitness will be selected is
higher than for the chromosomes with lower fitness.

1
 The final operation is mutation. In nature, mutation is the
result of miscoded genes being passed from a parent to a
child. New chromosomes may be better (or poorer) than old
ones. If they are poorer than old chromosomes, they will be
eliminated in the following generations. Generally, mutation
is a second-order effect (in relation to selection and
crossover), which helps to avoid reaching a local maximum.
As we already stated, the set of semantically equivalent
query trees can be generated for a given query. The task of
the query optimizer is to select an optimal query execution
plan out of all generated ones. On the other hand, genetic
algorithms can be used to solve optimization problems by
manipulating the patterns of bits in the chromosomes. For
this reason, genetic algorithms can be used to solve complex
problems of searching for and selecting an optimal plan out
of the set of all plans that are generated by query optimizer.
This paper shows that the implementation of the query
optimization component for ordering of join operations in
large join queries outperforms the traditional component,
i.e. the one, which uses exhaustive search, for queries with
more than twelve joins.
A. Related Work
The problem of ordering of relational joins has been
discussed in one of the first papers concerning the System R
[1]. The algorithm used in [1] belongs to deterministic
algorithms, because it constructs a solution step by step in
deterministic manner. Exhaustive search is one of
deterministic algorithms, where all possible combinations of
the joins in a given query are tested and the best solution in
relation to the costs is chosen. Almost all query optimizer
components of existing relational database systems use a
form of exhaustive search to find the optimal QEP.
The optimization technique called Iterative Improvement
starts with a random execution plan and searches for the
optimal execution plan similar to hill-climbing [2, 3]. In
other words, the execution plan that offers the best
improvement in relation to the costs is taken. The process is
repeated until one settles at the local minimum, where no
further improvement is possible. Another algorithm, which
belongs to the same class as the Iterative Improvement
algorithm is called Simulated Annealing [4]. It differs from
the former one, because it avoids the trapping in one of the
local minima by searching further for the global optimal
solution. The disadvantage of this algorithm is the increase
in costs. The Two-Phase optimization algorithm [5]
combines both previous algorithms: the Iterative
Improvement algorithm is run until a local minimum is
reached. After that the Simulated Annealing algorithm
attempts to find less obvious improvements. The algorithms
mentioned above are called randomized algorithms, because
each of them performs a random walk according to certain
rules and along the different solutions of the search space.
The paper [6] introduces genetic algorithms as a way to
solve the ordering problem of large join queries. In this
paper genetic algorithms are compared with the
corresponding component of System R for relational queries
up to 16 join operations. The experimental part of this paper
shows that GA can find a better execution plan than the
corresponding component of System R. The three
algorithm groups discussed above (deterministic,
randomized and GAs) were compared in [7]. The paper
concludes that randomized and genetic algorithms are
significantly better than the deterministic one, although they
need more time for the optimization process.
As we already stated, during the query optimization
process, every QEP is transformed in a query tree. Each
such tree can be viewed as a program, and a specialization
of GAs that is called genetic programming can be applied to
solve the optimization problem for large join queries. In this
case, the database relations are the terminals and algebraic
relational operations are the functions of the genetic
program.
Lahiri, in his article [8], compares GAs with genetic
programming and concludes from his experimentation that
genetic programming works better for the optimization
problem of large join queries than GAs. Another paper,
which uses genetic programming to solve the optimization
problem of large join queries is [9]. The conclusion of this
paper, where genetic programming is compared to the
Iterative Improvement algorithm, is that the latter is more
effective, while the former algorithm can find a better
solution.
Genetic algorithms were applied for the first time as a
technique for optimization of relational queries at the
beginning of the last decade of the twentieth century. Initial
papers showed that genetic algorithms are a viable solution
for optimization of large join queries. According to these
results, members of the PostgreSQL development
community implemented a genetic algorithm as an
alternative to the conventional module, which uses
exhaustive search to solve the optimization problem for
ordering of participating tables in large join queries. The
query optimization module of the PostgreSQL database
system, which uses GA to order tables in large join queries
is called GEQO [10]. Properties of the GEQO module will
be discussed in Section 2.
Besides the implementation of the GEQO module for the
PostgreSQL database system, there exist several papers
describing the implementation of GA for the IBM DB2
query optimizer. The papers of Muntes-Montero et al. [11,
12] introduce Carquinyoli Genetic Optimizer (CGO), which
can be used as a stand-alone optimizer or a post-optimizer,
together with IBM DB2. The experimental results of this
paper show that the CGO component generates QEPs that
are several times cheaper than those obtained by the IBM
DB2 query optimizer. Also, the average cost for QEPS with
CGO are circa four times cheaper.
2
B. Roadmap
The rest of the paper is organized as follows. Section 2
gives a short insight into the implementation of the genetic
algorithm called GENITOR used in the PostgreSQL
database system. The results of our experiments are shown
in Section 3. First, we run several large join queries using
the GEQO module and compare their execution times with
execution times, when the query component for exhaustive
search is used. Second, we investigate how the increasing
number of generations influences the execution time of
large join queries. Section 4 gives conclusions of this paper.
II. IMPLEMENTATION OF THE GENETIC ALGORITHM IN
POSTGRESQL
The particular genetic algorithm used to implement the
GEQO module is called GENITOR. GENITOR is an
acronym for GENetic ImplemenTOR, a genetic search
algorithm that differs in two major ways from standard
algorithms. The algorithm reproduces new chromosomes on
an individual basis. It does so in such a way that parents and
their offspring can exist together at the same time. The
GENITOR algorithm has been successfully used in solving
different optimization problems. The advantage of the
GENITOR algorithm is that it produces one new
chromosome at a time, so inserting a single new individual
is simple. Another advantage is that the insertion operation
automatically ranks the individual relative to the existing
ones. For this reason, no further measures of relative fitness
are needed.
The theoretical results show that GENITOR achieves faster
feedback relative to the rate at which new elements of the
search space are being sampled, because it uses a one-at-
time reproduction scheme. Also, this algorithm may be less
biased in relation to schemata with long defining lengths
than the standard one. The latter property is one of main
reasons why this algorithm is used to implement the GEQO
module, because large join queries belong to the group of
processes with long defining lengths.
The two GEQO system functions used in our tests are:
- geqo (boolean)
- geqo_generations(integer)
Each user of the PostgreSQL database system can
optionally use either the GEQO module or the exhaustive
search component for the execution of join queries. To
disable, i.e., enable the GEQO module, the geqo() system
function is used. Another important system function
concerning the GEQO module is geqo_generations(). This
function controls the number of generations (i.e., iterations)
used be the GEQO module. If the value is set to zero, the
system chooses the most appropriate value.
III. EXPERIMENTAL EVALUATIONS
For our experiments, we use the AdventureWorks 2008
sample database of the MS SQL Server system. This
database contains more than 60 tables, and hence suits very
well for testing large join queries [13]. (The original sample
database of the PostgreSQL system contains only 6 tables
[10], and cannot be used to test large join queries.) All
tables from the AdventureWorks 2008 database have been
exported from the SQL Server database system and
imported in the PostgreSQL database system Version 8.4.
For each query optimization component we run 18
different queries with up to 14 join operations.. The
following list shows the structures of some of the tables
from the AdventureWorks database used in our tests:
Product (productID: int, name char(50), productnumber:
char(25), color char(15), listprice dec(10,2),
size char(5), sizeunitmeasurecode char(3),
weightunitmeasurecode char(3), weight dec(8,2)
BusinessEntity (businessentityid: int, modifieddate: int)
Employee (businessentityid: int, nationalidnumber:
char(15), jobtitle: char(50), gender: char(1),
vacationhours: int)
EmployeePayHistory (businessentityid: int, rate: int)
JobCandidate (jobcandidateid: int, businessentityid: int)
Person (businessentityid: int, persontype: char(2),
firstname: char(50), lastname: char(50))
Product (productid: int, name: char(50),
productnumber: char(25),color: char(15),listprice: dec(10,2),
size: char(5), sizeunitmeasurecode: char(3),
weightunitmeasurecode: char(3), weight: dec(8,2),
productionsubcategoryid: int, productmodelid: int)
ProductCategory (productcategoryid: int, name: char(50))
ProductListPriceHistory (productid: int, startdate: date,
listprice: dec(9,2))
ProductReview (productreviewid: int, productid: int,
reviewername: char(50), rating: int)
ProductSubCategory (productsubcategoryid: int,
productcategoryid: int, name: char(50))
ProductVendor (productid: int, businessentityid: int,
minorderqty: int, maxorderqty: int, unitmeasurecode:
char(3))
PurchaseOrderDetail (purchaseorderid: int,
Purchaseorderdetailid: int, Orderqty: int, Productid: int)
PurchaseOrderHeader (purchaseorderid: int, status: int,
employeeid: int)
SalesOrderHeader (salesorderid: int, revisionnumber: int,
status: int, customerid: int, salespersonid: int, territoryid: int,
billtoaddressid: int, shiptoaddressid: int, shipmethodid: int)
SalesPerson (businessentityid: int, territoryid: int,
salesquota: int, bonus: int)
TransactionHistory (transcationid: int, productid: int,
quantity: int)
In processing the input queries, we considered the following
generic query. From it we derived other test queries:
SELECT p.productID, e.businessentityID,pod.orderqty,
pod.purchaseorderdetailID, eph.rate,
jc.jobcandidateID, pc.name, plph.startdate,
3
pv.unitmeasurecode, pod.productID, sp.bonus
FROM product p
INNER JOIN productSubCategory psc ON
p.productionsubcategoryid=psc.productsubcategoryid
INNER JOIN productCategory pc ON
pc.productcategoryid = psc.productcategoryid
INNER JOIN transactionHistory th ON
th.productID = p.productID
INNER JOIN productListPriceHistory plph ON
plph.productID = p.productID
INNER JOIN productVendor pv ON
pv.productID = p.productID
INNER JOIN purchaseOrderDetail pod ON
pod.productID = pv.productID
INNER JOIN purchaseOrderHeader poh ON
poh.purchaseOrderID = pod.purchaseOrderID
INNER JOIN employee e ON
e.businessEntityID = poh.employeeID
INNER JOIN salesPerson sp ON
sp.businessentityID = e.businessEntityID
INNER JOIN employeePayHistory eph ON
eph.businessentityID = e.businessentityID
INNER JOIN jobcandidate jc ON
jc.businessentityID = e.businessentityID
INNER JOIN businessentity be ON
be.businessentityID = e.businessentityID
WHERE p.productID= 317
AND pod.purchaseorderdetailID BETWEEN 100 and 200;
A. GEQO Module vs. Exhausted Search
As we already stated, the convenient component, which
uses exhaustive search, examines all possible combinations
of the joins in a given query and chooses the best one in
relation to costs. For this reason, exhaustive search is not
recommended for large join queries. In our tests (see Fig. 1)
we compare the execution time of the queries with 11, 12,
13 and 14 join operations. Each lefthand bar of the diagram
in Fig. 1 shows the execution time with the GEQO module,
while each right-hand bar represents the execution time of
the query optimization component, which uses exhaustive
search. Execution times represented in Fig. 1 show the
average time over all queries. As it can be seen from Fig. 1,
both components perform equally for queries up to 12 join
operations. When the number of join operations is higher
than 12, the GEQO module outperforms the conventional
component for about 15% and more. We did not use
queries, where the number of joins exceeds 14, but
according to the results of this paper, one can expect that
they are similar. Note also that our results are different than
the results obtained in [7]. The reason for this can be the use
of different class of queries and/or the use of different
constraints.
B. Increasing the Number of Generations
In the second test, we investigate how the increasing
number of generations, i.e., iterations influences the
execution time of a query. This test differs from the first one
because it uses only the GEQO module, i.e., it does not
compare the both components used in the first test. Queries
used for experimentation are the same as the queries from
Section 3.A. In this test, the parameter value of the
geqo_generations() system function was changed, starting
with 1, and increasing the value up to 13. The results are
shown in Fig. 2.
Figure 1: Comparison of the GEQO module and the exhaustive
search
The figure demonstrates that increasing the number of
generations decreases the execution time of a query. In the
initial generation (geqo_generations=1), the deviation from
the best value is around 25%. As generation proceed, the
deviation decreases and begins to converge beginning with
the 9th generation.
IV. CONCLUSIONS
Until now, the theoretical part of applying genetic
algorithms to the optimization of large join queries has been
set. For this reason, the emphasis in the future should be to
implement genetic algorithms in relational database systems
other than PostgreSQL. Therefore, future implementation
prospects can be divided in the following groups:
a) Future extensions of the GEQO module
b) Further development of the CGO module
c) Implementation of genetic algorithms in relational
database systems other than PostgreSQL and IBM
DB2
A. Future Extensions of the GEQO Module
As the results of Section 3.A of this paper show, the
GEQO module is a viable alternative for the execution of
large join queries in the PostgreSQL database system. On
4
 the other hand, the implementation of the GEQO module
supports only the left-deep tree query processing. The left-
deep tree query processing means that all internal nodes of a
query tree have at least one leaf as a child and inner nodes
of all join operations are always database relations. For this
reason, the query optimizer of the PostgreSQL system
would benefit from extensions of the GEQO module, where
the more general representation of query trees called bushy-
tree is supported. The reason for this is, as the paper [7]
shows that the bushy tree query processing is more
preferable than left-deep tree query processing in the case of
the hash join technique.
B. Further Development of the CGO Module
The future efforts in relation to the CGO module should
be twofold. The first effort should be to implement the
module as the integral part of the DB2 system software. One
way to do this is to implement the module as a new DB2
Extender module. Second, the CGO module should be
tested together with advanced features in DB2, such as
multidimensional clustering indexed, index ANDing and
index ORing, to get the complete picture of the viability of
the module. All these features were disabled during the tests
executed for the work in [11].
Figure 2: Modifying the number of generations
C. Implementation of Genetic Algorithms in Database
Systems other than PostgreSQL and IBM DB2
We are not aware of any efforts to implement genetic
algorithms as a component of relational database systems
other than PostgreSQL and DB2. As the first test of this
paper as well as the experimentation in [11] show, the
optimal plans generated by genetic algorithms are
competitive with corresponding optimal QEPs generated by
conventional query optimizer components in relation to
large join queries. For this reason, it is obvious that the
further research in applying and implementing of GA as an
extended component of query optimizers for other enterprise
View publication stats
database systems, such as Oracle and MS SQL Server, is
recommended.
REFERENCES
[1] Astrahan, M.M., Blasgen, M.W., Chamberlain, D.D., Eswaran,
K.P., Gray J.N. – Access Path Selection in a Relational Database
Management System, in Proc. of the ACM SIGMOD Conf. on
Management of Data, Boston, June 1979, pp.23-34.
[2] Swami, A. and Gupta, A. - Optimization of Large Join Queries,
Proc. of the 1988 ACM SIGMOD Int. Conf. on the Management of
Data, Vol. 17, No. 3, 1988. pp. 8-17
[3] Swami, A. – Optimizations of large join queries: Combining
Heuristic and Combinatorial Techniques, Proc. ACM SIGMOD
Conf. on Management of Data, Portland, 1989, pp.367-76.
[4] Ioannidis, Y. E; Wong, E. - Query Optimization by Simulated
Annealing, Proc. of the 1987 ACM SIGMOD Int. Conf. on the
Management of Data, San Francisco, 1987, pp. 9-22.
[5] Ioannidis, Y. E.; Kang, Y. C. - Randomized Algorithms for
Optimizing Large Join Queries, Proc. of the 1990 ACM-SIGMOD
Conference on the Management of Data. Atlantic City, NJ, 1990,
pp. 312-321.
[6] Bennett, K.; Ferris, M. C.; Ioannidis, Y. - A genetic algorithm
for database query optimization, Tech. Report TR1004, Univ.
Wisconsin, Madison, 1991
[7] Steinbrunn, M., Moerkotte, G., Kemper, A. - Heuristic and
randomized optimization for the join ordering problem. VLDB
Journal, 6, 3 (Aug. 1997), Springer, New York, pp. 191-208
[8] Lahiri, T. Genetic Optimization Techniques for Large Join
Queries, in Proc. of the 3rd Genetic Programming Conf., 1998,
pp.535-40.
[9] Stilger, M., Spiliopoulou, M. – Genetic Programming in
Database Query Optimization, in Proc. of the 1st Genetic
Programming Conference, 1996, pp. 388-93.
[10] PostgreSQL, http://www.postgresql.org, last access
24.12.2009.
[11] Muntes-Mulero, V.; Aguilar-Saborit, J.; Zuzarte, C; Larriba-
Pey, J. – SGO: A Sound Genetic Optimizer for Cyclic Query
Graphs, in Alexandrov, V.N. et al. Proceedings of ICCS 2006, Part
I, LNCS 3991, 2006, pp. 156-163.
[12] Muntes-Mulero, V.; Aguilar-Saborit, J.; Zuzarte, C; Larriba-
Pey, J. – Analyzing the Genetic Operations of an Evolutionary
Query Optimizer, in Bell, D. and Hong. J. (Eds.), Proc. of BNCOD
2006, LNCS 4042, 2006, pp. 240-244.
[13] Petković, D. – SQL Server 2008, A Beginner’s Guide,
McGraw Hill, 2008, ISBN: 0071546383.
[14] Ioannidis, Y. E. – Query Optimization,
http://infolab.stanford.edu/~widom/cs346/Ioannidis.pdf
[15] Goldberg, D. - Genetic Algorithms in Search, Optimization
and Machine Learning, Addison Wesley, 1989.