Evaluation Criteria for Public Procurement

František Ochrana1, Michal Plaček2, Veronika Abonyiová3, Milan Jan Půček4

Charles University in Prague
1

Faculty of Social Sciences

Smetanovo nábřeží 6, 110 01 Praha 1, Czech Republic
E-mail: ochrana@fsv.cuni.cz

2
Private College of Economics Studies Znojmo
Loucká 21, 669 02 Znojmo, Czech Republic
E-mail:placek@svse.cz

Charles University in Prague

3

Faculty of Social Sciences

Smetanovo nábřeží 6, 110 01 Praha 1, Czech Republic
E-mail: 29279467@fsv.cuni.cz
4 Private College of Economics Studies Znojmo
Loucká 21, 669 02 Znojmo, Czech Republic

Abstract: The paper analyzes evaluation criteria in public procurements. It examines a

data set of public procurements for construction that were awarded in Czech Republic in
2013. Evaluation criteria for public contracts was applied to a set of 5065 public
contracts. Two methods of evaluating bids can be used for the evaluation of public
contracts: single-criterion (the lowest bid price) and Multi-criterion (economically
advantageous tender). The paper analyzes the use criterion of the lowest tender price
and criteria economically advantageous tender with regard to the frequency of their use,
their average weight, the price differential (final price of tenders) and the type of
procurement procedure used. The predominant evaluation criterion is the lowest price
and respective bid price. The criteria of the quality and functionality of public contracts
was found to be infrequently used.
Keywords: public procurement, public contracts for construction, evaluation criteria, the
lowest price, the economic advantages of offers
JEL codes: D23, H57

1 Introduction
The public sector faces the problem of how to provide public goods and services. It can
secure these by the utilizing the methods of in-house and out-house (Prager, 1994).
Each of these methods has its economic advantages and limitations (Nemec, Mikušová
Měřičkova and Grega, 2014). If the public sector decides on the provision of public goods
and services through public procurement, it must respect a certain legal framework
(Jurčík, 2012, 2014). You can choose from a set of options that are set out by the Public
Procurement Act. One element of this option is the use of evaluation criteria and the
choice of the type of procurement procedures (see Strand, Ramada, Canton et al.,
2011).In this paper we focus on the problem of the evaluation criteria in public
procurement. The research is focused on public contracts in construction in the Czech
Republic in 2013. It is an analysis of the first year of the period after a major amendment
to the Public Procurement Act by Act no. 55/2012 Coll. In the paper, we aim to explore
what the frequency is of the use of assessment criteria in public contracts by a number of
evaluation criteria and the volume of funds, as well as whether the choice of evaluation
criteria had an impact on the average price difference of a public contract. We also ask at
the same time the question of how the tenders differ according to the type of
procurement procedure, as well as what the relationship between the average number of
offers and price differentials in relation to the evaluation criteria is. Consideration of
those issues has not only theoretical, but also practical importance. It concerns the
search for factors that affect the efficiency of public competition as well as possible

428
savings in public procurements (Půček, Ochrana, 2014). About 500 billion CZK have been
allocated via the institute of public contracts, so therefore it is important to look for
factors that may increase effectiveness of public procurement, or to reveal the factors
that cause waste in public procurement (see Bandiera, Prat, Valletta, 2008). Does the
choice of evaluation criteria belong among them?
2 Methodology and Data
The contracting authority may use the lowest bid price for the evaluation of offers, or it
can evaluate public contracts on the basis of the criterion of economically advantageous
tender (see Law no. 137/2006 on public procurement). In the case of the lowest tender
price the bid is evaluated by a single criterion. When applying the economically
advantageous tender, a multidimensional assessment of tenders is applied where
individual criteria are set by weight. The bid price is (can be) one of the evaluation
criteria.
The problem of the role of evaluation criteria does not belong to the predominant
research questions public contracts. In theory, we encounter studies that concentrate
more on other issues regarding public procurement. Bolton Dewtripont (2005) analyze
public awarding in terms of contractual theory. Examining procurement as being a barter
between a public authority, (the contracting entity) and the tenderer. This economic view
is complemented by a legal analysis of public procurement (Jurčík, 2012). A public tender
is a formalized procedure for which the Public Procurement Act has established rules and
procedures. These are compared to decisions of individual consumers as being so
complex that it can lead to inefficiency and a lack of transparency in public awards as
demonstrated, for example, in the empirical study of the OECD (2002). A factor of (in-
)efficiency may be the level of competition on the supply side (Kuhlman & Johnson,
1983). This is indicated by the number of offers in the competition. In theory, this
problem is referred to by the term "competitive effect" (Domberger & Rimmer, 1994).
The effect of competitive and open public competition is dealt with in most studies
dealing with public procurement. Among international research we would like to point
out, for example, the European Commission study (see Strand, Ramada & Canton, 2011)
and the study Iimi (2006). In the Czech Republic, empirical investigations into the effect
of competitive and open procurement procedure are dealt with by, for example, Pavel
(2010), Nikolovová & al. (2012), Kameník (ed., 2011), Soukopová & Malý (2013),
Ochrana and Stehlik (2014).
Other studies have examined the impact of selective factors that influence the outcome
of tenders (see eg. Nemec, Mikušová Měřičková and Grega, 2014). These authors
conducted a secondary data analysis of eight Central European countries (Slovakia,
Poland, the Czech Republic, Austria, Estonia, Latvia and Lithuania) and three Western
European countries (Great Britain, France and Germany). They examined, inter alia, the
role of the lowest bid price criteria in public awards. They concluded that Central
European countries prefer the criterion of the lowest tender price, whereas Western
European countries prefer multidimensional assessment of public contracts. In our
investigation, we will monitor whether this conclusion is valid for the Czech Republic,
even after the amendment to the Public Procurement Act (amended by Act no. 55/2012
Coll.). That conclusion is worth exploring due to the reason that we found out during the
implementation of professional training for several hundred contracting authorities (years
2005-2014) that the contracting authorities express the opinion that the use of a single
evaluation criterion is more appropriate than the use of multi-criteria evaluation of
tenders.
The contracting authorities consider that the single-criteria evaluation of bids leads to a
greater price difference between the final price and the estimated price. The contracting
authorities argue as follows: based on a rational assumption, the tenderer wants to win
the contest. They therefore choose a strategy that is supposed to bring victory in the
contest. The source for the creation of such a strategy is the available information about
the public contract. With a single-criterion evaluation of bids, the estimated cost of the
public contract is one of the key pieces of information. The candidate then tries to

429
minimize this price. He experiences an increase in hope that he will win the bid. With
multi-criteria evaluations, the state authorities create a situation where the bidder no
longer intensely feels the pressure of having the lowest bid price. He can indeed calculate
other criteria and their weight as well as the implied (expected) relative advantages of
his offer over the offers of others.
They often know their potential competitors in a public tender, supply the contracting
authorities, and from this the applicant assumes that the bidder should prevail over
potential competitors in some of the evaluation criteria. This leads (in the opinion of that
group of authorities), to the fact that bidders do not reduce the price as much as when
there is a single-criterion evaluation.
It infers that the mentioned group of contracting authorities conclude that the applicant
may offer a higher bid price for a multi-criteria evaluation than what they would offer
when the single-criterion evaluation is used. This higher price may reportedly
compensate other sub-criteria (e.g. quality, technical parameters, etc.). It can also
calculate the weights of individual criteria and assess their position when winning the bid.
It is true, that the use of a single evaluation criterion leads to a lower final price?
The difference between the final price and the estimated price can be monitored in
various ways, for example, as a normalized difference (see Nikolovová et al., 2012), or
as FC / EP (final price / estimated price), where the result is a measure of the final price
in relation to the estimated cost (%).
The starting point for the empirical analysis was to analyze the public construction
contracts that were awarded in 2013. Details on these public contracts were obtained
from the website of the Journal of Public Procurement (see
www.vestnikverejnychzakazek.cz). A total of 6 273 contracts 2013 were awarded. During
data collection and evaluation, however, a relatively high error rate in the database
became apparent. Some of the entries lacked some of the following signs of public
procurement: contracting authority, contractor (winning bid), type of award procedure,
the number of bids, the estimated price of the public contract, the final price of a public
contract, evaluation criteria. After adjusting for errors the resulting data set was 5 065
contracts.
3 Results and Discussion
The first analyzed problem was to examine the evaluation criteria. The results of the
used evaluation criteria according to the number of evaluation criteria and the financial
volume of public contracts is shown in Table 1.
Table 1 Usage of evaluation criteria according to the number (%)
and financial volume (%) of public contracts
Economically
The Offer
Advantageous
Price
tender
Number of Contracts 85 15
Volume of Public Contracts 76 24
Source: Journal of Public Procurement. Own analysis.
As seen by examining the data set, a clear majority of public contracts were evaluated
based on one criterion for evaluation, both in terms of Single-criterial ranked
procurement, and in terms of financial volume of public contracts. Single-criterial
evaluated bids, therefore, dominate. The research of Nikolovová et al. (2012), covering
the period 2006-2010, which examines public contracts for supplies, services and
constructions, finds approximate equality in the use of single evaluation criteria and
multiple criteria. The causes of this difference may be caused by the considerable
heterogeneity of the sample, especially when the examination includes all types of public
contracts (supplies, services and construction), as well as sub-limit and above the
threshold contracts. This may influence the choice of evaluation criteria.

430
If the contracting authority chooses the criterion of economically advantageous tender, it
provides sub-criteria which must be given weight. The findings of this criterion are shown
in Table 2.
Table 2 Selection of evaluation criteria (frequency of use and average weight)

Frequency Averaged
Criteria
of use (%) Weight (%)

Offer Price 47.96 80.44

Period of Implementation 34.21 17.15
Length of Guarantee 13.76 14.21
Response Time to Remove 0.98 15.00
Defects
Functional Properties 0.98 29.62
Amount of Guarantee 0.75 10.00
Operating Costs 0.61 10.00
Other 0.75 21.20
Total 100.00 47.16
Source: Journal of public procurement. Own analysis
As is evident from the results of the analysis, when bidding price is used as the
evaluation of procurement criteria, it dominates. This finding is not surprising. This
corresponds to the fact that public resources are scarce and, therefore, limited.
Contracting authorities react to this fact, by using the criterion of bidding price for the
tender evaluation, and by clearly giving it the most weight. In second place, in terms of
frequency of use, was the time of implementation. This finding can be interpreted to
mean that the authorities require that the benefits from realized investments begin to
flow as soon as possible. Surprisingly, however, there was a relatively low weight for this
criterion. This can be explained by the fact that some contracts regarding the time of
implementation of the work were used as two to four criteria and weighted differently
(for periods of implementing individual parts of the procurement). In third place was the
criterion dealing with the length of warranty which as shown, the dominant criteria that
relates to economical and efficient management of resources. On the lower end of the
criteria represented were the ones relating to the usability of the public procurement.
Here we can see, for example, the criterion of "functional properties". Another surprising
finding is that the criterion "quality" was so infrequent that it fell into the "other"
category. We believe that this is due to the fact that the contracting authorities are
unable to use qualitative criteria when evaluating bids, which include the criterion of
"quality". The findings from the training of contracting authorities show that evaluators
can not use the procedures to transform qualitative criteria into something that is
assessable. Therefore, their use is avoided. This can be explained in that public
procurement is dominated by the criteria that are more related to the cost of the public
contract. Those criteria, which reflect the effectiveness of public procurement, are seldom
used. The question arises whether contracting always act in the spirit of the idea of
"rational savings" or whether it is not a "pseudo-savings" (Půček, Ochrana, 2014). This
question, however, is not the subject of our investigation.
Another issue that our analysis focused on was to answer the question of what is the
average number of tenders using single-criterion and multi-criteria evaluation of tenders
and what is the price difference (ie. the price ratio between the final price and the
estimated price). The analysis results are shown in Table 3.

431
 Table 3 Average number of bids and the ratio between the final price
and the estimated price
Average
Average price
Criteria number of
difference
bids
Economically advantageous
6.03 0.7928
tender
Lowest Bidding Price 6.60 0.7656
Total 6.51 0.7697
Source: Journal of public procurement. Own analysis.
As is evident in the case of multi-criteria evaluations of public procurement where price is
only one of the evaluation criteria, the average number of tenders is approximately 0.5
offers smaller than in the case single-criterion assessment, while the ratio between the
final price and the estimated price indicates that the final price reaches approximately
79% of the level anticipated price, thus saving amounts of about one-fifth of the
estimated price. In the case of a single evaluation criterion greater savings are achieved.
The final price amounts to approximately 76% of the expected price level. The achieved
savings amounts to about one quarter of the estimated price. So it seems that with the
lowering of price as the only evaluation criterion, this reduces the number of offers. The
causes may be different. On the supply side, it can affect the behaviour of candidates
due to the multidimensional assessment because the offer must incur greater transaction
costs for the tender preparation as well as having to look for other winning strategies,
because it is not so easy to get a contract with a straight (single) strategy such as
focusing on the lowest bidding prices. The number of bidders may also influence the kind
of competition where non-transparent kinds contracting management may discourage
some potential bidders from participating in public tenders. The lower number of
competitors leads to lower savings in the final price.
This fact was also noted by Pavel (2008) who examined the influence of the number of
suppliers on the price of construction contracts in the field of transport infrastructure
(2004-2007). He believes that it is appropriate for construction work to prefer the
selection criterion of the lowest tender price. We believe that the problem is more
complicated. The criterion of the lowest tender price can be used in buildings that are not
structurally challenging. It is inappropriate to use them where they are relevant to the
assessment of technical specifications, terms, and guarantees. On the contrary,
experience shows that when choosing construction contracts based on the lowest bid
price in the evaluation,that according to this criterion there has been a failure to meet
the principles of economy as well as the incurring of consequent additional costs. But the
fact is that in the multi-criteria evaluation, the bidder with the lowest price may not
obtain the public contract. Another offer that is superior in the other criteria could win,
due to their ability to meet the specified criteria. Obviously, the problem of choice of
evaluation criteria may have different effects on the ultimate effect of public
procurement.
In the following table, we attempt to answer the question: what is the representation of
different types of tender procedures in connection with the use of evaluation criteria. The
research results are shown in Table 4.
The table shows that the criterion of the lowest price tender dominates all kinds of
procurement procedures (except negotiated with a call for tenders), while most public
contracts are awarded in an open procedure.

432
 Table 4 Type of award procedure and evaluation criteria used
(volume of public procurement)

Economically Lowest
Type of award procedure advantageous bidding
tender (%) price (%)
Open 45.64 56.92
Limited 45.98 24.25
Negotiated without
3.07 13.39
publication
Negotiated with a call for
4.22 3.91
tenders
Award the contract without
0.06 1.41
prior publication
Accelerated restricted 0.03 0.10
Accelerated negotiation 0.00 0.02
Others 0.75 21.20
Total 100.00 100,00
Source: Journal of public procurement. Own analysis.

Interesting results were seen compared to the kind of procurement procedure and
evaluation criteria based on the number of contracts (Table 5).
Table 5 Type of award procedure and evaluation criteria used (number of contracts)

Economically Lowest
Type of award procedure advantageous bidding
tender (%) price (%)
Open 65.83 67.13
Negotiated without
13.64 18.43
publication
Negotiated with a call for
10.33 9.02
tenders
Restricted 8.74 3.49
Award the contract without
1.19 1.56
prior publication
Accelerated Restricted 0.26 0.26
Accelerated Negotiation 0.00 0.12
Total 100.00 100.00
Source: Source: Journal of public procurement. Own analysis.

Comparing the results in Table 4 and Table 5 shows significant differences. If we examine
the representation of individual types of procurement procedures and criteria used by the
volume of funds allocated in the individual types of procedures, we see significant
differences in the use of the lowest bid price and economically advantageous tender for
individual types of procurement procedure. One possible explanation is that the volume
of allocated funds is related to the complexity of the public contract. It would therefore
be interesting in further research to explore what influences the contracting authority
when choosing evaluation criteria and the choice of procurement procedures.

433
If we follow the given problem from the perspective of public contracts (table 5), the
differences are minor. So it seems that the decision of candidates to compete is more
related to the type of procurement procedure. The entry of candidates into the public
tender (identified by the number of offers) does not significantly affect whether the
contracting authority provides as the evaluation criterion the lowest bid price or the
economically advantageous tender.
4 Conclusions
An analysis of public contracts for construction awarded in 2013 shows that the
predominant evaluation criterion is the lowest bid price. Also, the lowest bid price carries
the most weight when utilized in the multi-criteria evaluation. Among the sub-criteria (in
terms of frequency of use) then criteria that relate to the principles of prudent and
effective resource management are the most predominant. Criteria that express purpose
and use of a public contract (criterion of "functional characteristics" and the criterion of
"quality") are completely on the sidelines. The causes of this condition should be
explored in further research. This may be due to the fact that authorities purposely avoid
these criteria (especially the criterion of "quality") because, for example, they do not
control the methods of how such criteria are to be evaluated. Not utilizing the criteria
(quality, performance, satisfaction) can lead to the fact that the selection of a public
contract will be of lesser quality and functionality from the selected tender. A singular
focus on the cost criteria (and the underestimation of the criteria of quality and
functionality) can lead to pseudo-savings in public procurement. This will make the
bidder offer the lowest price (or offer the best service completed by cost considerations),
but these are often in a form of exploitation and lead to dysfunction within the system. It
would therefore be useful to examine the price differential (the difference between the
respective estimated price and the final price) in the context of usability and functionality
of public contracts. This economic theory is left out. Another perspective branch
examination is post-contract conduct which plays a crucial role here in information
asymmetry, assets of specificity and moral hazard, e.g. a situation where the contract is
implemented by the private sector, but the ultimate responsibility is borne by the public
sector. The public sector has therefore a weaker bargaining position, which can lead to
having to sign another contract, amendments, or more competitive work. Ultimately, any
savings may be purely virtual. The fact that we achieved savings in public procurement
(lower final cost, compared with an expected value) does not necessarily mean that it
certainly is an economically rational outcome. The path to finding answers may lead to
interdisciplinary (psycho-socio-economic) research that reveals the motives, for instance,
for the choice of evaluation criteria in public tenders.
Acknowledgments
This paper was prepared as an output of the research project P-17 PRVOUK – Programme
of the development of scientific branches at Charles University in Prague “Sciences of
Society, Politics, and Media under the Challenge of the Times”.

References
Bandiera, O., Prat, A., Valletti, T. (2008). Active and Passive Waste in Government
Spending: Evidence from a Policy Experiment. CEIS Tor Vergata. Research Paper Series,
vol. 6(4), No 115. Retrieved from: http://ssm.com/abstract=1115339.
Bolton,P., Dewatripont, M. (2005). Contract Theory. London: MIT Press.
Domberger, S., Rimmer, S. (1994). Competitive Tendering and Contracting in the Public
Sector: A Survey. International Journal of the Economics of Business. 1994. vol. 1. pp.
439–453.
Iimi, A. (2006). Auction Reforms for Effective Official Development Assistance. Review of
Industrial Organization, vol. 28(2), pp. 109-128.

434
Jurčík, R. (2014). Public Procurement in the Field of Public Administration in the Czech
Republic, Selected Aspects. WSEAS Transactions on Business and Economics, vol. 11,
Art. #57, pp. 615-624.
Jurčík, R. (2012). Discussion about the economic and legal aspects of the transparency
and anti-corruption amendment forced from 1 April 2012 and Proposals of new EC
procurement directives. Projednání ekonomických a právních dopadů novelizací velké
transparentní novely účinné od 1. 4. 2012 a návrhů nových zadávacích směrnic
v Evropské unii. Ekonomický časopis, vol. 60(7), pp. 766–768.
Kameník, M. (ed.)., Nikolovová, P., Palguta, J., Pertold, F., Soudek, J., Vozár, M.,
Andrlová, K., Prchal, P. (2011). Otevřenost zadávacích řízení v ČR. Praha: Oživení, o.s.
Kuhlman, J. R., Johnson, S. R. (1983). The Number of Competitors and Bid Prices.
Southern Economic Journal, vol. 50(1), pp. 213–220.
Nemec, J., Mikušová Meričková, B., Grega, M. (2014). Contracting, outsourcing,
procurement: selected factors limiting their success in the CEE region. In: XVIII IRSPM
Conference. Ottawa: Carleton University. Retrieved from:
http://www.irspm2014.com/index.php/irspm2014/IRSPM2014/paper/view/211.
Nikolovová, P., Palguta, J., Pertold, F., Vozár, M. (2012). Veřejné zakázky v ČR. Co říkají
data o chování zadavatelů? (Public Contracts in the Czech Republic. What the Data Say
on Behaviour of Contracting Authorities?). Study No. 5. Praha: Národohospodářský ústav
AV ČR, CERGE. Retrieved from: http://idea.cerge-ei.cz/documents/studie_2012_05.pdf.
Ochrana, F., Stehlík, P. (2014). Competitive Effect in Public Procurement. In: Proceedings
of the 19 th International Conference Theoretical and Practial Aspects of Public Finance
2014. Prague: Wotlters Kluwer, pp. 215-221.
Pavel, J. (2008). Vliv počtu nabízejících na cenu stavební zakázky v oblasti dopravní
infrastruktury 2004-2007. Praha: Transparency international.
Pavel, J. (2010). Analýza vlivu konkurence na cenu rozsáhlých staveb dopravní
infrastruktury. (Analysis of the impact of competition on the price of large-scale
constructions of transport infrastructure). Politická ekonomie: teorie modelování,
aplikace, vol. 3(58), pp. 343–356.
Prager, J. (1994). Contracting Out Government Services: Lessons from the Private
Sector. Public Administrative Review, vol. 54(2), pp. 176–184.
Půček, M., Ochrana, F. (2014). Theory and Practice of the Public Sector Savings: Case of
Czech Regions. Transylvanian Review of Administrative Sciences. No. 42 E/2014.
Soukopová, J., Malý, I. (2013). Competitive environment in waste management and its
impact on municipal expenditures. Acta Universitatis Agriculturae et Silviculturae
Mendelianae Brunensis, vol. 61(4), pp. 173–183.
Strand, I., Ramada, P., Canton. E. et al. (2011). Public Procurement in Europe. Cost and
Effectiveness. European Commission. Retrieved from:
http://ec.europa.eu/internal_market/publicprocurement/docs/modernising_rules/cost-
effectiveness_en.pdf.
OECD (2002). Jens Lundgaard: Competition and Efficiency in Publicly Funded Services.
Paris: OECD Economic Studies No. 35, 2002/2.
Věstník veřejných zakázek (Bulletin of Public Contracts). Retrieved from:
www.vestnikverejnychzakazek.cz.
Zákon č. 137/2006 o veřejných zakázkách ve znění pozdějších předpisů (platné novely z
1.4. 2012). (Act no. 137/2006 Coll., on Public Contracts, as amended (effective revisions
as of April 1, 2012).

435