Translated from German to English - www.onlinedoctranslator.

com

See discussions, stats, and author profiles for this publication at:https://www.researchgate.net/publication/346053222

Measures to increase software quality through continuous integration as

part of DevOps in Austrian SMEs

thesisFebruary 2019
DOI: 10.13140/RG.2.2.29227.67362

CITATIONS READS

0 515

1 author:

Markus Meierhofer

2PUBLICATIONS0CITATIONS

LAKE PROFILE

All content following this page was uploaded byMarkus Meierhoferon 21 November 2020.

The user has requested enhancement of the downloaded file.

 BACHELOR THESIS

MEASURES TO INCREASE SOFTWARE QUALITY THROUGH CONTINUOUS

INTEGRATION AS PART OF DEVOPS IN AUSTRIAN
SMEs

executed on the

on the course
business Informatics

By: Markus Meierhofer

Personal identification number: 1610319050

Graz, on … .............................................................

Signature
DECLARATION ON HONOR

I declare on my honor that I have written this thesis independently and without outside help, have not
used sources other than those indicated, have quoted the sources used verbatim and have marked
passages taken from them as such.

.............................................................

Signature

I
THANKSGIVING

At this point I would like to thank my supervisor, DI Markus Petelinc, for the excellent
support. Thanks to your support, all my questions could be clarified and the work could be
implemented correctly.

I would also like to thank my parents, Andrea and Gerhard, and especially my girlfriend, Nina,
for standing by my side during work and taking on many tasks.

II
SHORT VERSION

The practicecontinuous integrationhas established itself as a development standard in

software companies. However, this practice is often not implemented effectively. The aim of
this work is to present measures that can be implemented to effectively use continuous
integration to increase software quality, with the measures mentioned focusing on Austrian
SMEs. Also, continuous integration is becoming the more recent trendDevOpscompared,
which has similar objectives.

A systematic literature search on the topics of DevOps and Continuous Integration and how
these are related to high software quality serves to answer the question. First, the term
DevOps is defined and an overview of the methods, ideas, and practices on which DevOps is
based is given. After that, the connection between Continuous Integration and DevOps is
described. Furthermore, the components of the implementation of Continuous Integration
are presented and it is clarified how Continuous Integration contributes to achieving high
software quality.

Finally, on the one hand, challenges of Austrian SMEs with regard to high software quality are
researched and analyzed, which of these challenges are mastered by continuous integration.
On the other hand, concrete measures are presented through which continuous integration
can be used to increase quality.

It turns out that at least daily integration of source code changes between developers is an
important criterion for quality improvement. A special focus must also be placed on
automated tests at all levels in order to be able to test complex software effectively.
Ultimately, the quality-enhancing implementation of Continuous Integration also requires a
high level of team discipline in order to also take into account the knowledge gained with
regard to software quality.

III
ABSTRACT

continuous integrationis an established development practice in many software development

companies, but it is often not effectively implemented. The aim of this thesis is to present
measures by which Continuous Integration can be used effectively to raise the software quality in
Austrian small and medium-sized enterprises (SMEs). In addition, Continuous Integration is
compared to the more recent trendDevOps, which has similar goals.

The thesis uses systematic literature research on the topics DevOps and Continuous Integration
and how they are connected to software quality. First, the term DevOps is defined and an
overview over the methods, ideas and practices that DevOps bases on is given. Then the
connection between Continuous Integration and DevOps is defined. Also, the different
components of the implementation of Continuous Integration are shown and how they contribute
to higher software quality.

Finally, challenges of Austrian SMEs regarding software quality are researched and it is analyzed
which of those challenges can be overcome by using Continuous Integration. The thesis concludes
with the presentation of several measures to use Continuous Integration to raise software quality.

The results show that at least daily integration of developers' code changes is crucial to increase
software quality by Continuous Integration. A special focus should also be laid on having
automated tests on all layers to effectively test complex software. Quality increases ultimately also
need a high team discipline to use the insights given by Continuous Integration.

IV
TABLE OF CONTENTS

1 INTRODUCTION ................................................. .................................................. .................................1

1.1 Research question.................................................. .................................................. .............................1

1.2 Method ..................................... .................................................. ..........................................1

2 INTRODUCTION TO DEVOPS ................................................ .................................................. ............3

2.1 Definitions ..................................... .................................................. .......................................3

2.2 Origin.................................................. .................................................. ......................................5
2.3 Basic concepts, methods and tools................................................. .....................................6
2.3.1 Lean ..................................... .................................................. .....................................6
2.3.2 Agile software development........................................ .................................................. ....7
2.3.3 Version management ........................................ .................................................. .............8th
2.3.4 Test-driven development ..................................................... .................................................. .8th

2.4 Cultural shifts in adoption ........................................................ ......................................9

2.4.1 Improving cooperation ........................................................ ........................................9
2.4.2 Continuous improvement................................................. ................................................10
2.4.3 Developing a team culture ........................................ .................................................. ........11

2.5 Objectives ..................................... .................................................. ................................................11

3 CONTINUOUS INTEGRATION (CI)............................................. .................................................. 13

3.1 Definitions ..................................... .................................................. .......................................13

3.2 Distinction from similar practices ..................................................... .................................................. 15
3.2.1 Continuous Delivery/Deployment ........................................ .......................................15
3.2.2 Trunk based development ................................................ .................................................. ..16

3.3 Prerequisites for operating CI........................................................ .......................................17

3.3.1 Version management ........................................ .................................................. ..........17
3.3.2 Automated build process ............................................ ................................................18
3.3.3 Automated tests ..................................................... .................................................. ...........20
3.3.4 Method of continuous integration ........................................ ................................21

3.4 Additional components of a CI pipeline ............................................ .......................................22

3.4.1 Automatic database integration........................................................ .....................................22

V
 3.4.2 Automatic code inspections................................................. .......................................23
3.4.3 Automatic software deployment ................................................ ............................................24

3.5 Advantages of CI with regard to software quality ........................................ .......................................24

4 HIGH SOFTWARE QUALITY OF SMES THROUGH CI .......................................... .....................26

4.1 Characteristics of Austrian (software) SMEs ............................................ .................................26

4.2 Aspects of software quality ........................................ .................................................. .........26
4.3 Challenges of high software quality ............................................ .....................................27
4.4 Which challenges are overcome by CI......................................................... ......................28

5 MEASURES TO INCREASE SOFTWARE QUALITY THROUGH CI........................30

5.1 Daily and rapid integration of changes ........................................................ .......................30

5.2 High degree of automated testing at all levels ........................................................ ............................31
5.3 Useful automated inspections ........................................................ ................................................32
5.4 CI pipeline with multiple phases................................................. .................................................. .....33
5.5 High team discipline ..................................................... .................................................. .......................33

6 CONCLUSION AND OUTLOOK................................................. .................................................. .....35

LIST OF ABBREVIATIONS................................................. .................................................. .............37

LIST OF FIGURES ...................................................... .................................................. ..............38

BIBLIOGRAPHY................................................. .................................................. ..................39

VI
1 INTRODUCTION

The termDevOpshas since its popularization by theDevOpsDaysconference gained a high

importance in the software development industry. Many software development conferences
and magazines have sections on DevOps. Especially software solutions in the area of
Infrastructure as Code,containerizationand cloud computing are advertised here as tools, i.e.
software tools, for the implementation of DevOps. This thesis aims to define the term DevOps
and take a closer look at its components, history and objectives. It clarifies that DevOps does
not depend on specific tools, but represents a broader collection of ideas, practices and tools
with common goals.

Then continuous integration is highlighted as an important part for achieving the goals of
DevOps and important aspects of its implementation are described. The challenges of Austrian
software SMEs in achieving high software quality are also analyzed and it is clarified which of
these challenges can be mastered through continuous integration.

1.1 Research question

The following work is intended to clarify the question of which measures must be taken in
order to use continuous integration to increase the quality of software. The measures
presented are related to Austrian software SMEs and how they can generate advantages in
terms of software quality there. It should also be clarified how Continuous Integration
integrates into the larger context of DevOps and how these topics are related.

1.2 method

The data required to answer the research question were obtained through literature research
in online search engines. With regard to the theory of DevOps and Continuous Integration, the
best-known and most referenced works and articles were researched here and used as a basis
for processing the topics. The challenges of Austrian SMEs in achieving high software quality
were also obtained through internet research. Finally, the benefits of Continuous

1
Integration regarding software quality and effective measures for its implementation obtained
from the included sources.

2
2 INTRODUCTION TO DEVOPS

The concept, which emerged with the DevOps movement in 2009, is now of great importance
in the software development industry. Many companies therefore try to introduce the
underlying concepts and practices themselves. Not just because DevOps is in the software
industryin fashionbut also because its effectiveness has already been proven in studies.

Puppet Labs (2015) provides statistical evidence that implementing DevOps brings business
benefits. For example, companies that have effectively implemented DevOps have 60 times
fewer problems with system failures and resolve those failures 168 times faster.

This chapter is intended to provide an introduction to the DevOps movement and the
motivations, fundamental concepts and tools behind it.

2.1 Definitions

The term DevOps is a portmanteau of the terms "development" and "operations", i.e. the
development departments and IT operations departments of a company. The original goal of
DevOps is to improve collaboration between these departments, but DevOps should also
increase collaboration with other departments. (Davis & Daniels, 2016, p. 14; Verona, 2016, p. 2)

One standardized definition from DevOps is in the technical literature rarely.

Several definitions and explanations view DevOps as primarily cultural change:

Devops is a cultural movement that changes how individuals think about their work,

values the diversity of work done, supports intentional processes that accelerate the

rate by which businesses realize value, and measures the effect of social and
technical

change. It is a way of thinking and a way of working that enables individuals and

organizations to develop and maintain sustainable work practices. It is a cultural

framework for sharing stories and developing empathy, enabling people and teams to

practice their crafts in effective and lasting ways. (Davis & Daniels, 2016, p. 13)

3
Another definition places common goals, processes, and tools between departments at the
center of DevOps considerations:

DevOps is a mix of patterns intended to improve collaboration between development

other

operations. DevOps addresses shared goals and incentives as well as shared

processes

and tools. Because of the natural conflicts among different groups, shared goals and

Incentives may not always be achievable. However, they should at least be aligned
with

one another. (Hüttermann, 2012, p. 8)

In order to implement DevOps, changes must first and foremost be made to the company
culture, with additional practices and tools supporting the goals of DevOps. (Walls, 2013, p. 1)

Several sources also define DevOps in terms of what it isnotis to get a better differentiation
from other movements and frameworks in software development and IT operations. For
example, a separate team should not be used to implement DevOps, which mediates between
developers and IT operations employees. This approach contradicts the basic idea that
departmental thinking in companies has to be reduced and that common goals have to be
worked towards. The now common job designation "DevOps Engineer", which is supposed to
combine the work of development and IT operations in one job, makes just as little sense. This
combination does not work in large companies, since specialization in certain activities
becomes necessary. Also, the main goal of DevOps is not the merging of development and IT
operations activities, but to improve cooperation between these areas of activity. (Davis &
Daniels, 2016, p. 46f.; Hüttermann, 2012, p. 9f.)

Another misconception about implementing DevOps is the assumption that there is a rightway
for the implementation of DevOps. With the introduction of ready-made development practices
and tools, which are also used by pioneers of the DevOps movement, attempts are made to
achieve quick success. However, because a large part of DevOps is about cultural change,
which also involves constant learning, successes cannot simply be reproduced by copying the
practices of others. (Davis & Daniels, 2016, p. 49; Hüttermann, 2012, p. 10f.)

4
Mimicking the processes and practices of other companies can be consideredcargo cult be
designated. A religious movement from Melanesia, which arose after the end of the Second
World War, was originally called the cargo cult. Since the islanders there saw planes landing on
their island during World War II and this valuable and useful cargo (Cargo) brought, the
residents wanted to bring them back after the end of the war by imitating the general
conditions. Airplanes and runways were recreated without understanding why the planes came
back then. The same concept can be transferred to software companies, which adopt the
practices of leading DevOps companies directly without thinking about the underlying benefits
and sense in their own company. (Verona, 2016, p. 6)

2.2 Creation

Since software is no longer directly dependent on hardware, the development and operation of
IT solutions is separated in most companies. Helpful ideas and concepts were not shared
between companies because thesesecretshould be in order to achieve a business advantage
for one's own company. However, this attitude increased the overall complexity of IT systems
in most companies, requiring departments with specialized knowledge in many areas.
Additional departments such as system administration, quality assurance, databases and
network administration were created, which focused on a specific area but were quite isolated
from other departments and therefore did not work towards common goals. (Davis & Daniels,
2016, p. 22f.)

It consists of the different objectives of development and IT operations chronic conflict:

Developers want to bring changes to the user as quickly as possible in order to create benefits
for them. For IT operations, on the other hand, the main goal is to operate stable software and
IT systems with as few failures as possible (Kim, Humble, Debois & Willis, 2016, p. 17f.).

However, until a few years ago, the stability of software was often not compatible with the
implementation of changes and extensions to the software. One of the main activities of IT
operations staff associated with the introduction of new software versions was often the
stabilization of the system after an update. Installing software updates on production systems
often caused critical problems or failures. These issues then had to be fixed by IT operations
staff at problematic times such as nights or weekends (Kim et al., 2016), which only
compounded their reluctance to perform frequent software updates.

5
IT companies recognized this problem and took the first measures to counteract it. In 2009,
Paul Hammond and John Allspaw, the site's development and IT operations managersflickr, the
speech10+ Deploys per Day: Dev and Ops Cooperation at Flickr, which presented how this high
number of deployments, i.e. the installation of software changes in a production environment,
was possible per day (Davis & Daniels, 2016, p. 27f.). Due to the positive reception of the speech
by those present, the software developer Patrick Debois, who also aimed to improve the
cooperation between development and IT operations, organized the conferenceDevOpsDays.
On social networks, subsequent conversations about the conference used the abbreviation
DevOps, giving rise to the word as a term for this movement (Verona, 2016, p. 2).

2.3 Basic concepts, methods and tools

Many of the ideas of the DevOps movement are not new. In some cases, existing concepts
from other sectors were adopted (e.g. lean software development from lean manufacturing),
or concepts were applied to larger areas of software companies (e.g. retrospectives from the
agile software development model Scrum were also used in other areas of the company). An
excerpt of this and its connection to DevOps is presented here. These bases were taken from
Davis and Daniels (2016) and Kim et al. (2016) adopted.

2.3.1 Lean

From the study of the Toyota production system, the concept of the early 1990s emergedlean
production (in approximatelylean production) developed. The main goals of lean production
are to eliminate all types of waste (waste) and maximizing customer benefits. Since the concept
has a generalizable background, it was applied to other areas of companies or companies in
other industries. (Davis & Daniels, 2016)

Out of these efforts is finallyLean software developmentemerged, which is also based on the
same basis as lean production. One of the basic principles of this concept is also the
elimination of all types ofwaste. In terms of software development, waste includes unfinished
code, handovers to other employees, or delays (Poppendieck & Poppendieck, 2007). Reducing
the types of waste mentioned is also an important goal for DevOps. Unfinished code is
understood, for example, as untested code that must not be present in DevOps for the
implementation of automation.

6
2.3.2 Agile software development

The agile software development approach was based on the difficulties with traditional
development models such as the waterfall model or V-model, which put the entire specification
of a software product ahead of the actual development. Through agile development methods
that originate in the methodextreme programming changes to customer requirements should
be integrated better and faster into development and the quality of software should be
increased through practices such as test-driven development. (Davis & Daniels, 2016, p. 24)

A widespread agile development method today is Scrum, which contains a few role definitions
and processes in order to be able to react better to changes in the software development
process. The features to be developed are part ofsprintsdealt with, two to four week cycles in
which the development team plans and implements the changes. In addition, a daily short
meeting calledDaily Scrum held, in which the developers can clarify their status and address
problems. AProduct Ownertakes care of collecting and defining the requirements. Thescrum
masteris responsible for helping the development team to follow the Scrum process.
(Schwaber & Sutherland, 2017)

From the ideas of Extreme Programming, this arose in 2001Agile Manifesto, which combined the
ideas of agile software development in a few sentences:

We create better ways to develop software,

by doing it ourselves and helping others to do it.

Through this activity we have learned to appreciate these values:

Individuals and interactions over processes and tools

Working software more than comprehensive documentation

Cooperation with the customer more than contract negotiation

Responding to change more than following a plan

That is, while we find the values on the right important,

we estimate the values on the left to be higher. (Beck et al., 2001, paras. 1-3)

The DevOps movement mainly starts with “individuals and interactions more than processes
and tools” (Verona, 2016, p. 2). This is because with DevOps the

7
Cooperation is seen as very important and this should not only take place in rigid processes,
but also on an informal basis if necessary.

2.3.3 Version Management

A basis for many modern software practices is version controlversion control system=VCS).
Version management makes it easy to track changes to an application's code base. Every
change to the codebase is reflected in acommitdisplayed, which also contains a message
describing the change and an author. If changes cause problems, the associated commits can
also be easily rolled back. A contiguous codebase is called arepositorydenoted, and within
these different branches of development (branches) exist, in which further development can be
carried out independently. The main development branches, in which all finished changes to
the product should be included, depending on the VCSmainline,trunk, ormastercalled. For the
sake of consistency, references to the main development branch are always denoted below as
mainlinedesignated.

There are currently several implementations for version management. Known VCS are among
othersSVN,GitandMercurial.

According to Davis & Daniels (2016), the connection with DevOps is that, on the one hand,
teams can easily work together through version control, since the same code can be worked on
at the same time and changes can be merged. On the other hand, the structure of versioning
makes it very easy to roll back changes, which supports risk taking as part of a DevOps culture.

2.3.4 Test-Driven Development

Test-Driven Development (TDD) is a programming technique first developed by Kent Beck in

1999 as part of the agile development methodextreme programminghas been described
(Fowler, 2005). According to this technique, the development of code should proceed in three
steps:

1. Tests are written for the implementation to be developed, which initially fail.

2. The implementation of the source code is carried out until all tests are passed
successfully.

3. The structure of the resulting code is reworked to make it easier to understand. At the
end, all tests must be passed successfully again.

8th
The advantages of this technique include, among other things, that the resulting code can be
tested more easily, instead of converting the code for testability only after implementation. In
order to test components more easily, their responsibilities must be clear and the
dependencies of the components must be able to be entered from the outside. Therefore, test-
driven development also promotes good software design in terms of clear responsibilities and
the use ofinversion of control.

With regard to DevOps, test-driven development has the advantage that it can bring the test
coverage of the code and test automation to a high level. This increases the trust in and quality
of software releases, which makes the work of IT operations staff easier and thus improves the
collaboration between IT operations and development.

2.4 Cultural changes in adoption

As can already be seen from the definitions of DevOps, a change in corporate culture is
necessary to implement DevOps. In particular, effective collaboration between departments/
teams needs to be improved through various changes in culture. The culture of teams should
also be defined by values and continuous improvement in the company should be established
as a central principle.

2.4.1 Improving cooperation

One of the main goals to be achieved through the use of DevOps practices is closer
collaboration between developers and IT operations staff. Such an improvement can be
achieved by various approaches.

Communication between development and IT operations in companies can be made more

open, as described by Walls (2013). Communication should not only take place via formal
requests and ticket systems, but in a pragmatic way between the different departments.

An important aspect here is the creation of common goals for development and IT operations.
The goals of these departments are usually contradictory: developers want to make as many
changes and bug fixes as possible, while IT operations employees want to provide the most
stable system possible for customers (Kim et al., 2016). These conflicting objectives can cause
conflicts between departments. By focusing all departments or teams on common goals, such
as a system that is as user-friendly as possible, all employees can

9
contribute to the success of the company. This common definition also represents a good basis
for open communication between departments, since the product and its requirements can be
used to communicate with each other and not individual goals.

It is also essential to establish trust between the different departments of a company. For
example, developers must trust IT operations staff to correctly install and configure the
software system. On the other hand, IT operations staff must trust the developers that they
have carried out and tested their software changes conscientiously. Examples from Davis and
Daniels (2016) show the impact that a lack of trust can have. If a team of server administrators
does not trust other departments and only allows their own team access to the administered
servers, the workflow can be reduced because the team of administrators represents a
bottleneck for changing the server infrastructure.

2.4.2 Continuous Improvement

Continuous improvement is a practice that is an important aspect of the DevOps movement

due to the fundamental concept of Lean Development. Kim et al. (2016) try to establish
continuous learning through a few practices in companies. For example, human error should
not be penalized, but viewed as a process problem orPost Mortems, i.e. analyzes after the
occurrence of problems such as serious system failures.

However, copying practices alone is not effective. The concept of lean development originally
came from the Toyota production system. Here, many companies have tried to copy the
practices gained through years of experience and improvement, as well as Toyota's practice of
continuous improvement, and implement them in the company. However, often only parts
were implemented, meaning that the successes could not be copied (Poppendieck &
Poppendieck, 2007). In order to be able to really replicate the successes, not only one-time
changes and improvements must be made, but a culture of continuous improvement must be
established (Schwarz, 2014). This improvement culture must also be integrated into IT
companies in order to be able to achieve the desired effect in the long term.

Establishing a culture of continuous improvement is not easy to accomplish, as Schwarz (2014)

explains. In many companies, only selective efforts are made to

10
Improvement undertaken, which will be lost again if there is no culture for improvement.
Implementing blanket measures and programs to change culture is not effective. Instead,
there must be a clear target state of the culture and the ideas of the target culture should be
brought closer to people individually in order to gradually reach the target state. These
changes can be implemented most effectively if, from top management down, employees are
gradually convinced of an idea and then pass it on to lower levels.

A culture of continuous improvement cannot develop if, as in many companies, only standards
are defined in order to identify and eliminate deviations. New target states should be defined
again and again so that the different parts of the company can constantly improve. It must be
ensured here that the goals are aligned with a main goal or a vision. Otherwise the company
would develop in different directions, with no long-term success possible. (Black, 2014)

2.4.3 Develop team culture

In order to achieve a good corporate culture, a functioning team culture is necessary as a basis.
In order to be able to deal better with frequent changes in teams and to minimize conflicts,
teams should define common values. An example of common values would be how to deal
with tight project deadlines: should all the required features be implemented, even if they are
in a semi-finished or untested form, or should attention be paid to quality, even if not all
features can then be completed. Team values therefore describe the uniform way teams deal
with situations or their environment. (Davis & Daniels, 2016)

Care should be taken to ensure that team values relate to professional work and not to
external factors. (Davis & Daniels, 2016) describe a negative example of team culture here, in
which prospective new candidates were invited to lunch. Since the team members liked to
drink beer together, some candidates who did not order a beer at lunch were rejected.

2.5 Objectives

Implementing the concepts, practices, and cultural changes that come with DevOps is not
meant to be an end in itself. DevOps can and should bring several business benefits, which Kim
et al. (2016) in the form of study results
11
represent. Those that have implemented a DevOps culture can bring written code to
production more frequently and faster than companies without adopting DevOps practices.
This is reflected in 30x more frequent deployments and 200x shorter ones lead time, i.e. the
time between checking in changes and the time they are delivered. High-performance IT
organizations can do one herelead timereach from a few minutes to hours.

Despite this short period of time to release changes, the quality of the software produced is
also higher. This was reflected in the fact that DevOps-practicing companies were 60 times
more likely than the average to have a successful release.

12
3 CONTINUOUS INTEGRATION (CI)

The practice of continuous integration has become essential in software development in recent
years. Hardly any current software project does not use continuous integration in some form.
In many projects, however, the implementation of continuous integration does not correspond
to the original definition, which will be explained in the next chapter.

Continuous integration is an essential component for one of the goals to be achieved through
DevOps: high speed in the development of new features and at the same time high quality of
the resulting software. Through continuous integration, code changes are merged with
changes made by other developers as often as possible and checked for compilability and
against a series of automated tests. These measures are intended to prevent the merging of
major code changes on the one hand and the implementation of weeks of manual tests at the
end of a project or the release cycle on the other. Again, these CI practices can only be carried
out effectively when collaboration within a team or when multiple teams are working on the
same software product

The following sections aim to provide an overview of various definitions of continuous

integration and how it relates to known similar practices. The requirements for the
implementation of CI should also be listed as well as the typical components of a CIpipelineto
be discribed. Finally, various advantages of continuous integration in terms of software quality
are shown.

3.1 Definitions

Compared to the definitions of DevOps, those of continuous integration are more consistent in
what they say. That's because DevOps is more of acollectionof concepts, practices and tools
with common goals, while continuous integration represents a concrete practice in software
development.

Fowler (2006a) describes continuous integration as follows:

Continuous integration is a software development practice where members of a team

integrate their work frequently, usually each person integrates at least daily - leading
to multiple integrations per day. Each integration is verified by an automated build
(including test) to detect integration errors as quickly as possible. (paragraph 1)

13
Important points here are the frequent (at least daily) integration of code changes per
developer and that each integration runs through an automated build process that also
includes automated tests. These central cornerstones of continuous integration can be found
in many sources (Duvall, Matyas & Glover, 2007; Humble & Farley, 2011).

The sources have the following similarities in the definition:

• The daily integration of the changes must be done on the main development branch of the
VCS, otherwise the changes will only be integrated with a few and not with all developers
(Duvall et al., 2007; Fowler, 2006a; Humble & Farley, 2011).

• A separate PC for the automated build process is necessary to avoid the problem that
certain code changes only work on a certain development environment (Duvall et al., 2007;
Fowler, 2006a).

• The build process should run as quickly as possible in order to be able to receive direct
feedback. Ten minutes is usually given as the maximum value for the duration (Duvall et al.,
2007; Fowler, 2006a; Humble & Farley, 2011).

• Continuous integration does not work automatically with tools like CI Server, but is a
practice that should be lived by all developers. Therefore, continuous integration only
works if all or most developers agree to this practice (Duvall et al., 2007; Humble & Farley,
2011).

Several sources also point to the origin of continuous integration as a practice of the agile
process modelextreme programmingwhich was designed by Kent Beck in 1999 (Fowler, 2006a;
Humble & Farley, 2011).

In practice, it turns out that the definition of continuous integration is not clear for many
companies. Luke and Prince (n.d.) conducted a study in which participants had to provide
definitions for practices such as continuous integration and at the same time answer whether
and how they use these practices in their company. It turned out that most of the participants
could not fully define continuous integration, but the same participants also indicated that they
use this practice. 90% of the participants who do not make their changes in the dailymainline
integrate, but use longer existing branches (for examplefeature branches, in which a new
branch is created in the VCS for each feature), also indicated that they practice continuous
integration. Luke and Prince concluded that many companies equate using a CI server with
practicing continuous integration.

14
3.2 Differentiation from Similar Practices

In addition to the fact that the practice of continuous integration itself cannot be correctly
defined by all companies, similar practices have also emerged in the years since continuous
integration was popularized. These add to the confusion surrounding the term (Luke & Prince,
undated).

On the one hand, the continuous delivery and continuous deployment methods should be
mentioned here. These extend the idea of continuous integration with a component for
continuous deployment in production environments or the possibility of such. On the other
hand, the relatively new term existsTrunk based development, which focuses on the basic idea
of continuous integration.

3.2.1 Continuous Delivery/Deployment

Continuous delivery extends the classic definition of continuous integration, according to which
all changes go through an automated build and test process. When practicing continuous
integration, changes are automatically compiled and automated tests are carried out, but this
automated process is usually followed by a longer manual test process or deployment to
production environments is usually carried out manually by the IT operations team. (Humble &
Farley, 2011)

Continuous delivery is said to have the following results through further automation of the
software development process (Humble & Farley, 2011):

• Manual tests are reduced to a minimum through other types of automated tests, which,
for example, also test the entire system or non-functional requirements such as
capacity and response time. Additionally, testers should apply code changes to their
test environment after aPullDeploy the principle yourself.

• IT operations staff do not have to go through complicated, error-prone manual steps to

complete a software deployment, but can deploy applications to a production
environment with the click of a button.

Overall, these changes should lead to almost all steps in the software development process
being automated. Due to the fact that deployments are carried out more frequently compared
to the rare manual execution, these processes are tested more frequently and contain fewer
errors. Humble and Farley (2011, p. 26) call this principle "If It Hurts, Do It More Frequently, and
Bring the Pain Forward".

In contrast to continuous delivery, continuous deployment not only prepares every code
change so that it can be easily deployed in different environments. At

15
Practicing Continuous Deployment, every code change is automatically deployed in a
production environment. Therefore, in this sensecontinuous releasea more precise description
of this practice. (Humble, 2010)

Continuous deployment cannot be practiced for every software project. For example, this is not
possible with embedded systems. In the case of highly critical applications, a manual
assessment should also be carried out despite all automated tests in order to keep the system
as stable as possible. One advantage of continuous deployment, however, is that it is
convenient for developers to automatically deploy all changes to the production environment
forcedare able to write very good automated tests on all levels (unit tests, component tests,
system tests, ...). At the same time, with such a mechanism, in the presence of a simple rollback
option, the risk of a release is reduced to a single change. (Humble & Farley, 2011)

Continuous delivery and continuous deployment both build directly on the practice of
continuous integration. Both aim not only to automate the build and test of each code change,
but also to automate the deployment of the new software version. Continuous deployment
goes one step further and also carries out the software release for each individual code
change.

3.2.2 Trunk Based Development

Trunk Based Development attempts to make the original ideas of Continuous Integration more
understandable and visible (Hammant, n.d.):

• The developers should move all changes frequently to themainlinecommit to the VCS.

• The functionality of all changes must be confirmed by tests.

• Developers have confidence in the viability of all code changes because commits that
fail the build are automatically rolled back.

Fowler (2017) describes the problem that many misunderstand the term continuous
integration by asking three questions (this idea originally came from Jez Humble) during
lectures:

• Do all developers push at least daily on themainline?

• Does each commit trigger an automated build and test process?

• If the build fails, is it typically up and running again within 10 minutes?

16
After asking these questions, there are usually only a few participants left in a group who
believe they are practicing Continuous Integration. Fowler (2006b) mentions this problem
Semantic Diffusion: A term with an originally precise meaning becomes widespread and
popular. As many use the term without referencing the original authors, the meaning of the
term becomes vague and the term more difficult to use.

The problem that continuous integration is not properly understood has led to the emergence
of the term Trunk Based Development. This should clarify the basic idea of continuous
integration with a new term. It could be the same here however likewise with popularization of
the termSemantic Diffusionoccur, which would repeat the problem. (Fowler, 2017)

3.3 Requirements for running CI

In order to be able to use continuous integration in the company, certain practices are
necessary that must be applied correctly. Among other things, a version control system must
be used, software builds must be automated and relatively quick, and there must be an option
for integration, either using a separate PC or through a CI server.

3.3.1 Version Management

A VCS is necessary to enable developers to work together on a common code base in a

meaningful way. In order to operate continuous integration, it is important here that all
artifacts required for the build process are checked into the VCS (Fowler, 2006a). If files
required for the build are not checked in, integrating changes to those files cannot work.
Exceptions are certain system environment files that never or very rarely change, such as the
Java environment or the operating system environment.

Another critical point is the frequency of checking in changes to the main development branch,
which should happen once a day per developer on average. This practice is the only way to
ensure that changes are constantly integrated with changes made by other developers.
(Fowler, 2017)

A common problem in many continuous integration implementations is the use of long-lived

feature branches. These are often not included in themainlineintegrated until a customer
feature has been implemented. As a result, changes are only integrated after days or weeks,
which greatly reduces the advantages of continuous integration.

17
However, customer features should not be visible to customers until they have been fully
implemented. Because half-implemented features would suffer user experience, feature
branches were often used as a solution. (Fowler, 2009)

On the one hand, this problem can be circumvented by making the changesin the background,
For example, in the business logic and data model, for which users are not visible anyway and
the UI changes can often be made in one commit. On the other hand, with complex new
featuresFeature togglesbe used,who deactivate them in the productive application until they
are fully implemented. (Fowler, 2009)

3.3.2 Automated Build Process

One of the fundamental advantages of Continuous Integration is that it ensures that

applications can always be compiled, the process of compiling being mostly described as abuild
referred to as. Additional steps such as running automated tests are often performed as part of
a build, but in its basic form, source code is compiled. A build tool is required to carry out a
build consistently and independently of user interaction. (Duvall et al., 2007)

3.3.2.1. build tools

Many build tools for the most popular programming languages are now available here.
Examples are NAnt or MSBuild for .NET and Ant, Maven or Gradle for Java. In the case of Java,
Ant has been the dominant build tool for years. Ant builds are broken down by a series of tasks
specified in XML documents which represent tasks to be performed as part of the build. Tasks
for simple tasks like copying files or compiling source code are already available. However,
building with Ant has several problems. For example, build scripts must be written in XML,
which often makes large files difficult to read. Also often has toboiler plate code, i.e. the same
code modules that are repeated over and over again, can be written in order to handle simple
tasks such as creating JARs (Java archives, executable Java files) or running automated tests.
(Humble & Farley, 2011)

Maven was developed as a solution to many of Ant's problems. On the one hand, this build tool
tries toboiler plate codeavoided by specifying certain conventions. This ensures that little
configuration is required for projects that comply with the standard specifications. On the
other hand, Maven was also the first Java build tool to support automatic dependency
management. These can be specified with Maven

18
and, if necessary, automatically from centralrepositoriesbe downloaded. (Humble & Farley,
2011)

Figure 1: POM file of a Maven project (own figure)

Figure 1 shows an excerpt from the configuration of a simple Maven project, which is called
POM (Project Object Model). Projects in Maven are also described in XML, but the associated
files are based on the principle ofconvention over configurationmostly shorter than build
scripts in Ant. Dependent projects can just be in thedependenciesare specified and will be
downloaded if they are not available locally. Simple projects do not require additional
configuration for many standard Java application building tasks. Among other things, the
execution of automated tests, the creation of JARs or the uploading of the created JARs to a
central repository are supported.

3.3.2.2. Important implementation aspects

Running automated builds should be possible with just one command, for example mvn clean
installwhen using Maven. According to Duvall et al. (2007) ensures that all files necessary for
the build are directly available and makes it easier to perform an automated build via a CI
server. In addition, this ensures that the build always runs the same way, which increases its
stability.

It's also important that the build and test process is kept as short as possible. Humble and
Farley (2011) cite several consequences of a build that is too long. For example, developers no
longer run all tests before committing changes, leading to

19
leads to more frequent failed builds on the integration PC. Changes are also checked in less
frequently, since each check-in requires a long wait for the build process to be carried out.

3.3.3 Automated Tests

Running automated tests is an important part of applying continuous integration. When

changes are frequently integrated between developers, without automated testing, it can
happen that changes, while compilable and working for themselves, affect other parts of the
system. On the other hand, a complete manual test of the application by developers is not
possible with multiple daily commits. Automated tests that ensure the functional correctness of
the application when performing each integration build help here.

In this context is fromself testing codethe speech, i.e. source code that tests itself. Practices like
TDD help with thatself testing codeto produce, since the tests are already written before the
implementation. To facilitate the use of TDD, several frameworks for buildingunit testing, i.e.
tests of individual code components, such as JUnit for Java or NUnit for .NET. (Fowler, 2006a)

JUnit was the first popular implementation of the one designed by Kent BeckxUnitconcept
(Fowler, 2006c). With JUnit, automated tests can be easily written in code and launched either
from the development environment (IDE) or from popular Java build tools like Maven or Gradle.
Figure 2 shows the result of a JUnit test performed by Maven. The number of successful and
failed tests is displayed. In the case of failed tests, these are also stated explicitly. By default,
the Maven build also fails if individual tests fail.

20
Figure 2: Execution of JUnit tests in Maven (own figure)

Unit tests are often not enough to ensure the functionality of an entire application. On the one
hand, these only test individual classes, for example, and not the interaction between them, on
the other hand, parts of the application such as the graphical user interface (GUI) can be
difficult to test with unit tests. Because of this, frameworks forEnd-To-End Tests(E2E Tests)
developed, with the help of which tests can be carried out on the entire application.

Such E2E tests can also be used to automate acceptance tests. Acceptance tests test the
desired functionality of an application from the user's point of view and additional non-
functional requirements such as capacity, stability or performance. By additionally running
such tests, the confidence in the correct functionality of the entire application can be further
increased without performing manual tests. (Humble, 2010)

3.3.4 Continuous integration method

The central idea of continuous integration is that with every commit to the VCS, an automatic
build process is run to check the integration capability of the changes. This build process can
either be triggered manually by developers or automatically by a CI server. It is important that
a separate PC is always used for the integration in order to guarantee a successful build not
only on one development environment (Duvall et al., 2007).

Shore (2006) uses a simple procedure to show that continuous integration can also be
operated without a CI server. A discarded computer can serve as an integration PC where the
source code can be checked out. In addition, some kind of physical token is required so that it
is clear in the team who is currently integrating changes. If a developer
21
Once the changes have been tested locally, he/she gets the token, checks out the changes on
the integration PC and carries out an automatic software build there. If the integration was
successful, the token can be returned. Otherwise, an attempt can be made to rectify the error
within a short time and otherwise the changes can be undone.

CI Server facilitates this process by automatically starting a build of the application with each
commit to the main development branch. CI Server can also provide a graphical overview of
the result of a build and send this, for example, by email to the author of the respective
changes.

3.4 Additional components of a CI pipeline

With many CI servers, all activities that are carried out as part of an integration build are
referred to as a CI pipeline. In advanced applications, this includes not only compiling the
source code and running automated tests, but also other steps. Among other things, database
changes can be integrated, the source code can be checked for various metrics or automated
software deployments can be carried out in test environments, for example. Figure 3 shows the
visualization of a CI pipeline that includes these steps.

compilation database integration Testing

code inspections software deployment

Figure 3: Representation of a CI pipeline (based on Duvall, Matyas & Glover, 2007)

3.4.1 Automatic database integration

In many software company work software developers and

Database administrators (DBA) separately. When developers need to make changes to the
database structure, they need a DBA to do it, which is a bottleneck in software development.
Also, database scripts are often not managed centrally and only shared database environments
are made available, which means that changes can be difficult to make because the shared
database environment for

22
other developers becomes temporarily unusable. Automatic database integration aims to
minimize these problems. (Duvall et al., 2007)

A first step is to manage the database scripts in a VCS repository to make them easily available
within the build process and to make the changes easier to share and manage. This means that
these scripts can also be run and tested automatically in an integration build to ensure the
functionality of the database with every change. This automation in turn makes it easy to
create a database on a development computer, with which database changes can be tested
locally and shared databases are often superfluous. This also allows developers to make direct
changes to the database structure. (Duvall et al., 2007)

This automation also relieves the burden on the DBA, leaving more time for higher-level
database administration activities such as normalization or performance optimization (Duvall
et al., 2007). Automated database integration can therefore make the development of software
systems with database connection more agile and also increase the quality of the database
through automated tests of this and a higher focus on a good data structure through
normalization, for example.

3.4.2 Automatic Code Inspections

Performing manual code reviews for source code changes is now common in software
development. The changes are proofread by an uninvolved third party to ensure their accuracy
and quality. Aspects such as compliance with company coding guidelines and standard
programming language guidelines are also checked, but this can be error-prone manually. For
this and other reasons, automatic code inspections can promote high software quality and
more efficient development. (Duvall et al., 2007)

An important metric that code inspections can provide is the number of lines of source code
per method. Methods that have a large number of lines are often more nested and therefore
more error-prone. Metrics such as the amount of duplicated code and test coverage can also
be calculated automatically. Furthermore, guidelines such as the meaningful naming of
methods and variables can be checked by means of automatic code inspections, making these
checks in manual reviews superfluous. (Duvall et al., 2007)

Common inspection tools can also be easily added to an integration build. Integrations with
well-known build tools like Ant, Maven, Gradle or NAnt are supported by the

23
Most inspection tools are made available, allowing inspections to be performed on automatic
builds. (Duvall et al., 2007)

3.4.3 Automatic Software Deployment

If all other components of a CI pipeline have been implemented, i.e. automatic builds, tests,
database integrations and code inspections, it is theoretically possible to guarantee the
functionality of the software at all times and therefore to deploy it in a production environment
at any time (Duvall et al ., 2007). As already mentioned in chapter 3.2.1, this practice is called
continuous deployment. Continuous integration provides a basis for continuous deployment
here, since the software, together with the database, is already generated and tested
automatically.

The automation of deployments is therefore just an additional step in the CI pipeline. Since a
complete application must be installed when carrying out automatic E2E tests as part of
continuous integration (Duvall et al., 2007), continuous integration in this already provide an
option for automated deployment in test environments. This raises the question of how many
system levels the software and all its dependencies have to be reinstalled on (Duvall et al.,
2007). In test environments, it is usually sufficient to just reinstall the software. If the software
is heavily dependent on other components or the operating system, a deployment may require
reinstalling those components as well.

3.5 Benefits of CI in terms of software quality

Software quality can be viewed from two sides. On the one hand, it can be described as the
quality of software for customers. It is expressed how well the customer needs are met with
the software and that the system has as few errors as possibleexternalsoftware quality. On the
other hand, the quality of a software can also be understood as the internal structure of the
source code. This includes, among other things, the comprehensibility of the source code and
how well it follows a specified architecture, for example to make maintenance easier and
minimize the potential for errors. This kind of software quality can be calledinternal software
qualitybe designated. The implementation of continuous integration can increase external and
internal software quality in equal measure.

A quality advantage of using continuous integration is the fact that code changes are
constantly being made in themainlinecan be integrated, allowing for early detection and
resolution of problems (Duvall et al., 2007). On the one hand, at a high
24
Test coverage most issues already detected during check-in of the changes and the following
integration build. If the problems are not caught by automatic tests, developers work with this
change from check-in and there is a high probability that other developers will detect and fix
the problems early. Together, these aspects of continuous integration can help ensure that
code change issues are fixed during the development of a release, shortening manual testing
phases before a software release and increasing external software quality.

Through the practice of continuous integration, many repetitive tasks in software

development, such as build and test, can be automated, which reduces errors in their
execution (Duvall et al., 2007). An automatic build ensures that this is always done in the same
way and thus the quality of the software build is consistently at the same level. It must also be
possible to deploy the application automatically in order to carry out more complex automated
tests such as acceptance tests (Fowler, 2006a). This creates a basis for automated deployment,
even in production environments, which means that this step, which is often tricky with
software, achieves consistent quality and also increases the external software quality.

If a simple CI pipeline has already been introduced in a software project, it is easy to expand it
with further steps. Automated inspections and checks on the architecture can be carried out to
increase the internal software quality. Introducing a test coverage metric can ensure that the
test coverage of the entire application does not degrade as code changes. Since a clear
interface and a good design are often necessary for the testability of software components
(Fowler, 2005), this also increases the maintainability and changeability of the software.
Because all automated tests are carried out when changes are made, regressions are
prevented,

25
4 HIGH SOFTWARE QUALITY FOR SMES THROUGH CI

This chapter is intended to show what contribution Continuous Integration makes to achieving
high software quality in Austrian SMEs when used effectively. On the one hand, relevant
characteristics of Austrian SMEs are described in general and specifically in the software area.
On the other hand, software quality is separated into its different aspects and what challenges
there are in achieving them. Finally, it is clarified which of these challenges, especially in
Austrian SMEs, can be better managed through continuous integration.

4.1 Characteristics of Austrian (software) SMEs

SMEs make up the majority of companies across the EU and also in Austria. 99.7% of all
Austrian companies are SMEs, employing 67.7% of the workforce. Regarding the size of the
SMEs, 87% are micro-enterprises with fewer than ten employees and 36% of them are sole
proprietorships. 5.6% of SMEs are active in the field of information and communication, ie
Software SME. (Neubauer & Zoder, 2016)

The majority of SMEs are therefore micro-enterprises with 2 to 9 employees, accounting for
51% of all SMEs. With a company of this size, human and usually also financial resources are
limited (Neubauer & Zoder, 2016). On the other hand, according to the EU Commission, there
will be a shortage of 900,000 skilled workers in the field of ICT (information and communication
technology) by 2020, and Austria in particular has a shortage of people with ICT skills in an EU
comparison (Neubauer & Zoder, 2016). As a result, software SMEs also have difficulties in hiring
additional specialist staff if necessary.

4.2 Aspects of Software Quality

Software quality as a whole is difficult to define, but it can generally be broken down into the
following aspects (Perry, 1992):

• Correctness: The software meets the functional requirements of the customers.

• Reliability: The software has constant and error-free processes.

• Efficiency: The software uses as few resources as possible in its processes.

• Integrity: The software is protected against unauthorized access.

26
 • Ease of use: It is as easy as possible for users to interact with the software.

• Maintainability and flexibility: On the one hand, it is easy to find and fix errors in the
software and, on the other hand, it is easy to adapt the software to changing
requirements.

• Testability: The software can be easily tested for correct functionality.

• Portability: The software can be easily used on different hardware systems and the
effort involved in porting is low.

• Reusability: The software can easily be used within other applications.

• Interoperability: Communication between the software and other applications is simple

and standardized.

4.3 Challenges of high software quality

When developing software, whether as part of a project or continuous product development,

there is usually time pressure. Features must be completed within specified deadlines, as these
are of great benefit to customers and must be launched in their own software before the
competing products. Among other things, this is a factor that favors low internal software
quality and thustechnical debt elevated. (Kim et al., 2016)

Source code with technical debt is such where refactorings are not performed and the
structure and complexity of the source code is therefore increasingly difficult to understand.
These factors contribute to reduced maintainability and flexibility, but also reliability and
testability.

A common strategy to increase software quality is to perform extensive manual testing, but
this has several problems. It is often difficult to test the quality of a product afterwards (Perry,
1992). If high quality is only to be generated in the manual test phase, which is usually at the
end of a development cycle, the necessary changes and repetition of the tests will further delay
the release of the software and the costs for the release will increase (Humble & Farley, 2011) ,
which speaks against neglecting internal software quality when deadlines are tight.

27
The more complex software becomes, the more time-consuming manual testing of the
software becomes. In addition, beyond a certain level of complexity, it is no longer possible to
test all possible configuration options and processes at very high cost. However, changes to
parts of the code can destroy unforeseen other processes. In combination, this has the
consequence that some processes are no longer functional after changes, were not tested
manually due to the complexity of the software and errors with these processes only become
visible when updates are delivered to customers (Humble & Farley, 2011). These problems thus
degrade the correctness and reliability of software.

For Austrian SMEs, the challenges mentioned are particularly difficult to overcome. Since
human resources are limited here, as explained in Chapter 4.1, additional personnel cannot
simply be assigned to critical projects when deadlines are tight. The shortage of ICT specialists
in Austria increases this problem as new staff is harder to find. As a result, software quality
suffers under time pressure with changes, especially in SMEs. Since manual tests cause very
high costs with increasing software quality (Humble & Farley, 2011) and, as explained in
Chapter 4.1, the financial resources of SMEs are often lower, the extensive implementation of
manual tests is a challenge for SMEs in particular.

4.4 Which challenges are overcome by CI

Continuous integration offers the possibility of integrating software quality early in the
development cycle. For each code change checked into the VCS, automated tests for the
correctness of the software and checks for compliance with the software architecture and
programming guidelines are carried out, among other things. These measures ensure that
technical debt cannot build up. Adhering to programming guidelines and software architecture
guidelines is also easier and less time-consuming if you don't have to pay attention to it
manually, but instead constantly check tools such as CheckStyle or Sonarqube for compliance
with the guidelines (Duvall et al., 2007). As a result, a certain quality standard of the code can
be maintained even in phases with high time pressure, or it is clear which areas are affected
and need to be improved afterwards. Continuous integration can therefore ensure the
maintainability and flexibility of software.

Software with a good design and no technical debt can also be better automated, for example
automated acceptance tests can be implemented more easily (Humble & Farley, 2011). From
this it can be concluded that the correct application of

28
Continuous integration also increases the automated testability of software. By regularly
running automated tests and constantly integrating changes, errors are found and fixed faster.
This early correction of errors in turn shortens manual test phases, since fewer errors occur
and need to be corrected.

Automated tests, which are produced when continuous integration is used correctly, in
connection with automated software inspections, ensure a high test coverage of the code. In
turn, the complexity of software becomes manageable through high test coverage, since all
possible processes in the software are tested. Through these measures, Continuous
Integration ensures a high degree of correctness and reliability of the software.

Especially for software SMEs, manual tests are a major burden due to a lack of personnel, as
explained in Chapter 4.3. Either release cycles become longer due to the volume of tests to be
carried out and the company loses competitiveness, or the tests are not carried out extensively
enough, which causes problems for customers. In Austrian SMEs in particular, hiring new
employees in the ICT sector is difficult (Neubauer & Zoder, 2016). After the initial effort for the
introduction, continuous integration can keep the software quality at a high level through
constant checks.

29
5 MEASURES TO INCREASE THE SOFTWARE
QUALITY BY CI

As Luke and Prince (oD) describe in their study, the definition of continuous integration is not
clear for many companies. Most of the participants understood this to mean the use of a CI
server. However, the need for daily commits in themainlinenot seen by 90% of respondents.
Fowler (2017) also describes that most developers do not carry out these essential components
of continuous integration, such as daily commits and the automated triggering of a build with
each commit, although they claim to practice continuous integration. However, the benefits of
continuous integration in terms of software quality cannot be realized if these principles are
not followed. This chapter is intended to provide an overview of effective measures through
which continuous integration can be used to increase software quality.

5.1 Daily and rapid integration of changes

As explained in chapter 3.3.1, an essential practice of continuous integration is the daily

committing of changes by developers to a common onemainline. This is the only way to ensure
that changes work together with changes made by other developers. In addition, it makes it
easier to find bugs by constantly testing the changes by other developers, which was described
in chapter 3.5.

However, many software enhancements are not fully implemented within a day. Therefore,
when using continuous integration, the required changes must be spread over several days.
With each commit, it must be ensured that the mainlineremains in working condition. The
changes to the source code should be incremental and after each increment they can then be
committed. (Humble & Farley, 2011)

In order to be able to use continuous integration sensibly and to make it easier for developers
to deal with, attention must be paid to keeping the integration build as short as possible in
order to prevent consequences such as those described in Chapter 3.3.2.2. If a build takes
longer than the required limit, changes can be made to the build process to make it faster.

30
Duvall et al. (2007) suggest several tactics to decrease build times. Simple improvements are
the independent integration PC, which is often necessary anyway for effective continuous
integration, and its hardware upgrade. If the automated tests or inspections take a long time,
measures can be taken to improve their performance. If a system consists of multiple
components and changes to one component do not affect other components, separate
integration builds can be performed for each component individually, which can greatly reduce
the build time for a project. Finally, with many modern CI servers, several PCs can be used
simultaneously for the integration and parts of the CI pipeline can also be executed in parallel,

5.2 High level of automated testing at all levels

In chapter 3.3.3 the execution of automated tests was mentioned as a central part of the
execution of continuous integration. In order to achieve high software quality, it is important
not only to introduce unit tests, but also component tests and E2E tests. If it is ensured that all
classes of a code base work independently, but not the functionality of the overall system,
many manual tests are still necessary.

When performing unit tests, care must be taken to ensure that they really only test one code
unit and that dependent code parts are replaced by pseudo-implementations (Duvall et al.,
2007). Practices like TDD ensure easy unit testability by writing the tests before implementation
and knowing how the unit will be tested. The use of TDD in the context of continuous
integration is generally desirable, but not absolutely necessary (Fowler, 2006a). Unit tests that
test larger units with their dependent code parts should also be continuously written and
executed by developers.

In any case, high test coverage of the source code is a key task for achieving high software
quality. Automated inspection tools can calculate the test coverage of the code base as part of
an integration build, making it easy for software developers to trace their development. If
negative trends are recognizable, developers can intervene manually and add test cases for
poorly tested code units. On the other hand, the integration build can also be configured in
such a way that it fails if the test coverage falls below a limit,

31
which forces developers to test their changes automatically. (Humble & Farley, 2011)

In addition todeveloper-orientedTests such as unit and component tests must also be carried
out to achieve high software qualitycustomizedAutomated acceptance tests are written, which
was also mentioned in chapter 3.3.3. These are often not implemented in companies because
their implementation can be very expensive. This is the case, for example, when acceptance
tests interact directly with the GUI and therefore fail on small changes to the GUI. However,
when acceptance testing is aimed at an abstraction layer above the GUI, it can be robust and
cost less than manual testing. (Humble & Farley, 2011)

If the GUI layer is only responsible for display, as in many modern applications, tests against
theApplication Driverimplemented layer below, with which the GUI also communicates. For
example, this can be an abstraction of a REST interface through which the business logic is
available. The acceptance tests should be derived directly from the acceptance criteria of the
requirements, for example from user stories in agile process models. On the one hand, there is
the possibility to describe acceptance criteria in a directly executable way. Tools such as
Cucumber or Twist describe the acceptance tests in the most legible form possible, which is
easy to learn even for non-developers. These acceptance tests are then linked directly to test
implementations such as in JUnit. (Humble & Farley, 2011)

In this form of representation, however, the descriptions of the acceptance tests must be kept
in sync with the implementations, which represents an additional effort. If the testers in a
software project are familiar with programming or the tests are written by the developers,
xUnit frameworks such as JUnit can also be used directly. This has the advantage that the
additional effort for the specification of the acceptance tests in tools like Cucumber is not
required. However, specification documents that are difficult to read can be created from such
test implementations. (Humble & Farley, 2011)

5.3 Useful Automated Inspections

As described in Chapter 3.4.2, automated inspections can also be run as part of an integration
build. In addition to the test coverage inspection described above, checks for the number of
duplicated code or the length of methods for monitoring the internal software quality also
contribute. The metrics obtained from this should be checked regularly, and problematic code
sections analyzed and, if necessary

32
be improved. Often the knowledge of problematic code points represents an improvement
compared to no metrics, since special care can be taken when changes are made at these
points.

5.4 CI pipeline with multiple stages

Within a CI pipeline, many steps can be automated, such as tests at different levels, inspections
or database integration. However, parts such as acceptance tests or inspections often cannot
be performed at an acceptable speed for rapid feedback. In order to keep the time for
feedback on changes low and at the same time to preserve the quality gains for software
through additional automated steps, the integration builds can be divided into several phases
with different speeds (Fowler, 2006a).

A common way to split up integration builds is a two-phase approach, such as that described
by Duvall et al. (2007) or Humble and Farley (2011). In a first phase, the application is compiled
and simple unit tests are carried out. Even simple inspections, such as compliance with code
guidelines, can be run there to fail the build and provide feedback to the developer in the event
of non-compliance. A second phase then carries out more extensive tasks such as automated
acceptance tests or complex inspections for, for example, architectural conformity. The second
phase is performed either for each commit after the first phase has been successful, or due to
the long duration at regular intervals or at night.

5.5 High team discipline

Many aspects of continuous integration such as writing and maintaining automated tests or
incorporating feedback from automated inspections cannot be done once, but require ongoing
attention from developers. For example, the performance of the integration builds must always
be taken into account, since the source code is usually larger and the tests as part of CI are
more extensive. From these facts it can be concluded that many parts of continuous
integration require a high level of discipline from team members in their maintenance, as also
stated by Humble and Farley (2011).

Aspects of continuous integration can be enforced by CI advocates on a team with the support
of CI servers, such as failure of integration builds due to insufficient test coverage or non-
compliance with code guidelines. Around

33
However, in order to be able to implement continuous integration in a target-oriented manner,
the consent of all people involved is necessary (Duvall et al., 2007). This is the only way to
achieve the positive effects on software quality. In order to maintain a quality-enhancing CI
pipeline, it must also be constantly improved, which requires a culture of continuous
improvement, as described in Chapter 2.4.2.

34
6 CONCLUSION AND OUTLOOK

As explained in Chapter 2, the DevOps movement represents a collection of cultural changes,

practices and tools. These are intended to increase collaboration between developers and IT
operations staff, which makes software deployable more often and with higher security and
quality. The definitions of DevOps in different sources are often different. On the other hand,
continuous integration is a clearly defined practice that aims to improve collaboration between
developers on a common code base and increase software quality.

Continuous integration can be seen as practice in the implementation of DevOps, since

continuous integration focuses on the rapid integration of changes while constantly
maintaining quality. This is a necessary basis to ensure the frequent and rapid release
capability of changes, which is seen as the goal of DevOps. The terms Continuous Delivery and
Continuous Deployment expand Continuous Integration to ensure that applications are always
releasable, for example by focusing more on automated, customer-oriented tests and the
software development process beyond the boundaries of the development team. DevOps adds
cultural aspects to these approaches, which are necessary to implement such changes within
teams and departments.

Continuous integration is used by most companies, but often not according to its actual
definition. The same problem with DevOps and continuous integration is that many companies
understand the use of certain tools as meaning, but this does not guarantee the successful use
of these practices (Luke & Prince, n.d.; Verona, 2016). However, continuous integration can
make a significant contribution to software quality if the measures described in Chapter 5 are
observed.

Austrian SMEs in particular can make good use of the quality advantages of continuous
integration, since, as explained in Chapter 4.1, they often have fewer human and financial
resources at their disposal, so manual tests are more difficult to carry out here. By constantly
ensuring high software quality, the duration and costs of manual test cycles can be reduced.

The conclusions presented regarding the quality-enhancing measures of continuous

integration and how these are advantageous for Austrian software SMEs are based exclusively
on literature research. A quantitative study could start here, which measures the effectiveness
of continuous integration with regard to software quality in several Austrian SMEs. On the
other hand, a case study of an Austrian SME can also be created, in which individual measures
to achieve high software quality through Continuous

35
Integration is implemented and metrics are created to measure the software quality before
and after the implementation of these measures. After that, a hypothesis test using the data
obtained can be used to check whether a significant improvement in software quality can be
identified.

36
List of abbreviations

LIST OF ABBREVIATIONS

CI… continuous integration

VCS… Version control system Designations for the main
mainline, trunk, master... development branch of a VCS Test-Driven Development
TDD...
IDE... Integrated development environment Graphical User Interface
GUI…
SMEs… Small and Medium Business Information and
ICT... Communication Technology Database
DBA... Administrator
E2E tests… End-To-End Testing

37
List of Figures

LIST OF FIGURES

Figure 1: POM file of a Maven project (own figure) ........................................ .................... 19

Figure 2: Execution of JUnit tests in Maven (own figure) ........................................ .............. 21
Figure 3: Representation of a CI pipeline (based on Duvall, Matyas & Glover, 2007) .................. 22

38
bibliography

BIBLIOGRAPHY

Beck K, Martin RC, Fowler M, Cunningham W, Cockburn A, Beedle M et al. (2001).Manifesto for
Agile software development. Accessed on December 13, 2018 at

http://agilemanifesto.org/iso/de/manifesto.html

Davis, J & Daniels, K (2016).Effective DevOps. Building a culture of collaboration, affinity and tooling
at Scale.Sebastopol, USA: O'Reilly.

Duvall, P.M., Matyas, S. & Glover, A. (2007).continuous integration. Improving Software Quality and
reducing risk.Boston, USA: Pearson Education.

Fowler, M. (2005).Test-Driven Development. Accessed on 01/07/2019 at

https://www.martinfowler.com/bliki/TestDrivenDevelopment.html

Fowler, M. (2006a).continuous integration. Accessed on 1/12/2019 at

https://martinfowler.com/articles/continuousIntegration.html

Fowler, M. (2006b).Semantic Diffusion. Accessed on 1/12/2019 at

https://martinfowler.com/bliki/SemanticDiffusion.html

Fowler, M. (2006c).Xunit. Accessed on January 17, 2019 at https://www.martinfowler.com/bliki/Xunit.html

Fowler, M. (2009).FeatureBranch. Accessed on 2/3/2019 at

https://www.martinfowler.com/bliki/FeatureBranch.html

Fowler, M. (2017).Continuous Integration Certification. Accessed on January 13, 2019 at

https://martinfowler.com/bliki/ContinuousIntegrationCertification.html

Hammant, P. (oD).Five-minute overview - Trunk Based Development. Accessed on 1/12/2019 at

trunkbaseddevelopment.com: https://trunkbaseddevelopment.com/5-min-overview/

Humble, J (2010).Continuous Delivery vs Continuous Deployment. Accessed on 1/12/2019 at

continuousdelivery.com: https://continuousdelivery.com/2010/08/continuous-delivery-
vscontinuous-deployment/

Humble, J & Farley, D (2011).continuous delivery. Reliable Software Releases Through Build, Test,
and Deployment Automation.Boston, USA: Pearson Education.

Hüttermann, M. (2012).DevOps for Developers. Integrate Development and Operations, the Agile Way.
New York, USA: Springer.

Kim G, Humble J, Debois P & Willis J (2016).The DevOps Handbook. How To Create World Class
Agility, Reliability, & Security in Technology Organizations.Portland, USA: IT Revolution Press.

39
 bibliography

Luke, E. & Prince, S. (n.d.).No One Agrees How to Define CI or CD. Accessed on 1/12/2019 at
https://www.gocd.org/2017/05/09/continuous-integration-devops-research/

Neubauer, T & Zoder, MC (2016).SME report 2016.Vienna, Austria: Federal Ministry of

science, research and economy.

Perry, WE (1992). Quality concerns in software development.Information Systems Management,9(2),

48-52.

Poppendieck, M. & Poppendieck, T. (2007).Implementing lean software development. From Concept to

Cash.Boston, USA: Pearson Education.

Puppet Labs. (2015). Puppet Labs 2015 State of DevOps Report.

Schwaber, K & Sutherland, J (2017).The Scrum Guide.

Schwarz, T. [Lernzone GmbH]. (2014, October 6).Toyota Kata - on the way to lean culture
[YouTube]. Available at https://www.youtube.com/watch?v=Y4tTVO9Oz_w

Shore, J (2006).James ShoreContinuous Integration on a Dollar a Day. Accessed on 1/12/2019 at

https://www.jamesshore.com/Blog/Continuous-Integration-on-a-Dollar-a-Day.html

Verona, J. (2016).Practical DevOps. Harness the power of DevOps to boost your skill set.Birmingham,
UK: Packt Publishing.

Walls, M. (2013).Building a DevOps Culture. DevOps is as much about culture as it is about tools.
Sebastopol, USA: O'Reilly.

40

View publication stats