Professional Documents
Culture Documents
Ares(2017)13571 - 03/01/2017
DOCUMENT
CHANGE LOG
Reason for change Issue Revision Date
CHANGE RECORD
Issue 1 Revision 0
Reason for change Date Pages Paragraph(s)
Page 2/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
Table of contents:
TABLE OF CONTENTS: ............................................................................................................................................. 3
1. INTRODUCTION ............................................................................................................................................. 4
1.1. SCOPE ............................................................................................................................................................ 4
1.2. APPLICABLE DOCUMENTS .............................................................................................................................. 4
1.3. ACRONYMS ................................................................................................................................................... 4
2. SOFTWARE STANDARD (ECSS-E-40) ............................................................................................................... 7
2.1. INTRODUCTION ............................................................................................................................................. 7
2.2. ECSS-E-40....................................................................................................................................................... 8
2.2.1. Processes.............................................................................................................................................................. 9
2.2.2. Reviews .............................................................................................................................................................. 11
2.2.3. Software documentation ................................................................................................................................... 12
2.2.4. Software life cycles ............................................................................................................................................ 13
3. CONTINUOUS INTEGRATIONS ...................................................................................................................... 13
3.1. PRACTICES OF CI .......................................................................................................................................... 14
3.1.1. Maintain a single source repository ................................................................................................................... 14
3.1.2. Automate the Guild............................................................................................................................................ 14
3.1.3. Automate unit test ............................................................................................................................................. 14
3.1.4. Everyone commits to the mainline every day .................................................................................................... 15
3.1.5. Every commit should build the mainline on an integration machine ................................................................ 15
3.1.6. Fix broken builds immediately ........................................................................................................................... 15
3.1.7. Keep the build Fast............................................................................................................................................. 15
3.1.8. Other practices................................................................................................................................................... 16
3.2. ADVANTAGES .............................................................................................................................................. 17
3.3. JENKINS ....................................................................................................................................................... 17
3.3.1. Features ............................................................................................................................................................. 18
4. SOFTWARE METHODOLOGY ........................................................................................................................ 20
4.1. AGILE DEVELOPMENT: SCRUM ..................................................................................................................... 22
4.1.1. Roles................................................................................................................................................................... 22
4.1.2. Components of the Scrum ................................................................................................................................. 23
4.1.3. Working meetings .............................................................................................................................................. 24
4.2. SCRUM WITH JIRA SOFTWARE ..................................................................................................................... 25
4.3. UNIT TESTING .............................................................................................................................................. 27
4.3.1. Unit test structure .............................................................................................................................................. 28
4.3.2. Advantages......................................................................................................................................................... 28
4.3.3. Tools ................................................................................................................................................................... 29
4.4. CODE COVERAGE ......................................................................................................................................... 30
4.4.1. Sonarcube .......................................................................................................................................................... 30
5. SOFTWARE CONFIGURATION MANAGEMENT .............................................................................................. 32
5.1. GIT............................................................................................................................................................... 34
5.2. JIRA ............................................................................................................................................................. 35
5.3. DEVELOPMENT AND PRODUCTION ENVIRONMENT ..................................................................................... 35
5.3.1. Eclipse ................................................................................................................................................................ 36
5.3.2. NetBeans ............................................................................................................................................................ 37
5.3.3. MS Visual Studio ................................................................................................................................................ 37
6. DOCUMENT MANAGEMENT CONFIGURATION CONTROL ............................................................................. 38
6.1. CONFLUENCE ............................................................................................................................................... 39
6.2. DOCUMENT MANAGEMENT AT JIRA / CONFLUENCE.................................................................................... 40
7. CONCLUSIONS ............................................................................................................................................. 41
8. REFERENCES ................................................................................................................................................ 41
Page 3/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
1. Introduction
1.1. Scope
This document is the deliverable “Software Quality assurance Plan (SQAP)” for the EU H2020
(COMPET-5-2015 – Space) project “A Gaia and Herschel Study of the Density Distribution and
Evolution of Young Massive Star Clusters” (Grant Agreement Number: 687528), with
abbreviated code name StarFormMapper (SFM) project. The structure and content is based on
[AD1].
The following documents contain requirements and references, which are applicable to the
work:
• [AD1] SFM proposal: A Gaia and Herschel Study of the Density Distribution and
Evolution of Young Massive Star Clusters. Reference 687528. Reference: Space2015-
StarFormMapper-Parts1-3, Issue: 1.1, Date: 04/08/2015.
• [AD2] SFM-MAN-SDP-1.0: SFM System Developmen Plan.
1.3. Acronyms
Page 6/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
2. Software Standard (ECSS-E-40)
2.1. Introduction
In June 1994, the ESA Council adopted a resolution to confirm the Agency’s commitment to the
transfer of the PSS (Procedures, Specifications &. Standards) system of ESA space standards to
the new set of standards prepared by the European Cooperation for Space Standardisation [1].
The structure of the ECSS standards is shown in Figure 1. As can be seen, it has three main
branches:
Page 7/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
The ECSS system is organized into three levels that are interconnected [2]. In this way, the third
level is inside the second and the second is inside the first. The first one is "Policy and principles".
This level encompasses the general statements of policy and principles for standardisation in
that domain. For example in Table 1, the E-00 is the nomenclature for the engineering domain.
The second one is "Requirements". The requirements for the specific disciplines in that domain
are described in this level. Continuing with the previous example, E-40B is the nomenclature for
software. Finally, the third level is "Normative documents and handbooks" where are the
guidelines to interpreting the requirements for specific applications. In the case of our example,
it could be E-40-03 for the case of ground segment software.
The software standardisation within ECSS spans over the Engineering branch (the E-40 family)
and the Quality branch (the Q-80 family) 1. ECSS software standards, in particular ECSS-E-40
(Software Engineering) and ECSS-Q-80 (Software Product Assurance), have been in use since
1999 and 1996 respectively and were reissued in improved "B" and "C" (ECSS-E-ST-40, 2007)
versions [3]. They are complemented by ECSS-E40-07, a specific standard on ‘software
simulation’, and by the following level three handbooks:
Throughout the next section, a brief review of the ECSS-E-40 family will be conducted. For this
purpose, [1] will be used as the base of the content.
2.2. ECSS-E-40
QSR has set the use of ECSS-E-40 in this project. ECSS-E-40 is based on the customer–supplier
model [4, 5]. This model begins with customer requirements and emphasizes active measures
1
http://www.esa.int/TEC/Software_engineering_and_standardisation/TECMKDUXBQE_0.html
Page 8/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
for customer satisfaction. Thus, the feedback is provided and the process is improved. As shown
in Figure 2, this concept may be applied recursively.
Several processes, reviews and documentation are required in ECSS-E-40. These are described
below.
2.2.1. Processes
System Engineering: This is carried out by the “customer” and involves such activities as:
The software is a part of a system that put requirements on it – the surrounding system could
be onboard hardware and other software systems, and need not necessarily include a human
user.
Software Requirements Engineering: This is carried out by the “supplier” and in essence
involves:
• Software-requirements analysis.
• Software top-level architectural design.
Software Design Engineering: This is also carried out by the supplier and involves:
Page 9/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
• Designing of software items.
• Coding and unit testing.
• Integration.
• Validation with respect to the technical specification.
• Validation with respect to the requirements baseline: the milestone is the Qualification
Review (QR), which is carried out in the supplier’s environment and is often referred to
as the “Factory Acceptance Test (FAT)”. Milestones are tools used in project
management to mark specific points along a project timeline and baselines are defined
as a point in the software life cycle in which the configuration control is applied to a
specific configuration element.
• Software acceptance: the milestone is the Acceptance Review (AR) and is carried out in
the operational environment. This is also referred to as the Site Acceptance Test (SAT),
and may be preceded by a Preliminary SAT (PSAT). Acceptance is carried out by the
customer.
• Preparation of plans for operational testing (i.e. of new releases coming from the
maintenance process).
• User support, including what is usually called "first-line support", e.g. help desk.
• Software migration.
• Software retirement.
Page 10/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
In Figure 3 the processes of ECSS-E-40 is shown.
2.2.2. Reviews
Page 11/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
2.2.3. Software documentation
The main categories of documentation are shown in Figure 4. It is arranged in "folders", into
which the various output documents are aggregated. The main folders are:
The contents of these folders are built up in the course of the project, as shown in Figure 4.
Page 12/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
2.2.4. Software life cycles
The software life cycle defines the sequencing and dependencies of the processes. No particular
life-cycle model is imposed, but its selection is an essential management activity. The supplier
must document this choice in the Software Development Management Plan (SDMP).
3. Continuous integrations
According to Martin Fowler, the Continuous Integration (CI) is a software development practice
where members of a team integrate their work frequently. Usually each person integrates at
least daily - leading to multiple integrations per day. Each integration is verified by an automated
build (including test) to detect integration errors as quickly as possible [6]. QSR has set the use
of CI in this project. An scheme of CI is shown in Figure 5.
Page 13/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
3.1. Practices of CI
To carry out a good continuous integration some practices are necessary [6]. These are
discussed below.
The code repository is a fundamental tool that allows the team to work in a totally synchronized,
but at the same time simple way [7]. Each one works on his own. So it is possible to update the
changes without other developers having to wait for them and vice versa. At present there are
many free quality solutions 2, such as Apache Subversion (SVN), Git [8]. This is a fundamental
piece and it will be virtually impossible to perform continuous integration if it is not available. In
our case, the GIT repository was proposed by QSR, Section 5.1.
Any developed project must be build on any machine without the need of complex
environments. Not all the machines will have the development environment, even in some cases
they may have relatively expensive licenses, so it would not be a viable option. The condition is
that it must be a very simple tool that can be run on any machine and that can automate the
compilation process, the execution of the tests of the application and even the deployment in
the application server. There are many different solutions depending on the programming
languages. For example, in Java they are Ant [9] or Maven [10], in Ruby is Rake [11], etc.
In order to verify that the project works automatically unit tests (often even integration tests)
are necessary. The fundamental premise is that all new functionality must have a set of tests
that verify that its behavior is correct.
There are many different ways to include tests in the project. In particular the rise of Extreme
Programming (XP) [12] and Test Driven Development (TDD) [13]. Over the last few years the rise
of TDD has popularized the XUnit family of open-source tools which are ideal for this kind of
testing [14]. This point is dealt with in Section 4.3.
2
https://en.wikipedia.org/wiki/Comparison_of_version_control_software
Page 14/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
3.1.4. Everyone commits to the mainline every day
Not all requirements are based on having a certain type of tool. Some are based on knowing
how to use a particular tool. Once the version control software is chosen, ideally a developer
should make even several uploads to the code repository a day. If the project has a code
repository, but each developer raises their changes every two weeks, the same problem is
appeared. The integration of the project is delayed during all that time, losing any option to
obtain information immediately. In addition, delaying the integration generates more conflicts
between the files edited by the developers. Because with each update of the repository a
complete project integration is done, every minimum change that does not destabilize the
project must be uploaded to verify that everything is still working correctly.
There are two main ways: using a manual build or a continuous integration server [6]. The first
one it is similar to the local build that a developer does before the commit into the repository.
The developer goes to the integration machine, checks out the head of the mainline and kicks
off the integration build. A continuous integration server acts as a monitor to the repository.
Every time a commit against the repository finishes the server automatically checks out the
sources onto the integration machine, initiates a build, and notifies the committer of the result
of the build.
QSR has chosen the second option. Assembling an integration server, such as Jenkins (which is
chosen by QSR, see Section 3.3), or Cruise Control 3, is free and as simple as deploying a file on a
server. On the contrary the advantages are very important. On the one hand, the process is fully
automated, which avoids human error. On the other hand, it reduces the amount of work, since
a prefabricated solution is taken without having to create complex customized solutions.
The goal is to automate the simplest tasks, to detect the most basic faults as soon as possible.
The longer the time elapses from the production of the error to its detection, the more difficult
it is to solve it.
This is one of the first tasks to define, to know how the development is, what is the criterion for
the code to promote from one environment to another, and what is done in each environment.
3
http://cruisecontrol.sourceforge.net/
Page 15/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
To implement continuous integration, a pipeline, a set of stages, phases through which the
software passes and are automated is usually defined. An example is shown in Figure 6.
With the steps shown so far, a fairly complete process and that will improve the quality of the
software is described. However, it is recommended the use of the integration server to do
certain relatively complex tasks. For example, it is possible to configure the server to do static
code analysis, so that you can search the entire code for suspicious blocks, duplicate blocks or
references that are not used, etc. There are many options such as FindBugs [15] or PDM [16].
Another thing could be to include tasks that allow the application to be deployed on a server.
Thus, a demo server with the latest version of the project is always available. Another operation
that we can automate is to perform system tests on the application. It possible to create tests
with a navigation recording tool, such as Selenium 4, and launch them with every construction
that the server does [17].
Finally, other simple practices help ensure the proper functioning of the software such as to
have a clone of the production environment, the latest executable available for everyone and
scripts that allow to deploy the application into any environment easily.
4
http://docs.seleniumhq.org/
Page 16/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
3.2. Advantages
The advantages of the use of IC can be summarized in the following points [18]:
3.3. Jenkins
This tool is inherited from another similar called Hudson 6, devised by Kohsuke Kawaguchi 7, who
worked at Sun. A few years after Oracle bought Sun, the Hudson community decided to rename
the project to Jenkins, migrate the code to Github and continue the work from there. However,
Oracle has continued to maintain and work on Hudson.
5
https://wiki.jenkins-ci.org/display/JENKINS/Meet+Jenkins
6
http://hudson-ci.org/
7
https://en.wikipedia.org/wiki/Kohsuke_Kawaguchi
Page 17/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
3.3.1. Features
• Continuous integration and continuous delivery: As an extensible automation server,
Jenkins can be used as a simple CI server or turned into the continuous delivery hub for
any project.
8
https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins
Page 18/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
• Easy configuration: It can be easily set up and configured via its web interface, Figure
10, which includes on-the-fly error checks and built-in help.
• Extensible: It can be extended via its plugin architecture, providing nearly infinite
possibilities for what Jenkins can do.
Page 19/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
• Distributed: It can easily distribute work across multiple machines, helping drive builds,
tests and deployments across multiple platforms faster.
4. Software Methodology
The methodology for software development is a systematic way of realizing and managing a
project to carry it out with high chances of success. In other words, within a methodology for
software development are the processes to systematically follow to design, implement and
maintain a software product from the time the need for the product arises until we fulfill the
objective for which it was created [19]. Nowadays there are different methodologies. Taking
into account the philosophy of development of methodologies, those with greater emphasis on
project planning and control, in precise specification of requirements and modeling, receive the
appellation of heavyweight methodologies [20]. These traditional methodologies impose a work
discipline on the software development process, in order to achieve a more efficient software.
For this, emphasis is placed on the overall planning of all the work to be done and once it is all
detailed, the software product development cycle begins. On the other hand, a lightweight (or
Agile) methodology, Section 4.1, has only a few rules and practices which are easy to follow.
Agile processes emphasizes both the rapid and flexible adaptation to changes in the process, the
product, and the development environment [21]. The main differences between both are
collected in Table 3 [20]. The choice of one or the other is made according to the characteristics
of the project [22].
Among the most used of both types are waterfall development [23], prototyping [24], iterative
and incremental development [25, 26], spiral development [27], rapid application development
Page 20/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
[28], etc. The scheme of some of these methodologies are shown in Figure 12. Throughout the
development of this project, QSR has set the use of an agile methodology [29], specifically Scrum
which will be explained below.
Figure 12: Examples of heavy and agile software development methodologies schemes. (a) Waterfall, (b)
prototyping, (c) iterative and incremental development, (d) spiral and (e) rapid application.
Page 21/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
4.1. Agile development: Scrum
There are certain development projects that cannot be completely defined at the beginning of
the project because they require a constant revision and modification process. Our project is
located in this case so the methodology most indicated is the Agile. It allows maintaining
sufficient flexibility to adapt to a changing context maintaining a set of general rules that all
participants must know. In this sense, Scrum can be considered as the paradigm of Agile
development methodology, defining the way to approach a software development process in
an agile and light way, through the description of a set of roles, components and organization
of daily activity [30]. The fundamental basis of this methodology consists in the division of the
complete work (Product Backlog) in different sections or blocks that can be approached in short
periods of time (1-4 weeks) that are denominated Sprints. This organization of the process of
creation of software allows strengthening the following aspects:
• Agile: The division of labor into small functional units (sprints) allows maintaining a
policy of frequent software deliveries that offer a clear view of the state of the process
and allows the introduction of modifications.
• Collaborative: The approach, from the point of view of the organization of the team, is
quite horizontal giving the members of the development team a high degree of
autonomy and self-organization of their work.
4.1.1. Roles
As shown in Figure 13, different roles are defined within the Scrum framework:
• Product Owner: It is the person responsible for transmitting to the development team
the vision of the product to be created, providing the business perspective. It represents
the rest of people of the project (stakeholders, customers, managers, etc.) in product
development. The Product Owner is responsible for defining the Product Backlog,
prioritizing them, and finally validating them.
• Stakeholders: A group of people who are not directly involved in the development
process, but who should be taken into account, because they are people interested in
Page 22/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
it, such as directors, managers, commercials, etc. The Product Owner will be responsible
for collecting the opinions and suggestions and deciding whether to apply them to the
Product Backlog, as well as to decide if to invite any of these people to the process of
reviewing deliverables.
• Users: Like stakeholders, they are not part of the creation process directly (they could
be in the revision phase of deliverables if deemed necessary). They are the final
recipients of the application to be developed, the target audience of the application.
• Scrum master: Person who guarantees that the work team does not find impediments
or obstacles to approach their tasks within the project.
• Scrum team: Team responsible for developing and delivering the product. It maintains
a horizontal organization in which each member of the team self-manages and organizes
freely in the definition and execution of the different sprints
In the particular case of our project, the product owner is the set of consortium members and
the stakeholders are the scientists who will provide their opinions and suggestions for
interacting with their algorithms. At the end of the project, anyone will be able to use the final
software therefore any person receives the role of user. Finally, both master and team Scrum
roles are related to the QSR team.
• Product Backlog: It is the definition of the project. It consists of a document that collects
the set of requirements that are associated with the project. It is the responsibility of
the Product Owner to make this definition and establish the priorities of each
Page 23/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
requirement. It is a high level document, containing generic (not detailed) descriptions,
and subject to modifications throughout development.
• Running the Sprint: It would be the period of 1 to 4 weeks (period previously defined
based on the tasks collected in the sprint backlog) during which the work team would
address the corresponding development tasks. Once the execution of a defined sprint
has begun, it cannot be modified, and in case of being necessary to make changes these
will be done once the period ends through the definition of another sprint backlog.
• Delivery: Once the execution of the sprint is completed, a portion of the potentially
definitive application will be available.
• Burn down: It is a document that reflects the status of the project, indicating the volume
of requirements that are currently being addressed (in the product backlog), the
requirements that are currently being developed (sprint backlog) and the requirements
whose development has already been completed in its entirety.
• Sprint planning meeting: It is done at the beginning of each sprint cycle, and it is
directed to select the set of requirements of the product backlog that will be addressed,
the work team that will be necessary and the estimated time (between 1 and 4 weeks)
for its development.
• Daily scrum: This is done at the beginning of each day that the sprint is running. It is a
short meeting (no more than 30 minutes) in which the team members answer the
following questions: “What have you done since the last meeting”, “What problems did
you encounter to get the job done”, “What do you plan to do before the next meeting”.
• Sprint review meeting: Once the sprint cycle is completed, a meeting is held which
defines which part of the planned work has been completed and which part remains
pending. As for the work completed, a review of the same is done to the product owner
and other users who may be involved.
• Sprint retrospective: A meeting in which all team members make an assessment of the
work done in the last sprint, identifying points of improvement for the next to perform.
Page 24/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
The main objective is to introduce a continuous improvement component into the
process.
Jira software has been chosen by QSR to develop the project by means of Scrum methodology.
Atlassian, software owner, provides a tutorial which is shown below 9:
• Create a Scrum project: Once a JIRA Software account is started and logged in, it is
possible to create a project. Subsequently, the project type and the Scrum methodology
must be selected. A Scrum card is created automatically with a Scrum software
development project and it possible to view the empty backlog of the Scrum card.
• Create user stories or tasks in the backlog: In JIRA, the tasks, and bugs "issues" are
called user stories 10. There is a quick create option on the backlog as is shown in Figure
15. Once it is created 6 or more user stories, the priorization the stories in the backlog
is allowed.
9
https://es.atlassian.com/agile/how-to-do-scrum-with-jira-software
10
In an agile framework, user stories are the smallest units of work. As a (type of user), I want (goal) so
that I (receive benefit).
Page 25/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
• Estimate your stories: Before to begin the first sprint, it is important to estimate the
stories in the backlog and to determine the execution time. Estimates will also help
gauge how much work it should add to the next sprint based on the number of team
members. In JIRA Software, it possible to estimate issues by going to the backlog and
entering a number in the Estimate field for each issue.
• Create a sprint: To create the first sprint in the backlog, Figure 16, and to organize issues
into sprints. Before starting a sprint, a sprint planning meeting must be held. Afterwards,
the stories are dragged to the sprint created. Finally, name the sprint and to add the
duration and start and end dates of the sprint. In Figure 17, an example the sprint in Jira
is shown.
Page 26/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
• View the burndown Chart. A good idea to check the Burndown Chart during a sprint. A
Burndown Chart, Figure 18 shows the actual and estimated amount of work to be done
in a sprint. The horizontal x-axis in a Burndown Chart indicates time, while the vertical
y-axis indicates issues. In JIRA, the Burndown Chart shows the actual and estimated
amount of work to be done in a sprint. To view this chart, click Reports from the sidebar,
and then select the Burndown Chart from the reports dropdown.
• Complete the sprint. At the end of the sprint, a situation similar to in Figure 19 is shown.
After you complete the sprint, reports should be done. QSR has proposed the
Confluence software for this, Section 6.1.
The rapid correction of errors is one of the premises of the Agile methodology. In this sense, it
is convenient to have a method to systematize this process. QSR has set the creation and use of
Page 27/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
unit tests. There are many definitions about what really is a unit test. However, some are
ambiguous and others have a particular focus according to the situation in which they are used
or developed. Unit testing is the testing of individual hardware or software units or groups of
related units where a unit is a software component that is not subdivided into other components
[31]. Unit testing is often simple and quick to code. In addition, they must have the following
characteristics [32]:
• Arrange: This is the part of the unit test where all the code is configured to run the test.
• Act: This is the phase of the unit test where the code to be tested is executed.
• Assert: It is the section of the unit test where the result of the test is checked.
4.3.2. Advantages
Unit testing seeks to isolate each part of the program and show that the individual parts are
correct, providing some advantages:
1. Encourage change. Unit testing facilitates code restructuring (refactoring). It allows testing
on changes and verifying that the modifications have not introduced errors (regression).
2. Simplified integration. They allow reaching the integration phase by ensuring that the
individual parts work correctly. In this way, integration tests are facilitated.
3. Document the code. The tests themselves can be considered documentation, since they are
a reference implementation of how to use the code.
4. Separation of interface and implementation. The only interaction between the test cases
and the units under test are the interfaces of the latter. It is possible to change either one
without affecting the other through, for example, the use of mock objects [33].
5. Less errors and easier to locate. Unit testing reduces the number of errors and time to locate
them.
6. They can improve the design. The use of Test-Driven Design and Development practices
(TDD) [13], allows defining the expected behavior in a step prior to coding.
Page 28/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
7. It may be the simplest way to verify operation in situations such as the development of an
API or a component that provides services that are not yet available to a customer.
4.3.3. Tools
Currently there are a variety of tools to perform unit tests. Among the most outstanding are the
following:
Page 29/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
Figure 20: Unit testing tools.
Another quality measure that QSR proposes to implement is the Code Coverage. It was one of
the first methods invented for software testing [34]. It is loosely defined as the percentage of
some code structure or artifact which is executed or covered by at least one test [35]. This data
can be used to determine how many lines of code have been executed by the tests and the lines
of code that have yet to be tested. Code coverage analysis allows [36]:
4.4.1. Sonarcube
Page 30/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
• Jcov, https://wiki.openjdk.java.net/display/CodeTools/jcov: is a pure java
implementation of a code coverage tool which provides a means to measure and
analyze dynamic code coverage of Java programs. JCov provides functionality to collect
method, linear block and branch coverage, as well as showing uncovered execution
paths. It is also able to show a program's source code annotated with coverage
information.
It is an open platform to manage code quality. As such, it covers the 7 axes of code quality:
Page 31/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
• The platform provides the ability to track quality on new code: recently introduced
issues, lack of test coverage on new code, etc. Once found, issues can easily be assigned
for mop-up.
• It is possible the integration with other software such as Maven, Ant, SVN, Git,
Mercurial, JIRA, Mantis, Google Analytics, Piwik, Fortify…
The computer infrastructure of this project is too broad and complex to be managed by people
or with systems of manuals, spreadsheets, etc. QSR has set the use of the Software Configuration
Management (SCM). SCM is an organizational framework for managing the evolution of
computer systems throughout all stages of systems development [37]. The configuration
management facilitates the maintenance of the application, providing precise information on
the impact of the requested changes and reducing the time of implementation of a change, both
evolutionary and corrective. It also allows the control of the product throughout its
development, obtain reports on the status of the project and reduce the number of errors of
adaptation of the system, which results in an increase in product quality. Its main objective is to
Page 32/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
maintain the integrity of the versions throughout the development of the product, ensuring that
no uncontrolled changes are made.
The SCM practices that will be carried out by QSR are [38]:
• Control and audit changes [39]. Change control is a mechanism for evaluating and
approving changes made to elements of software configuration during the life cycle.
• Identify and store elements in a secure repository.
• Require Change Requests and Change Packages according to the established procedure.
Page 33/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
• Organize and integrate consistent sets of versions using activities.
• Support simultaneous changes in elements and components.
• Maintain stable and consistent workspaces.
• Ability to do continuous build and deployments.
• Integrate early and frequently.
• Ensure absolute reproducibility for all elements.
Then the tools chosen by QSR for this purpose will be described.
5.1. Git
Git, https://git-scm.com/, is a free and open source distributed version control system designed
to handle everything from small to very large projects with speed and efficiency. It is easy to
learn and has a tiny footprint with lightning fast performance. It outclasses SCM tools like
Subversion, CVS, Perforce, and ClearCase with features like cheap local branching, convenient
staging areas, and multiple workflows.
• Branching and Merging: Git allows and encourages you to have multiple local branches
that can be entirely independent of each other.
• Distributed: One of the nicest features of any Distributed SCM, Git included, is that it's
distributed. This means that instead of doing a "checkout" of the current tip of the
source code, you do a "clone" of the entire repository.
• Data Assurance: The data model that Git uses ensures the cryptographic integrity of
every bit of the project. Every file and commit is checksummed and retrieved by its
checksum when checked back out.
• Staging Area: Unlike the other systems, Git has something called the "staging area" or
"index". This is an intermediate area where commits can be formatted and reviewed
before completing the commit.
• Git comes with built-in GUI tools for committing (git-gui) and browsing (gitk), but there
are several third-party tools for users looking for platform-specific experience 11.
11
https://git-scm.com/downloads/guis
Page 34/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
5.2. Jira
• Plan: Create user stories and issues, plan sprints, and distribute tasks across a software
team.
• Track: Prioritize and discuss in full context with complete visibility.
• Release: Information is always current.
• Report: Improve team performance based on real-time, visual data.
• Compatible with Scrum methodology as shown in Section 4.2.
Note: An independent database Server has to be configured in order for Atalassian SW to run.
5.3.1. Eclipse
Its current version, Eclipse Luna, includes official support for Java 8 in the Java development
tools, Plug-in Development Tools, Object Teams, Eclipse Communication Framework, Maven
integration, Xtext, Xtend, Web Tools Platform, and Memory Analyzer.
Page 36/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
5.3.2. NetBeans
NetBeans, https://netbeans.org/, is the official IDE for Java 8. With its editors, code analyzers,
and converters, it can quickly and smoothly upgrade applications to use new Java 8 language
constructs, such as lambdas, functional operations, and method references.
Batch analyzers and converters are provided to search through multiple applications at the
same time, matching patterns for conversion to new Java 8 language constructs.
Page 37/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
Figure 33: MS Visual studio logo.
When several authors collaborate in a shared space as in this project, they may need to edit and
revise common documents repeatedly. Keeping track of the changes made by the different
authors means an additional effort. That is why it is necessary to use specialized tools similar to
the case of the software development, Section 5. In this case QSR has chosen the Confluence
tool that is compatible with any established procedure such as the show in Figure 35. It has also
established the use of the coding of document names described in [AD2].
Page 38/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
Figure 35: Example of document management work flow.
6.1. Confluence
Confluence, https://www.atlassian.com/software/confluence is the collaborative platform of
the company Atlassian. It allows connecting teams, and creating and sharing content and
knowledge to develop a better and faster work.
• Powerful editor: Confluence gives the power to create anything - meeting notes,
project plans, product requirements, etc. thanks to a simple, but powerful editor.
• Feedback in context: It is possible to leave feedback on the work itself with inline
comments on any Confluence page.
• File collaboration: It works with files every day – images, PDFs, spreadsheets, and
presentations. It can give feedback directly on the files and it keeps tracks of versions
automatically.
• Organized workspaces.
• Blogs and discussions: Share news and announcements and it is possible to have active
discussions with comments, @mentions and likes.
• User and content permissions.
• Page and file versioning: Track every version and change that you make to a page. Files
are automatically versioned.
Page 39/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
Figure 37: Confluence software.
Each project present in Jira, Section 5.2, has a space reserved to save the documentation in
Confluence. The project documentation is structured by default as is shown in Figure 38:
The association between the JIRA project and the Confluence space is done automatically
(Creating space in confluence). This configuration can be edited from the project management.
Page 40/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
7. Conclusions
This document discusses the techniques, procedures, and methodology that will be
implemented by QSR for software development and that should be adopted by the rest of the
consortium. Given the characteristics of the project, QSR proposes the adoption of the ECSS-E-
40 standard and the Scrum methodology for software implementation and development. To do
this, we have chosen a set of tools, which include Jenkins, Sonarcube, Git, Jira and Confluence.
These tools will also ensure a good interaction with the rest of the consortium as they can be
made accesible to everyone, and imposes a good development praxis beneficial for the success
of the SFM project.
8. References
Page 41/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0
17. A. Holmes and M. Kellogg, "Automating functional tests using selenium," in AGILE 2006
(AGILE'06), (2006), 6 pp.
18. C. Kaner, J. Bach, and B. Pettichord, Lessons Learned in Software Testing: A Context-
Driven Approach (Wiley, 2011).
19. H. Zhu, Software Design Methodology: From Principles to Architectural Styles (Elsevier
Science, 2005).
20. M. Awad, "A comparison between agile and traditional software development
methodologies," University of Western Australia (2005).
21. A. I. Khan, R. J. Qurashi, and U. A. Khan, "A comprehensive study of commonly practiced
heavy and light weight software methodologies," arXiv preprint arXiv:1111.3001 (2011).
22. R. N. Burns and A. R. Dennis, "Selecting the appropriate application development
methodology," ACM Sigmis Database 17, 19 (1985).
23. K. Petersen, C. Wohlin, and D. Baca, "The waterfall model in large-scale development,"
in International Conference on Product-Focused Software Process Improvement, (2009),
386.
24. P. Tavolato and K. Vincena, "A prototyping methodology and its tool," in Approaches to
prototyping (Springer, 1984), pp. 434.
25. R. Victor, "Iterative and incremental development: A brief history," IEEE Computer
Society, 47 (2003).
26. C. Larman, Agile and iterative development: a manager's guide (Addison-Wesley
Professional, 2004).
27. B. W. Boehm, "A spiral model of software development and enhancement," Computer
21, 61 (1988).
28. T. S. Jones and R. C. Richey, "Rapid prototyping methodology in action: A developmental
study," Educational Technology Research and Development 48, 63 (2000).
29. P. Abrahamsson, O. Salo, J. Ronkainen, and J. Warsta, "Agile software development
methods: Review and analysis," (VTT Finland, 2002).
30. K. Schwaber, "Scrum development process," in Business Object Design and
Implementation (Springer, 1997), pp. 117.
31. J. Radatz, A. Geraci, and F. Katki, "IEEE standard glossary of software engineering
terminology," IEEE Std 610121990, 3 (1990).
32. P. Runeson, "A survey of unit testing practices," IEEE software 23, 22 (2006).
33. T. Mackinnon, S. Freeman, and P. Craig, "Endo-testing: unit testing with mock objects,"
Extreme programming examined, 287 (2001).
34. J. C. Miller and C. J. Maloney, "Systematic mistake analysis of digital computer
programs," Communications of the ACM 6, 58 (1963).
35. H. Zhu, P. A. Hall, and J. H. May, "Software unit test coverage and adequacy," Acm
computing surveys (csur) 29, 366 (1997).
36. S. Desikan and G. Ramesh, Software Testing: Principles and Practice (Pearson Education
Canada, 2006).
37. J. Keyes, Software configuration management (CRC Press, 2004).
38. S. P. Berczuk and B. Appleton, Software Configuration Management Patterns: Effective
Teamwork, Practical Integration (Addison-Wesley, 2003).
39. E. F. Ferraby and L. Ferraby, Change control during computer systems development
(Prentice Hall, 1991).
40. M. Nagl, Building Tightly Integrated Software Development Environments: The IPSEN
Approach (Springer, 1996).
Page 42/42
Software Quality assurance Plan (SQAP)
Date 01/12/2016 Issue 1 Rev 0