Future Direction Categorization (Final)

Study ID
random number
S48 2
S7 3
S20 3
S32 4
S28 5
S22 7
S33 8
S35 9
S18 10
S39 10
S56 10
S27 11
S32 13
S30 14
S7 15
S7 17
S38 22
S39 22
S18 24
S7 25
S50 25
S13 27
S7 29
S48 32
S29 34
S30 36
S31 37
S3 42
S38 42
S19 43
S46 43
S54 44
S22 45
S40 45
S29 46
S13 47
S8 48
S39 48
S39 48
S22 49
S35 52
S26 54
S29 54
S31 56
S5 57
S9 58
S20 61
S22 61
S8 63
S30 65
S2 66
S9 66
S20 66
S47 67
S7 68
S14 68
S29 68
S8 71
S43 71
S2 73
S7 74
S44 76
S15 78
S22 78
S31 78
S2 79
S50 79
S39 81
S36 82
S19 83
S36 83
S7 84
S30 85
S56 85
S19 86
S24 87
S46 87
S9 88
S37 88
S19 89
S30 89
S35 89
S29 90
S22 91
S37 91
S7 92
S32 92
S55 92
S32 94
S38 94
S33 95
S50 95
S50
S21 96
S46 96
S7 99
S23 99
S15 102
S3 105
S11 107
S24 107
S9 109
S12 109
S19 109
S59 109
S31 111
S38 111
S9 112
S7 114
S7 114
S33 114
S7 116
S20 116
S12 119
S4 123
S7 123
S49 123
S6 127
S20 128
S23 129
S41 129
S52 129
S57 129
S60 130
S5 135
S25 136
S39 136
S4 137
S50 137
S7 139
S9 139
S46 139
S43 140
S25 144
S25 145
S46 146
S20 147
S46 147
S49 150
S10 154
S9 157
S8 158
S56 161
S39 162
S50 162
S50 163
S56 163
S14 164
S54 164
S7 165
S8 165
S19 165
S5 166
S11 167
S35 167
S21 168
S46 168
S7 169
S29 169
S3 171
S17 172
S39 173
S39 173
S7 174
S45 175
S36 176
S54 177
S18 178
S29 178
S36 178
S31 180
S18 183
S33 183
S36 183
S41 183
S1 184
S40 184
S42 184
S50 184
S8 186
S46 186
S51 186
S4 187
S20 187
S58 187
S33 188
S12 190
S17 192
S9 194
S46 194
S5 196
S23 196
S7 197
S14 198
S30 198
S39 198
S53 199
S7 200
S29 202
Text Segment
The process map and its activities characteristics description presented in our study are based on primary academic s
However, it can be used as basis for the conduction of empirical studies to investigate OSS process in a more practica
We believe that our results can contribute to improve the understanding of OSS process activities and consequently h
mitigate OSS project failure.
While there are a few studies outside the scope of this review focusing on software selection [46, 56, 105, 184] and k
sharing within OSS communities [119, 173, 195], none of these are directed towards studying actual practice in organ
A few studies have started to look at some of the challenges in the borderlands between integrating an OSS componen
contributing to the development of it [106, 130, 186], but further research is needed to solve the maintenance challeng
developers who integrate a large number of components into their products.
Studies that can quantitatively infer OSS maintenance effort from size-related metrics are needed. To mitigate the diff
acquiring actual effort data from incomplete development records, Yu et.al [40][41][23] focused on predicting the si
related metrics to indirectly estimate the maintenance effort. The strong correlation between size-related metrics and t
effort has been confirmed in closed-source projects [28]. However there still exists a gap between size-related metrics
time-aware effort for maintaining OSS projects. There is a need for studies that can quantitatively infer OSS maintena
effort from size-related
we commented in Sect. metrics. Furthermore,
4.2 the lack thefor
of measures effort drivers
process used inbecause
maturity generalinmaintenance
this case theeffort estimation
assessment needsmodels
to be
as an example to improve OSS maintenance effort estimation. For example, Nguyen [28] developed
qualitative evaluations of the community. Since we have focused on quantitative measures, there may be other characan extension to
COCOMO II [9]
of the quality sizethat
model andrequire
effort estimation
or that maymodels to capture various
be complemented characteristics
with qualitative of software maintenance in gene
evaluations.
through a number of enhancements to the COCOMO II models to support the cost estimation of software maintenanc
effort drivers such as DATA (Database size), CPLX (Product Complexity), and PVOL (Platform Volatility) in his stu
contribute greatly to OSS maintenance effort estimation.
We have in fact identified these motivations, strategies, advantages and difficulties in other software projects that are
but have the goal to meet customers’ expectations. For this reason, many of the lessons learned from OSS projects ca
adopted in other types of software projects regarding the use of rapid releases.
As future work, we are considering using the results of this study to build a meta-model for the mining of open source
view of gathering data that lead to assessments of the quality of projects adopting the Frequent Release approach.
As Kirk and Miller stated in [22], “although no one defends a positivistic ontology, but scholars in social science has
that much research makes sense only in terms of a set of unexamined positivist assumptions.” Research in the field of
success has the same problem. We want to precisely point to variables like:“general viewpoint of audience society” a
use of software3” as measurements of success and contextual parameters such as “availability of knowledgeable deve
“legal support and level of IT development in the development environment” as affecting factors. That’s why we reco
mixed-methodology
One research
of our main findings in the
is that most field of OSS
studies are success.
not deeply concerned with research methods: among the relevant sel
papers, we may cite two case studies, one action research, but no experiments, which seems to be in sharp contrast w
recent growth of evidence-based SE. Software engineering research is slowly adopting increasing scientific rigour in
years. Moreover, SE education is an interdisciplinary area. As such, it can strongly benefit not only from SE research
but also from research methods from areas such as sociology, anthropology, pedagogy and communication. Thus, it s
relevant
As futuretowork,
identify
we which
plan toresearch
developmethods
a methodare more
that appropriate
extracts in this intersection,
users’ software requirements in order to achieve
automatically better
from results
internet re
interdisciplinary area.
and to design automated processing to support requirement prioritization.
The reviewed articles show that self-determined participation motives are most relevant for FLOSS developers’ comm
Moreover, both relational (e.g. trust) and structural (e.g. network centrality) group aspects foster FLOSS developers’
commitment. Also, the chosen code license affects developers’ commitment. Beyond these aspects, the reviewed artic
suggest other factors which yet need dedicated analysis. For example, the effects of members’ cultural background [5
their geographic proximity [29] on their development intensity. Further, the interrelations between the research perspe
need further evolution
Sub-project scrutiny. In particular,
with the relationships
their community. between
Large open individual
source projects or team
often factors and
encompass project
many characteristics
sub-projects. Such(se
as
1).
projects in Eclipse, GNU, Linux, and Apache. Often ecology of sub-communities formed around these sub-projects,an
Such cross-perspective analysis is necessary to understand fully how FLOSS projects can incentivize individual w
factors
governed which
by aincrease
common developers’
governance participation.
[50]. Study Future
on the research
formationmay
anddraw
evolu-ontion
research results by Gallivan
of sub-projects and their [21] and e
communi
and how many
revealed governance processes foster
key characteristics, FLOSS
which developers’
are listed in Table personal relationships.
V and Table Also, further
VI, respectively. research
Yet the is needed toinf
interdependency
understand if and how project governance can stimulate individuals’ participation motives.
evolution between the two and their impact on the overall project evolution remain untouched. The following would b
to
Asinvestigate.
a future work, we plan to provide a more formal definition of the open IoT platform rather than a perspective by c
•different
Does there exist a correlation
stakeholders in detail. between
The opentheIoTevolution
platform(growth, complexity,
term is frequently change)
used of the
in public andsub-projects andunderstand
without clear their asso
sub-communities?
cause problems likeDoes the community
misunderstanding change
or even withstrategic
wrong the change in the sub-project?
decisions. Our future work intends to provide a clearer
•deeper
How does a community form around a newly added sub-project?
understanding of open IoT platforms for the communities. The open issues listed above are also some possible
•directions
What attributes of a sub-project attract new developers to join?
of investigation
• What happens to the sub-community when a sub- project is deleted or merged to other sub-project?
in the study with a development component but oriented to a specific requirement, 16.13% adds tools to the central ax
• What dependencies lead to inter project communica-
GIS Web architecture and 9.68% integrates new methods and algorithms to improve different aspects of architecture.
tion?
Open Source Web Software Architecture Components. Hence, the need to carry out new research aimed at evaluating
• What kind and level of communication and collabo-
improving the components of the Open Source software architecture of a geographic information system in a Web
ration takes place between sub-communities?
environment.
• Does there exist a correlation between the project
This quality
evolution andmodel can be used
the sub-project as a starting point for the quality assessment of an OSS ecosystem, and it is in our pla
evolution?
future work to define a complete quality assessment process (as described in Sect. 5) and to apply it in a real quality
assessment. As consequence new measures may be needed for the assessment, but this is the best way to improve, an
complete the quality model, and a way to prove its capabilities in quality assessment.
Another aspect of the use of OSS in dentistry is education: 10,000 participants from 138 countries used the Supercour
module entitled, “Sterilization and cross-infection control in a dental practice”, which makes this software one of the
popular e-lectures on this topic (31). Supercourse is a network of 56,000 scientists from 174 countries who share a fre
of 5,802 lectures in thirty-three languages, but it contains only eleven lectures on dental topics. These numbers are im
but several issues regarding the use of open intellectual property, a unified rating system and a standard citation syst
require proper
To enable regulationto(19;32).
organizations reap benefits from their participation in OSS communities, the research community shou
dedicate much more attention to questions concerning this [48, 165]. While there are a few examples [50, 101, 176, 1
work is needed to aid organizations in participating in communities and collaborating with other organizations throug
collaboratively working on OSS products [7] and to solve questions like:
􏰅 When, how, and with what should organizations participate in the development of OSS products controlled by othe
(including
Most inter- organizational
organizations seem to havecollaborations)?
rather limited contributions to the OSS communities [33, 43, 91, 98]. The most com
􏰅 How
of can companies
participation is being(effectively) allow
an active user thatproducts to be partlybugs
reports occasional OSStoand
thepartly commercial
community products?
[43, 98, 99]. Only one of 32
respondents from a sample of tertiary education institutions had participated actively by writing code, while 14 had
contributed to an OSS community through reporting bugs [91].
As observed in the study, code quality is seen as the most important criterion for measuring quality. On the other hand
success and developer activity seem to be the most important criteria for measuring success. Therefore, in future, rese
have to study how code quality may be used to measure software success, or how market success and developer activ
be used to measure software quality. Only then, it might be possible to talk about significant relationship between OS
and quality.
ON CO-EVOLUTION
It is turned out from our review that the understanding of co-evolution of the code and the community in OSS project
received little attention in literature (Figure 2). As a consequence, the community dimension and corresponding
communication channels (e.g., mailing archives, bug tracking systems) are explored seldom, as can be seen from Figu
Figure 6 respectively. Study on co-evolution in OSS projects, however, is becoming increasingly popular. Because, in
projects
For a hightheretention
code evolution
rate, it is is important
dependentthat on the
FLOSScontribution
developers of community
perceive their members,
project and
workthatas aself-
successful evolution
determined. In add o
code is required for the survival of the community. The following research directions
members’ project continuance is influenced by relational and structural characteristics of the team. Also, less restrictican be considered relevant. Exp
socio-technical
licenses and thecongruence.
modularity of In the
the code
OSS affect
projects contributions
members’ projectmade by theHowever
retention. community members
as shown not only
in table driveisthe
3, there sy
little
evolutionon
research buthowalsoteam
redefine
levelthe role of
aspects these
affect contributing
developers’ members
retention. Withandregard
thus change
to the the
keysocial
role ofdynamics of thefor
group aspects OSSdeveco
[53]. In this connection, it will be very interesting to investigate the phenomenon socio-technical
commitment, future studies should examine this aspect closely. In addition, very few articles use more than one resea congruence in OSS
Socio-technical
perspective. congruence whichresearch.
is a conceptualization of Conway’s
studiesaslaw [67]draw
states
onthat there should exists a match
We classifiedThis 53 ofcalls
thefor112further
empirical papersInidentified
particular,infuture
this review should
experience research
reports. Hence,by Stewart
the mostand Gosai
common
the
whichcoordination
suggests needs
that established
members’ by
retention theistechnical
a product domain
of (i.e.,
motivational the architectural
and relational dependency
factors. in
Further,the software)
future and the
research is
of studying the OSS phenomenon in organizations is through experience reports. These experience reports lack explic
coordination
to understand activities carried
the interaction out by project members properties and project characteristics. To do so, future studies shouc
(i.e., within the members of the development team) [67]. This
research questions, and most of alsostructural network
lack a method description.
was already explored in closed source
on research by Oh and Jeon [40] and analyze the ways projects, and reported
in whicha high
FLOSScorrelation
projectswith
cansoftware
actively build
utilizesuccess, quality,ne
the interaction a
rate of modification [68]. Thus socio-technical congruence plays a pivotal role in conceptualizing
their members to foster their retention. Finally, further research is necessary to understand the ways in which project the co-evolution in
Surprisingly,
characteristicsthis notion individuals’
influence as a researchretention.
area has not been iven much attention among open source researchers. Althoug
Some researchers
identified and reportedhave as studied the joining
a desired property process of OSS com-
for collaborative munities. But
development more research
activities like OSSisprojects
required[69].
to discover
Consider th
which
lack ofmayfocushelp the developers
in this direction, we to propose
alleviatethethefollowing
issues experienced by them when they migrate to the other projects, to
to investigate.
investigate the changes in the degree of socialization
• Does the essence of socio-technical congruence as a conceptualization over time, to examine the association
of Conway’s between
law holds similarities
for OSS project? inCan
new
socialization patterns and distinction in joining scripts,
stated as an implicit characteristics or property of successful OSS project? to analyze the effect of the difference in the joining process o
retention. Besides, more
• What quantitative significant research
approach/method is required
can be utilized to investigate
to verify contributor
the existence roles in the ecosystem,
of socio-technical congruencetoinanalyze
OSS pro th
of resolved
Whatreview
Our issues
repositories
has foundon onboarding
can that
be used success, to
for this purpose?
the evolution examine the relationship between joining process and
trends and patterns is the most focused research area with 23 papers publishe role transition, and
discover
• WhatThere
topic. effective
correlation
were tenways betoderived
canpapers organize
on the project
between
role information
socio-
of process supportthat
technical may
congruence
in support
evolution. andnewcomers thereduring the onboarding
the quality/sustainability
However, are quiet fewofnumbers period.
OSS projects?
of pap
addressing the characteristics of evolvability and architecture, with five and three papers respectively.
The general costs related to such a migration are unclear [62, 187], and there are very few studies showing complete
calculations of the true costs and savings of (1) introducing OSS products into organizations, and (2) keeping the OSS
products operational over a longer period of time. One paper reports cost savings from an OSS migration project at B
Hospital [81], but it is published just after the initial stage of the project is finished.
Despite this lack of clarity, many organizations seem to be blinded by the perceived advantages of OSS and have the
adopted
As futureitwork,
without
we per- forming
intend anythe
to apply cost-benefit
findings ofevaluations
our study ininmore
theirrecent
own context [91, 187,
OSS projects and190]. Theaadoption
provide validationofofO
furthermore frequently bottom-up, in the sense that it is introduced by engineers rather than strategic top-level
proposed OSS macro process though a practical point of view analyzing OSS projects process activities, their charact decisio
and understanding how roles are involved in each activity, fact that still not clear yet in the literature. Furthermore, w
to investigate how researchers perform OSS process analysis in OSS projects, including approaches, techniques and t
have used to retrieve OSS process information.
In response to RQ2, we found that the OSS studies focusing on evolution process support used different methods, to
approaches for OSS evolution process support. OSS Evolution process support studies have usually focused on evolu
models, exogenous factors, maintenance support, fault detection and change propagation aspects. A very little effort
paid to the other aspects such as Configuration Management, Growth, Complexity and Control, possible evolutionary
etc. SVN/CVS is again found to be the largest explored dataset. The detailed explanation about these aspects is presen
Sect.
During4.4.
our research, we found only a small number of studies on the use of OSS in dentistry. Although the conclusio
from studies support the use of OSS, most of the articles presented a low level of evidence (3b-5) and poor quality o
ing, which makes it difficult to recommend OSS as a clinically useful software. The only study with high-quality repo
a case-controlled study but the conclusions were based on very small research group, three teeth, in what way does it
it possible to say that the results are representative.
This study could help researchers to identify essential quality attributes with which to develop more robust quality mo
are applicable in the various software domains. Also, researchers can compare the existing selection methods in or
determine the most effective. As future work, we intend to model OSS quality assessment as a MCDM problem. This
afford us the opportunity to choose from a range of MCDM methods one (or more) that can be used to evaluate qualit
across multiple domains.
a vital need to improve the quality of reporting empirical studies of OSS. We assert that an improvement in the empir
studies of OSS will help the community to better understand the results and limitations of the reported research.
We have presented a set of guidelines that are expected to help improve the quality of reported studies in OSS-related
We do not claim that the set of guidelines we have proposed is exhaustive or complete. However, we believe that sign
improvements can be made in the quality of reporting empirical research if the future papers on empirical studies of O
provide
Our answerall the information
to RQ1.1 suggested
indicates that thebymajority
the guidelines.
of the articles contribute experimental (case) study to deal with evalu
quality and success of OSS. There are a similar number of articles that contribute new methods / techniques and very
number of articles that introduce tools. The lack of tool development can be interpreted as a lack of agreement on a co
method for measuring the relationship between the success and quality of OSS. Response to RQ1.2 supports that of R
large number of articles are classified as solution proposals. There are relatively less articles with strong empirical ba
indicating
Since the relatively
we were not able to low research
identify anyrigor in this
existing area. that indicated the early effort model for OSS Web project, we
studies
believe that there is a need for researchers to further explore this field. This is particularly relevant as OSS is being
increasingly used nowadays by software provider organizations. This can be supported by the paper of [78], even tho
author in this paper only focuses on effort estimation for software development. However, the author strongly believe
there is a need to develop an effort estimation model especially for OSS projects.
Fur- thermore, the architecture-sensitive metrics for code anomalies discovery provides the majority of awareness to
engineers for the existence of the smells code elements that are more significant to the architecture design than the mo
traditional metrics that are depending on source code and based on static code metrics combination. This means that t
developers and engineers could detect and repair such anomalies promptly. Therefore, more studies are needed in this
other metrics to be analyzed in order to provide the most appropriate architecture without any impact of the size bias.
Furthermore,
As part of ourthere
futureis research,
a need forwemetrics
plan tothat have aa great
conduct ability
compre- to discover
hensive surveythe inconsistent classes
of practitioners affected
to identify by challe
the key the de
from the consistent classes. In addition, there is a need to identify the effort required for the metrics
implementing inner source and propose resolution strategies to overcome. The survey could be also performed to v strategy to archi-
detect related
the signifi- anomalies
cance and also agenda.
of our research to derive more metrics that may have an impact on the quality relationships of other
that are closely related to architectural problems.
As could be observed in fig 3, most of factors which affect the success of OSS are related to developers and product a
of success indicators are related to product. Our study shows that user related factors have been studied less than othe
and researchers limited these factors to number of downloads in both success factors and indicators. On top of this res
gap, we highlight some other gaps that may help future researchers better define and conduct their study in the field:
More empirical and theoretical research is needed on its applicability in context of OSS certification. Important resear
questions include “How do certification concerns shape and impact OSS communities?” and “How to organize open s
communities for effective and economic certification? “
Figure 3 depicts that around 62% of the articles used statistical methods such as regression, time series analysis, cor
analysis, Pearson coefficient, Spearman’s rank correlation, principal component analysis and Bayesian belief network
Statistical analysis methods are found to be more reliable for predicting different aspects for OSS studies as compar
other methods. The second large set of methods employed falls into the category of machine learning algorithms. Ot
methods such as mathematical models, probability methods, Chaos theory and SRGM’s (Software Reliability Growth
have very low exploration.
Regarding the category of examining OSS evolution at software architecture level, we have found that although an in
amount of attention is being paid to the architecture of software systems due to its recognized role in fulfilling the qua
requirements of a system [20], only few papers address OSS evolution at architectural level. Software evolution can b
examined at different levels such as architectural level, detailed design and source code level. We have noticed from t
review that most papers address OSS evolution at source code level. However, software architectures are inevitably s
evolution.
Each metric They
usedexpose the dimensions
for prediction, alongpositively
either being which a system is expected
or negatively to evolve
associated with[22] and provide
prediction basis
results. Forfor softwar
example,
evolution
fault prediction, a metric signifies a module as either being faulty or not faulty. In either case, the metric’s predic-OSS
[37]. Therefore, it is of major importance to put more focus on managing OSS evolution and assessing tion
evolvability at thesignificant
judged as a best, software architecture level In
or bad predictor. besides the code-level
this regard, our reviewevolution.
results show inconsistency on some metric’s
performance. For instance, the metric LOC (Line of Code) was evaluated as a best or good predictor in [1][9], wherea
it was noted as a bad predictor. Moreover, DIT (Depth of Inheritance Tree) was noted as a significant predictor in [6]
classified
Metric setas fora software
bad predictor in [1][4].
evolution. Possible
Software causes studies
evolution behind mostly
these differences in results
utilize metrics might
that are be the variations
empirically validatedininOSp
systems [9], differences in implementations of the metrics [9], or different prediction models used.
studies (as presented in Table III). These metrics are derived for closed source projects, and are primarily used to However, an veri
ind
investigation
Lehman’s lawand resolutionevolution.
of software of such conflicting
Though theseissues wouldprovide
metrics be a future research
valuable agenda.
insight to OSS evolution, they do not co
community dynamics. Thus an empirically validated set of metrics in favor of explicit representation of the communi
required to complement the existing metric set.
Framework for the data collection and representation. As discussed in RQ6, OSS projects often produce large volume
representing their development and evolution history. Research to date, explores the repositories that maintain these d
of which is provided in Figure 6. However, data collection and representation in these repositories vary significantly f
project to project. Furthermore, data from the same source may have different formatting (e.g., emails are often free o
even in listing the senders credentials). Due to these facts, it is a challenging task to collect relevant data following a s
format
Althoughfromall OSS repositories.
research in the fieldIn of
thisOSS
context, researchers
success have triedoften employ
to study their own
previous work,means
but weto observe
collect and represent
little connectiondatab
research. This reduces the compatibility and comparability of the reported results even if they use
them. One exception is reference [1] which has mentioned four previous works in the model and studied them in a lon same data sources.
these
study issues in consideration,
and found a framework
some inconsistencies for uniform
between original data collection
and current and representation
study.We believe thatcan be of
study developed
other work to make th
and com
cohesive and comparable to each other.
the results may lead to considering new factors (such as contextual or longitudinal factors) in study of OSS success.
Regarding RQ 2 and RQ 3, we found that there seems to a lack of research on the use of OI for requirements prioritiz
requirements validation as there was only one paper dealing with these topics each, i.e., papers R_11 and R_19, respe
In addition, we found only one paper (paper R_18, dealing with OI in the context of requirements extraction) that pre
solution approach with tool support. This indicates that there is little automation support mentioned in the literature o
of OI strategies in RE.
tization of designs and fail at espousing an alternative knowledge sharing economy. This points to a gap in open desig
research, where the ways of keeping design solutions open, accessible, replicable, and adaptable while conforming to
regulations and standards is a challenging topic and remains mostly unresolved. Co-operatives and similar models ma
community-based ownership and responsibility, but this model is not as open as open design is espoused to be enacte
affects the reliability of these design solutions, especially when they are not widely reviewed online. Although larger
transitions towards
Researchers alternative economic
have contradictions models are
on the predictive discussed
power on theused
of metrics macro
for level, researchofonOSS
the evolution howstudies.
they will be enact
These cont
development, iteration and dissemination of open designs is still an important area of interest.
are discussed in Sect. 4.2. There is a need of further research to empirically evaluate predictive power of different me
In themetrics
Most reviewed areliterature,
applied onopen
the design is indeed
file level for the framed as pre-
evolution a better alternative
diction of OSSby manyasauthors,
studies especially
highlighted on topics
in Table 19. Wep
new ways to do business, prototype alternative economies, and foster sustainability. From a strictly
analyzed that class level metrics are applied by few studies but method-level metrics are applied by none of the select business perspect
potential of openMoreover,
primary studies. design is observed mainlythat
we also reveal as acode
value-capture strategy
level metrics and a way
are applied to achieve
by most rapidfor
researchers innovation
evolutioncycles
predicf
development
OSS and wide-scale testing. However, open design’s relation to enterprise is still largely considered within th
Fromstudies.
Fig. 5, itLittle
can beattention
observedis paid
that to
47%requirements,
of the quality design and architectural
assessment level metrics
models considered do notformention
predictingtheevolu-
domain tion
of o
paradigm,
studies. while the potential of an open design ‘sharing economy’ is not yet generally discussed as a way to transfor
application. This implies that most of the models were designed to be domain-independent. As such, domain-indepen
way businesses operate. Toward the manufacturing side, companies open up their initial processes but do not develop
should be the focus of model developers (Wagner et al. 2015). A domain independent model is one that is able to asse
alternative models befitting the sharing economy as suggested by open design. Moreover, it remains to be seen if ope
ity in various category of OSS including those that are data-dominant, system software, control-dominant and comp
as a research framing remains semantically and ontologically tied to trajectories of business-as-usual (as has been see
dominant. It should also be able to this with little or no customization. By following this particular consideration, the
software; see Morozov, 2013), and therefore not a true alternative nor necessarily democratizing; if it is increasingly e
proposed can tend to be widely adopted and possibly standardized.
by research on alternative post-capitalist and postcolonial practices; or if a new term becomes more appealing to the r
community and replaces it entirely.
Interestingly, the aerospace domain represented in our study in only 3 papers was the top domain in both related surve
[11]. At the same time, the most represented domain (automotive) in our study was not the most represented in the oth
surveys (ranked the 4th and the 3rd) [10], [11].
Less explored, however still represented by primary studies, is work on OSS for safety-critical automation systems an
maritime systems. Process industries and rail industry (mentioned in the top five domains in the evidence provided i
[10] and [11]) are not represented among the primary studies4. Finally, oil and gas, off-highway equipment and minin
industries
Although we represented in the
considered the previous
barriers assurvey aboutthat
something compliance with
can hinder safety
new- standards
comers’ [11] are not
contributions, represented
some among
barriers can be u
primary studies. Hence, it could conceivably be hypothesized that these industries have yet not been intensified
filters by the projects. Findings from a Halfaker et al. [19] study on Wikipedia newcomers revealed that some entry by sob
systems
to or explored
improved by open
contributions source
in the solutions.
future. More- over, research conducted in the OSS domain [33, 13] demonstrated tha
cialization barriers are useful for maintaining community integration and the quality of the community’s product. A c
direction
The resultsforfrom
future work is totopics
classifying explore how the
showed thatcommunities
studies aim toperceive
predict these
effortbarriers and howactivity
of maintenance they impact theconcentra
mainly quality o
contributions from newcomers.
bug fixing time prediction, while less efforts contributed to other types of activities
Except initial research by Crowston et al. [23], and Crowston et al. work on the definition of OSS success [21] , we d
any general model of OSS success. In fact many researches in the field have just tried to validate their partial model o
success. We believe that according to wide range of social, cultural and technical factors that may have an effect on s
OSS, developing a general model is not reasonable but we recommend contingency practicesin this regard. In other w
suggest researchers to develop general models for specific contexts and believe that these models would be more help
practice.
What sets open source development apart from the traditional proprietary software is the developer community behi
Although the social structure and communication of OSS communities have gained significant research interest, the r
efforts to the community in relation to prediction appear quite the opposite. Evolution of communities is of interest st
from the paper introducing the community structure [13] but our search did not find much focus on community evol
to prediction. In [14] the authors investigate the impact of social structures between developers and end-users on softw
quality
The and their
usability results
of OSS wasgive supportonly
evaluated to thinking
in a fewthat
ar- social structures
eas: mental in thelocalization,
foramen community do hold
upper prediction
airway power
calculation in
in gr
to the source code centric approaches. It is also suggested that combining metrics focusing on code and
patients, an experimental assessment of hard debris in the root canal system after root canal treatment, information g social aspects
a(RSS),
betterpractice
prediction model thanand
management, either
usealone. This gives
for educa- tionalsupport
purposes.thatNone
the question has research
of the studies revealedvalue andof
the use is OSS
worthinlookin
prost
further:iswhat
which does the
currently the most
community and the
intensively community
developing areastructure predict
of digital for theVirtual
dentistry. software?
planning and design of prosthetic
reconstructions provide many opportunities for OSS solutions.
As future work, we intend to review some of the concepts related to ISO 9241-210 [14] and participatory design to r
the main principles and how they were addressed or not in the papers surveyed. Moreover, with the result obtained in
systematic mapping, the gaps and the lack of studies involving the areas of interaction design and development of FL
intend to advance in the research on this subject. Therefore, the next step of our research will be to expand this system
mapping to identify approaches, methods, techniques, and tools for participatory interaction design in distributed soft
development
In the future, environments. Withsome
we aim to conduct this, qualitative
we intend to develop
studies a participatory
to confirm interaction
the problems designby
evidenced process model for
the literature. Wedistri
are
software development environments. Finally, we must extend the Open Source Maturity Model
conducting some interviews with experienced OSS developers and newcomers to verify the main problems faced by [54] with the propose
interaction
newcomersdesign model.
from their Considering
perspective. Wethealsoinher-
plan toentrefine
advantages of softwaremodel
the classification development
based onfollowing
the resultsa software proces
of the interview
we believe that interaction design activities will be considered during the different stages of the FLOSS
analysis. Addition- ally, based on the model, it is possible to propose awareness mechanisms and tools to offer better development
particularly
for newcomers in the early stages.
New evaluation methods are needed to validate the correctness of these estimation methods. With the growth of more
companies developing or collaborating with OSS projects, estimating maintenance effort has become a major interest
researchers have been focusing on improving the estimation towards the direct effort of OSS projects from both peop
activity aspects by developing maintenance effort estimation methods. However, since most OSS projects lack of com
development records and actual effort data, it is very difficult to evaluate and validate the results of these methods by
comparing the estimated results with the actual effort. This can be a significant threat to these estimation methods and
the risks to effectively validate of their results. This is an issue where we need new evaluation methods that can valid
correctness of these estimation methods.
As a result of our findings, we propose the following directions for future research in this area. Focus on the definitio
common model (which may be obtained by merging multiple available approaches) and favor its adoption through rig
and extensive validation in industrial settings. This could increase the validity of the model and thus its dissemination
industry, where OSS is
still not widely adopted. Several models already exist but, according to the results of our SLR, they have not been stro
validated
The overalland, as aofconsequence,
rigor adoption has
the studies performed been both
on OSS, limited.
within organizations and in general, is furthermore not good
Try to target the models at quality factors that are of
Consequently, we should strive to do better work and to present real interestthis
for stakeholders.
work in more Most
detailof the available
[180]. models
In particular, wefocus
agree
overall
Kitchenham et al. [118] in that the context of the organizations being studied should be given much more attention. o
quality of the product, but few of them are able to assess each single factor that composes the overall quality
OSSobserved
We product. that
Thisfewcanof
complicate
the studiesthe assessment
were of OSS
longitudinal andproducts
that few by stakeholder,
publi- who areoninterested
cations focused providinginin-depth
specific detai
qual
factors:
one or a few organizations. To really understand the profound consequences of approaching OSS, we believe theremi
e.g., developers are likely more interested in reliability or testability aspects, while business people may be
interested
for bothshowsin cost
more or maintenance
longitudinal and of factors,case
in-depth etc..
Results that “prediction properties”,studies.
“aggregate metrics” and “changevevolution analysis” are the most eme
Develop
Finally, tools
we foundthat support the research directions listed above (i.e., tools
infor-able to support and simplify the applicability
open issues in OSSevidence that OSS is not
evolution,vtogether withthat
OSS different from other
integrability and licensing. mation technologies. OSS researchers shou
proposed
therefore increasingly rely on research and theories from related fields (see Section 2.4). Software engineering andpro
models during the evaluation of OSS products). Most of the tools mentioned in the primary studies are in
and mostresearchers
systems of them areshould
not available
see OSSorasmaintained anymore.
an opportunity to investigate general software engineering and information sy
research challenges.
There is a need of further research to empir ically evaluate predictive power of different metrics. Most metrics are app
the file level for the evolution prediction of OSS studies as highlighted in Table 19. We also analyzed that class leve
are applied by few studies but method-level metrics are applied by none of the selected primary studies. Moreover, w
reveal that code level metrics are applied by most researchers for evolution prediction of OSS studies. Little attentio
to requirements, design and architectural level metrics for predicting evolution of OSS studies.
This research question evolved due to the fact that most of the reviewed articles (67%) admitted the necessity of ext
validity of the prediction models studied. To be specific, in [4] generalization of the findings was not done because th
subjective and is dependent on how the errors are classified in the project. Again in [9], it is acknowledged that furthe
replication across many OSS projects is required to establish the cross project validity of the prediction model. It is al
that the prediction models are not general and are not applicable to different software systems [10]. Specially for defe
prediction
We mod-
envision els work
future there could
exists perform
very littlea deeper
evidence on their
analysis cross
and projecton
synthesis applicability [5].research
the empirical Thus a on
comprehensive stud
ISS developmen
generalizability issue of the prediction models across the domain of OSS projects is an area of future research.
on this analysis and synthesis, we will further investigate the limitations of the current research on ISS development a
establish a research agenda on inner source. To enhance the findings of this review, we intend to conduct a compre- h
survey of practitioners to identify the key challenges involved in ISS development and propose some resolution strate
overcome the challenges.
there are few studies on MTTSA of interaction design proposed or used for/in FLOSS development;
• methods of interaction design proposed specifically for the development of FLOSS were not found; the studies found
used existing methods of interaction design in the context of FLOSS;
• techniques of interaction design, proposed specifically for the development of FLOSS, were not found; one of the selected papers, Lichtner
used pre-existing techniques and did not consider the distributed development environment of FLOSS;
• the principal interest of the selected studies is in the activities of prototyping and evaluating; few studies have addressed the activities of e
requirements and designing alternatives;
•an
theoverview
majority ofofthethe research
selected methods
studies reported
do not present in these
any type papers,through
of validation a briefempirical
description of these methods, and the number o
studies.
that were classified into each category. We observed a focus on case studies and surveys.
Can a new pharmaceutical be developed entirely through an open source model? Likely not. However, a new drug for
neglected disease may be shepherded up to clinical trials utilizing a hybrid open source model combining open source
other development models such as fee-for-service outsourcing. To assist with this development, we believe that furt
research is needed on business modeling, incentive development and the impact of the use of the public domain. It i
important that this research includes expert input from researchers, the pharmaceutical industry and PDPs to assess th
practicality and relevance of open source drug discovery at a task level.
The areas are important for research and it is interesting to see that research is available in all these areas. The questio
to use open source practices within a closed company (iv) is for example an interesting area for further research.
Based on this review we also propose that further research is conducted on how companies can transform their propri
software to open source and build a community on it. Further research related to all four research questions in Sectio
could involve more case studies on implementation of specific methodologies for dealing with different aspects of op
in industry.
OSS development takes place in an environment which is highly affected by socio-cultural parameters and specificati
users and development teams of OSS may affect or alter the success parameters of OSS. That’s while context of deve
is usually ignored while studying success of OSS. Except [20] that studies specific kind of software and [4] that verif
model in Korean software context, we do not find any other research that was based on a specific context. Even these
papers have tried to generalize their findings and the later one mentioned the context based research as a limitation. S
that
Basedlocalizing the issue ofofsuccess
on the comparison and paying
the existing qualityattention to parameters
assessment suchisas:
models, there social,
clearly nocultural
suitableand economicalmodel
model—each state of
h
development community would be beneficial point of view in future research.
own limitations. As a result, the findings of this analysis have implications especially for practitioners who work towa
coming up with new assessment models. They should note the following points in line with the research questions po
this study: Emphasis should shift from trying to build comprehensive models (containing all the possible software
characteristics) to building models that include only essential quality characteristics. This study has shown that these
quality
The characteristics
results include:mapping
of this systematic maintainability, usability
suggest the andbroad
need for maintenance
support capacity
for FLOSS of software community.
projects and By narrow
communities by th
down
community, through research efforts in the area of interaction design for the availability of MTTSA of interactioneval
to these three essential quality characteristics, model developers would help to reduce the burden of OSS des
via existing quality
considering assessmentof
the characteristics models,
FLOSSwhich has been Therefore,
development. referred to it
largely as being
is necessary tolaborious
develop andandpublish
time consuming to c
research on
interaction design in the context of FLOSS.
This study also indicates the areas such as joining process and abandonment where research is lacking. Moreover, m
is another field that needs to be explored in the future. Furthermore, significant research is required to explore the too
practices, and processes that could be helpful to improve community participation. The research community may us
findings to understand the issues in the area and select the topic for further research. Additionally, we also observed f
results that in majority of the studies, combination of survey and questionnaire was used as a research methodology to
research.
Study theFrom theseofresults,
existence we observed
SOC. Another that few
direction studies used
of research would machine learning
be to study methods.
the notion In future
of SOC (Selfuse of these Criti
Organized tech
will be helpful to solve the problems related to task distribution, selecting a task to start contribution,
OSS projects. SOC dynamics articulate that the current state of a project is determined (or at least, heavily influenced and managemen
project information.
events that took placeFurthermore,
long time ago. findings indicate
Existential that majority
exploration of SOCof the studies
in the and of
domain researchers
OSS projectswho reveals
investi-contradict
gated com
dynamics
results belong
(Table V).to USA.
Thus We research
future also observed
can takethatfurther
most ofstep
theinstudies (75%)
validating thewere conducted
existence of SOC by the
andresearch group of
its implication ono
country. Thus more collaboration
evolution of open source software. is required among research groups of different countries to conduct more research i
area.
Notice that there are a huge number of publications, which report and interpret results from qualitative and quantitativ
to identify possible risks. It stands out that only [SLR40] calculates threshold values for defining bug risks, and evalu
performance, while no paper identifies risks based on quantitative data of project failure or created losses and revenue
correlates project failures and losses with quantitative data such as the number of bug reports and bug fixes.
Few papers consider quantitative measures on community qualities (number of contributors, activity, presence of hero
[SLR39]
None etc.),
of the whilediscussed
studies no work empirically evaluatesanthe
the need to develop existence
early of causal relationships
effort estimation for OSS Webbetween the metrics
projects.From applied,
the findings,
identified and the actual faults happening. Moreover, an empirical evaluation of the effectiveness
shown that there have been no studies that discussed the need to develop early effort estimation for OSS Web project of mitigation activit
their
per seen in several papers recommending future work, the authors only mention the need to improve the model orAlso
influence on the retrieved metrics is also missing in the works, which propose specific mitigation activities. to c
complete works,
more detailed such as theorones
comparisons by the
propose thegroup [SLR12][SLR19][SLR20][SLR47],
involvement of different effort measurement whose surveyssuch
attributes, retrieve data in
as [28], forwh ris
mitigation
future workactivities, do not
is to conduct show
more any link
detailed between these
comparisons two: different
by using typical mitigation
estimation activities adopted in in
tools. Meanwhile, software compa
the paper [10
very general
authors (see
indicate a Section
plan to 1.2.5)
compare and would
different reduce various
estimation risks.
methods and different effort measurement attributes such as th
Some of the analysed approaches propose quantitative measures and analyse their effectiveness (e.g. [SLR4][SLR6][S
technologies (e.g., J2EE versus .NET), and the development processes (e.g. RUP versus
in the SLR for API metrics and code changes, [SLR14] for code executability, [SLR25] for business values). [SLR17 Spiral). In another paper [28
authors
proposesonly expressed the intention to enlarge the dataset and to develop a tool to automate Cosmic Function Point (C
a reliability
counting
model combiningbased
process on theand
qualitative UWE models. Therefore,
quantitative metrics, butthis
doeslednot
to us to further
consider investigatecommunity
OSS-specific how OSS can andimpact the e
repository
estimation.
We would in particular recommend investigating two issues: (1) topics related to integration of OSS components an
topics concerning participation in organization-community or inter-organizational OSS collaborations. We find thes
important because integration of OSS components concerns most software-intensive organizations [98] and because
participation in collaborative software development is increasing [7, 185]. The research could focus on identifying the
characteristics of successful approaches to OSS, the challenges these organizations met, and how they solved them.
Deploying
Our reviewOSS:
resultsMany claim
showed thatthat
thereducing
medical costs is one
literature onofthe
thetopic
advantages
of OSS of in deploying
dentistry isOSS server
limited andsoftware,
includes infrastru
mostly e
applications.
opinion and case-control studies.The second suggestion is to include OSS as a control group in experimental studiesOo
However, a recent study by Fitzgerald [80] is one of few studies with a longitudinal view on deployed
products. This highlights
software validation. Suchacompara-
need for tivemoreanalysis
studies can on: have positive effects for the commercial programming vendors
􏰅 What are
showing long-term
them the mostcosts and consequences
advantageous of deploying
OSS solutions that canand keeping OSS
be deployed into products
commercialoperational?
software packages.
It may also be beneficial for customers who, apart from obtaining detailed information on the performance of softwar
packages, might are
types in general be able to decide
directly if thefrom
influenced risk oftheusing OSS with-ofout
interpretation technical
different support and
stakeholder requiringBelow
viewpoints. greaterwecomputer
highligh
justified
open issues:in specific cases.
• Since none of the analyzed papers define openness, identifying openness dimensions of IoT platforms remains a cha
be addressed.
• It is of utmost importance to find a consensus regarding openness among the different stakeholders to avoid confusi
preferably
Since agree only
this study on a focuses
formal definition.
on the involvement of OSS tools towards the development, another future research area
•beInvestigating openness not only
investigated is measuring the other fromaspect
IoT platform perspective,
of the effort but also
measurement considering
attribute IoTthe
such as middleware and framework
year of experience of dev
•toward
To understand how much openness of IoT platforms has penetrated the field and in which
the OSS as well the error fixing time. As is well known, a year of experience of the expert’s programming domains, a mapping study
ski
application
are involveddomains of the identified open
in the projectdevelopment canIoT platforms
contribute to awould be useful.
different effort, therefore how about a year of experience o
expert toward the tools. Does this affect the effort estimation? Since OSS is an open source which everyone can acces
therefore
This studya suggests
bug that can
some occur duringfor
directions thefuture
implementation cannot be COSS
research. Disruptive avoided. As such,
revenue what organizational
models, would be the time and of
aspects effC
needed to fix the bug and how this can affect the effort can also be further investigated.
localization of COSS, and creation and maintenance of the user community are among them. On the other hand, pract
should notice that the points of different aspects of the COSS business model presented here to improve the cost-bene
offs of their business.
In the anomaly’s prioritization strategy, the agglomeration flood standard and most optimal models showed that some
agglomerations are overburden with false positives and not precise enough to identify architectural inconsistencies in
leading to the inability to capture several various architectural problem types. In contrast, the recommended heuristics
architecture blueprints, and the context-based smell prioritization techniques show the ability to rank and improve in
identifying the prioritization of anomalies related to architectural problems. This reveals the need for providing an in
enrichment
We analyzedofthe thecharacteristics
possible resultsandto goals
adoptof thethe
solution with aHowever,
newcomers. tendency inmanythe of
ideal
thecombination
papers did notto explicitly
prioritize architect
profile t
anomalies. Consequently, there is a need to provide various prioritization criteria for seizing the diverse
newcomers they analyzed. This is probably related to the type of data analyzed and the type of study conducted, as m architectural
types
studiesand enhancing
only used datathe essen-from
coming tial techniques used for
software repos- discovering
itories code
and did not goanomalies.
deeper in theMore- over,of
analysis thethe
integration of twp
subjects. The
heuristics
that the term newcomer can be used in a loose way, which can bias the results. Newcomers can be novice developersto
would get better ranking results’ effectiveness. Additionally, it is possible to introduce the new strategies
ranking
starting on numerous
their criteriawho
in orderexperienced
to provide visualizationfromcapabilities
industrythat are most relevant to architectural probl
we claim thatcareer, people
emergent researcharead- dressing thedevelopers
current mobile-platforms but areconsidering,
is not not used to OSS projects, previous
or exploiting or people
the developers.
migrating from other OSS projects.
works on open-source platforms, as These
it oftenthree profiles are different and can face different barriers or experience bar
should.
differently. Therefore, it would be a better approach to assess how these different types of developers see the barriers
their impressions of them are. For example, does a novice developer find more issues to contribute than an experience
developer without an OSS background?
As part of our future work, we are currently conducting a study investigating the effectiveness of current estimation m
especially Bayesian Network (BN) towards OSS Web application development. In the paper [79], the author had imp
Fuzzy Logic into Tukutuku dataset, and proclaimed that the result of implementing Fuzzy logic is better than other m
Another paper [80], also focused on the implementation of Fuzzy Logic toward effort estimation Web application, bu
study the author only emphasized how well Fuzzy Logic could be implemented into the field. In other words, the auth
highlighted the effectiveness level of fuzzy but does not cover the OSS perspectives as well. Therefore, in addition, w
looking into how well the level of effectiveness can be achieved by implementing fuzzy logic conditional probabilitie
characteristics into BN network for Web application projects that involve the usage of OSS.
The evaluation of OSS in medicine should not be overlooked. Professional medical certification (FDA and CE) of- t
be applied to OSS because it requires a legal commercial entity for distribution liability and support availability. It
important that FDA and CE certification confirm that the software and its provider ensure an error-free work- flow, b
accuracy of software actions/calculations, and provide appropriate documentation and support. Therefore, ex- perime
studies and algorithm evaluations should be conducted and published.
Acknowledging the lack of published research on the use of OI strategies in specific RE activities, i.e., prioritization a
validation, as well as the lack of reported tool support, we see new opportunities for research on automated and thus n
intrusive and low-cost methods for applying OI strategies in RE. More specifically, we think this mapping study prov
with a motivation to conduct more research on capitalizing upon OI to automatically extract and prioritize requiremen
In both cases of evolution prediction and evolution process support, the reviewed articles admitted the necessity of ex
validity, whereas the ratio of articles not addressing validation process is considerably higher (56% for evolution pred
and 68% for evolution process support). We real ized that vast heterogeneity of evolution prediction models building
make their evaluation difficult. Generalization of evolution prediction models regardless of their applicability on pro
requires the attention of researchers.
Reviewing the related work in the field of OSS success, we observe different measures and factors for success and no
different methods are used in research in the field but source of data is mainly repositories of OSS projects such as
sourceforge.net and freshmeat.net. We mainly recommend using variety of methods for research in the field and also
draw attention of potential research to context of OSS development in future research.
When contrasting previous literature on older “platforms-wars”, such as the ones from the PC and game-console indu
with the current and under-studied mobile platforms-war, we empirically notice that many of the market players rema
same (Microsoft and Apple). There is a scenario of convergence: same firms push for similar technological standards
different platforms, i.e. Microsoft Windows within X-box, Surface Tablets, PC, Netbooks and Mobile phones. This
convergence between industries remains unexplored by academia. Interesting research questions dealing with the imp
of such convergence
Despite having a closeremain unexplored,
relation i.e research
to the CASE “should firms
field, concentrate on one platform-war
only seven experience or runthe
reports discuss several platform-w
use of OSS CASE
parallel?
the context of organizations. Given the number of OSS CASE tools available, it is surprising that the use of such too
been studied in any empirical research papers.
Assesment Process: It is worth mentioning that to perform a complete quality assessment of a software ecosystem we
would need to define the assessment process which is out of the scope of this paper. The quality assessment proces
have to deal with, e.g., How are the values of each measure interpreted (i.e., defining what are the good and the bad v
How can the measures be merged to provide the assessment for a particular sub- characteristic of the quality model?;
are the principles to perform the assessment with missing, incorrect, and/or inconsistent measure data? We are will pr
answeron
Based to the
these and other
findings questions
of this as part
research, of our
we have future the
planned work in this topic.
following research objectives, to be carried out in the nea
to more strengthen the existing research work, and to stress the OSS vendors’ community to meet the maximum bene
OSS paradigm.
• To identify the practices for addressing the identified success factors
• To identify the potential risk factors in adapting open-source software development from vendors’ perspective
•Unbalanced
To identifyDistribution
the practicesoffor addressing
Measures: justthe
byidentified success
looking into factors and
the measure riskitfactors
tables, is easy to observe that the amount of
•forTosome
develop open-source software development maturity model (OSSDMM)
characteristics is high (e.g., activeness with 17 mfeasures, visibility with to measure the maturity
11 measures) while level of vendor
for other is ver
organization in implementing open-source development strategy
(e.g., heterogeneity with 1 measure, information consistency with 1 measure). This unbalanced situation could be an i
•that
To more
conduct multiple
research case studies
is needed for theatcharacteristics
software vendor organizations
with a low amount to of
evaluate the efficacy of the model
measures.
As a future work, this relationship will be reviewed in-depth with respect to RQ4 in order to expand the information g
by this SM study and elicit further insights.
Despite the scarce references to learning assessment in the selected papers, some studies assessed the experiences fro
the teacher and the student perspective. They also used various different instruments. The main issues with the teache
perspective to assessment is the absence of clear definitions of criteria to assess students’ products, performed tasks, a
expected skills and attitudes. Therefore, it is important to state that student assessment deserves more thorough work.
Morelli, de Lanerolle, and Hislop (2007) point out that students can perform various types of contributions. Since the
different
We plan to back-
extendgrounds and previous
this work experiences,
in the future they fulfil
by concentrating ondifferent roles and
the identified perform
research gapsComputer Science Education
and by introducing more re
contributions. Therefore, the authors recognize that grading is not an easy task.
questions to acquire in-depth knowledge about community participation and engagement. In the future,They suggest the need to establish
this studyacas
metrics
extendedfortoeach rolesoftware
include a studentecosystem,
plays. community structure, mentoring, project governance, difficulties associated
Learning how to solve complex
finding a task,and project characteristics problems, andwill
that knowing
providehowmoreto insights
work in inteams are relevantdynamics
the community skills thatdomain.
are typical requi
Furtherm
in
study can be repeated by using other research techniques such as a combination of systematic mapping study and systi
SE education. Thus, students should develop some skills such as communication and leadership. Peer assessment
important
literature instrument to used
approach this need.
betterMorelli and de Lanerolle (2009)aresustain that
ningstudents should assessbytheir
Thus, pro-review can be
vides more insightstoabout
obtain research results. Moreover,
topics and issues in thethe
authors plan-helpful
area that will tofor
expand the search
researchers. Furthermoinc
conjunction
more terms with
in the teacher
search assessment
strings and in courses
additional on
dataOSP, even
sources, though,
not the
considered adoption
by this of this
study practice
to find is
more still a challenge.
relevant studie
research should focus on newcomers’ orientation and reception to identify the problems in initial contribution as well
In any activedynamics.
community learning approach, students are responsible to conduct their own learning. A formative evaluation that in
provide guidance regarding barriers in advance. Moreover, significant research is required to explore the means to sup
self-, peer and faculty assessment can play an important role in this process. According to Ellis et al. (2012), the iterat
newcomer initial participation, enhancing motivation to increase participation, and reduce the hindering factors that
development process present in OSP, where any artefact can be reviewed by different people, and in multiple times, p
improve retention of new developers and One Time Contributors (OTC). Abandonment is another area in OSS comm
an intrinsic and valuable environment of formative evaluation.
and
As has many issuesVthat
andneed
TabletoVI,
be explored in future research. Additionally, further research is required to explore v
All shown in Table
those issues represent research the SLR found
opportunities relatively
to be a few
more thoroughly studies providing
explored in thesufficient
future. details regarding the
factors that lead developers to stay or leave a project.
advantages and challenges of using OS in computer science education. Moreover, some of the advantages and challen
supported by a few of the papers found. For instance, only 4 papers (19%) included specific evidence regarding the w
range of skills acquired through the use of OS, which suggests that there is a lack of data regarding the advantages an
challenges of OS use in the practical situations. Consequently, the results of this study may not be easily generalised a
further
This investigations
reveals must bemay
that architecture planned in practical
be recovered situations
with where
acceptable OS is unlike
accuracy currently
theused
priorinintuition
computer science
based educatio
on its claime
hypothesis in an application difficulty to recover conceptual architecture. However, there is a clear opportunity for ma
researchers to shed light on checking more efficient approaches to architecture recovery by enriching ARCADE’s to
further different current code-level analyses to it. Besides, there is also a need to conduct more experiments on a wide
more systems, especially industrial systems as well as to increase approach accuracy through documentations, pull re
commit messages,
We furthermore comments,
advise tests,to
researchers andputmuch more.on how the studied organizations actually use OSS, and on proble
emphasis
really matter to practitioners. Practitioners should be open to OSS and see that it offers several opportunities. Howeve
must first evaluate the implications of adopting OSS in their own context.
Another interesting topic that deserves attention in future research is the emerging body of literature on agile and dist
development (Angioni et al., 2005; Ramesh et al., 2006). The coming together of these two worlds was not explored i
study, but we consider that they can also contribute to facilitate reconciliation among plan-driven, agile and FOSS dev
opment models.Likewise, the idea of processes diversity (where different processes may be running concurrently on t
project—in multi-team projects or changing over time during the phases of the development and maintenance cycle) (
2001;
Many Lindvall and Rus,
of the studies 2000;
are in Siebel
the form ofetsurveys,
al., 2003) alsogives
which should be investi-
a broad gated in future
and necessary research, Based
understanding. since the
on need
this ittowo
m
this diversity can be an important motivation for the reconciliation among software development models.Finally,
probably be possible to conduct more studies investigating specific cases of implementation of methodologies for dea appr
that deal with
different reconcil-
aspects iation
of open considering
source organization
in industry. More caseand project
studies contexts
could and be
probably needs, appearsontoall
conducted beaspects
a promising
of thepath
res
reconciliation in the future (Jaufman and Munch, 2005; Xu and Ramesh, 2008). This is the focus
questions. More case studies could probably also provide more knowledge of research question 3 and research of the next section.
questio
is, research could be carried out to understand more about the cost and advantages of different approaches, and why
approaches are chosen. It is also worth noticing that there are no controlled experiments at all in the identified articles
Furthermore, our study has revealed that the quality of reported empirical research on OSS has significant room for improvement. To that en
proposed a set of guidelines for reporting empirical research on OSS. We claim that these guidelines can help the OSS research community
improve the quality of designing and reporting empirical studies.
The analysis of the results allows us to state that OS- SECO is a growing research area in software engineering [R16
R50]. Due to this, there are several new research opportunities in the empirical examination, modelling, analysis, me
quality evaluation, etc. of OSSECOs. Along with this argumentation, in this section we provide two initial proposals
improve the current structure of the knowledge on OSSECOs: a definition for OSSECOs and a taxonomy of OSSECO
terms.
COSS is comparable to opensourcing. In opensourcing, the companies outsource to the open source community outsi
company [82]. This lowers the cost of development, because on the one hand, volunteer developers code for free, and
other hand, users report flaws in the software [43]. Hence, the existence of an open source community is vital to the s
the open sourcing company [82]. In COSS also exists a user community and somehow developer is effective and imp
[3]. But, the problem is creating and sustaining such communities [41, 43]. Therefore, it is imperative that the manage
the creation
On the otherand hand, maintenance
studies such of asthethose
user conducted
communitybyinSteinmacher
future research be[PS14]
et al. investigated;
and Jensen an issue
et al.which
[PS9]has not beensim
presented se
considered in the literature of COSS.
views of the problem when they drew conclusions from only analyzing the first messages from newcomers and their
The next point
retention. is that, as
The context Riehle noted,
is important: Why open
didsource
they sendsoftware can possess
the messages? Whatestablished
motivatedmarkets,
them? Did provided that itwant
they really is suffic
to
disruptive
or [1].some
just clarify For this disruptiveness,
doubt? we suggest
Did they contribute working
at the end but onnever
the revenue
got back model.
to theInmailing
fact, since
list? the
To business
answer such model quesis
and
need to merge information from different sources (issue tracker, mailing lists, documentation, code repository) and Tv
every change in one component also affects the rest [4], the initiation of this disruption can be a revenue model.
it is suggested
context by talking that tofuture studies inAnother
practitioners. helping possibility
to describeismore revenueobservational
to conduct models examine ethnographic
the role of new revenue model
Since 2008, synthesizers of research have introduced frameworks and platforms toand perform OSS research studies by analyz
paving the
adding
barriers to the disruptiveness
and effects of
for newcomers the open
in real source software.
settings. This is because the explanation of open source software rev
future work. The analysis of non-cited papers indicates that significant research has not been exploited, yet. Therefore
models has been difficult, especially because of free distribution [83]. In addition, this study suggests that the provisio
recommend the OSS community to exploit further the potential provided by the OSS conference series while maintain
complementarities has been an integral part of the COSS business model as a way of earning money. So, new disrupt
interest in its major research streams.
revenue models can help new configuration of the COSS business model and probably possess established markets.
Due to the cultural, economic, institutional, geographic and other characteristics of developing countries with emergin
Most
markets used sources
[84], the use development
of businesstoolsmodels of the data are not
of developed properly
and matured documented.
markets is oftenNoneunsuccessful
of the study data [58].documented
Therefore, the w
there was any
the creation and involvement of OSSvalue
capture of import features in the may
of COSS development.From
need to be adjusted. the result
Thiswe knowthe
implies thatneed
the most used sources
to localize the COS
development tools mentioned in the data are not properly documented, as can
business model [85]. In addition, countries are demanding indigenous and localized software, but due to expensivebe seen from the result, with the highes lic
percentage
some of them are looking for open source software [86]; a localized open source software. But, in the literature of
being unidentified development sources. Due to the poor documentation regarding the involvement of the
the
development
business model, tools,notwe onlyhave
thenot been able to determine
commercialization of openwhether in each ofasthe
source software projects,model
a business was there
is notany involvement
considered, but ofth
Since this
also fewwork
Future SLR
studies
could is focused
oninclude on OSS,
the openthesource therefore
software
expansion we can say
localization.
of this that to the best
Therefore, itsome
research, overcoming of our knowledge,
is suggested. there
Finally, there
stated limitations has been very
is not much
by increasing little
researc
number o
documentation
the organization
universities or by ofandOSS usage
about
considering in
what the
only datasets, which
is happening
experienced inside
Open led us to further
a Source
COSS company. investigate
practitioners. Thishow
Therefore, one
way, OSS
we canimplications
of would
the affect
be ablethe accuracy
to for future
provide ofa erm
estimation.
that it is suggested
effective feedback to that
thefuture researchAnother
researchers. examinepossibility
the structural
is todimensions and content
assess the survey dimensions
participants [87] ofregarding
motivations the COSSthe
organization.
provided. With a greater amount of participants, this could be achieved without extending the consumed time, which
affect the quality of the results.
Newer models should incorporate selection meth-ods that are amenable to automation as this is not the case in most o
existing OSS quality assessment models reviewed in this study. The selection methods mostly adopted are the model
process (21%) and other (16%) such as guidelines, which are not easily amenable to automation (Fahmy et al. 2012).
developers should thus turn their focus to data mining techniques (Leopairote et al. 2013), framework or tool-based se
methods, which are currently among the least considered options. The advantage this offers is that it will help quicken
evaluation process resulting in faster decision- making. Following this advice could also bring about increased adopti
models in practice
As discussed in Section(Wang et al.there
IV.D, 2013). In addition,
is no model developers
concrete relationship can also
set between consider
quality and modeling qualityofassessment
success criteria OSS in thea
criteria
papers. decision-making
The lack of studies (MCDM)
examining problem so as to facilitate
the relationship between automation
quality and assuccess
seen in criteria
recent studies
and metrics(Fakirshould
and Canbola
encour
Cavus
researchers to conduct further studies in this context. In addition, there is no satisfactory number of studies in the alter
2010, 2011). A MCDM problem in this context can be regarded as a process of choosing among available con
(i.e.
typesdiffer-
of model,ent OSS metricalternatives)
or tool, andbased on a number
it is observed thatof attrib-
there is anutes (quality
evident criteria).
necessity to Considering
fill the gap for thisthese
option opens
types. the
Since
developer
measuringto theseveral
success well-known
or quality is MCDM methodstask,
a challenging thatespecially
amenable tool to automation
contribution such as: DEA,
is quite minimal.AnalyticThisHierarchy
may lead Pro pra
(AHP), and Technique for Order of Preference by Similarity to Ideal Solu- tion (TOPSIS)
to collaborate with researchers in order to clarify terminology, identify metrics, and develop tools that are capable to mention a few (Zavadsk of
2014).
this need.
It is worth noting that the main focus of analysis were large projects with a high number of developers and more than
years of existence. Moreover, projects that focused on products used during the development cycle and developed in
C were preferred. Such projects can be classified as clearly successful projects which, combined with the historical da
available, provide an easy target to search for newcomers. We observed that, although projects gain several newcome
small percentage
Maturing are successful
the research field on OSSin contributing someand
in organizations source code.with
dealing Because
some the identification
of its of barriers
limitations may be donefaced and sur
through fou
such
steps: newcomers is important, projects with a high number of developers (and newcomers) are easier to analyze to fin
evidence of such barriers. However, a high number of OSS projects
1. Focus research on topics that are relevant to how organizations approach OSS present different characteristics, such as small tea
short
2. lifetime,
Strive and were
to increase not considered
the rigor for evaluation.
of the empirical studies Naturally, such projects provide less data and are less attractiv
large
3. Conduct more longitudinal, in-depth studies newcomers, they can account for different problems than those identi
successful projects, but when considering
our
4. model
Align orresearch
our modify their
with importance. Further
relatedofresearch investigation is required regarding such projects to improve the model
fields
Navigating through the amounts OSS components and related information available across the Internet is a signif
barriers described in this paper.
challenge [143]. However, the information offered over the Internet through OSS communities, web fo- rums, and so
constitutes at the same time a valuable resource. Due to the easy access to reusable software components, we see that
systems are constantly growing. Software developers are integrating an increasingly larger number of OSS and comm
components into their products. In doing so they have to relate, adapt, and possibly contribute to a large number of p
Therefore,
Some weand
trends believe
issuesresearch
emergedcouldfromfocus on theanalysis
a detailed following questions:
of the studies: (i) solution proposals are the main research a
􏰅 How may organizations most efficiently navigate through available
(ii) very few papers focus on specific SE areas; (iii) the traditional project information
method is and selectlearning
the main OSS components?
approach; (iv)
studies use previously chosen OSP in regular courses; (v) there is a balance between inside and no control approaches
very few papers use criteria to evaluate students’ learning based on either outcomes or developed skills. We also foun
main combinations of OSP use: (a) full control and predefined projects, (b) no control and free choice projects and (c
control with papers
n fact, most no or almost nomore
do little projectthanchoice for students.
mention These
such issues trends and
as research issues provide
questions, future
limitations, directions
data analysis,for
andresearch
so on
furthermore found that many of the publications lack details about the research methods and findings. As a consequen
several papers have limitations when it comes to how they describe these issues. Moreover, many of the research pap
explorative and they are therefore lacking a precise focus and clear contributions.
It will be worthwhile to explore the capability model for OSS developers. Since most OSS projects rely on task or iss
tracking systems to maintain the projects, recognizing the time of specific maintenance tasks can provide better decisi
port for task assignment as well as OSS project management. A large amount of studies are devoted to predicting bug
time while a small amount focused on other activities such as code review and duplication identification. These studie
commonly used metrics from source code changes and issue reports as predictors, which indicate that the prediction r
basically
Our future related
workto theextend
will characteristics of the
this analysis to tasks. Therehubs
OSS article are two
likekinds of targets
for example among these predictions.
http://flosshub.org or One is the
numerical days of an activity,
http://pascal.case.unibz.it evaluated
or the by PRED(25) or PRED(50). Another is the bins of categorical time evaluated
MIT repository.
We plan toprecision,
accuracy, use morerecall,
sophisticated techniques
or f-measure. of string results
The prediction similarity
forand a betterdays
numerical dataarecleansing
not verytosatisfying,
get finer result
while the r
categorical bins are relatively high. In OSS projects, the time recorded on issue tracking system and repository may n
correlate with the actual effort because the developers are voluntary and self-determined when implementing the task
be worthwhile to explore the capability model for OSS developers and consider the developer-related metrics to be on
The results shows that security areas in construction and verification(secure architecture, code review, and security testing) are followed by r
sources
with moreof predictors,
interests as an
than other opportunity
areas to improve
in Governance the prediction
and deployment. Next basedresults.
on our research, the security studies in OSS development are
technical driven. The socio-technical perspective has not gained much attention in this research area (2 out of 42 papers). According to the re
socio-technical analysis on the selected papers, the discussions between technical and social aspects seem quite unbalanced, either (Coverage
versus 16% in average). The socio-technical perspective has as the main target to blend both the technical and the social systems in an organ
This can be viewed as a necessary condition within a security management framework as both aspects are of equal importance. Technical sec
practice considering different social aspects (e.g., culture and structure) of open source develoopment will assure the effectiveness and effice
A few other issues are worth mentioning. First, all of the eight empirical research papers from the public sector focus
implementation of the tool.
deployment of OSS products. Besides [37], which has a mixed sample, no paper focuses on deploying OSS in the p
sector. Second, 27 of the 59 empirical research papers report findings from samples of several organizations from the
sector. However, as few as eight papers report findings from one single private organization. Hence, most research p
dedicate relatively little space to describing the individual organizations.
First, regarding research types, it is worth highlighting that the high number of evaluation papers and the increase of v
idation contributions reflects the maturity FLOSS adoption studies. However, this research area is not yet to the point
contributing experience reports. Second, regarding FLOSS adoption factors we observe that most of the studies have
focusing on the organizational and technological factors leaving the economic factors not so well covered. We suspec
lack of research results in economic factors is due the reluctance of companies to provide economic details. Also, FLO
experts consider
Software size hasthatbeen organizations
the most common are already aware
attribute of the hidden
to analyze evolutioncostsofwhen
OSSadopting
projects. FLOSS,
Several typesand therefore,
of metricsthey havt
focus more on researching technological and organizational factors.
employed to measure software size. These metrics range from coarse grained level metrics such as number of files, m Additionally, we only found two solution propo
related
and functions, with economic factors level
to fine grained and one with such
metrics tech-as nological
number and of LOC, organizational
methods, and factors. We also
classes. observed
Several that validatio
approaches, other
research,
source codeopinion
analysispapers
using and philosophical
metrics, to analyze papers
OSSare gaining have
evolution maturityalsoin the employed
been FLOSS adoption area because
in the research we found
literature;
taxonomies,
• Lately, metrics literature
related reviews
to change and activity
systematic have maps.
also been included to understand OSS evolution. These metrics meas
changes in source code such
Individual contribution and performance measurementas number of program elements
also has (functions/ /classes/methods)
been receiving attention. Gousios changed [15]in consecutive
defined a contr ve
Change activity as recorded in SCM systems is also used in a few cases.
ratio by considering various type of parameters from the OSS community, and Rastogi et al. defined the contribution Most of the work deals with finding change
change
of four differenteffort distributions. A few studies
roles of stakeholder. Since dothe
change profileof
importance analysis as OSSmeasurement
contribution systems evolve. But that
has been is restricted
realized, it mighttoba
the change categories e.g.
promising research topic in the coming years. adaptive v/s non-adaptive changes, or corrective v/s non-corrective changes. A fine-grained
the changes can help to answer amount of progressive/ regressive work performed in a software system as it evolves.
also be used to validate Lehman’s 2nd law as Gonzalez-Barahona (2013) points to the lack of information available in
The first opportunity for future research lies in reexecution of the protocol, to capture references to more recent work
regard in their study of the glibc system;
extends the search space chronologically. This could also include other search engines, such as ACM (Association fo
• Techniques and tools have been devised to tackle large amounts of data generated in software evolution analysis and
Computing Machinery), in an attempt to retrieve documents only indexed by these machines, which would extend the
prediction. Software evolution visualization helps in understanding the transitions in complex and large systems in an
space geographically. Finally, the search can also be expanded with manual searches to include: books; conferences
way. Big data analytics can also help to analyze large sets of data generated during software evolution. Data analytics
and dissertations; technical reports; and other search engines, such as Google Scholar and AISeL (Association for Inf
used to manage and understand the complex web of software evolution as it happens in source code and other related
Systems
Consequently, Electronic Library). Although the systematic approach adopted ensures the reliability and com- pleteness of
repositories. the healthcare industry is “lagging behind in terms of adoption of modern ICT tools and infrastructure”
study,
compared to be it can amplified
other sectors by these extensions.In
(Karopka et al., 2014; Section
Munoz-Cornejo7 (SQ2),etweal., discussed
2008). how studies that showed how to com-
development models could be extended to the third. This discussion included ideas like: (i) the reconciliation of FOSS
plan-driven configuration management practices to agile model; (ii) the need for further studies on the reconciliation
practice of continuous code integration between agile and FOSS models and extension of this search to the context of
driven model; (iii) investigate how the knowledge management practices in agile model can contribute to improve kn
A mapping study review provides a structure for a research report type, which enables categorizing and giving a visua
management in organizations or in FOSS communities; (iv) analyze if the use of explicit knowledge management pr
summary of results that have been published in papers of a research area[8]. This map aids to identify gaps in a rese
coming from plan-driven and FOSS development, can be beneficial in an agile context; (v) extrapolate the use of tes
becoming a basis to guide new research activities[7]. The current mapping review captured the current state of researc
development to a plan-driven context. These ideas are all opportunities for future research.An area that still deserves t
report severity prediction, characterized related problems and identified the main approaches employed to solve them
explored is the search for studies to reconcile the three models of software development, since only one study was ide
objectives were reached by conducting a mapping of existing literature. In total, the review identified 27 relevant pa
this quasi-systematic review. First of all, it can be present in areas that were not the focus of this work. Some of these
analyzed them along 12 dimensions. Although these papers have made valuable contributions in bug report severity p
understudied areas are indicated below to guide future research. In addition, as stated before, the reconciliation resear
the
This panorama alsopresented insome
this mapping study review suggests that there are potential research opportunities for furth
still atstudy an early identifies
stage, but it challenging
is expected areas
that in future for there
future work.
are more studies and results on the topic to be investigated.
improvements
1. Append newinfindings
this topic.intoAmong
the body them, the following
of knowledge research directions appear to be more promising:
there also remains the possibility that organizations oronother
OS forking
researchers behaviour. Applying
have already the combined
achieved positive approaches
results on the of
•CAM Thererevealed
is an apparent
seven lack
forking of investigation
types interpreted on bug
by report
academic severity pre-
researchers diction
and thein other
latest relevant
interpretation FLOSS foundsuchis as,
file for ex
langu
reconciliation among agile, FOSS, and plan-driven models, but have not written about it yet.
Linux Kernel, Ubuntu Linux, and MySQL, and in others BTS, for example,
repository fork. This novel insight will assist researchers on how forking is presented and interpreted and industry pra Github.
•inOften, technical users reporthealth,
most bugs. Thus,projects
the influence of user experience infile predicting outcomes is less
still adopte
overlo
Asreviewing
future work, project forking
it is necessary especially
to investigate in real software with programing
projects, in language repositories
software organizations and thatin are
open-source so
•forked
Bug reports
by labeled
developers. with default severity level (often “normal”) were prevalent in the most datasets used in reviewed
projects, what are the influence factors observed by their developers. Comparing the results of this SLR with real obs
However,
2. Understandingthey areforking
considered unreliable [26], studies
consequences. and justare discarding themway also to
does not seem appropriate.
learnt byThen, efforts
may clarify the importance of influencingCase factors within an context
the important highlight
of productivity withinlessons
organizations researchers.
and in open-
researching
paper on novel
identified forking ap-impacts
proaches andto consequences,
handle this typewith of report
one ofshouldthe worstbe considered
impacts being to improve the
a political state-of-the-art
strategy that divideo
communities.
severity prediction algorithms.
project community and forms a new community. Forming a new community results in less contributions by develope
•original
Most approaches
file repository,werebug based on or
fixes unstructured text featuresAllowing
feature enhancement. (summary and description).
accumulated bugs and To feature
handle enhancements
them, researchers to re
Most OSS
use the tra- projects used in the primary studies are still active however, and often contain codebases with moderate [66])too
unfixed for ditional
a periodbag-of-words
of time can affect approachprojectinstead
healthofrisk. more recent text mining methods (e.g., word-embedding
high
driven activity. In general, therefore, it seems that
maythe relationship between the adaptation
so far.of OSS in safety-critical and
3. Morefeature research engineering
is required methods
on forking which
sustainability.likely improve outcomes
Reviewing these 21 yielded
papers revealed the importance of forking
history
There is is rather
•sustainability a clear clear. At the
research same time,
opportunity tofurther
investigateresearch should be done to investigate the relationship between the the tra
ad
investigation as a top priority with twowhether specific state-of- the-art
areas of interest.4. MLStudying
algorithms mightsustainability
forking outperform using
of OSS
algorithms in safety-critical and OSS project activity. Moreover, it would be interesting to investigate the number of dow
for softwareused in all reviewed
development papers for
with GitHub. bug report
Valentio, Javier,severity
Izquierdoprediction.
and CabotThe investigation
(2017) used aofSLR the touseshowof Deep learnin
that forking
this project which
algorithms or its adoption
perform within
very industry
well when classifying audio, text, and image data [67] seems to be a promising resear
good indicator of project longevity and the chance of forking is highly dependent on the project, where developers pr
direction.
additional contact information (e.g., emails, personal website URLs that are clearly active or aligned with popular pro
• Researchers
owners) shouldsocial
to increase investigate more recent
connections between techniques
a project(e.g.,ownercontinuous
and forker,learning [68]) to
and increase provide an
developer approachsize
community for bu
fo
prediction which could be employed in real-world scenarios.
medium-size projects and projects that are written in a forker’s preferred programming language. Future work could i
•developing
Many bugareports predictionare resolved
model for infork
a few days (or in afrom
effectiveness few forking
hours) [69].Efforts to predict severity
motivation classifications level fortothese
in response groupr
language
reports
files, do not
where seem very useful.
programming language Thus, an investigation
survival time is critical to confirm
to an OS thisprojects’
hypothesishealthandandto determine
survivability. when the severity
prediction is more appropriate in bug report lifecycle is of critical importance.
• The primary objective of our mapping study was to review the research approaches for severity level prediction in
However, it would be a promising research venue to extend this study to commercial systems, and verify whether th
Further research is required to explore suitable KR practices applicable in OSS projects as indicated by one of the que
raised on mechanisms and team norms that are used to store knowledge contributed by team members. In CSS organ
KR mainly comes into focus when an employee is leaving an organisation (Lindvall & Rus, 2003). On the contrary, i
projects the unpredictable nature of commitment from contributors creates an element of risk (Robles & Gonzalez-Ba
2006). In OSS, contributors can leave since they are not under any contractual binding as in CSS organisations.
In this literature
Migration review, weand
of responsibility found that KM relevant
sustainability. It has activities of knowledge
been reported creation
that migration and knowledge
of developers from sharing
one re- are evide
lease to t
OSS projects as discussed in Section 6.1. Furthermore, literature examination directed us to 10 mitigations
high and that the developers take more responsibility as they gain experience. Yet it is a common phenomenon in ope to reduce
impact
domainof knowledge
that developersloss due join
freely to contributor turnover
or leave the project.inAnd
OSSwhen
projects discussed
a developer in Section
leaves, 6.2.
his responsibilities must be assi
someone else. For instance, the codebase maintained by a outgoing developer should be taken care of by others. Else
abandoned and discarded from subsequent releases. Thus it will be beneficial to explore the followings,
•Furthermore,
How responsibility migrates
the result of this among the developers?
SLR study also shows theDoesgapthis
thatmigration
there is afollow
lack ofpreferential-attachment?,
knowledge managementi.e., is the
aspect of
responsibility handed over to the developers who are in close connection to the outgoing developer.
source security. Several researchers did mention the knowledge problems in securing OSS development, however, we
•identify
What impact suchtackle
any study migration has on the
this security project
issue from evolution?
knowledge management perspectives.
In order to reduce dropouts of developers who abandon an OSS project (especially the ones who disappear after their
contribution), there is a need to look for effective ways to engage such developers in the community and providing g
community support when they are taking baby steps in an OSS project. Reception of new developers is another chal
the open source community. To overcome this issue, research is needed to develop tools, which assist in better recep
new developers. Moreover, more significant research is required to study the impact of community dynamics (stagnan
dynamic
There arecommunity)
relatively fewon empirical
developerpublications
turnover andonthe effect
OSS of turnover onand
in organizations, project performance.
the quality of published work is not goo
enough. Much of the published
research lacks relevance and a clear focus, and does not draw enough support from related literature. These observati
not particular to research on OSS. For instance, Kitchenham et al. [117], Vessey et al. [192], and Zelkowitz and Walla
observe a lack of relevant empirical research of high quality within both the software engineering and information s
fields. Finally, we would also like to see more research from outside Europe and the USA.
OSS researchers can also benefit from these results by using them to conceive strategies for newcomer support. To ac
this, it is necessary to put more effort on specific research topics, such as understanding and creating ways to measure
influence of the barriers in newcomers’ experience, identifying and creating different strategies to lower each barrier,
proposing metrics to grade the support offered for each barrier. To gain a better understanding of the barriers and to w
extent they need to be lowered, it is important to conduct more qualitative studies because this phenomenon occurs in
complex, social environment
In the architectural in which
smells group, the context
architectural bad of its occurrence
smells is important.
(architecture Moreover,
anti-patterns) a qualitative
stood out as the mostview comple
effective s
the existing literature, which relies mostly on quantitative evidence.
compared to architectural change (instability) and architectural hotspots smells (as shown in Fig. 10). This reveals tha
architectural bad smells were studied in isolation and not combined with more than one smell, which was covered in
code smells. Consequently, further research in this domain should be conducted to identify the effect of architectural
agglomeration and its correlation with architectural problems rather than the architectural smells in isolation in orde
its inclusion
show a great or exclusion asofone
concentration of the key
empirical indicators
research on theof study
the architectural
of how or- decay symp-adopt
ganisations toms.ISS development into thei
software development processes. Other research areas receive much less attention. Among the frameworks/methods,
and tools identified, none of them have been empirically validated in real industry settings. One of the implications of
findings for research and practice is the need for more empirical studies on engineering practices to support ISS dev
Specifically, while ISS development is highly influenced by OSS development, there is a need to translate OSS practi
suit
The the organisational
future direction forcontext to achieve
our work the many
is to evaluate benefits
further associated
practices that canwith OSS. Furthermore,
be incorporated future
in the OSS research
project workisstru
req
empirically validate the proposed frameworks/methods, models and tools. To advanced our understanding
The work structure of OSS projects is informal, varies in each project, and is dynamic due to transient contributors; of the inn
phenomenon, researchers
it is a challenging need to draw
task to generalise a setonoftheoretical foundations
practices for that have been used in prior research on OSS, as w
all OSS projects.
other theoretical lens that are considered relevant to ISS approach.
The implication for practice also lie in the evidence of the benefits and challenges of ISS development. The findings h
shown that the adoption of ISS development helps organisations to improve better quality, time-to-market and in-
novativeness. However, as suggested by Brown et al. [4], that newcomers should understand the reality of the method
an appropriate enculturation, so that they can recog- nise what works and what does not work, and thus be aware of c
working processes.
Organisations invest in KM activities to organise, create, share, reuse, transfer, and retain knowledge. We found know
relevant activities in OSS projects namely knowledge creation and knowledge sharing. Knowledge sharing was found
abundant but there was no evidence of knowledge retention to reduce the impact of knowledge loss in OSS projects.
Moreover, knowledge sharing is reactive in nature, initiated by the contributor while looking for task relevant knowle
suggest that there is insufficient attention paid to KM in general in OSS, in particular, there would appear to be an abs
proactive
It measures to
can be concluded reduce
that the potential
the problems impact of knowledge
of architectural loss. We
erosion within the also
OSSpropose
projects,the need foridentifying,
including a KM evaluation
addresm
OSS projects similar to the ones that evaluate the health of online communities. KM evaluation metrics
avoiding and predicting are still open research issues, which need further analysis and investigation. Consequently, should be bas
m
extent
efforts on this domain should be focused on identifying the other reasons that are still unclear and suggesting other so
of knowledge sharing activities observed in a project. Such a metric could help to inform potential consumers
OSS
to of themore
provide KM status on a project,
performance something
and accuracy that is non-existent
to address architecturaltoday.
decay.We consider it a vital ability for OSS projec
sustain a knowledge-sharing culture that will support the long-term survival and competitiveness of OSS projects.
In the future work, we plan to address the in-depth investigation of estimation accuracy by applying the estimation mo
one typical OSS database, and identify benchmarks.
An SLR study provides directions for researchers and practitioners on architectural decay within the OSS domain as
1) There are reasons that could contribute excessively to increasing architectural erosion such as rapid development
software, frequent changes, and lack of developers’ awareness. Therefore, further studies should be conducted as a r
deep study to find out other causes and to identify architecture erosion whether on the OSS or industrial systems. Prac
should follow guidelines to avoid architectural contradictions in the new and subsequent versions of the system in te
identifying the potential reasons within the system environment.
2) Sincearchitecturaldegradationsymptomspresentthat
However, according to the facts detected in the research code smells
studies, ITagglomeration has a considerable
managers are neither using any tool correlation to
nor procedure
architectural
allows them to problems
evaluatecompared
the adoptionto code smells solutions.
of FLOSS individual,For researchers
this reason, should conduct further
this contribution caninves-
motivatetigation on archt
researchers
bad smells in combination. Practitioners can change their detection way depending on
on the creation and publication of guidelines for adopting FLOSS. The purpose of this study is to guide future researc the code smells agglomeration
identify
application degrada-
of FLOSStion symptoms
in new domains effectively.
as a guide for the correct selection of FLOSS to help IT managers make appro
3) The
decisions findings of the current study serve asfor
evident that a metrics-basedothers.strategy is the most commonly used solution
Due to thefor organizations,
rising dominancedefineof the policies
OSS in the FLOSS
software adoption,
industry; among
not only practitioners, but researchers as well as
compared to other proposed solutions. However, more studies are needed in this field for other metrics to be analyze
academicians are also keen to understand the OSS software development process. Several studies have been conducte
provide the most architecturally appropriate solution and identify the effort required for the metrics to detect architect
past in this regard. This paper presents a systematic literature review of the studies performed to understand OSS evo
related smells. The essential techniques of ranking used should enhance the possibility to get better effectiveness res
set of 190 primary studies are identified for analysis and discussion. The studies are characterized on the basis of the
the identification of critical cores of architectural violations. Also, there is a clear opportunity for many researchers
questions they address. The main findings are as follows:
highlight enriching ARCADE’s tool for efficient approaches to architecture recovery. Additionally, there are solutio
•WeOSS evolution prediction studies studies
use ARIMA modeling of time series analysis forThe prediction of
cansoftware evolution
showalso thatnoticed
it is nota effective
lack of in-depth on technical
at all in the problems of deep issues faced
analysis bythe
and newcomers. reason such
difficulty of address be attributed
refactoring to th
strateg
such
number as size,
of defects,
qualitative change
studies requests
found etc.
because However,
it cannot as software
be evolution
quantitatively in general
extracted from and OSS
mailing evolution
lists. For in particular
example, tech
has no considerably a positive impact to address architectural erosion.
discontinuous
hurdles are evidenced by only five studies analyzed. Issues related to workspace setup is reported in only one study,shb
phenomenon, the use of prediction techniques that just extrapolate the historic trends in to the future
conscious
subject in atask. Furthermore,
debrief Herraiz
session. These et al.
kinds of (2007c) observed
issue deserve more that there arefrom
attention, no long
bothterm correlations
practitioners and in the time series
researchers.
representing OSS activity. The idea of fuzzy time series to deal with the uncertain evolutionary behavior of OSS syste
also been explored. The results show that a fuzzy based method for time series analysis is rather a promising approach
Traditionally defect prediction models rely on metrics that represent the state of the software system at a specific mom
rigorous prediction methods may be explored in future;
time [11].These metrics are used to capture a particular snapshot or release of a project to predict the next one. But m
• Lehman’s laws of software evolution for OSS systems have been validated in several studies. Only two laws (I and
capturing changes over time in projects also play a significant role in prediction. For example, metrics presenting the
been confirmed so far in different studies on OSS evolution. There is need to look into the change activity of these pr
evolution were used to predict the need of refactoring [12] and quality of OSS projects with significant accuracy. Thu
validate the laws using the change related information available in the SCM systems
research direction would be to explore a comparative study for identifying either (a) which form of metrics are more s
• A shift in the programming languages, from procedural to object oriented, has been noticed as
for prediction
Another resultasofmodels in terms
our study is thatofthere
accuracy,
are noreproducibility,
clear definitions and generalizability,
related to thethe or (b) are
openness these metrics complemen
OSS systems, subject systems in the corresponding studies, evolved over period of of IoT
time;platforms. One of the p
each
attemptsother and
to defineshould be used
the platform in combina-
aspects only tion to get better prediction results.
• Techniques and tools have been devised to[33],
tackleanother paper categorizes
large amounts the openness
of data generated dimensions
in software of platforms
evolution analysis [34
and
final study categorizes
prediction. some open-source
Software evolution automationplatforms but without
offers to collect volumes providing
of dataa indefinition [35].manner.
a consistent Thus, none of them
Software explic
evolutio
define what an
visualization openinIoT
helps platform is.the
understanding Moreover,
transitions in our study weand
in complex were alsosystems
large interested to easy
in an identify
way.theBigopenness types ofc
data analytics
platforms. Our results suggest that the most common openness types of most IoT platforms
help to analyze large sets of data generated during software evolution. Data analytics can be used to manage and unde are related to open-source
types identified
the complex webareofopen standards,
software openasAPIs,
evolution open data
it happens and open
in source codelayer-based platforms.
and other related However,
repositories to further invest
(Gonzalez-Barahon
openness types of IoT platforms we believe it is important to look from a stakeholder view, as identified above [34,52
it is essential to further analyze how important these openness types are from different IoT stakeholder perspectives.
ON RESEARCH METHOD
A number of issues related to the research approach can be improved to increase the acceptability of the reported resu
pointed out the followings,
External validity of the results. Empirical study is the most popular research approach employed in evolution studies
4). These studies, however, are horizontal in nature (as reported in RQ5) considering only flagship OSS projects. Due
approach
In order toofincrease
studyingtheOSS projects, the
participation reported
of new results suffer
developers, from
there is generalizability
a need to devise tools, threat, as reported
which in Figure
may eliminate 7. Ye
contribu
these
barriers and provide onboarding support. It is necessary to examine the impact of social interaction on newcomerinter
finding applicable and hold for the extended region of OSS projects, explicit measure should be taken. An succ
route
effect to
ofdeal
doc-with this is tooncategorize
umentation the findings
task accuracy, technical(current or future)
and coding according
issues, culturaltodifferences,
the projectand domain,
issuesorrelated
similartoorgan
crea
structure
local and practices,
workspace. or similarmore
Fur- thermore, product size and
research complexity.
is required Thisthe
to study will revealofthe
impact men-broader
toringpicture
on thewhich
success canofthen
onboa be
compared and possibly merged for proposing a more general evolutionary pattern and behavior
process and provide suggestions to better support newcomer onboarding, and contribution barriers in virtual commu for OSS.
Besides,
Studies aremore researchon
conducted is the
needed to develop
joining process strategies to alleviate
and abandonment. the issuesfuture
Therefore, related to choosing
research shoulda address
task to start initial o
the issues
contribution,
two areas. The authors also noticed that some studies investigated more than one research topic. Additionally, our of
to design tools to enhance retention of One Time Contributors (OTC), and to explore the motivation stu
developers to work as a mentor.
identified the gaps in community dynamics area that could be useful for researchers. Moreover, mapping study also
insights of some areas that need more exploration for instance mentoring of newcomers and finding a task to start are
the open research areas, which need more study. There is a need to develop tools to better support newcomers’ onbo
and easy
Our migration
results suggest ofthatthethedevelopers
most common among different
openness projects.
types of mostThere is a lack ofare
IoT platforms evidence onopen-source.
related to how to remove technic
Other typ
barriers,
identified are open standards, open APIs, open data and open layer-based platforms. However, to further investigateto
how gender and age of developers influence their retention in a project. It shows that there is still need of t
practices, and processes
openness types for betterwe
of IoT platforms community participation
believe it is important toandlookengagement in OSS projects.
from a stakeholder view, as identified above [34,52
it is essential to further analyze how important these openness types are from different IoT stakeholder perspectives.
As far as research community concerns, the results from the systematic mapping study prove that many researchers d
perform empirical studies or replications for solving the open issues.
The review establishes the state of the research on ISSD in terms of focused knowledge areas and contributions. Majo
research centred on the adoption and adaptation of inner source in various context, while other KAs in SWEBOK rec
attention. Thus our review calls for more studies on these areas to advance our knowledge on the inner source phenom
Furthermore, to advance our understanding of inner source, researchers need to draw on theoretical foundations that h
used in prior research on OSS, as well as other theoretical lens that are considered relevant to inner source. With th
the OSS framework,
researchers need to pay ourmore
studyattention
also hasto
identified theare
issues that characteristics
interesting toofprac-
the inner source
titioners. phenomenon,
Hence, including
we recommend its typ
focusing
projects or products, the motivation for adopting inner source, its environment, inner source processes
related to the ways in which organizations actually approach OSS, and issues that could benefit practitioners, rather and tools andth
actors involved in inner source. We also highlight the challenges as well as the benefits for organisa-
general “adoption issues”. Researchers and practitioners should increasingly collaborate to define a common research tions that are int
inner source.
and study For research,
research questions wethat
identified a research
matter to agenda
practitioners. to further
These advanced
research ourshould
questions knowledge in the inner
be answered source
through are
severa
In addition, it is possible that
studies from different contexts. adding non-software and information system related databases may yield similar or diff
findings. The comparison of findings from different databases and the findings presented in this study can po- tentia
SLR concerning software fault prediction was first conducted by [3] and was extended with new results in [7]. Howev
considered as future work.
works were limited to fault prediction of closed source projects and fall short of exploring OSS domain.
This SLR will help researchers to investigate prediction studies from the perspective of metrics, methods, datasets, t
in an effective manner. Future research should focus on establishing external validity and consistent accuracy of pred
models, incorporation of social aspects of OSS projects, and building tool support to automate the prediction process.
None of the studies discussed early effort estimation for OSS Web projects.
• Most of the source projects used as the datasets are in CMS form, and none of the development projects in the datas
involved any open source framework as there are no recorded details.From the findings it is shown that there has been
proper documentation detailing the involvement of OSS in the development, hence we can conclude that, none of the
have discussed early effort estimation for OSS Web projects.
Concerning the domains (RQ1) where OSS systems have been used, we found 22 studies originating from research o
different areas. Examples of areas include medical devices, the nuclear industry, and the aerospace industry. However
also found that there were some areas where safety is important but where no published research was found, such as i
areas of rail traffic, and the oil and gas industry.
Concerning the functionality (RQ2) provided by open source software, it was found that more research report on syst
are subsystems
Q1. What are the ofdemographic
larger systems than completeofsystems.
characteristics the studies That is, only
about rather few papers were found that reported on o
OSSECOs?
source software making up a complete system.
RQ1.1 In which type of sources are articles mostly published? Our resultsInstead most published research concernsthat
have revealed openresearch
source components
on OSSECOstha i
included in larger systems.
published in conference proceed- ings. The approximate ratio of publication in journals with respect to conferences is
Concerning
This indicates thethat
open source communities
OSSECOs are considered (RQ3)
to bethat are investigated
a valuable softwareinen- research
gineering it was foundtopic.
research that most were active a
mature (> 5 years). However, there were examples of communities that
RQ1.2 How has the number of publications evolved over the years? OSSECOs have been an increasingly addressed r are no longer active.
As a final
topic since conclusion
2006. Publication it can bepeaksseen that open in
occurred source
2011 software today is is
used as partthatof safety criticalhave
systems. Thisanis er
We are interested in using a nonintrusive approach to and
OI in 2013. There
the context ofevidence
RE by using OS-
fullySECOs become
automated approaches he
in both
research research
domain. that investigate complete systems and open source software components. It is reported from a numbe
companies fuel their RE processes with innovative ideas created outside their company boundaries. Following this lin
different
RQ1.3 How domains, and communities aredistributed?
active and mature.
research, we are
have papers
designedgeographically
an automatic sorting algorithm based on the Kano Model [35]. We believe that the algorit
The
be usedresults in this
in the process study ofsuggest that theprioritization
automatically current out- put users'of requirement
OSSECO papers and thusis strongly
eventuallysupported by Euro-
may become an pean and
essential
American researchers. However, in the
cornerstone in a fully automated, non-intrusive approach to OI used for requirements extraction and prioritization.
last
The four years, authors
advantages from other
and potential continents
challenges of OShave been contributing
in computer with publications
science education have been related to the using
explored OSSECO topic.
SLR method
review shows that the United States and The Netherlands are currently the
particular, the impact of OS in computer science education curriculum has been explored focusing on the four RASE leading countries in terms of undertaking
OSSECOs.
course design and delivery. The findings represent a starting point in evaluating the use of OS in computer science fr
RQ1.4 Who of
perspective areeducation
the predominantproviders. researchers?
In addition, Wepossible
observed that six authors
directions for future have been the
research have predominant
been suggested researchers
such as di
OSSECOs. These authors and their clusters account for a considerable fraction
effective strategies for proper alignment of OS projects and courses as well as efficient evaluation approaches fitting i of all papers covered in this systemati
mapping.
specifc context of using OS in computer science education.
These
RQ1.5reasons
How areare considereddistributed
publications the most contributing
between academy to degradation,
and industry?while Ittheis rest of the reasons
no surprise that thedemonstrated
pub- licationsinwritteTabl
less important than the three stated reasons depending
academic authors by far our number papers that have at least one industry author. on what was declared in the selected primary studies. However
studies
RQ1.6 What shouldtype be conducted
of papers are to find out the Although
published? other causes thereas are
a rooted-deep
more empirical studyresearch
in digging up new
papers thancauses
papersthatfrom could
otherh
significant contribution to identifying the architecture erosion,
categories (i.e., experience reports and non- empirical papers), the difference is not significant.whether over the OSS or industrial systems.
8.2. RQ2. What is an OSSECO?
RQ2.1 Whatatdefinitions
Succeeding providing are an OSSrelated to the isOSSECO
product definition?
not necessarily easyRegarding
as there arethe definitions
challenges related
related to to OS- SECOs,with
collaborating we a
encountered
munity, five majorand
like attracting concepts
relating (i.e., ECO, BECO,requirements
to contributors, DBECO, SECO, and OSSECO),
engineering and we builtbalancing
from a community, a genealogical
focus on tree
their evolution.
community and paying customers, and so on [109, 200]. We hope to see more research on these topics like e.g. [7, 20
RQ2.2 Aretopics:
following there specific definitions of OSSECO? Our results show that there are only three definitions of OSSECOs
paper
􏰅 How proposes
are OSSa providers
definitionable of OS- SECOs,
to attract andintegrating the different definitions related to OSSECOs: a SECO placed
sustain a community?
heterogeneous
􏰅 What are the envi-
success ronment,
criteria whose
for boundary
incorporating
Interests of researchers toward predicting and supporting evolution is a set of
contributionsniche (require-
players and of
process whose
OSSkeystone
projects areplayer
shownis aninOSSTablescommu
11 a
around
ments, a set
code, of
bug projects in
reports/fixes,an open-common
etc.) from a platform.
community?
Research interest toward predicting OSS projects has dominantly focused defect prediction. Change propaga-
RQ2.3 What elements
tion, ,maintenance belong
effort and SOC to an(self-organized
OSSECO? We criticality)
obtained have up togot 64 elements
slightly betterbelong to OSSECOs
attention. The rest in of
ourthe
review.
aspectsA
them,
addressed project, community,
considerably very andlow.source
In thecode are the
context most evo-
of OSS used.lution
Furthermore, we sketched
process support, a taxonomy
researchers paid with
majorthree categ
attention
(i.e., OSS community, ecosystem network, and software platform) to classify
evolution models and exogenous factors contributing to support evolution process. Maintenance support is the secon the OSSECOs terms.
RQ2.4
aspect, What
and instances
fault detection of OSSECOs
and change have been reported
propagation are the in the largest
thewell-known
third literature? We identified
explored aspects. 27 instances of OSSECOs th
We intend to extend our SR to include more studies by searching digital literature databases. We will also extend our data an
in
we our
plansystematic
to discover the mapping.
trends andAmong them, Eclipse
future directions and GNOME are the most frequently used.
of OSS research.
8.3. RQ3. Which representations have been proposed for OSSECOs?
RQ3.1 Which primary studies use models to represent OSSECOs? Our study showed that most of the papers adapt
modelling techniques or use ad hoc models to support their works, without proposing new modelling techniques.
RQ3.2 Which of the proposed models, if any, are specific for OSSECOs? None of the primary studies developed a
Broadlyspeaking,thesystematicreviewrevealsthat:(1)themajorityofstudiesarenormative
technique, notation, or guidelines for modelling OSSECOs.
andlackempiricalortheoreticalfoundations(2)noneofthestudiesfocusontheperspectiveofBI
RQ3.3 Which notation and guidelines have been used for modelling OSSECOs? We found a lack of specific modell
experts(3)therelatedbodyofknowledgeisscattered;thus,itislackinganall-encompassed,integrated
techniques for OSSECOs. However, we identified several modelling techniques to describe them in general. The mos
framework.Itisimportanttorememberthatthelastobservationisconsistentwiththeresultsofa
commonly applied notations were: ad hoc, tabular, and conceptual maps. Other OSSECOs were modelled using class
recentreviewofresearchon“gettingvaluefromBI”conductedbyTrieu(2017,p.1).
diagrams, metamodels, or mathematical models.
RQ3.4 What type of analysis was conducted using the models identified in RQ3.3? We found that most of the papers
models for OSSECOs do not conduct any OSSECO analysis. In addition, the analysis techniques used in the remainin
such as mathematical, visual, statistical, and SNA were used to analyze specific cases.
Predicting the future. Prediction of OSS projects is one area that is least popular among the study facets (Figure 2). Y
research should focus on developing reliable prediction models and methods supporting error prediction, measuring
maintenance effort and cost of OSS projects. Because, the commercial organizations, for instance, requires such pred
models to assess an open source component for adoption [66].
ON COMMUNITY EVOLUTION
Study on the community evolution identifies several key properties (reported in Table VI), which lay the foundation f
research in this direction. We propose the followings to be investigated.
Community building. Studies reported that the majority of OSS projects failed to attract members to attain the critical
Only few flagship projects are able to attract developers. Factors influencing the motivation to join a community has b
studied (e.g.,is[62]
While there quite[50]),
a lotand several phenomena
of research on specific are proposed. For
development instance,
processes rich communities
in OSS gets richer phenomenon. Yet itisislittle
e.g. [167], there no
identified what exclusive properties initiate the community
on using these processes and practices inside organizations. building process at the nebula stage of the project. Follow
research questions can be considered relevant,
• Why some projects are able to attract contributors during the nebula stage of the project, while most of them can not
• What formation of the community refers to a balance one, and how the community structure changes towards a ba
structure during its evolution? Can a visible pattern be identified within the domain of OSS projects for the above two
In this systematic mapping, we found the publication on the intrinsic (social) motivation factors of open source softw
also found the publication on the extrinsic motivation (economic) factors of open source software. We also found sele
open source software license on the base of economic, social and commercial (managerial) perspectives. In future, we
to find out some other motivation factors both economic and social perspectives for selecting the open source license.
will be our research question in future what are the motivation factors in selecting an open source software license wi
to
Feweconomic and social
publications in thisperspectives in software
field speak about community?
risk mitigation. Are the
Mostly, results of
mitigation ofRQ1 are in accordance
the various with perceptio
risks encountered in OSS
(Pakistani) open source software community?
is only mentioned informally, in form of general hints, such as to train the people, i.e. to develop the existing stock of
capital [SLR21], to follow general COTS adoption decision processes, to evaluate the community [SLR24], to evalua
similarity to previous projects [SLR44], to evaluate the OSS project’s roadmap and possible future directions [SLR46
make managers aware of risks and opportunities [SLR37]. Empirical experiments were used to identify risks and to id
effective mitigation
However, as described activities. However,
in Section none isofathe
5.4.4, there works
lack showedinevidences
of research this area. for causal
Thus, relationships
we need more studybetween
to shedthese
lighr
concrete measures and the effectiveness of the mitigation
to manage an effective inner source community in an organisation. activities. Only for more concrete risks, as e.g. for lowering
of introducing errors when upgrading to new versions, concrete mitigation activities were proposed, such as automati
checking API compatibility [SLR4] or exploring code executability with test cases [SLR14] to ensure correct function
In contrast, developers are commonly driven by extrinsic motives to join a FLOSS project. Also, structural characteri
FLOSS developers’ contact network influence their joining behavior. Moreover, the application domain and the deve
phase are relevant characteristics for developer attraction. However, it is unclear how relational factors influence deve
attraction. While Stewart and Gosain [55] view trust as essential, there is no attraction centric research on this aspect.
future evaluations should focus on this aspect. Moreover, as shown in table 2 there is very few attraction specific rese
which combines
Future individual
research should also and
focusteam factors. Considering
on predicting that ideological
change propagation, and status motives
size, refactoring, are dependent
maintenance on the fee
effort, contribu- ti
others, future studies should examine closely this interaction for extending our understanding of developer
evolution, SOC and clone evolution besides defect prediction. The architectural and requirements change evolution is attraction.
Similarly,
undermined there
areaisthat
veryneeds
littleattention
literatureofwhich combines
researchers. Theteam
toolsand
andproject characteristics.
approaches proposed Considering
for evolutionthe key
also role the
admit of stn
properties [26, 40], future studies are necessary to understand how FLOSS projects can utilize the
of external validation. Future research should focus on the above mentioned issues and try to make them more generasocial contacts of th
members
regardlesstoofreach out or new
the domain members.
of OSS Finally, drawing on research by Shah [48], further studies are essential to und
projects.
fully how FLOSS initiatives can stimulate individuals’ extrinsic and intrinsic motives to join the project.
Notice that there are a huge number of publications, which report and interpret results from qualitative and quantitativ
to identify possible risks. It stands out that only [SLR40] calculates threshold values for defining bug risks, and evalu
performance, while no paper identifies risks based on quantitative data of project failure or created losses and revenue
correlates project failures and losses with quantitative data such as the number of bug reports and bug fixes.
Few papers consider quantitative measures on community qualities (number of contributors, activity, presence of hero
[SLR39] etc.), while no work empirically evaluates the existence of causal relationships between the metrics applied,
identified and the actual faults happening. Moreover, an empirical evaluation of the effectiveness of mitigation activit
their influence on the retrieved metrics is also missing in the works, which propose specific mitigation activities. Also
complete works, such as the ones by the group [SLR12][SLR19][SLR20][SLR47], whose surveys retrieve data for ris
mitigation activities, do not show any link between these two: typical mitigation activities adopted in software compa
very general (see Section 1.2.5) and would reduce various risks.
The most commonly used selection method is the model approach and the least considered are the tool- based and dat
approaches. Another interesting result is that nearly half (47%) of the selected papers do not mention an application d
the models in their research. More attention should be paid to building models that incorporate only essential quality
characteristics. Also, framework, tool-based and data mining selection methods should be given more attention in futu
proposals.
factors for FLOSS developers’ attraction and retention. Moreover, there is even less dedicated research on these two
management aspects. Often this is because of the use of ambiguous measures such as “team size”. With such measure
possible to tease out distinct lessons for the attraction and retention of FLOSS developers. Thus, future research shou
specific measures for this particular aspect, for example the number of new, respectively retained developers. Finally,
visualized in table 1 – 3, our literature review shows that there is few dedicated research on these aspects (especially o
developer
We plan toattraction andexploratory
perform an retention) which combines research
mixed-methods more than one OSP
using research
in anperspective. Considering
undergraduate course intheSE,interrelation
in order to
the three research perspectives, however, an isolated research perspective seems too narrow. For example,
new insights with an experience with this approach. We will experiment with a combination of inside control FLOSS
and adepr
extrinsic motives are stimulated by receiving appreciation and by particular project characteristics (e.g. corporate
project in a software evolution course, using a combination of different learning approaches. We also plan to investig spon
[32]. With
methods torespect to these interrelations,
assess students’ future
learning in this studies on FLOSS management should adopt more than one research
context.
perspective in order to fully understand the effects of the examined aspects.
Our aim is to define a process that improves the selection, management and maintenance of OSS components. We wi
existing COTS quality assessment processes, including the ones enacted in the project partner companies in practice.
particular focus on the differences between OSS and closed source components identified in the SLR, we try to under
causal relationships between risks and failures, risk metrics, and risk mitigation. To propose such measures we invest
two directions: (i) evaluating the OSS project community ecosystem and the adopting company, by organisational mo
and
OSSanalysis,
has longand (ii) evaluating
passed the market the data available
introduction stageinbut
thehas
OSS
notprojects repositories,
yet reached suchstage”.
the maturity as codeThis
repositories,
has been bug trac
particul
mailing lists, applying measurements based on a statistical analysis. Techniques and tools for this analysis,
relevant for public administration where FLOSS’ technological immaturity, lack of interoperability with existing soft the priorit
and multi-criteria
decision decision making
making influenced will be poses
by politicians presented in the next
a significant section.
barrier for its wider adoption
There has been a steady but yet small body of research that addresses the pedagogical use of FLOSS projects in SEE;
includes 105 primary studies, developed by a small number of groups and researchers, and mostly published in confe
computer science education. From 105 studies in 20 years, only one was classified as Experiment/Quasi-experiment i
Research Type facet. Despite the increase in the number of experience reports, evidence shows that the research area
mature yet.
The
The updated SMS
availability of is a valuable
source asset
code for OSSboth to researchers
components inter ested
provides in the identified
opportunities trends
for scrutiny by and
thirdgaps,
partyand to instructor
certification bo
interested in trying out their own experiences in their classes.
However, the complexity, size and evolving nature of many OSS projects severely limit the practicality of such effort
the software is developed “with certification in mind”. Cotroneo’s pre- certification kit [7], Comar et al.’s Open-DO
continuous certification process [6], Fusani and Marchetti’s virtual certification repository [10], Kakarontzas et al.’s O
SME reuse process [11] are examples for approaches to develop “for certification”. Some of these proposals can be c
complimentary,
Software engineersothers are decade
in last alternatives. Littleinterested
have been empirical in
evidence is available and
agile methodology to-date
openabout their
source effectiveness
software in pract
development.
increasing
them present some new features and they seem beneficial for better and faster software development. By doing anwill
number of OSS systems are subject to certification and may consider these proposals, the community SL
more empirical
were looking forevidence on their
relationship effectiveness.
between ASD and OSSD. Fortunately our study shows that both ASD and OSSD can he
other and collaborate in doing software projects by sharing their practices. There are enough evidences that agile and
source practices can support each other, mainly because of some of their common concepts and principles. Also, how
there are adynamics
mmunity few successful
studiesexperiences
focus on theoninternal
integration of ASD motivation
and external and OSSD,of but, most of the
developers andstudies
factorsare optimistic
that influenceinthe
posr
of their integration, but there is no empirical successful case study for supporting this idea in software
of developers in projects. However, there are some areas, which need research such as retention of veteran developers producing indu
impact of contribution barriers on abandonment, use of gamification in student engagement and its effect on retentio
effect of project characteristics on retention, the impact of the project governance on motivation, development of strat
maintain loyalty, the impact of leadership styles on developer turnover, and impact of corporate sponsorship on FLOS
communities. Moreover, there is a need to design tools to assess the health of OSS projects and develop strategies to r
new developers.
Researchers studied the effectiveness and accuracy of several metric suites using data from one or more OSS projects
of their esteemed contribution in predicting OSS projects, they suffer from lack of generalizability due to diverse natu
OSS projects and the project specific nature of the metric suites. Also it is quite difficult to ensure the availability and
of metric data, which makes the results incomparable [10]. Thus, a future research agenda would be to perform an ind
analysis on (a) cross project validity of the studied metric suits, and (b) to propose methods to ensure the quality of m
In terms of using the architectural rules violations solutions, we observed that many violations were not restricted by
architectural principles of the system, which may not have defined the necessary rules to reduce the severity of major
violations. This means that violations still appear over again and over again despite the good violation solutions that w
introduced and the approaches that have a significant role in capturing violations with thorough accuracy. Therefore,
important to highlight the identification of the necessary rules and identification of critical cores through a broad stud
architectural
After examiningconformance
more thanusing multipleresearch
69 different frameworks.
papers and taking the survey findings into account, we constructed a
map that can be used by any educational institutedespecially high schoolsdto strategically integrate OSS into the edu
system. The roadmap is designed to target three main aspects of implementation: using OSS in the curriculum itself th
the semester, using OSS in final projects at the end of the semester, and using OSS in extra-curricular activities. The f
direction of this research is to implement this roadmap and measure its effectiveness. The roadmap can be applied via
phases:onchanges
Based to curriculum,
the findings implementation
of this research, we have in finaltocourse
come project that
the conclusion and the
establishment of extra-curricular
existing software activitie
security practices ha
limitations in supporting secure open source development. Secure architecture, code review and security testing do he
OSS products. However, as there is less research on socio-technical security aspects and no discussion of security kno
management in the context of OSS development, these practicies, and software security knowledge cannot be effectiv
spread within the open source community. Since OSS parcticipants are not experts on security in general and the dom
knowledge
The results of software
from security
classifying is vast
topics and extensive,
showed that studiesitaim
is suggested
to predictthat future
effort research should
of maintenance explore
activity soio-technica
mainly concentra
approaches in helping OSS developers learn the necessary security knowledge
bug fixing time prediction, while less efforts contributed to other types of activities to fulfill the need of their work, furthe
reinforce their behaviors towards OSS security.
Considering that there are sentiments associated with positive and negative polarity that were marked as not specified
selected studies regarding software practices, there is still room for further investigation on the associated sentiments
specific impacts. Moreover, there is a tendency of a considerable set of open source software projects to have regular
cycles and to adopted the so called frequent releases. We plan to investigate sentiments in this context and to which e
influence software productivity. We also want to investigate how programmers sentiments vary between releases.
Given the large variety of OSS systems, with different sizes, domains and complexity, we believe they are an importa
of examples to teach software design, architecture and quality (Brown & Wilson, 2012). Nonethe- less, we found ver
studies to describe these issues in, say, a case-based learning approach. Only three reported this type of experience: tw
them related to software design and architecture, and one of them, to software evolution.
No selected study focuses on learning the areas of requirements or configuration management, despite the large use o
configuration
There management
are themes and issue
still little cited tracking
that might tools
have in OSP.
some futureWe believe OSS
potential: that these specific
process areas
(meta-) may benefit
modelling, OSSfrom an
security
engagement with OSP and their associated tools.
and OSS development methods, and teaching OSS in universities.
Meiszner et al. (2009) point out some learning features of the OSP experience: self-learning, project-/problem-/inquir
learning, collaborative learning and reflective practice. However, very few studies cite these learning approaches, an
them provides details on how to design and implement pedagogical practices that result in effective learning of SE sk
From a practical standpoint, this research provides IS managers, as well as open source BI providers and consultants
initial structured lens to better understand the most important
barriersthatpreventorganizationsfromadoptingopensourceBItools.Thesebarriersrequirefurther
considerationbyallstakeholdersinterestedintheadoptionordeploymentofopensourceBI.
Fromamethodologicalstandpoint,followingPoba-Nzaouetal.(2016),thisresearchprovides
onemaincontributionthatisarigorousanalysisofQualitativeSurveydata,basedontwoprinciples
ofinterpretiveresearch(Klein&Myers,1999):thefundamentalprincipleofHermeneuticCircle,
andtheprincipleofAbstractionandGeneralization.
Toconclude,theauthorsacknowledgesomeareasoflimitations,andcallforfurtherstudiesof
opensourcebusinessintelligencetobeconducted.First,thoughitisadequateforaqualitativesurvey
andmethodologicallysufficient,thesizeofthepanelwassmall.Second,itcouldbeinterestingto
comparethesebarrierswiththosepreventingorganizationsfromadoptingopensourcesoftwareinareas
It is possible to see the lack of studies conducting qualitative analysis as supporting the existence of problems that h
newcomers’ contributions. Quantitatively analyzing historical data can bring highlights of the barriers faced by newc
but conducting qualitative analysis can enrich the evidence, reveal new facts, and help in finding the issues faced by
newcomers. There is still room available for studies based on observations, interviews and experiments that can help
the barriers faced in practice.
The findings showed that 17 of the most commonly occurred causes contribute to the architectural decay of the OSS
community. Essentially, architectural degradation has numerous causes, which have been discussed by several researc
their studies [11], [12], [23], [30]. However, these reasons have been discussed from limited aspects such as aging be
changes over time [11], identifying the reasons through only one case study [12] or based on their investigation of in
case studies [27]. Consequently, these causes need to be further identified in terms of the frequency of their occurrenc
especially
To summarize, in thethe
scope of the
review OSSindicate
results projects.thatMoreover,
requiredidentifying the mostisimportant
OSS functionality reasonsintegrated
more frequently will indeed in contribute
the safety ct
erosion according to the chosen primary studies, which contain several case
systems rather than the entire OSS solution is used. The complete OSS solutions found in the review are ratherstudies in different domains for thesmall
OSS
community.
low activity. These
On thecauses
contrary,differ
OSSin their impacts
solutions with regard
integrated to the
in safety actualsystems
critical contribution
have overto thefive
architectural decay and
years of history prom h
medium activity. Hence, we can conclude that long history of OSS projects facilitates their use in as integral parts of
critical systems. However, the relationship between the activity of the OSS project and their adaption in safety-critica
is less
Our clear from
research the on
agenda studied papers
this topic and therefore
involves requires
investigating further
soft- wareinvestigation.
development process tailoring (Pedreira et al., 200
way to achieve a balance between characteristics of software development models. Process tailoring is the act of
particularizing a general process description to derive a new process applicable to a specific situation (Ginsberg and Q
1995).
We claim that it is necessary to tailor these processes to suit specific project and organization contexts. This tailoring
also
Using consider
OSS CASE the key distinctive
tools: features
The research on of
OSS plan-driven,
CASE tools agile
hasand
been FOSSverymodels:
limited.collaboration
However, Wicks and discipline
and Dewar(Magda propos
2010a).
agenda for research on tool integration, requesting a more business-oriented approach to future research [207]. The us
Collaboration
development of can be defined
OSS CASE tools as a and
group of two on
research or more people
such tools working
could easilytofitachieve
into thisa common
new agenda. goalRobbins
(Vreede provides
and Brigga
Collaboration is an important factor for software organizations to achieve their
extensive overview of OSS tools for development, and claims that CASE research has a lot to learn from OSS [157]. productivity, quality and knowledge-s
goals.
shouldIn particular, be
furthermore software development
particularly interestingis atocomplex
academia process
since that
theyinvolves
have access collaboration amongstate-of-the-art
to professional several peopletools ove
achieve
tools’ a common
source code. goal
Thisof(Cugola
enables and Ghezzi,
them to extend 1998). Therefore,and
existing software development is a typical example of collab
A detailed description results from the mapping study tools test new
and classification ideas
of the in collaboration
risk factors in the with each other.
adoption of OSS i
work (DeMarco
Increased and
participation Lister,
in 1999).
OSS Discipline,
projects, increased meanwhile, relates
collaboration to
betweenthe planning level
organizations, adopted
and in
increasedsoftware
use of process
OSS prade
business context will be published soon by the authors.
and the rigidity of control employed in process execution (Boehm and Turner,
will most likely require improved collaborative development tools. Hence, there is a potential for research on: 2003b).
Both
􏰅 What arekinds
complementary
of tools areand essential
needed in any project,
for collaborative but in differing
software development proportions, depending on and
across organizational project characteristic
community borde
balanced mix between collaboration and discipline,
􏰅 Howdoorganizationscollaborateusingsuchsoftwaredevelopmenttools? it is necessary to understand how these aspects vary and distingu
software development models (Magdaleno et al., 2010).In order to facilitate process tailoring, it is possible to support
Four
projectof manager
five studies with the highest
by automating somelevelof theofsteps,
evidence (2b) examined
possibly reducing the clinical
effortchanges
requiredintoorthodontics
conduct this(influence
activity and of im
he
orientation on directional changes in 3D space; orbital volume and aperture width changes
the quality and appropriateness of the resulting process (Park et al., 2006). Thus, we intend to develop an infrastruct after rapid maxillary (expa
(18;21) and endodontics
an optimization- (canal transportation)
based approach (22;23), but none
(Magdaleno, 2010b).Finally, ofalso
it is the identified
important studies included
to investigate a compar-
whether ison betw
the results of
and any other software.The only level-2b study that included a comparative analysis
quasi-systematic review are consistent with what is observed in practice, i.e., in organizational routine. To accompli of a commercial and open-sourc
software
we intendwas assessing
to plan software
and conduct anprecision
experimental in airway This
study. volume measurement
experimental study(27).
willThe studyawas
involve surveybasedof on 33 participa
industry and a
lead to the
experts
Comprehension conclusion
to validate of the
thesethat
resultstheand
results use of Osirix
discuss
suggest theand
that the ITK-Snap
conclusions.
laws AOSS presented
similar
and theory appear to clinical
strategy be was value.
applied
breaking downby (Bekkers
through non- et al.,conformin
2008) to
determine
and findings the(Table
factorsIV).thatThus
mostLehman’s
influence laws the selection
of softwareof software
evolutiondevelopment model. based on the study of the large
which is primarily
source systems, is not sufficient to justify or account for the evolutionary pattern and behavior of the open source soft
none-the-less these laws did not consider the community dimension of the OSS projects which is an integral part of su
evolution of the open source softwareTo deal with this problem, a viable route would be to examine the underlying on
for
As asoftware
follow-up evolution [5] considering
to this research, we expect the OSS specific
to deepen thecharacteristics,
study on the aspects and thenthatre-assess the laws
may influence theofparticipation
software evolut of w
in OSS domain.
open source software projects and software development projects, as well as to propose ways of addressing the identi
problems regarding issues of gender inequality in open source communities and software factories.
It is more and more difficult to talk about “OSS practices” as the practices used in OSS communities are heterogeneo
organizations are increasingly getting involved in, and influenced by, the development of OSS. Nevertheless, there ar
opportunities for further research on the use of development practices for distributed software development. OSS dev
in large communities and in and between organizations, are areas where researchers could have an impact on practic
research has so far focused mainly on processes in communities of volunteers [167], but some of this research could t
focus towards
It reflects that the application
most of their findings
authors performed withinonorganizations
experiments andsource
different open questions like:and comparison of results of dif
projects
􏰅 How can development practices from OSS communities be adopted within organizations?
studies become arduous. There is a need of common corpus regarding evolution prediction of open source projects t
􏰅
be How
used may
by theorganizations successfully
researchers for comparingcollaborate
results. through community- or consortium-based software development?
Year SMS/SLR Extracted From Extracted by Classification
2020 SLR Discussion Ms. Saima Imtiaz Future Direction
Future Research Ms. Saima Imtiaz Future Direction
Discussion and
Ms. Saima Imtiaz Future Direction
Future Work
2014 SLR Discussion Ms. Saima Imtiaz Gap
2017 SLR Conclusion Ms. Saima Imtiaz Author Future Intent
Discussion Ms. Saima Imtiaz Future Direction
Conclusion and
Ms. Saima Imtiaz Author Future Intent
Future Work
Conclusion Ms. Saima Imtiaz Author Future Intent
2018 SMS Conclusion Ms. Saima Imtiaz Future Direction
Results Ms. Saima Imtiaz Gap
Conclusion Ms. Saima Imtiaz Future Direction

SLR Overview of studies Ms. Saima Imtiaz Gap
Summary and
Ms. Saima Imtiaz Gap
Conclusion

Conclusion and
Future Work
Implications for
2009 SLR OSS Research and Ms. Saima Imtiaz Future Direction
Practice
2019 SMS Ms. Saima Imtiaz Gap
Summary of Finding
Conclusion and
Future Work
2016 SMS Discussion Ms. Saima Imtiaz Future Direction
Results and
Discussion
Conclusion Ms. Saima Imtiaz Gap
Open Questions and

Research Agenda
Discussion and
2019 SLR Ms. Saima Imtiaz Future Direction
Conclusion
Summary and
Conclusion
Summary and
Discussion
Results and
Discussion
Open Questions and

Research Agenda
Discussion Ms. Saima Imtiaz Gap
Discussion and
Future Work
2020 SLR Future Research Ms. Saima Imtiaz Future Direction
Results from
2013 SMS Mapping Study Ms. Saima Imtiaz Gap
Summary and
Conclusion
Open Questions and

Research Agenda
Conclusion and
Future Work
Main Finding and

Discussion
2011 SLR Conclusion Ms. Saima Imtiaz Future Direction

Summary and
Discussion
Main Finding and

Discussion
2014 SLR Paper Analysis Ms. Saima Imtiaz Gap
Paper Analysis Ms. Saima Imtiaz Gap

Future Work Ms. Saima Imtiaz Future Direction
Revisiting Research
2014 SLR Questions Ms. Saima Imtiaz Gap
Future Work Ms. Saima Imtiaz Author Future Intent

Conclusion and
Future Work
Summary and
Conclusion
Conclusion and
Future Work
Conclusion and
2020 SLR Ms. Saima Imtiaz Author Future Intent
Future Work

2015 SMS Findings Ms. Saima Imtiaz Future Direction
Directions for
Future Research

2017 SMS Future Research Ms. Saima Imtiaz Future Direction
Directions for
Future Research
2019 SMS Conclusion Ms. Saima Imtiaz Future Direction
Summary and
Discussion

Implications of
Findings
Discussion and
Future Work
2011 SMS Discussion Ms. Saima Imtiaz Author Future Intent
2017 SLR Conclusion Ms. Saima Imtiaz Gap
2010 SLR Results Ms. Saima Imtiaz Gap

2020 SLR Ms. Saima Imtiaz Gap
Research Scope in
Floss Adoption
Conclusion and
Future Work
Discussion and
Future Work
Directions for
Future Research
Conclusion and
2019 SMS Ms. Saima Imtiaz Future Direction
Research Direction
Conclusion and
Future Work
2019 SLR Conclusion Ms. Saima Imtiaz Future Direction
Results and
Discussion
Saima imtiaz Ms. Saima Imtiaz Future Direction
Directions for
Future Research
Directions for
Future Research
Implications for
OSS Research and Ms. Saima Imtiaz Future Direction
Practice
Conclusion and
Future Work
Open Questions and

Research Agenda

Results from
Mapping Study Ms. Saima Imtiaz Gap
Conclusion and
Future Work

Discussion Ms. Saima Imtiaz Author Future Intent
Results and
Discussion
Adoption of BI tools
2019 SLR by organizations Ms. Saima Imtiaz Gap
2011 SMS Conclusion Ms. Saima Imtiaz Author Future Intent
Summary and
Conclusion

Conclusion and
Future Work
Discussion Ms. Saima Imtiaz Author Future Intent
2018 SMS Conclusion Ms. Saima Imtiaz Gap
Saima imtiaz Ms. Saima Imtiaz Future Direction

Open Questions and
Research Agenda
Research
Contributions and Ms. Saima Imtiaz Future Direction
Limitations
2014 SLR Overview of studies Ms. Saima Imtiaz Gap
Results and
Discussion
Future Work Ms. Saima Imtiaz Author Future Intent
Results from
Mapping Study Ms. Saima Imtiaz Gap
2013 SLR Future Research Ms. Saima Imtiaz Future Direction

Results and
Discussion
Comments in case of
Verfication disagreements
new added
Study ID
S48
S7
S20
S32
S28
S22
S33
S35
S18
S39
S56
S27
S32
S30
S7
S7
S38
S39
S18
S7
S50
S13
S7
S48
S29
S30
S31
S31
S3
S38
S19
S46
S54
S22
S40
S29
S13
S8
S39
S39
S22
S35
S26
S29
S31
S5
S9
S20
S22
S8
S30
S2
S9
S20
S47
S7
S14
S8
S43
S2
S44
S15
S22
S31
S2
S50
S50
S39
S36
S19
S36
S7
S30
S56
S19
S24
S46
S9
S37
S19
S30
S35
S29
S22
S37
S7
S32
S55
S32
S38
S33
S50
S50
S50
S21
S46
S7
S23
S15
S3
S11
S24
S9
S12
S19
S59
S31
S38
S9
S7
S7
S33
S7
S20
S12
S4
S7
S49
S6
S6
S20
S23
S41
S52
S57
S60
S5
S25
S39
S4
S50
S7
S7
S9
S46
S43
S25
S25
S46
S20
S46
S49
S10
S9
S8
S56
S39
S50
S50
S56
S14
S54
S7
S8
S19
S5
S11
S35
S21
S46
S7
S29
S3
S17
S39
S39
S7
S45
S36
S54
S18
S29
S36
S31
S18
S33
S36
S41
S1
S40
S42
S50
S8
S46
S51
S4
S20
S58
S33
S12
S17
S9
S46
S5
S23
S7
S30
S39
S53
S7
S29
Text Segment
The process map and its activities characteristics description presented in our study are based on primary academic
sources. However, it can be used as basis for the conduction of empirical studies to investigate OSS process in a more
practical view. We believe that our results can contribute to improve the understanding of OSS process activities and
consequently help to mitigate OSS project failure.
While there are a few studies outside the scope of this review focusing on software selection [46, 56, 105, 184] and
knowledge sharing within OSS communities [119, 173, 195], none of these are directed towards studying actual
practice in organizations. A few studies have started to look at some of the challenges in the borderlands between
integrating an OSS component and contributing to the development of it [106, 130, 186], but further research is
needed to solve the maintenance challenges facing developers who integrate a large number of components into their
products.
Studies that can quantitatively infer OSS maintenance effort from size-related metrics are needed. To mitigate the
difficulty of acquiring actual effort data from incomplete development records, Yu et.al [40][41][23] focused on
predicting the size-related metrics to indirectly estimate the maintenance effort. The strong correlation between size-
related metrics and the actual effort has been confirmed in closed-source projects [28]. However there still exists a
gap between size-related metrics and time-aware effort for maintaining OSS projects. There is a need for studies that
can quantitatively infer OSS maintenance effort from size-related metrics. Furthermore, the effort drivers used in
general maintenance effort estimation models can serve as an example to improve OSS maintenance effort
estimation. For example, Nguyen [28] developed an extension to COCOMO II [9] size and effort estimation models
to capture various characteristics of software maintenance in general through a number of enhancements to the
COCOMO II models to support the cost estimation of software maintenance. Some effort drivers such as DATA
(Database size), CPLX (Product Complexity), and PVOL (Platform Volatility) in his study might contribute greatly
to OSS maintenance effort estimation.
we commented in Sect. 4.2 the lack of measures for process maturity because in this case the assessment needs to be
done with qualitative evaluations of the community. Since we have focused on quantitative measures, there may be
other characteristics of the quality model that require or that may be complemented with qualitative evaluations.
not OSS but have the goal to meet customers’ expectations. For this reason, many of the lessons learned from OSS
projects can also be adopted in other types of software projects regarding the use of rapid releases.
As future work, we are considering using the results of this study to build a meta-model for the mining of open
source bases in view of gathering data that lead to assessments of the quality of projects adopting the Frequent
Release approach.
As Kirk and Miller stated in [22], “although no one defends a positivistic ontology, but scholars in social science has
find out that much research makes sense only in terms of a set of unexamined positivist assumptions.” Research in
the field of OSS success has the same problem. We want to precisely point to variables like:“general viewpoint of
audience society” and “actual use of software3” as measurements of success and contextual parameters such as
“availability of knowledgeable developers”, “legal support and level of IT development in the development
environment” as affecting factors. That’s why we recommend mixed-methodology research in the field of OSS
success.
One of our main findings is that most studies are not deeply concerned with research methods: among the relevant
selected papers, we may cite two case studies, one action research, but no experiments, which seems to be in sharp
contrast with the recent growth of evidence-based SE. Software engineering research is slowly adopting increasing
scientific rigour in the latest years. Moreover, SE education is an interdisciplinary area. As such, it can strongly
benefit not only from SE research methods, but also from research methods from areas such as sociology,
anthropology, pedagogy and communication. Thus, it seems relevant to identify which research methods are more
appropriate in this intersection, in order to achieve better results in this interdisciplinary area.
As future work, we plan to develop a method that extracts users’ software requirements automatically from internet
resources and to design automated processing to support requirement prioritization.
The reviewed articles show that self-determined participation motives are most relevant for FLOSS developers’
commitment. Moreover, both relational (e.g. trust) and structural (e.g. network centrality) group aspects foster
FLOSS developers’ commitment. Also, the chosen code license affects developers’ commitment. Beyond these
aspects, the reviewed articles suggest other factors which yet need dedicated analysis. For example, the effects of
members’ cultural background [57] or their geographic proximity [29] on their development intensity. Further, the
interrelations between the research perspectives need further scrutiny. In particular, the relationships between
individual or team factors and project characteristics (see table 1). Such cross-perspective analysis is necessary to
understand fully how FLOSS projects can incentivize individual and group factors which increase developers’
participation. Future research may draw on research results by Gallivan [21] and examine if and how governance
processes foster FLOSS developers’ personal relationships. Also, further research is needed to fully understand if and
how project governance can stimulate individuals’ participation motives.
Sub-project evolution with their community. Large open source projects often encompass many sub-projects. Such
as, sub-projects in Eclipse, GNU, Linux, and Apache. Often ecology of sub-communities formed around these sub-
projects, which are governed by a common governance [50]. Study on the formation and evolution of sub-projects
and their communities have revealed many key characteristics, which are listed in Table V and Table VI,
respectively. Yet the interdependency in evolution between the two and their impact on the overall project evolution
remain untouched. The following would be worth to investigate.
• Does there exist a correlation between the evolution (growth, complexity, change) of the sub-projects and their
associated sub-communities? Does the community change with the change in the sub-project?
• How does a community form around a newly added sub-project?
• What attributes of a sub-project attract new developers to join?
• What happens to the sub-community when a sub- project is deleted or merged to other sub-project?
tion?
• What kind and level of communication and collabo-
ration takes place between sub-communities?
• Does there exist a correlation between the project
evolution and the sub-project evolution?
As a future work, we plan to provide a more formal definition of the open IoT platform rather than a perspective by
consulting different stakeholders in detail. The open IoT platform term is frequently used in public and without clear
understanding could cause problems like misunderstanding or even wrong strategic decisions. Our future work
intends to provide a clearer and deeper understanding of open IoT platforms for the communities. The open issues
listed above are also some possible directions of investigation
in the study with a development component but oriented to a specific requirement, 16.13% adds tools to the central
axis of the GIS Web architecture and 9.68% integrates new methods and algorithms to improve different aspects of
architecture.
Open Source Web Software Architecture Components. Hence, the need to carry out new research aimed at evaluating
and improving the components of the Open Source software architecture of a geographic information system in a
Web environment.
This quality model can be used as a starting point for the quality assessment of an OSS ecosystem, and it is in our
plans for the future work to define a complete quality assessment process (as described in Sect. 5) and to apply it in a
real quality assessment. As consequence new measures may be needed for the assessment, but this is the best way to
improve, and complete the quality model, and a way to prove its capabilities in quality assessment.
Another aspect of the use of OSS in dentistry is education: 10,000 participants from 138 countries used the
Supercourse e-module entitled, “Sterilization and cross-infection control in a dental practice”, which makes this
software one of the most popular e-lectures on this topic (31). Supercourse is a network of 56,000 scientists from 174
countries who share a free library of 5,802 lectures in thirty-three languages, but it contains only eleven lectures on
dental topics. These numbers are impressive, but several issues regarding the use of open intellectual property, a
unified rating system and a standard citation system may require proper regulation (19;32).
To enable organizations to reap benefits from their participation in OSS communities, the research community
should dedicate much more attention to questions concerning this [48, 165]. While there are a few examples [50, 101
176, 186], more work is needed to aid organizations in participating in communities and collaborating with other
organizations through collaboratively working on OSS products [7] and to solve questions like:
􏰅 When, how, and with what should organizations participate in the development of OSS products controlled by
others (including inter- organizational collaborations)?
􏰅 How can companies (effectively) allow products to be partly OSS and partly commercial products?
Most organizations seem to have rather limited contributions to the OSS communities [33, 43, 91, 98]. The most
common way of participation is being an active user that reports occasional bugs to the community [43, 98, 99].
Only one of 32 respondents from a sample of tertiary education institutions had participated actively by writing
code, while 14 had contributed to an OSS community through reporting bugs [91].
As observed in the study, code quality is seen as the most important criterion for measuring quality. On the other
hand, market success and developer activity seem to be the most important criteria for measuring success. Therefore,
in future, researchers have to study how code quality may be used to measure software success, or how market
success and developer activity may be used to measure software quality. Only then, it might be possible to talk about
significant relationship between OSS success and quality.
ON CO-EVOLUTION
It is turned out from our review that the understanding of co-evolution of the code and the community in OSS
projects has received little attention in literature (Figure 2). As a consequence, the community dimension and
corresponding communication channels (e.g., mailing archives, bug tracking systems) are explored seldom, as can be
seen from Figure 5 and Figure 6 respectively. Study on co-evolution in OSS projects, however, is becoming
increasingly popular. Because, in such projects the code evolution is dependent on the contribution of community
members, and that a successful evolution of the code is required for the survival of the community. The following
research directions can be considered relevant. Exploring socio-technical congruence. In the OSS projects
contributions made by the community members not only drive the system evolution but also redefine the role of these
contributing members and thus change the social dynamics of the OSS community [53]. In this connection, it will be
very interesting to investigate the phenomenon socio-technical congruence in OSS projects. Socio-technical
congruence which is a conceptualization of Conway’s law [67] states that there should exists a match between the
coordination needs established by the technical domain (i.e., the architectural dependency in the software) and the
actual coordination activities carried out by project members (i.e., within the members of the development team) [67]
This concept was already explored in closed source projects, and reported a high correlation with software build
success, quality, and faster rate of modification [68]. Thus socio-technical congruence plays a pivotal role in
conceptualizing the co-evolution in a project. Surprisingly, this notion as a research area has not been iven much
attention among open source researchers. Although it is identified and reported as a desired property for collaborative
development activities like OSS projects [69]. Considering the lack of focus in this direction, we propose the
following to investigate.
• Does the essence of socio-technical congruence as a conceptualization of Conway’s law holds for OSS project? Can
it be stated as an implicit characteristics or property of successful OSS project?
• What quantitative approach/method can be utilized to verify the existence of socio-technical congruence in OSS
projects? What repositories can be used for this purpose?
• What correlation can be derived between socio- technical congruence and the quality/sustainability of OSS projects?
For a high retention rate, it is important that FLOSS developers perceive their project work as self- determined. In
addition, members’ project continuance is influenced by relational and structural characteristics of the team. Also,
less restrictive code licenses and the modularity of the code affect members’ project retention. However as shown in
table 3, there is little dedicated research on how team level aspects affect developers’ retention. With regard to the
key role of group aspects for developers’ commitment, future studies should examine this aspect closely. In addition,
very few articles use more than one research perspective. This calls for further research. In particular, future studies
should draw on research by Stewart and Gosain [55] which suggests that members’ retention is a product of
motivational and relational factors. Further, future research is necessary to understand the interaction of structural
network properties and project characteristics. To do so, future studies should draw on research by Oh and Jeon [40]
and analyze the ways in which FLOSS projects can actively utilize the interaction network of their members to foster
their retention. Finally, further research is necessary to understand the ways in which project characteristics influence
individuals’ retention.
We classified 53 of the 112 empirical papers identified in this review as experience reports. Hence, the most common
method of studying the OSS phenomenon in organizations is through experience reports. These experience reports
lack explicit research questions, and most also lack a method description.
Some researchers have studied the joining process of OSS communities. But more research is required to discover
the tools which may help the developers to alleviate the issues experienced by them when they migrate to the other
projects, to investigate the changes in the degree of socialization over time, to examine the association between
similarities in newcomer socialization patterns and distinction in joining scripts, to analyze the effect of the
difference in the joining process on retention. Besides, more significant research is required to investigate contributor
roles in the ecosystem, to analyze the impact of resolved issues on onboarding success, to examine the relationship
between joining process and role transition, and to discover effective ways to organize project information that may
support newcomers during the onboarding period.
Our review has found that the evolution trends and patterns is the most focused research area with 23 papers
published on this topic. There were ten papers on the role of process support in evolution. However, there are quiet
few numbers of papers addressing the characteristics of evolvability and architecture, with five and three papers
respectively.
The general costs related to such a migration are unclear [62, 187], and there are very few studies showing complete
calculations of the true costs and savings of (1) introducing OSS products into organizations, and (2) keeping the
OSS products operational over a longer period of time. One paper reports cost savings from an OSS migration
project at Beaumont Hospital [81], but it is published just after the initial stage of the project is finished.
Despite this lack of clarity, many organizations seem to be blinded by the perceived advantages of OSS and have
therefore adopted it without per- forming any cost-benefit evaluations in their own context [91, 187, 190]. The
adoption of OSS is furthermore frequently bottom-up, in the sense that it is introduced by engineers rather than
strategic top-level decisions [188].
As future work, we intend to apply the findings of our study in more recent OSS projects and provide a validation of
the proposed OSS macro process though a practical point of view analyzing OSS projects process activities, their
characteristics and understanding how roles are involved in each activity, fact that still not clear yet in the literature.
Furthermore, we intend to investigate how researchers perform OSS process analysis in OSS projects, including
approaches, techniques and tools they have used to retrieve OSS process information.
In response to RQ2, we found that the OSS studies focusing on evolution process support used different methods,
tools and approaches for OSS evolution process support. OSS Evolution process support studies have usually focused
on evolution models, exogenous factors, maintenance support, fault detection and change propagation aspects. A
very little effort has been paid to the other aspects such as Configuration Management, Growth, Complexity and
Control, possible evolutionary paths etc. SVN/CVS is again found to be the largest explored dataset. The detailed
explanation about these aspects is presented in Sect. 4.4.
During our research, we found only a small number of studies on the use of OSS in dentistry. Although the
conclusions drawn from studies support the use of OSS, most of the articles presented a low level of evidence (3b-
5) and poor quality of reporting, which makes it difficult to recommend OSS as a clinically useful software. The
only study with high-quality reporting was a case-controlled study but the conclusions were based on very small
research group, three teeth, in what way does it not make it possible to say that the results are representative.
This study could help researchers to identify essential quality attributes with which to develop more robust quality
models that are applicable in the various software domains. Also, researchers can compare the existing selection
methods in order to determine the most effective.
s future work, we intend to model OSS quality assessment as a MCDM problem. This will afford us the opportunity
to choose from a range of MCDM methods one (or more) that can be used to evaluate quality in OSS across multiple
domains.
a vital need to improve the quality of reporting empirical studies of OSS. We assert that an improvement in the
empirical studies of OSS will help the community to better understand the results and limitations of the reported
research.
We have presented a set of guidelines that are expected to help improve the quality of reported studies in OSS-related
research. We do not claim that the set of guidelines we have proposed is exhaustive or complete. However, we
believe that significant improvements can be made in the quality of reporting empirical research if the future papers
on empirical studies of OSS provide all the information suggested by the guidelines.
Our answer to RQ1.1 indicates that the majority of the articles contribute experimental (case) study to deal with
evaluating the quality and success of OSS. There are a similar number of articles that contribute new methods /
techniques and very small number of articles that introduce tools. The lack of tool development can be interpreted as
a lack of agreement on a concrete method for measuring the relationship between the success and quality of OSS.
Response to RQ1.2 supports that of RQ.1.1. a large number of articles are classified as solution proposals. There are
relatively less articles with strong empirical base, indicating the relatively low research rigor in this area.
Since we were not able to identify any existing studies that indicated the early effort model for OSS Web project, we
therefore believe that there is a need for researchers to further explore this field. This is particularly relevant as OSS
is being increasingly used nowadays by software provider organizations. This can be supported by the paper of [78],
even though the author in this paper only focuses on effort estimation for software development. However, the author
strongly believes that there is a need to develop an effort estimation model especially for OSS projects.
Fur- thermore, the architecture-sensitive metrics for code anomalies discovery provides the majority of awareness to
engineers for the existence of the smells code elements that are more significant to the architecture design than the
most traditional metrics that are depending on source code and based on static code metrics combination. This means
that the developers and engineers could detect and repair such anomalies promptly. Therefore, more studies are
needed in this field for other metrics to be analyzed in order to provide the most appropriate architecture without any
impact of the size bias. Furthermore, there is a need for metrics that have a great ability to discover the inconsistent
classes affected by the degradation from the consistent classes. In addition, there is a need to identify the effort
required for the metrics strategy to architecturally detect related anomalies and also to derive more metrics that may
have an impact on the quality relationships of other software that are closely related to architectural problems.
As part of our future research, we plan to conduct a comprehensive survey of practitioners to identify the key
challenges in implementing inner source and propose resolution strategies to overcome. The survey could be also
performed to validate the signifi- cance of our research agenda.
As could be observed in fig 3, most of factors which affect the success of OSS are related to developers and product
and most of success indicators are related to product. Our study shows that user related factors have been studied less
than other factors and researchers limited these factors to number of downloads in both success factors and indicators
On top of this research gap, we highlight some other gaps that may help future researchers better define and conduct
their study in the field:
More empirical and theoretical research is needed on its applicability in context of OSS certification. Important
research questions include “How do certification concerns shape and impact OSS communities?” and “How to
organize open source communities for effective and economic certification? “
Figure 3 depicts that around 62% of the articles used statistical methods such as regression, time series analysis,
correlation analysis, Pearson coefficient, Spearman’s rank correlation, principal component analysis and Bayesian
belief network. Statistical analysis methods are found to be more reliable for predicting different aspects for OSS
studies as compared with other methods. The second large set of methods employed falls into the category of
machine learning algorithms. Other methods such as mathematical models, probability methods, Chaos theory and
SRGM’s (Software Reliability Growth Models) have very low exploration.
Regarding the category of examining OSS evolution at software architecture level, we have found that although an
increasing amount of attention is being paid to the architecture of software systems due to its recognized role in
fulfilling the quality requirements of a system [20], only few papers address OSS evolution at architectural level.
Software evolution can be examined at different levels such as architectural level, detailed design and source code
level. We have noticed from the review that most papers address OSS evolution at source code level. However,
software architectures are inevitably subject to evolution. They expose the dimensions along which a system is
expected to evolve [22] and provide basis for software evolution [37]. Therefore, it is of major importance to put
more focus on managing OSS evolution and assessing OSS evolvability at the software architecture level besides the
code-level evolution.
Each metric used for prediction, either being positively or negatively associated with prediction results. For example,
in case of fault prediction, a metric signifies a module as either being faulty or not faulty. In either case, the metric’s
prediction recital is judged as a best, significant or bad predictor. In this regard, our review results show
inconsistency on some metric’s performance. For instance, the metric LOC (Line of Code) was evaluated as a best or
good predictor in [1][9], whereas in [11] it was noted as a bad predictor. Moreover, DIT (Depth of Inheritance Tree)
was noted as a significant predictor in [6], but classified as a bad predictor in [1][4]. Possible causes behind these
differences in results might be the variations in OSS systems [9], differences in implementations of the metrics [9],
or different prediction models used. However, an indeepth investigation and resolution of such conflicting issues
would be a future research agenda.
Metric set for software evolution. Software evolution studies mostly utilize metrics that are empirically validated in
prior studies (as presented in Table III). These metrics are derived for closed source projects, and are primarily used
to verify the Lehman’s law of software evolution. Though these metrics provide valuable insight to OSS evolution,
they do not consider the community dynamics. Thus an empirically validated set of metrics in favor of explicit
representation of the community is required to complement the existing metric set.
Framework for the data collection and representation. As discussed in RQ6, OSS projects often produce large volume
of data representing their development and evolution history. Research to date, explores the repositories that maintain
these data, a list of which is provided in Figure 6. However, data collection and representation in these repositories
vary significantly from project to project. Furthermore, data from the same source may have different formatting
(e.g., emails are often free of format even in listing the senders credentials). Due to these facts, it is a challenging task
to collect relevant data following a standard format from OSS repositories. In this context, researchers often employ
their own means to collect and represent data for research. This reduces the compatibility and comparability of the
reported results even if they use same data sources. Taking these issues in consideration, a framework for uniform
data collection and representation can be developed to make the results cohesive and comparable to each other.
Although all research in the field of OSS success have tried to study previous work, but we observe little connection
between them. One exception is reference [1] which has mentioned four previous works in the model and studied
them in a longitudinal study and found some inconsistencies between original and current study.We believe that study
of other work and comparing the results may lead to considering new factors (such as contextual or longitudinal
factors) in study of OSS success.
Regarding RQ 2 and RQ 3, we found that there seems to a lack of research on the use of OI for requirements
prioritization and requirements validation as there was only one paper dealing with these topics each, i.e., papers
R_11 and R_19, respectively. In addition, we found only one paper (paper R_18, dealing with OI in the context of
requirements extraction) that presented a solution approach with tool support. This indicates that there is little
automation support mentioned in the literature on the use of OI strategies in RE.
Utization of designs and fail at espousing an alternative knowledge sharing economy. This points to a gap in open
design research, where the ways of keeping design solutions open, accessible, replicable, and adaptable while
conforming to safety regulations and standards is a challenging topic and remains mostly unresolved. Co-operatives
and similar models may suggest community-based ownership and responsibility, but this model is not as open as
open design is espoused to be enacted. This affects the reliability of these design solutions, especially when they are
not widely reviewed online. Although larger transitions towards alternative economic models are discussed on the
macro level, research on how they will be enacted as development, iteration and dissemination of open designs is still
an important area of interest.
In the reviewed literature, open design is indeed framed as a better alternative by many authors, especially on topics
proposing new ways to do business, prototype alternative economies, and foster sustainability. From a strictly
business perspective, the potential of open design is observed mainly as a value-capture strategy and a way to achieve
rapid innovation cycles for further development and wide-scale testing. However, open design’s relation to enterprise
is still largely considered within the current paradigm, while the potential of an open design ‘sharing economy’ is not
yet generally discussed as a way to transform the way businesses operate. Toward the manufacturing side, companies
open up their initial processes but do not develop alternative models befitting the sharing economy as suggested by
open design. Moreover, it remains to be seen if open design as a research framing remains semantically and
ontologically tied to trajectories of business-as-usual (as has been seen in software; see Morozov, 2013), and
therefore not a true alternative nor necessarily democratizing; if it is increasingly embraced by research on alternative
post-capitalist and postcolonial practices; or if a new term becomes more appealing to the research community and
replaces it entirely.
Researchers have contradictions on the predictive power of metrics used for the evolution of OSS studies. These
contractions are discussed in Sect. 4.2. There is a need of further research to empirically evaluate predictive power of
different metrics. Most metrics are applied on the file level for the evolution prediction of OSS studies as
highlighted in Table 19. We also analyzed that class level metrics are applied by few studies but method-level metrics
are applied by none of the selected primary studies. Moreover, we also reveal that code level metrics are applied by
most researchers for evolution prediction of OSS studies. Little attention is paid to requirements, design and
architectural level metrics for predicting evolution of OSS studies.
From Fig. 5, it can be observed that 47% of the quality assessment models considered do not mention the domain of
application. This implies that most of the models were designed to be domain-independent. As such, domain-
independence should be the focus of model developers (Wagner et al. 2015). A domain independent model is one tha
is able to assess quality in various category of OSS including those that are data-dominant, system software,
control-dominant and computation-dominant. It should also be able to this with little or no customization. By
following this particular consideration, the model proposed can tend to be widely adopted and possibly standardized.
Interestingly, the aerospace domain represented in our study in only 3 papers was the top domain in both related
surveys [10], [11]. At the same time, the most represented domain (automotive) in our study was not the most
represented in the other two surveys (ranked the 4th and the 3rd) [10], [11].
Less explored, however still represented by primary studies, is work on OSS for safety-critical automation systems
and maritime systems. Process industries and rail industry (mentioned in the top five domains in the evidence
provided in both [10] and [11]) are not represented among the primary studies4. Finally, oil and gas, off-highway
equipment and mining industries represented in the previous survey about compliance with safety standards [11] are
not represented among our primary studies. Hence, it could conceivably be hypothesized that these industries have
yet not been intensified by software systems or explored by open source solutions.
Although we considered the barriers as something that can hinder newcomers’ contributions, some barriers can be
used as filters by the projects. Findings from a Halfaker et al. [19] study on Wikipedia newcomers revealed that some
entry barriers led to improved contributions in the future. More- over, research conducted in the OSS domain [33, 13]
demonstrated that socialization barriers are useful for maintaining community integration and the quality of the
community’s product. A clear direction for future work is to explore how the communities perceive these barriers and
how they impact the quality of contributions from newcomers.
The results from classifying topics showed that studies aim to predict effort of maintenance activity mainly
concentrated on bug fixing time prediction, while less efforts contributed to other types of activities
Except initial research by Crowston et al. [23], and Crowston et al. work on the definition of OSS success [21] , we
do not find any general model of OSS success. In fact many researches in the field have just tried to validate their
partial model of OSS success. We believe that according to wide range of social, cultural and technical factors that
may have an effect on success of OSS, developing a general model is not reasonable but we recommend contingency
practicesin this regard. In other words we suggest researchers to develop general models for specific contexts and
believe that these models would be more helpful in practice.
What sets open source development apart from the traditional proprietary software is the developer community
behind it. Although the social structure and communication of OSS communities have gained significant research
interest, the research efforts to the community in relation to prediction appear quite the opposite. Evolution of
communities is of interest starting from the paper introducing the community structure [13] but our search did not
find much focus on community evolution tied to prediction. In [14] the authors investigate the impact of social
structures between developers and end-users on software quality and their results give support to thinking that social
structures in the community do hold prediction power in addition to the source code centric approaches. It is also
suggested that combining metrics focusing on code and social aspects work as a better prediction model than either
alone. This gives support that the question has research value and is worth looking into further: what does the
community and the community structure predict for the software?
The usability of OSS was evaluated only in a few areas: mental foramen localization, upper airway calculation in
growing patients, an experimental assessment of hard debris in the root canal system after root canal treatment,
information gathering (RSS), practice management, and use for educational purposes. None of the studies revealed
the use of OSS in prosthodontics which is currently the most intensively developing area of digital dentistry. Virtual
planning and design of prosthetic reconstructions provide many opportunities for OSS solutions.
As future work, we intend to review some of the concepts related to ISO 9241-210 [14] and participatory design to
reflect on the main principles and how they were addressed or not in the papers surveyed. Moreover, with the result
obtained in this systematic mapping, the gaps and the lack of studies involving the areas of interaction design and
development of FLOSS, we intend to advance in the research on this subject. Therefore, the next step of our research
will be to expand this systematic mapping to identify approaches, methods, techniques, and tools for participatory
interaction design in distributed software development environments. With this, we intend to develop a participatory
interaction design process model for distributed software development environments. Finally, we must extend the
Open Source Maturity Model [54] with the proposed interaction design model. Considering the inher- ent advantages
of software development following a software process model, we believe that interaction design activities will be
considered during the different stages of the FLOSS development process, particularly in the early stages.
In the future, we aim to conduct some qualitative studies to confirm the problems evidenced by the literature. We are
conducting some interviews with experienced OSS developers and newcomers to verify the main problems faced by
newcomers from their perspective. We also plan to refine the classification model based on the results of the
interview analysis. Addition- ally, based on the model, it is possible to propose awareness mechanisms and tools to
offer better support for newcomers
New evaluation methods are needed to validate the correctness of these estimation methods. With the growth of more
companies developing or collaborating with OSS projects, estimating maintenance effort has become a major interest
More researchers have been focusing on improving the estimation towards the direct effort of OSS projects from both
people and activity aspects by developing maintenance effort estimation methods. However, since most OSS projects
lack of complete development records and actual effort data, it is very difficult to evaluate and validate the results of
these methods by comparing the estimated results with the actual effort. This can be a significant threat to these
estimation methods and raises the risks to effectively validate of their results. This is an issue where we need new
evaluation methods that can validate the correctness of these estimation methods.
As a result of our findings, we propose the following directions for future research in this area. Focus on the
definition of a common model (which may be obtained by merging multiple available approaches) and favor its
adoption through rigorous and extensive validation in industrial settings. This could increase the validity of the mode
and thus its dissemination in industry, where OSS is still not widely adopted. Several models already exist but,
according to the results of our SLR, they have not been strongly validated and, as a consequence, adoption has been
limited. Try to target the models at quality factors that are of real interest for stakeholders. Most of the available
models focus on the overall quality of the product, but few of them are able to assess each single factor that composes
the overall quality of the OSS product. This can complicate the assessment of OSS products by stakeholder, who are
interested in specific quality factors: e.g., developers are likely more interested in reliability or testability aspects,
while business people may be more interested in cost or maintenance factors, etc..Develop tools that support the
research directions listed above (i.e., tools able to support and simplify the applicability of the proposed models
during the evaluation of OSS products). Most of the tools mentioned in the primary studies are prototypes and most
of them are not available or maintained anymore.
The overall rigor of the studies performed on OSS, both within organizations and in general, is furthermore not
good enough. Consequently, we should strive to do better work and to present this work in more detail [180]. In
particular, we agree with Kitchenham et al. [118] in that the context of the organizations being studied should be
given much more attention.
We observed that few of the studies were longitudinal and that few publications focused on providing in-depth
details from one or a few organizations. To really understand the profound consequences of approaching OSS, we be-
lieve there is a need for both more longitudinal and in-depth case studies.
Finally, we found evidence that OSS is not that different from other information technologies. OSS researchers
should therefore increasingly rely on research and theories from related fields (see Section 2.4). Software engineering
and information systems researchers should see OSS as an opportunity to investigate general software engineering
and information systems research challenges.
Results shows that “prediction of properties”, “aggregate metrics” and “changevevolution analysis” are the most
emergent open issues in OSS evolution,vtogether with OSS integrability and licensing.
This research question evolved due to the fact that most of the reviewed articles (67%) admitted the necessity of
external validity of the prediction models studied. To be specific, in [4] generalization of the findings was not done
because the study is subjective and is dependent on how the errors are classified in the project. Again in [9], it is
acknowledged that further replication across many OSS projects is required to establish the cross project validity of
the prediction model. It is also noted that the prediction models are not general and are not applicable to different
software systems [10]. Specially for defect prediction models there exists very little evidence on their cross project
applicability [5]. Thus a comprehensive study on the generalizability issue of the prediction models across the domain
of OSS projects is an area of future research.
We envision future work could perform a deeper analysis and synthesis on the empirical research on ISS
development. Based on this analysis and synthesis, we will further investigate the limitations of the current research
on ISS development and establish a research agenda on inner source. To enhance the findings of this review, we
intend to conduct a comprehensive survey of practitioners to identify the key challenges involved in ISS
development and propose some resolution strategies to overcome the challenges.
there are few studies on MTTSA of interaction design proposed or used for/in FLOSS development;
• methods of interaction design proposed specifically for the development of FLOSS were not found; the studies found
• techniques of interaction design, proposed specifically for the development of FLOSS, were not found; one of the selected papers, Lichtner
et al. [32], used pre-existing techniques and did not consider the distributed development environment of FLOSS;
• the principal interest of the selected studies is in the activities of prototyping and evaluating; few studies have addressed the activities of
establishing requirements and designing alternatives;
• the majority of the selected studies do not present any type of validation through empirical studies.
Can a new pharmaceutical be developed entirely through an open source model? Likely not. However, a new drug for
a neglected disease may be shepherded up to clinical trials utilizing a hybrid open source model combining open
source with other development models such as fee-for-service outsourcing. To assist with this development, we
believe that further research is needed on business modeling, incentive development and the impact of the use of the
public domain. It is important that this research includes expert input from researchers, the pharmaceutical industry
and PDPs to assess the practicality and relevance of open source drug discovery at a task level.
The areas are important for research and it is interesting to see that research is available in all these areas. The
question of how to use open source practices within a closed company (iv) is for example an interesting area for
further research.
Based on this review we also propose that further research is conducted on how companies can transform their
proprietary software to open source and build a community on it. Further research related to all four research
questions in Section 3.1 could involve more case studies on implementation of specific methodologies for dealing
with different aspects of open source in industry.
OSS development takes place in an environment which is highly affected by socio-cultural parameters and
specifications of users and development teams of OSS may affect or alter the success parameters of OSS. That’s
while context of development is usually ignored while studying success of OSS. Except [20] that studies specific kind
of software and [4] that verifies the model in Korean software context, we do not find any other research that was
based on a specific context. Even these two papers have tried to generalize their findings and the later one mentioned
the context based research as a limitation. So it seems that localizing the issue of success and paying attention to
parameters such as: social, cultural and economical state of development community would be beneficial point of
view in future research.
Based on the comparison of the existing quality assessment models, there is clearly no suitable model—each model
has its own limitations. As a result, the findings of this analysis have implications especially for practitioners who
work towards coming up with new assessment models. They should note the following points in line with the
research questions posed in this study: Emphasis should shift from trying to build comprehensive models (containing
all the possible software characteristics) to building models that include only essential quality characteristics. This
study has shown that these essential quality characteristics include: maintainability, usability and maintenance
capacity of software community. By narrowing down to these three essential quality characteristics, model devel-
opers would help to reduce the burden of OSS evaluation via existing quality assessment models, which has been
referred to largely as being laborious and time consuming to conduct
The results of this systematic mapping suggest the need for broad support for FLOSS projects and communities by
the HCI community, through research efforts in the area of interaction design for the availability of MTTSA of
interaction design considering the characteristics of FLOSS development. Therefore, it is necessary to develop and
publish research on interaction design in the context of FLOSS.
This study also indicates the areas such as joining process and abandonment where research is lacking.
Moreover, mentoring is another field that needs to be explored in the future. Furthermore, significant research is
required to explore the tools, practices, and processes that could be helpful to improve community participation. The
research community may use these findings to understand the issues in the area and select the topic for further
research. Additionally, we also observed from the results that in majority of the studies, combination of survey and
questionnaire was used as a research methodology to conduct research. From these results, we observed that few
studies used machine learning methods. In future use of these techniques will be helpful to solve the problems related
to task distribution, selecting a task to start contribution, and management of project information. Furthermore,
findings indicate that majority of the studies and researchers who investigated community dynamics belong to USA
We also observed that most of the studies (75%) were conducted by the research group of one country. Thus more
collaboration is required among research groups of different countries to conduct more research in the area.
Study the existence of SOC. Another direction of research would be to study the notion of SOC (Self Organized
Criticality) in OSS projects. SOC dynamics articulate that the current state of a project is determined (or at least,
heavily influenced) by events that took place long time ago. Existential exploration of SOC in the domain of OSS
projects reveals contradictory results (Table V). Thus future research can take further step in validating the existence
of SOC and its implication on the evolution of open source software.
Notice that there are a huge number of publications, which report and interpret results from qualitative and
quantitative studies to identify possible risks. It stands out that only [SLR40] calculates threshold values for defining
bug risks, and evaluates their performance, while no paper identifies risks based on quantitative data of project failure
or created losses and revenues, or correlates project failures and losses with quantitative data such as the number of
bug reports and bug fixes.
Few papers consider quantitative measures on community qualities (number of contributors, activity, presence of
heroes [SLR39] etc.), while no work empirically evaluates the existence of causal relationships between the metrics
applied, the risks identified and the actual faults happening. Moreover, an empirical evaluation of the effectiveness of
mitigation activities by their influence on the retrieved metrics is also missing in the works, which propose specific
mitigation activities. Also for very complete works, such as the ones by the group [SLR12][SLR19][SLR20][SLR47]
whose surveys retrieve data for risks and mitigation activities, do not show any link between these two: typical
mitigation activities adopted in software companies are very general (see Section 1.2.5) and would reduce various
risks.
None of the studies discussed the need to develop an early effort estimation for OSS Web projects.From the findings,
it is shown that there have been no studies that discussed the need to develop early effort estimation for OSS Web
projects. As can per seen in several papers recommending future work, the authors only mention the need to improve
the model or to conduct more detailed comparisons or propose the involvement of different effort measurement
attributes, such as [28], in which the future work is to conduct more detailed comparisons by using different
estimation tools.
Some of the analysed approaches propose quantitative measures and analyse their effectiveness (e.g. [SLR4][SLR6]
[SLR15] in the SLR for API metrics and code changes, [SLR14] for code executability, [SLR25] for business
values). [SLR17] proposes a reliability
model combining qualitative and quantitative metrics, but does not consider OSS-specific community and repository
measures.
We would in particular recommend investigating two issues: (1) topics related to integration of OSS components
and (2) topics concerning participation in organization-community or inter-organizational OSS collaborations. We
find these issues important because integration of OSS components concerns most software-intensive organizations
[98] and because participation in collaborative software development is increasing [7, 185]. The research could focus
on identifying the characteristics of successful approaches to OSS, the challenges these organizations met, and how
they solved them.
Deploying OSS: Many claim that reducing costs is one of the advantages of deploying OSS server software,
infrastructure, and applications. However, a recent study by Fitzgerald [80] is one of few studies with a longitudinal
view on deployed OSS products. This highlights a need for more studies on:
􏰅 What are long-term costs and consequences of deploying and keeping OSS products operational?
Our review results showed that the medical literature on the topic of OSS in dentistry is limited and includes mostly
expert opinion and case-control studies.The second suggestion is to include OSS as a control group in experimental
studies on software validation. Such comparative analysis can have positive effects for the commercial pro-
gramming vendors by showing them the most advantageous OSS solutions that can be deployed into commercial
software packages.
It may also be beneficial for customers who, apart from obtaining detailed information on the performance of
software packages, might be able to decide if the risk of using OSS without technical support and requiring greater
computer skills is justified in specific cases.
types in general are directly influenced from the interpretation of different stakeholder viewpoints. Below we
highlight several open issues:
• Since none of the analyzed papers define openness, identifying openness dimensions of IoT platforms remains a
challenge to be addressed.
• It is of utmost importance to find a consensus regarding openness among the different stakeholders to avoid
confusion, and preferably agree on a formal definition.
• Investigating openness not only from IoT platform perspective, but also considering IoT middleware and
frameworks.
• To understand how much openness of IoT platforms has penetrated the field and in which domains, a mapping
study of the application domains of the identified open IoT platforms would be useful.
Since this study only focuses on the involvement of OSS tools towards the development, another future research area
that can be investigated is measuring the other aspect of the effort measurement attribute such as the year of
experience of development toward the OSS as well the error fixing time. As is well known, a year of experience of
the expert’s programming skills that are involved in the projectdevelopment can contribute to a different effort,
therefore how about a year of experience of the expert toward the tools. Does this affect the effort estimation? Since
OSS is an open source which everyone can access freely, therefore a bug that can occur during the implementation
cannot be avoided. As such, what would be the time and effort needed to fix the bug and how this can affect the effor
can also be further investigated.
This study suggests some directions for future research. Disruptive COSS revenue models, organizational aspects of
COSS, localization of COSS, and creation and maintenance of the user community are among them. On the other
hand, practitioners should notice that the points of different aspects of the COSS business model presented here to
improve the cost-benefit trade- offs of their business.
In the anomaly’s prioritization strategy, the agglomeration flood standard and most optimal models showed that some
agglomerations are overburden with false positives and not precise enough to identify architectural inconsistencies in
classes, leading to the inability to capture several various architectural problem types. In contrast, the recommended
heuristics, architecture blueprints, and the context-based smell prioritization techniques show the ability to rank and
improve in identifying the prioritization of anomalies related to architectural problems. This reveals the need for
providing an initial enrichment of the possible results to adopt the solution with a tendency in the ideal combination
to prioritize architectural anomalies. Consequently, there is a need to provide various prioritization criteria for seizing
the diverse architectural problem types and enhancing the essential techniques used for discovering code anomalies.
More- over, the integration of two or more heuristics would get better ranking results’ effectiveness. Additionally, it
is possible to introduce the new strategies to produce ranking on numerous criteria in order to provide visualization
capabilities that are most relevant to architectural problems for the developers.
We analyzed the characteristics and goals of the newcomers. However, many of the papers did not explicitly profile
the newcomers they analyzed. This is probably related to the type of data analyzed and the type of study conducted,
as most of the studies only used data coming from software repositories and did not go deeper in the analysis of the
subjects. The problem is that the term newcomer can be used in a loose way, which can bias the results. Newcomers
can be novice developers who are starting their career, people who are experienced developers from industry but are
not used to OSS projects, or people who are migrating from other OSS projects. These three profiles are different and
can face different barriers or experience barriers differently. Therefore, it would be a better approach to assess how
these different types of developers see the barriers and what their impressions of them are. For example, does a
novice developer find more issues to contribute than an experienced developer without an OSS background?
we claim that emergent research addressing the current mobile-platforms is not considering, or exploiting previous
semi- nal works on open-source platforms, as it often should.
As part of our future work, we are currently conducting a study investigating the effectiveness of current estimation
method especially Bayesian Network (BN) towards OSS Web application development. In the paper [79], the author
had implemented Fuzzy Logic into Tukutuku dataset, and proclaimed that the result of implementing Fuzzy logic is
better than other methods. Another paper [80], also focused on the implementation of Fuzzy Logic toward effort
estimation Web application, but in the study the author only emphasized how well Fuzzy Logic could be
implemented into the field. In other words, the author highlighted the effectiveness level of fuzzy but does not cover
the OSS perspectives as well. Therefore, in addition, we are also looking into how well the level of effectiveness can
be achieved by implementing fuzzy logic conditional probabilities characteristics into BN network for Web
application projects that involve the usage of OSS.
The evaluation of OSS in medicine should not be overlooked. Professional medical certification (FDA and CE) of-
ten cannot be applied to OSS because it requires a legal commercial entity for distribution liability and support
availability. It is also important that FDA and CE certification confirm that the software and its provider ensure an
error-free work- flow, but not the accuracy of software actions/calculations, and provide appropriate documentation
and support. Therefore, experimental studies and algorithm evaluations should be conducted and published.
Acknowledging the lack of published research on the use of OI strategies in specific RE activities, i.e., prioritization
and validation, as well as the lack of reported tool support, we see new opportunities for research on automated and
thus nonintrusive and low-cost methods for applying OI strategies in RE. More specifically, we think this mapping
study provides us with a motivation to conduct more research on capitalizing upon OI to automatically extract and
prioritize requirements.
In both cases of evolution prediction and evolution process support, the reviewed articles admitted the necessity of
external validity, whereas the ratio of articles not addressing validation process is considerably higher (56% for
evolution prediction and 68% for evolution process support). We real ized that vast heterogeneity of evolution
prediction models building data make their evaluation difficult. Generalization of evolution prediction models
regardless of their applicability on project size requires the attention of researchers.
Reviewing the related work in the field of OSS success, we observe different measures and factors for success and
noticed that different methods are used in research in the field but source of data is mainly repositories of OSS
projects such as sourceforge.net and freshmeat.net. We mainly recommend using variety of methods for research in
the field and also want to draw attention of potential research to context of OSS development in future research.
When contrasting previous literature on older “platforms-wars”, such as the ones from the PC and game-console
industries, with the current and under-studied mobile platforms-war, we empirically notice that many of the market
players remain the same (Microsoft and Apple). There is a scenario of convergence: same firms push for similar
technological standards across different platforms, i.e. Microsoft Windows within X-box, Surface Tablets, PC,
Netbooks and Mobile phones. This convergence between industries remains unexplored by academia. Interesting
research questions dealing with the implications of such convergence remain unexplored, i.e “should firms
concentrate on one platform-war or run several platform-wars in parallel?
Despite having a close relation to the CASE research field, only seven experience reports discuss the use of OSS
CASE tools in the context of organizations. Given the number of OSS CASE tools available, it is surprising that the
use of such tools has not been studied in any empirical research papers.
Assesment Process: It is worth mentioning that to perform a complete quality assessment of a software ecosystem we
first would need to define the assessment process which is out of the scope of this paper. The quality assessment
process will have to deal with, e.g., How are the values of each measure interpreted (i.e., defining what are the good
and the bad values)?; How can the measures be merged to provide the assessment for a particular sub- characteristic
of the quality model?; or What are the principles to perform the assessment with missing, incorrect, and/or
inconsistent measure data? We are will provide the answer to these and other questions as part of our future work in
this topic.
Based on the findings of this research, we have planned the following research objectives, to be carried out in the
near future, to more strengthen the existing research work, and to stress the OSS vendors’ community to meet the
maximum benefits of OSS paradigm.
• To identify the potential risk factors in adapting open-source software development from vendors’ perspective
• To identify the practices for addressing the identified success factors and risk factors
• To develop open-source software development maturity model (OSSDMM) to measure the maturity level of vendor
• To conduct multiple case studies at software vendor organizations to evaluate the efficacy of the model
Unbalanced Distribution of Measures: just by looking into the measure tables, it is easy to observe that the amount of
measures for some characteristics is high (e.g., activeness with 17 mfeasures, visibility with 11 measures) while for
other is very low (e.g., heterogeneity with 1 measure, information consistency with 1 measure). This unbalanced
situation could be an indicator that more research is needed for the characteristics with a low amount of measures.
As a future work, this relationship will be reviewed in-depth with respect to RQ4 in order to expand the information
gathered by this SM study and elicit further insights.
Despite the scarce references to learning assessment in the selected papers, some studies assessed the experiences
from both the teacher and the student perspective. They also used various different instruments. The main issues with
the teacher perspective to assessment is the absence of clear definitions of criteria to assess students’ products,
performed tasks, and expected skills and attitudes. Therefore, it is important to state that student assessment deserves
more thorough work. Ellis, Morelli, de Lanerolle, and Hislop (2007) point out that students can perform various types
of contributions. Since they have different back- grounds and previous experiences, they fulfil different roles and
perform Computer Science Education different contributions. Therefore, the authors recognize that grading is not an
easy task. They suggest the need to establish a set of metrics for each role a student plays.
Learning how to solve complex problems, and knowing how to work in teams are relevant skills that are typical
requirements in SE education. Thus, students should develop some skills such as communication and leadership. Peer
assessment is one important instrument to approach this need. Morelli and de Lanerolle (2009) sustain that students
should assess their peers in conjunction with teacher assessment in courses on OSP, even though, the adoption of this
practice is still a challenge.
In any active learning approach, students are responsible to conduct their own learning. A formative evaluation that
includes self-, peer and faculty assessment can play an important role in this process. According to Ellis et al. (2012),
the iterative development process present in OSP, where any artefact can be reviewed by different people, and in
multiple times, provides an intrinsic and valuable environment of formative evaluation.
All those issues represent research opportunities to be more thoroughly explored in the future.
We plan to extend this work in the future by concentrating on the identified research gaps and by introducing more
research questions to acquire in-depth knowledge about community participation and engagement. In the future, this
study can be extended to include software ecosystem, community structure, mentoring, project governance,
difficulties associated with finding a task,and project characteristics that will provide more insights in the community
dynamics domain.Moreover, the authors are planning to expand the search by including more terms in the search
strings and additional data sources, not considered by this study to find more relevant studies on community
dynamics. Thus, provides more insights about research topics and issues in the area that will helpful for researchers.
Furthermore, this study can be repeated by using other research techniques such as a combination of systematic
mapping study and systematic literature review can be used to obtain better results.
Furthermore, future research should focus on newcomers’ orientation and reception to identify the problems in initia
contribution as well as provide guidance regarding barriers in advance. Moreover, significant research is required to
explore the means to support newcomer initial participation, enhancing motivation to increase participation, and
reduce the hindering factors that will improve retention of new developers and One Time Contributors (OTC).
Abandonment is another area in OSS community and has many issues that need to be explored in future research.
Additionally, further research is required to explore various factors that lead developers to stay or leave a project.
As shown in Table V and Table VI, the SLR found relatively a few studies providing sufficient details regarding the
advantages and challenges of using OS in computer science education. Moreover, some of the advantages and
challenges are supported by a few of the papers found. For instance, only 4 papers (19%) included specific evidence
regarding the wider range of skills acquired through the use of OS, which suggests that there is a lack of data
regarding the advantages and challenges of OS use in the practical situations. Consequently, the results of this study
may not be easily generalised and further investigations must be planned in practical situations where OS is currently
used in computer science education.
This reveals that architecture may be recovered with acceptable accuracy unlike the prior intuition based on its
claimed hypothesis in an application difficulty to recover conceptual architecture. However, there is a clear
opportunity for many researchers to shed light on checking more efficient approaches to architecture recovery by
enriching ARCADE’s tool to add further different current code-level analyses to it. Besides, there is also a need to
conduct more experiments on a wide scope on more systems, especially industrial systems as well as to increase
approach accuracy through documentations, pull requests, commit messages, comments, tests, and much more.
We furthermore advise researchers to put emphasis on how the studied organizations actually use OSS, and on
problems that really matter to practitioners. Practitioners should be open to OSS and see that it offers several
opportunities. However, they must first evaluate the implications of adopting OSS in their own context.
Another interesting topic that deserves attention in future research is the emerging body of literature on agile and
distributed development (Angioni et al., 2005; Ramesh et al., 2006). The coming together of these two worlds was
not explored in this study, but we consider that they can also contribute to facilitate reconciliation among plan-driven
agile and FOSS development models.Likewise, the idea of processes diversity (where different processes may be
running concurrently on the same project—in multi-team projects or changing over time during the phases of the
development and maintenance cycle) (Deck, 2001; Lindvall and Rus, 2000; Siebel et al., 2003) also should be investi
gated in future research, since the need to manage this diversity can be an important motivation for the reconciliation
among software development models.Finally, approaches that deal with reconciliation considering organization and
project contexts and needs, appears to be a promising path for reconciliation in the future (Jaufman and Munch, 2005
Xu and Ramesh, 2008). This is the focus of the next section.
Many of the studies are in the form of surveys, which gives a broad and necessary understanding. Based on this it
would probably be possible to conduct more studies investigating specific cases of implementation of methodologies
for dealing with different aspects of open source in industry. More case studies could probably be conducted on all
aspects of the research questions. More case studies could probably also provide more knowledge of research
question 3 and research question 4. That is, research could be carried out to understand more about the cost and
advantages of different approaches, and why different approaches are chosen. It is also worth noticing that there are
no controlled experiments at all in the identified articles.
Furthermore, our study has revealed that the quality of reported empirical research on OSS has significant room for improvement. To that
end, we have proposed a set of guidelines for reporting empirical research on OSS. We claim that these guidelines can help the OSS
research community to improve the quality of designing and reporting empirical studies.
The analysis of the results allows us to state that OS- SECO is a growing research area in software engineering
[R16, R49, R50]. Due to this, there are several new research opportunities in the empirical examination, modelling,
analysis, measuring, quality evaluation, etc. of OSSECOs. Along with this argumentation, in this section we provide
two initial proposals to improve the current structure of the knowledge on OSSECOs: a definition for OSSECOs and
a taxonomy of OSSECO- related terms.
COSS is comparable to opensourcing. In opensourcing, the companies outsource to the open source community
outside of the company [82]. This lowers the cost of development, because on the one hand, volunteer developers
code for free, and on the other hand, users report flaws in the software [43]. Hence, the existence of an open source
community is vital to the success of the open sourcing company [82]. In COSS also exists a user community and
somehow developer is effective and improving [3]. But, the problem is creating and sustaining such communities [41
43]. Therefore, it is imperative that the management of the creation and maintenance of the user community in future
research be investigated; an issue which has not been seriously considered in the literature of COSS.
The next point is that, as Riehle noted, open source software can possess established markets, provided that it is
sufficiently disruptive [1]. For this disruptiveness, we suggest working on the revenue model. In fact, since the
business model is a system and every change in one component also affects the rest [4], the initiation of this
disruption can be a revenue model. Therefore, it is suggested that future studies in helping to describe more revenue
models examine the role of new revenue models in adding to the disruptiveness of the open source software. This is
because the explanation of open source software revenue models has been difficult, especially because of free
distribution [83]. In addition, this study suggests that the provision of complementarities has been an integral part of
the COSS business model as a way of earning money. So, new disruptive revenue models can help new configuration
of the COSS business model and probably possess established markets.
Due to the cultural, economic, institutional, geographic and other characteristics of developing countries with
emerging markets [84], the use of business models of developed and matured markets is often unsuccessful [58].
Therefore, the logic of the creation and capture of import value of COSS may need to be adjusted. This implies the
need to localize the COSS business model [85]. In addition, countries are demanding indigenous and localized
software, but due to expensive licenses, some of them are looking for open source software [86]; a localized open
source software. But, in the literature of the COSS business model, not only the commercialization of open source
software as a business model is not considered, but there are also few studies on the open source software
localization. Therefore, it is suggested. Finally, there is not much research about the organization and about what is
happening inside a COSS company. Therefore, one of the implications for future research is that it is suggested that
future research examine the structural dimensions and content dimensions [87] of the COSS organization.
On the other hand, studies such as those conducted by Steinmacher et al. [PS14] and Jensen et al. [PS9] presented
simplistic views of the problem when they drew conclusions from only analyzing the first messages from new-
comers and their retention. The context is important: Why did they send the messages? What motivated them? Did
they really want to contribute or just clarify some doubt? Did they contribute at the end but never got back to the
mailing list? To answer such questions, we need to merge information from different sources (issue tracker, mailing
lists, documentation, code repository) and verify the context by talking to practitioners. Another possibility is to
conduct observational and ethnographic studies by analyzing the barriers and effects for newcomers in real settings.
Since 2008, synthesizers of research have introduced frameworks and platforms to perform OSS research paving the
way for future work. The analysis of non-cited papers indicates that significant research has not been exploited, yet.
Therefore, we recommend the OSS community to exploit further the potential provided by the OSS conference series
while maintaining the interest in its major research streams.
Most used sources development tools of the data are not properly documented. None of the study data documented
whether there was any involvement of OSS features in the development.From the result we know that the most used
sources development tools mentioned in the data are not properly documented, as can be seen from the result, with
the highest percentage being unidentified development sources. Due to the poor documentation regarding the
involvement of the sources development tools, we have not been able to determine whether in each of the projects,
was there any involvement of OSS. Since this SLR is focused on OSS, therefore we can say that to the best of our
knowledge, there has been very little documentation of OSS usage in the datasets, which led us to further investigate
how OSS can affect the accuracy of effort estimation.
Future work could include the expansion of this research, overcoming some stated limitations by increasing number
of universities or by considering only experienced Open Source practitioners. This way, we would be able to provide
a more effective feedback to the researchers. Another possibility is to assess the survey participants motivations
regarding the rates provided. With a greater amount of participants, this could be achieved without extending the
consumed time, which could affect the quality of the results.
Newer models should incorporate selection meth-ods that are amenable to automation as this is not the case in most
of the existing OSS quality assessment models reviewed in this study. The selection methods mostly adopted are the
model (32%), process (21%) and other (16%) such as guidelines, which are not easily amenable to automation
(Fahmy et al. 2012). Model developers should thus turn their focus to data mining techniques (Leopairote et al.
2013), framework or tool-based selection methods, which are currently among the least considered options. The
advantage this offers is that it will help quicken the evaluation process resulting in faster decision- making. Following
this advice could also bring about increased adoption of the models in practice (Wang et al. 2013). In addition, model
developers can also consider modeling quality assessment as a multi- criteria decision-making (MCDM) problem so
as to facilitate automation as seen in recent studies (Fakir and Canbolat 2008; Cavus 2010, 2011). A MCDM problem
in this context can be regarded as a process of choosing among available alternatives (i.e. different OSS
alternatives) based on a number of attributes (quality criteria). Considering this option opens the model developer to
several well-known MCDM methods that amenable to automation such as: DEA, Analytic Hierarchy Process (AHP),
and Technique for Order of Preference by Similarity to Ideal Solu- tion (TOPSIS) to mention a few (Zavadskas et al.
2014).
As discussed in Section IV.D, there is no concrete relationship set between quality and success criteria of OSS in the
reviewed papers. The lack of studies examining the relationship between quality and success criteria and metrics
should encourage the researchers to conduct further studies in this context. In addition, there is no satisfactory
number of studies in the contribution types of model, metric or tool, and it is observed that there is an evident
necessity to fill the gap for these types. Since measuring the success or quality is a challenging task, especially tool
contribution is quite minimal. This may lead practitioners to collaborate with researchers in order to clarify
terminology, identify metrics, and develop tools that are capable of meeting this need.
It is worth noting that the main focus of analysis were large projects with a high number of developers and more than
five years of existence. Moreover, projects that focused on products used during the development cycle and
developed in Java and C were preferred. Such projects can be classified as clearly successful projects which,
combined with the historical data available, provide an easy target to search for newcomers. We observed that,
although projects gain several newcomers, just a small percentage are successful in contributing some source code.
Because the identification of barriers faced and surpassed by such newcomers is important, projects with a high
number of developers (and newcomers) are easier to analyze to find evidence of such barriers. However, a high
number of OSS projects present different characteristics, such as small teams and short lifetime, and were not
considered for evaluation. Naturally, such projects provide less data and are less attractive than large successful
projects, but when considering newcomers, they can account for different problems than those identified by our
model or modify their importance. Further investigation is required regarding such projects to improve the model of
barriers described in this paper.
Maturing the research field on OSS in organizations and dealing with some of its limitations may be done through
four main steps:
1. Focus research on topics that are relevant to how organizations approach OSS
2. Strive to increase the rigor of the empirical studies
3. Conduct more longitudinal, in-depth studies
4. Align our research with related research fields
Navigating through the amounts of OSS components and related information available across the Internet is a
significant challenge [143]. However, the information offered over the Internet through OSS communities, web fo-
rums, and so on, constitutes at the same time a valuable resource. Due to the easy access to reusable software
components, we see that software systems are constantly growing. Software developers are integrating an
increasingly larger number of OSS and commercial components into their products. In doing so they have to relate,
adapt, and possibly contribute to a large number of providers. Therefore, we believe research could focus on the
following questions:
􏰅 How may organizations most efficiently navigate through available information and select OSS components?
Some trends and issues emerged from a detailed analysis of the studies: (i) solution proposals are the main research
approach; (ii) very few papers focus on specific SE areas; (iii) the traditional project method is the main learning
approach; (iv) most studies use previously chosen OSP in regular courses; (v) there is a balance between inside and
no control approaches; and (vi) very few papers use criteria to evaluate students’ learning based on either outcomes
or developed skills. We also found three main combinations of OSP use: (a) full control and predefined projects, (b)
no control and free choice projects and (c) inside control with no or almost no project choice for students. These
trends and issues provide future directions for research.
n fact, most papers do little more than mention such issues as research questions, limitations, data analysis, and so on
We furthermore found that many of the publications lack details about the research methods and findings. As a
consequence, several papers have limitations when it comes to how they describe these issues. Moreover, many of the
research papers are explorative and they are therefore lacking a precise focus and clear contributions.
It will be worthwhile to explore the capability model for OSS developers. Since most OSS projects rely on task or
issue tracking systems to maintain the projects, recognizing the time of specific maintenance tasks can provide better
decision support for task assignment as well as OSS project management. A large amount of studies are devoted to
predicting bug fixing time while a small amount focused on other activities such as code review and duplication
identification. These studies commonly used metrics from source code changes and issue reports as predictors, which
indicate that the prediction results are basically related to the characteristics of the tasks. There are two kinds of
targets among these predictions. One is the numerical days of an activity, evaluated by PRED(25) or PRED(50).
Another is the bins of categorical time evaluated by accuracy, precision, recall, or f-measure. The prediction results
for numerical days are not very satisfying, while the results of categorical bins are relatively high. In OSS projects,
the time recorded on issue tracking system and repository may not correlate with the actual effort because the
developers are voluntary and self-determined when implementing the tasks. It will be worthwhile to explore the
capability model for OSS developers and consider the developer-related metrics to be one of the sources of
predictors, as an opportunity to improve the prediction results.
Our future work will extend this analysis to OSS article hubs like for example http://flosshub.org or
http://pascal.case.unibz.it or the MIT repository.
We plan to use more sophisticated techniques of string similarity and a better data cleansing to get finer result
The results shows that security areas in construction and verification(secure architecture, code review, and security testing) are followed by
researchers with more interests than other areas in Governance and deployment. Next based on our research, the security studies in OSS
development are mostly technical driven. The socio-technical perspective has not gained much attention in this research area (2 out of 42
papers). According to the result of socio-technical analysis on the selected papers, the discussions between technical and social aspects seem
quite unbalanced, either (Coverage rate:98% versus 16% in average). The socio-technical perspective has as the main target to blend both the
technical and the social systems in an organization. This can be viewed as a necessary condition within a security management framework as
both aspects are of equal importance. Technical security practice considering different social aspects (e.g., culture and structure) of open
source develoopment will assure the effectiveness and efficency of the implementation of the tool.
A few other issues are worth mentioning. First, all of the eight empirical research papers from the public sector focus
on deployment of OSS products. Besides [37], which has a mixed sample, no paper focuses on deploying OSS in
the private sector. Second, 27 of the 59 empirical research papers report findings from samples of several
organizations from the private sector. However, as few as eight papers report findings from one single private
organization. Hence, most research papers dedicate relatively little space to describing the individual organizations.
First, regarding research types, it is worth highlighting that the high number of evaluation papers and the increase of
validation contributions reflects the maturity FLOSS adoption studies. However, this research area is not yet to the
point of contributing experience reports. Second, regarding FLOSS adoption factors we observe that most of the
studies have been focusing on the organizational and technological factors leaving the economic factors not so well
covered. We suspect that this lack of research results in economic factors is due the reluctance of companies to
provide economic details. Also, FLOSS experts consider that organizations are already aware of the hidden costs
when adopting FLOSS, and therefore, they tend to focus more on researching technological and organizational
factors. Additionally, we only found two solution proposals related with economic factors and one with tech-
nological and organizational factors. We also observed that validation research, opinion papers and philosophical
papers are gaining maturity in the FLOSS adoption area because we found taxonomies, literature reviews and
systematic maps.
Software size has been the most common attribute to analyze evolution of OSS projects. Several types of metrics
have been employed to measure software size. These metrics range from coarse grained level metrics such as number
of files, modules, and functions, to fine grained level metrics such as number of LOC, methods, and classes. Several
approaches, other than source code analysis using metrics, to analyze OSS evolution have also been employed in the
research literature;
• Lately, metrics related to change activity have also been included to understand OSS evolution. These metrics
measure changes in source code such as number of program elements (functions/ /classes/methods) changed in
consecutive versions. Change activity as recorded in SCM systems is also used in a few cases. Most of the work deals
with finding change size, change effort distributions. A few studies do change profile analysis as OSS systems
evolve.
But that is restricted to a few of the change categories e.g. adaptive v/s non-adaptive changes, or corrective v/s non-
corrective changes. A fine-grained view of the changes can help to answer amount of progressive/ regressive work
performed in a software system as it evolves. It can also be used to validate Lehman’s 2nd law as Gonzalez-Barahona
(2013) points to the lack of information available in this regard in their study of the glibc system;
• Techniques and tools have been devised to tackle large amounts of data generated in software evolution analysis
and prediction. Software evolution visualization helps in understanding the transitions in complex and large systems
in an easy way. Big data analytics can also help to analyze large sets of data generated during software evolution.
Data analytics can be used to manage and understand the complex web of software evolution as it happens in source
code and other related repositories.
Individual contribution and performance measurement also has been receiving attention. Gousios [15] defined a
contribution ratio by considering various type of parameters from the OSS community, and Rastogi et al. defined the
contribution in terms of four different roles of stakeholder. Since the importance of contribution measurement has
been realized, it might be a promising research topic in the coming years.
The first opportunity for future research lies in reexecution of the protocol, to capture references to more recent work
that extends the search space chronologically. This could also include other search engines, such as ACM
(Association for Computing Machinery), in an attempt to retrieve documents only indexed by these machines, which
would extend the search space geographically. Finally, the search can also be expanded with manual searches to
include: books; conferences; theses and dissertations; technical reports; and other search engines, such as Google
Scholar and AISeL (Association for Information Systems Electronic Library). Although the systematic approach
adopted ensures the reliability and com- pleteness of this study, it can be amplified by these extensions.In Section 7
(SQ2), we discussed how studies that showed how to combine two development models could be extended to the
third. This discussion included ideas like: (i) the reconciliation of FOSS and plan-driven configuration management
practices to agile model; (ii) the need for further studies on the reconciliation of the practice of continuous code
integration between agile and FOSS models and extension of this search to the context of the plan-driven model; (iii)
investigate how the knowledge management practices in agile model can contribute to improve knowledge
management in organizations or in FOSS communities; (iv) analyze if the use of explicit knowledge management
practices, coming from plan-driven and FOSS development, can be beneficial in an agile context; (v) extrapolate the
use of test driven development to a plan-driven context. These ideas are all opportunities for future research.An area
that still deserves to be explored is the search for studies to reconcile the three models of software development, since
only one study was identified in this quasi-systematic review. First of all, it can be present in areas that were not the
focus of this work. Some of these understudied areas are indicated below to guide future research. In addition, as
stated before, the reconciliation research area is still at an early stage, but it is expected that in future there are more
studies and results on the topic to be investigated. Finally, there also remains the possibility that organizations or
other researchers have already achieved positive results on the reconciliation among agile, FOSS, and plan-driven
models, but have not written about it yet.
Consequently, the healthcare industry is “lagging behind in terms of adoption of modern ICT tools and
infrastructure”, compared to other sectors (Karopka et al., 2014; Munoz-Cornejo et al., 2008).
A mapping study review provides a structure for a research report type, which enables categorizing and giving a
visual summary of results that have been published in papers of a research area[8]. This map aids to identify gaps in
a research area, becoming a basis to guide new research activities[7]. The current mapping review captured the
current state of research on bug report severity prediction, characterized related problems and identified the main
approaches employed to solve them. These objectives were reached by conducting a mapping of existing literature.
In total, the review identified 27 relevant papers and analyzed them along 12 dimensions. Although these papers have
made valuable contributions in bug report severity prediction, the panorama presented in this mapping study review
suggests that there are potential research opportunities for further improvements in this topic. Among them, the
following research directions appear to be more promising:
• There is an apparent lack of investigation on bug report severity prediction in other relevant FLOSS such as, for
example, Linux Kernel, Ubuntu Linux, and MySQL, and in others BTS, for example, Github.
• Often, technical users report most bugs. Thus, the influence of user experience in predicting outcomes is still
overlooked.
• Bug reports labeled with default severity level (often “normal”) were prevalent in the most datasets used in
reviewed papers. However, they are considered unreliable [26], and just discarding them also does not seem
appropriate. Then, efforts in researching on novel approaches to handle this type of report should be considered to
improve the state-of-the-art of severity prediction algorithms.
• Most approaches were based on unstructured text features (summary and description). To handle them, researchers
chose to use the traditional bag-of-words approach instead of more recent text mining methods (e.g., word-
embedding [66]) or data-driven feature engineering methods which may likely improve outcomes yielded so far.
• There is a clear research opportunity to investigate whether state-of- the-art ML algorithms might outperform the
traditional algorithms used in all reviewed papers for bug report severity prediction. The investigation of the use of
Deep learning algorithms which perform very well when classifying audio, text, and image data [67] seems to be a
promising research direction.
• Researchers should investigate more recent techniques (e.g., continuous learning [68]) to provide an approach for
bug report prediction which could be employed in real-world scenarios.
• Many bug reports are resolved in a few days (or in a few hours) [69].Efforts to predict severity level for these group
of bug reports do not seem very useful. Thus, an investigation to confirm this hypothesis and to determine when the
severity prediction is more appropriate in bug report lifecycle is of critical importance.
• The primary objective of our mapping study was to review the research approaches for severity level prediction in
FLOSS. However, it would be a promising research venue to extend this study to commercial systems, and verify
This study also identifies some challenging areas for future work.
1. Append new findings into the body of knowledge on OS forking behaviour. Applying the combined approaches of
SLR and CAM revealed seven forking types interpreted by academic researchers and the latest interpretation found is
file language repository fork. This novel insight will assist researchers on how forking is presented and interpreted
and industry practitioners in reviewing project forking health, especially projects with programing language file
repositories that are less adopted or forked by developers.
2. Understanding forking consequences. Case studies are an important way to highlight lessons learnt by researchers.
This paper identified forking impacts and consequences, with one of the worst impacts being a political strategy that
divides a project community and forms a new community. Forming a new community results in less contributions by
developers to the original file repository, bug fixes or feature enhancement. Allowing accumulated bugs and feature
enhancements to remain unfixed for a period of time can affect project health risk.
3. More research is required on forking sustainability. Reviewing these 21 papers revealed the importance of forking
sustainability investigation as a top priority with two specific areas of interest.4. Studying forking sustainability using
a SLR for software development with GitHub. Valentio, Javier, Izquierdo and Cabot (2017) used a SLR to show that
forking is a good indicator of project longevity and the chance of forking is highly dependent on the project, where
developers provide additional contact information (e.g., emails, personal website URLs that are clearly active or
aligned with popular project owners) to increase social connections between a project owner and forker, and increase
developer community size for medium-size projects and projects that are written in a forker’s preferred programming
language. Future work could include developing a prediction model for fork effectiveness from forking motivation
classifications in response to language repository files, where programming language survival time is critical to an
OS projects’ health and survivability.
As future work, it is necessary to investigate in real software projects, in software organizations and in open-source
software projects, what are the influence factors observed by their developers. Comparing the results of this SLR with
real observations may clarify the importance of influencing factors within the context of productivity within
organizations and in open-source communities.
Most OSS projects used in the primary studies are still active however, and often contain codebases with moderate to
very high activity. In general, therefore, it seems that the relationship between the adaptation of OSS in safety-critical
and their long history is rather clear. At the same time, further research should be done to investigate the relationship
between the adaptation of OSS in safety-critical and OSS project activity. Moreover, it would be interesting to
investigate the number of downloads of this project or its adoption within industry
Further research is required to explore suitable KR practices applicable in OSS projects as indicated by one of the
questions raised on mechanisms and team norms that are used to store knowledge contributed by team members. In
CSS organisations KR mainly comes into focus when an employee is leaving an organisation (Lindvall & Rus, 2003)
On the contrary, in OSS projects the unpredictable nature of commitment from contributors creates an element of risk
(Robles & Gonzalez-Barahona, 2006). In OSS, contributors can leave since they are not under any contractual
binding as in CSS organisations.
In this literature review, we found that KM relevant activities of knowledge creation and knowledge sharing are
evident in OSS projects as discussed in Section 6.1. Furthermore, literature examination directed us to 10
mitigations to reduce the impact of knowledge loss due to contributor turnover in OSS projects discussed in Section
6.2.
Migration of responsibility and sustainability. It has been reported that migration of developers from one release to
the next is high and that the developers take more responsibility as they gain experience. Yet it is a common
phenomenon in open source domain that developers freely join or leave the project. And when a developer leaves, his
responsibilities must be assigned to someone else. For instance, the codebase maintained by a outgoing developer
should be taken care of by others. Else it will be abandoned and discarded from subsequent releases. Thus it will be
beneficial to explore the followings,
• How responsibility migrates among the developers? Does this migration follow preferential-attachment?, i.e., is the
responsibility handed over to the developers who are in close connection to the outgoing developer.
• What impact such migration has on the project evolution?
Furthermore, the result of this SLR study also shows the gap that there is a lack of knowledge management aspect of
open source security. Several researchers did mention the knowledge problems in securing OSS development,
however, we cannot identify any study tackle this security issue from knowledge management perspectives.
In order to reduce dropouts of developers who abandon an OSS project (especially the ones who disappear after their
first contribution), there is a need to look for effective ways to engage such developers in the community and
providing good community support when they are taking baby steps in an OSS project. Reception of new
developers is another challenge in the open source community. To overcome this issue, research is needed to
develop tools, which assist in better reception of new developers. Moreover, more significant research is required to
study the impact of community dynamics (stagnant v/s dynamic community) on developer turnover and the effect of
turnover on project performance.
There are relatively few empirical publications on OSS in organizations, and the quality of published work is not
good enough. Much of the published
research lacks relevance and a clear focus, and does not draw enough support from related literature
. These observations are not particular to research on OSS. For instance, Kitchenham et al. [117], Vessey et al. [192]
and Zelkowitz and Wallace [214] observe a lack of relevant empirical research of high quality within both the
software engineering and information systems fields. Finally, we would also like to see more research from outside
Europe and the USA.
OSS researchers can also benefit from these results by using them to conceive strategies for newcomer support. To
achieve this, it is necessary to put more effort on specific research topics, such as understanding and creating ways to
measure the influence of the barriers in newcomers’ experience, identifying and creating different strategies to lower
each barrier, and proposing metrics to grade the support offered for each barrier. To gain a better understanding of the
barriers and to what extent they need to be lowered, it is important to conduct more qualitative studies because this
phenomenon occurs in a complex, social environment in which the context of its occurrence is important. Moreover,
a qualitative view complements the existing literature, which relies mostly on quantitative evidence.
In the architectural smells group, architectural bad smells (architecture anti-patterns) stood out as the most effective
smells compared to architectural change (instability) and architectural hotspots smells (as shown in Fig. 10). This
reveals that the architectural bad smells were studied in isolation and not combined with more than one smell, which
was covered in the prior code smells. Consequently, further research in this domain should be conducted to identify
the effect of architectural smells agglomeration and its correlation with architectural problems rather than the
architectural smells in isolation in order to prove its inclusion or exclusion as one of the key indicators of the
architectural decay symptoms.
show a great concentration of empirical research on the study of how organisations adopt ISS development into
their internal software development processes. Other research areas receive much less attention. Among the
frameworks/methods, models and tools identified, none of them have been empirically validated in real industry
settings. One of the implications of these findings for research and practice is the need for more empirical studies on
engineering practices to support ISS development. Specifically, while ISS development is highly influenced by OSS
development, there is a need to translate OSS practices to suit the organisational context to achieve the many benefits
associated with OSS. Furthermore, future research is required to empirically validate the proposed
frameworks/methods, models and tools. To advanced our understanding of the inner source phenomenon,
researchers need to draw on theoretical foundations that have been used in prior research on OSS, as well as other
theoretical lens that are considered relevant to ISS approach.
The implication for practice also lie in the evidence of the benefits and challenges of ISS development. The findings
have shown that the adoption of ISS development helps organisations to improve better quality, time-to-market and
in- novativeness. However, as suggested by Brown et al. [4], that newcomers should understand the reality of the
method through an appropriate enculturation, so that they can recog- nise what works and what does not work, and
thus be aware of changing working processes.
The future direction for our work is to evaluate further practices that can be incorporated in the OSS project work
structure. The work structure of OSS projects is informal, varies in each project, and is dynamic due to transient
contributors; therefore, it is a challenging task to generalise a set of practices for all OSS projects.
Organisations invest in KM activities to organise, create, share, reuse, transfer, and retain knowledge. We found
knowledge relevant activities in OSS projects namely knowledge creation and knowledge sharing. Knowledge
sharing was found to be abundant but there was no evidence of knowledge retention to reduce the impact of
knowledge loss in OSS projects. Moreover, knowledge sharing is reactive in nature, initiated by the contributor while
looking for task relevant knowledge. We suggest that there is insufficient attention paid to KM in general in OSS, in
particular, there would appear to be an absence of proactive measures to reduce the potential impact of knowledge
loss. We also propose the need for a KM evaluation metric in OSS projects similar to the ones that evaluate the health
of online communities. KM evaluation metrics should be based on the extent of knowledge sharing activities
observed in a project. Such a metric could help to inform potential consumers of the OSS of the KM status on a
project, something that is non-existent today. We consider it a vital ability for OSS projects to sustain a knowledge-
sharing culture that will support the long-term survival and competitiveness of OSS projects.
It can be concluded that the problems of architectural erosion within the OSS projects, including identifying,
addressing, avoiding and predicting are still open research issues, which need further analysis and investigation.
Consequently, more efforts on this domain should be focused on identifying the other reasons that are still unclear
and suggesting other solutions to provide more performance and accuracy to address architectural decay.
In the future work, we plan to address the in-depth investigation of estimation accuracy by applying the estimation
models to one typical OSS database, and identify benchmarks.
An SLR study provides directions for researchers and practitioners on architectural decay within the OSS domain as
follows:
1) There are reasons that could contribute excessively to increasing architectural erosion such as rapid development
of software, frequent changes, and lack of developers’ awareness. Therefore, further studies should be conducted as
a rooted-deep study to find out other causes and to identify architecture erosion whether on the OSS or industrial
systems. Practitioners should follow guidelines to avoid architectural contradictions in the new and subsequent
versions of the system in terms of identifying the potential reasons within the system environment.
2) Sincearchitecturaldegradationsymptomspresentthat code smells agglomeration has a considerable correlation to
architectural problems compared to code smells individual, researchers should conduct further investigation on
architectural bad smells in combination. Practitioners can change their detection way depending on the code smells
agglomeration to identify degradation symptoms effectively.
3) The findings of the current study serve as evident that a metrics-based strategy is the most commonly used solution
as compared to other proposed solutions. However, more studies are needed in this field for other metrics to be
analyzed to provide the most architecturally appropriate solution and identify the effort required for the metrics to
detect architecturally related smells. The essential techniques of ranking used should enhance the possibility to get
better effectiveness results and the identification of critical cores of architectural violations. Also, there is a clear
opportunity for many researchers to highlight enriching ARCADE’s tool for efficient approaches to architecture
recovery. Additionally, there are solutions, which show that it is not effective at all in the problems of deep analysis
and the difficulty of address such refactoring strategy that has no considerably a positive impact to address
architectural erosion.
However, according to the facts detected in the research studies, IT managers are neither using any tool nor
procedures that allows them to evaluate the adoption of FLOSS solutions. For this reason, this contribution can
motivate researchers to work on the creation and publication of guidelines for adopting FLOSS. The purpose of this
study is to guide future research in the application of FLOSS in new domains as a guide for the correct selection of
FLOSS to help IT managers make appropriate decisions for organizations, define policies for FLOSS adoption,
among others.
Due to the rising dominance of the OSS in the software industry; not only practitioners, but researchers as well as
academicians are also keen to understand the OSS software development process. Several studies have been
conducted in the past in this regard. This paper presents a systematic literature review of the studies performed to
understand OSS evolution. A set of 190 primary studies are identified for analysis and discussion. The studies are
characterized on the basis of the research questions they address. The main findings are as follows:
• OSS evolution prediction studies use ARIMA modeling of time series analysis for prediction of software evolution
attributes such as size, defects, change requests etc. However, as software evolution in general and OSS evolution in
particular is a discontinuous phenomenon, the use of prediction techniques that just extrapolate the historic trends in
to the future should be a conscious task. Furthermore, Herraiz et al. (2007c) observed that there are no long term
correlations in the time series representing OSS activity. The idea of fuzzy time series to deal with the uncertain
evolutionary behavior of OSS systems has also been explored. The results show that a fuzzy based method for time
series analysis is rather a promising approach. More rigorous prediction methods may be explored in future;
• Lehman’s laws of software evolution for OSS systems have been validated in several studies. Only two laws (I and
VI) have been confirmed so far in different studies on OSS evolution. There is need to look into the change activity
of these projects and validate the laws using the change related information available in the SCM systems
• A shift in the programming languages, from procedural to object oriented, has been noticed as
OSS systems, as subject systems in the corresponding studies, evolved over the period of time;
• Techniques and tools have been devised to tackle large amounts of data generated in software evolution analysis
and prediction. Software evolution automation offers to collect volumes of data in a consistent manner. Software
evolution visualization helps in understanding the transitions in complex and large systems in an easy way. Big data
analytics can also help to analyze large sets of data generated during software evolution. Data analytics can be used to
manage and understand the complex web of software evolution as it happens in source code and other related
repositories (Gonzalez-Barahona, 2017).
We also noticed a lack of in-depth studies on technical issues faced by newcomers. The reason can be attributed to
the small number of qualitative studies found because it cannot be quantitatively extracted from mailing lists. For
example, technical hurdles are evidenced by only five studies analyzed. Issues related to workspace setup is reported
in only one study, by one subject in a debrief session. These kinds of issue deserve more attention, from both
practitioners and researchers.
Traditionally defect prediction models rely on metrics that represent the state of the software system at a specific
moment in time [11].These metrics are used to capture a particular snapshot or release of a project to predict the next
one. But metrics capturing changes over time in projects also play a significant role in prediction. For example,
metrics presenting the software evolution were used to predict the need of refactoring [12] and quality of OSS
projects with significant accuracy. Thus a future research direction would be to explore a comparative study for
identifying either (a) which form of metrics are more suitable for prediction models in terms of accuracy,
reproducibility, and generalizability, or (b) are these metrics complementary to each other and should be used in
combination to get better prediction results.
Another result of our study is that there are no clear definitions related to the openness of IoT platforms. One of the
papers attempts to define the platform aspects only [33], another paper categorizes the openness dimensions of
platforms [34], and a final study categorizes some open-source platforms but without providing a definition [35].
Thus, none of them explicitly define what an open IoT platform is. Moreover, in our study we were also interested to
identify the openness types of IoT platforms. Our results suggest that the most common openness types of most IoT
platforms are related to open-source. Other types identified are open standards, open APIs, open data and open layer-
based platforms. However, to further investigate the openness types of IoT platforms we believe it is important to
look from a stakeholder view, as identified above [34,52]. Thus, it is essential to further analyze how important these
openness types are from different IoT stakeholder perspectives.
ON RESEARCH METHOD
A number of issues related to the research approach can be improved to increase the acceptability of the reported
results. We pointed out the followings,
External validity of the results. Empirical study is the most popular research approach employed in evolution studies
(Figure 4). These studies, however, are horizontal in nature (as reported in RQ5) considering only flagship OSS
projects. Due to this approach of studying OSS projects, the reported results suffer from generalizability threat, as
reported in Figure 7. Yet to make these finding applicable and hold for the extended region of OSS projects, explicit
measure should be taken. An interesting route to deal with this is to categorize the findings (current or future)
according to the project domain, or similar organizational structure and practices, or similar product size and
complexity. This will reveal the broader picture which can then be compared and possibly merged for proposing a
more general evolutionary pattern and behavior for OSS.
In order to increase the participation of new developers, there is a need to devise tools, which may eliminate
contribution barriers and provide onboarding support. It is necessary to examine the impact of social interaction on
newcomer success, the effect of documentation on task accuracy, technical and coding issues, cultural differences,
and issues related to creation of local workspace. Fur- thermore, more research is required to study the impact of
mentoring on the success of onboarding process and provide suggestions to better support newcomer onboarding,
and contribution barriers in virtual communities. Besides, more research is needed to develop strategies to alleviate
the issues related to choosing a task to start initial contribution, to design tools to enhance retention of One Time
Contributors (OTC), and to explore the motivation of developers to work as a mentor.
Studies are conducted on the joining process and abandonment. Therefore, future research should address the issues
of these two areas. The authors also noticed that some studies investigated more than one research topic.
Additionally, our study identified the gaps in community dynamics area that could be useful for researchers.
Moreover, mapping study also provides insights of some areas that need more exploration for instance mentoring of
newcomers and finding a task to start are some of the open research areas, which need more study. There is a need to
develop tools to better support newcomers’ onboarding and easy migration of the developers among different
projects. There is a lack of evidence on how to remove technical barriers, how gender and age of developers influence
their retention in a project. It shows that there is still need of tools, practices, and processes for better community
participation and engagement in OSS projects.
Our results suggest that the most common openness types of most IoT platforms are related to open-source. Other
types identified are open standards, open APIs, open data and open layer-based platforms. However, to further
investigate the openness types of IoT platforms we believe it is important to look from a stakeholder view, as
identified above [34,52]. Thus, it is essential to further analyze how important these openness types are from differen
IoT stakeholder perspectives.
As far as research community concerns, the results from the systematic mapping study prove that many researchers
don’t perform empirical studies or replications for solving the open issues.
The review establishes the state of the research on ISSD in terms of focused knowledge areas and contributions.
Majority of research centred on the adoption and adaptation of inner source in various context, while other KAs in
SWEBOK receive less attention. Thus our review calls for more studies on these areas to advance our knowledge on
the inner source phenomenon. Furthermore, to advance our understanding of inner source, researchers need to draw
on theoretical foundations that have been used in prior research on OSS, as well as other theoretical lens that are
considered relevant to inner source. With the help of the OSS framework, our study also has identified the
characteristics of the inner source phenomenon, including its types of projects or products, the motivation for
adopting inner source, its environment, inner source processes and tools and the actors involved in inner source. We
also highlight the challenges as well as the benefits for organisations that are interested in inner source. For
research, we identified a research agenda to further advanced our knowledge in the inner source area.
In addition, it is possible that adding non-software and information system related databases may yield similar or
different findings. The comparison of findings from different databases and the findings presented in this study can
po- tentially be considered as future work.
researchers need to pay more attention to issues that are interesting to practitioners. Hence, we recommend focusing
on topics related to the ways in which organizations actually approach OSS, and issues that could benefit
practitioners, rather than general “adoption issues”. Researchers and practitioners should increasingly collaborate to
define a common research agenda and study research questions that matter to practitioners. These research questions
should be answered through several related studies from different contexts.
SLR concerning software fault prediction was first conducted by [3] and was extended with new results in [7].
However these works were limited to fault prediction of closed source projects and fall short of exploring OSS
domain.
This SLR will help researchers to investigate prediction studies from the perspective of metrics, methods, datasets,
tool sets in an effective manner. Future research should focus on establishing external validity and consistent
accuracy of prediction models, incorporation of social aspects of OSS projects, and building tool support to automate
the prediction process.
None of the studies discussed early effort estimation for OSS Web projects.
• Most of the source projects used as the datasets are in CMS form, and none of the development projects in the
dataset involved any open source framework as there are no recorded details.From the findings it is shown that there
has been no proper documentation detailing the involvement of OSS in the development, hence we can conclude that,
none of the studies have discussed early effort estimation for OSS Web projects.
Concerning the domains (RQ1) where OSS systems have been used, we found 22 studies originating from research
on different areas. Examples of areas include medical devices, the nuclear industry, and the aerospace industry.
However it was also found that there were some areas where safety is important but where no published research was
found, such as in the areas of rail traffic, and the oil and gas industry.
Concerning the functionality (RQ2) provided by open source software, it was found that more research report on
systems that are subsystems of larger systems than complete systems. That is, only rather few papers were found that
reported on open source software making up a complete system. Instead most published research concerns open
source components that can be included in larger systems.
Concerning the open source communities (RQ3) that are investigated in research it was found that most were active
and mature (> 5 years). However, there were examples of communities that are no longer active.
As a final conclusion it can be seen that open source software today is used as part of safety critical systems. This is
reported in both research that investigate complete systems and open source software components. It is reported from
a number of different domains, and communities are active and mature.
Q1. What are the demographic characteristics of the studies about OSSECOs?
RQ1.1 In which type of sources are articles mostly published? Our results have revealed that research on OSSECOs
is mostly published in conference proceed- ings. The approximate ratio of publication in journals with respect to
conferences is 1 to 2. This indicates that OSSECOs are considered to be a valuable software engineering research
topic.
RQ1.2 How has the number of publications evolved over the years? OSSECOs have been an increasingly addressed
research topic since 2006. Publication peaks occurred in 2011 and 2013. There is evidence that OS- SECOs have
become an established research domain.
RQ1.3 How are papers geographically distributed?
The results in this study suggest that the current out- put of OSSECO papers is strongly supported by Euro- pean and
North American researchers. However, in the
last four years, authors from other continents have been contributing with publications related to the OSSECO topic.
This review shows that the United States and The Netherlands are currently the leading countries in terms of
undertaking OSSECOs.
RQ1.4 Who are the predominant researchers? We observed that six authors have been the predominant researchers
in OSSECOs. These authors and their clusters account for a considerable fraction of all papers covered in this
systematic mapping.
RQ1.5 How are publications distributed between academy and industry? It is no surprise that the publications
written only by academic authors by far our number papers that have at least one industry author.
RQ1.6 What type of papers are published? Although there are more empirical research papers than papers from other
categories (i.e., experience reports and non- empirical papers), the difference is not significant.
8.2. RQ2. What is an OSSECO?
RQ2.1 What definitions are related to the OSSECO definition? Regarding the definitions related to OS- SECOs, we
encountered five major concepts (i.e., ECO, BECO, DBECO, SECO, and OSSECO), and we built a genealogical tree
with their evolution.
RQ2.2 Are there specific definitions of OSSECO? Our results show that there are only three definitions of
OSSECOs. This paper proposes a definition of OS- SECOs, integrating the different definitions related to OSSECOs:
a SECO placed in a heterogeneous environment, whose boundary is a set of niche players and whose keystone
player is an OSS community around a set of projects in an open-common platform.
RQ2.3 What elements belong to an OSSECO? We obtained up to 64 elements belong to OSSECOs in our review.
Among them, project, community, and source code are the most used. Furthermore, we sketched a taxonomy with
We are interested in using a nonintrusive approach to OI in the context of RE by using fully automated approaches
helping companies fuel their RE processes with innovative ideas created outside their company boundaries.
Following this line of research, we have designed an automatic sorting algorithm based on the Kano Model [35]. We
believe that the algorithm can be used in the process of automatically prioritization users' requirement and thus
eventually may become an essential cornerstone in a fully automated, non-intrusive approach to OI used for
requirements extraction and prioritization.
The advantages and potential challenges of OS in computer science education have been explored using SLR
methodology. In particular, the impact of OS in computer science education curriculum has been explored focusing
on the four RASE areas of course design and delivery. The findings represent a starting point in evaluating the use of
OS in computer science from the perspective of education providers. In addition, possible directions for future
research have been suggested such as developing effective strategies for proper alignment of OS projects and courses
as well as efficient evaluation approaches fitting into the specifc context of using OS in computer science education.
These reasons are considered the most contributing to degradation, while the rest of the reasons demonstrated in
Table 6 are less important than the three stated reasons depending on what was declared in the selected primary
studies. However, further studies should be conducted to find out the other causes as a rooted-deep study in digging
up new causes that could have a significant contribution to identifying the architecture erosion, whether over the OSS
or industrial systems.
Succeeding at providing an OSS product is not necessarily easy as there are challenges related to collaborating with a
community, like attracting and relating to contributors, requirements engineering from a community, balancing
focus on community and paying customers, and so on [109, 200]. We hope to see more research on these topics like
e.g. [7, 205] on the following topics:
􏰅 How are OSS providers able to attract and sustain a community?
􏰅 What are the success criteria for incorporating contributions (require-
ments, code, bug reports/fixes, etc.) from a community?
Interests of researchers toward predicting and supporting evolution process of OSS projects are shown in Tables 11
and 12. Research interest toward predicting OSS projects has dominantly focused defect prediction. Change propaga-
tion, ,maintenance effort and SOC (self-organized criticality) have got slightly better attention. The rest of the aspects
are addressed considerably very low. In the context of OSS evolution process support, researchers paid major
attention to evolution models and exogenous factors contributing to support evolution process. Maintenance support
is the second largest aspect, and fault detection and change propagation are the third largest explored aspects.
We intend to extend our SR to include more studies by searching the well-known digital literature databases. We will also extend our data
analysis as we plan to discover the trends and future directions of OSS research.
Predicting the future. Prediction of OSS projects is one area that is least popular among the study facets (Figure 2).
Yet future research should focus on developing reliable prediction models and methods supporting error prediction,
measuring maintenance effort and cost of OSS projects. Because, the commercial organizations, for instance, requires
such prediction models to assess an open source component for adoption [66].
ON COMMUNITY EVOLUTION
Study on the community evolution identifies several key properties (reported in Table VI), which lay the foundation
for further research in this direction. We propose the followings to be investigated.
Community building. Studies reported that the majority of OSS projects failed to attract members to attain the critical
mass. Only few flagship projects are able to attract developers. Factors influencing the motivation to join a
community has been studied (e.g., [62] [50]), and several phenomena are proposed. For instance, rich gets richer
phenomenon. Yet it is not identified what exclusive properties initiate the community building process at the nebula
stage of the project. Following research questions can be considered relevant,
• Why some projects are able to attract contributors during the nebula stage of the project, while most of them can
not?
• What formation of the community refers to a balance one, and how the community structure changes towards a
balance structure during its evolution? Can a visible pattern be identified within the domain of OSS projects for the
above two cases?
While there is quite a lot of research on specific development processes in OSS communities e.g. [167], there is little
research on using these processes and practices inside organizations.
In this systematic mapping, we found the publication on the intrinsic (social) motivation factors of open source
software. We also found the publication on the extrinsic motivation (economic) factors of open source software. We
also found selecting of open source software license on the base of economic, social and commercial (managerial)
perspectives. In future, we will try to find out some other motivation factors both economic and social perspectives
for selecting the open source license. These will be our research question in future what are the motivation factors in
selecting an open source software license with respect to economic and social perspectives in software community?
Are the results of RQ1 are in accordance with perception of local (Pakistani) open source software community?
Few publications in this field speak about risk mitigation. Mostly, mitigation of the various risks encountered in OSS
adoption is only mentioned informally, in form of general hints, such as to train the people, i.e. to develop the
existing stock of human capital [SLR21], to follow general COTS adoption decision processes, to evaluate the
community [SLR24], to evaluate similarity to previous projects [SLR44], to evaluate the OSS project’s roadmap and
possible future directions [SLR46], or to make managers aware of risks and opportunities [SLR37]. Empirical
experiments were used to identify risks and to identify effective mitigation activities. However, none of the works
showed evidences for causal relationships between these risks, concrete measures and the effectiveness of the
mitigation activities. Only for more concrete risks, as e.g. for lowering the risk of introducing errors when upgrading
to new versions, concrete mitigation activities were proposed, such as automatically checking API compatibility
[SLR4] or exploring code executability with test cases [SLR14] to ensure correct functionality.
However, as described in Section 5.4.4, there is a lack of research in this area. Thus, we need more study to shed ligh
on how to manage an effective inner source community in an organisation.
In contrast, developers are commonly driven by extrinsic motives to join a FLOSS project. Also, structural
characteristics of FLOSS developers’ contact network influence their joining behavior. Moreover, the application
domain and the development phase are relevant characteristics for developer attraction. However, it is unclear how
relational factors influence developers’ attraction. While Stewart and Gosain [55] view trust as essential, there is no
attraction centric research on this aspect. Hence, future evaluations should focus on this aspect. Moreover, as shown
in table 2 there is very few attraction specific research which combines individual and team factors. Considering that
ideological and status motives are dependent on the feedback of others, future studies should examine closely this
interaction for extending our understanding of developer attraction. Similarly, there is very little literature which
combines team and project characteristics. Considering the key role of structural properties [26, 40], future studies are
necessary to understand how FLOSS projects can utilize the social contacts of their members to reach out or new
members. Finally, drawing on research by Shah [48], further studies are essential to understand fully how FLOSS
initiatives can stimulate individuals’ extrinsic and intrinsic motives to join the project.
Future research should also focus on predicting change propagation, size, refactoring, maintenance effort, contribu-
tion evolution, SOC and clone evolution besides defect prediction. The architectural and requirements change
evolution is also undermined area that needs attention of researchers. The tools and approaches proposed for
evolution also admit the necessity of external validation. Future research should focus on the above mentioned issues
and try to make them more generalized regardless of the domain of OSS projects.
Notice that there are a huge number of publications, which report and interpret results from qualitative and
quantitative studies to identify possible risks. It stands out that only [SLR40] calculates threshold values for defining
bug risks, and evaluates their performance, while no paper identifies risks based on quantitative data of project failure
or created losses and revenues, or correlates project failures and losses with quantitative data such as the number of
bug reports and bug fixes.
Few papers consider quantitative measures on community qualities (number of contributors, activity, presence of
heroes [SLR39] etc.), while no work empirically evaluates the existence of causal relationships between the metrics
applied, the risks identified and the actual faults happening. Moreover, an empirical evaluation of the effectiveness of
mitigation activities by their influence on the retrieved metrics is also missing in the works, which propose specific
mitigation activities. Also for very complete works, such as the ones by the group [SLR12][SLR19][SLR20][SLR47]
whose surveys retrieve data for risks and mitigation activities, do not show any link between these two: typical
mitigation activities adopted in software companies are very general (see Section 1.2.5) and would reduce various
risks.
The most commonly used selection method is the model approach and the least considered are the tool- based and
data mining approaches. Another interesting result is that nearly half (47%) of the selected papers do not mention an
application domain for the models in their research. More attention should be paid to building models that incorporate
only essential quality characteristics. Also, framework, tool-based and data mining selection methods should be given
more attention in future model proposals.
factors for FLOSS developers’ attraction and retention. Moreover, there is even less dedicated research on these two
management aspects. Often this is because of the use of ambiguous measures such as “team size”. With such
measure, it is not possible to tease out distinct lessons for the attraction and retention of FLOSS developers. Thus,
future research should use specific measures for this particular aspect, for example the number of new, respectively
retained developers. Finally, as visualized in table 1 – 3, our literature review shows that there is few dedicated
research on these aspects (especially on developer attraction and retention) which combines more than one research
perspective. Considering the interrelations between the three research perspectives, however, an isolated research
perspective seems too narrow. For example, FLOSS developers’ extrinsic motives are stimulated by receiving
appreciation and by particular project characteristics (e.g. corporate sponsors) [32]. With respect to these
interrelations, future studies on FLOSS management should adopt more than one research perspective in order to
fully understand the effects of the examined aspects.
We plan to perform an exploratory mixed-methods research using OSP in an undergraduate course in SE, in order to
obtain new insights with an experience with this approach. We will experiment with a combination of inside control
and a predefined project in a software evolution course, using a combination of different learning approaches. We
also plan to investigate better methods to assess students’ learning in this context.
Our aim is to define a process that improves the selection, management and maintenance of OSS components. We
will review existing COTS quality assessment processes, including the ones enacted in the project partner companies
in practice. Giving a particular focus on the differences between OSS and closed source components identified in the
SLR, we try to understand the causal relationships between risks and failures, risk metrics, and risk mitigation. To
propose such measures we investigate into two directions: (i) evaluating the OSS project community ecosystem and
the adopting company, by organisational modelling and analysis, and (ii) evaluating the data available in the OSS
projects repositories, such as code repositories, bug trackers, and mailing lists, applying measurements based on a
statistical analysis. Techniques and tools for this analysis, the prioritisation and multi-criteria decision making will be
presented in the next section.
OSS has long passed the market introduction stage but has not yet reached the maturity stage”. This has been
particularly relevant for public administration where FLOSS’ technological immaturity, lack of interoperability with
existing software, and decision making influenced by politicians poses a significant barrier for its wider adoption
There has been a steady but yet small body of research that addresses the pedagogical use of FLOSS projects in SEE;
it includes 105 primary studies, developed by a small number of groups and researchers, and mostly published in
conferences on computer science education. From 105 studies in 20 years, only one was classified as
Experiment/Quasi-experiment in the Research Type facet. Despite the increase in the number of experience reports,
evidence shows that the research area is not mature yet.
The updated SMS is a valuable asset both to researchers inter ested in the identified trends and gaps, and to
instructors interested in trying out their own experiences in their classes.
The availability of source code for OSS components provides opportunities for scrutiny by third party certification
bodies. However, the complexity, size and evolving nature of many OSS projects severely limit the practicality of
such efforts, unless the software is developed “with certification in mind”. Cotroneo’s pre- certification kit [7],
Comar et al.’s Open-DO continuous certification process [6], Fusani and Marchetti’s virtual certification repository
[10], Kakarontzas et al.’s OPEN-SME reuse process [11] are examples for approaches to develop “for certification”.
Some of these proposals can be considered complimentary, others are alternatives. Little empirical evidence is
available to-date about their effectiveness in practice. As an increasing number of OSS systems are subject to
certification and may consider these proposals, the community will need more empirical evidence on their
effectiveness.
Software engineers in last decade have been interested in agile methodology and open source software development.
Both of them present some new features and they seem beneficial for better and faster software development. By
doing an SLR we were looking for relationship between ASD and OSSD. Fortunately our study shows that both ASD
and OSSD can help each other and collaborate in doing software projects by sharing their practices. There are enough
evidences that agile and open source practices can support each other, mainly because of some of their common
concepts and principles. Also, however, there are a few successful experiences on integration of ASD and OSSD, but
most of the studies are optimistic in possibility of their integration, but there is no empirical successful case study for
supporting this idea in software producing industry.
mmunity dynamics studies focus on the internal and external motivation of developers and factors that influence the
retention of developers in projects. However, there are some areas, which need research such as retention of veteran
developers, the impact of contribution barriers on abandonment, use of gamification in student engagement and its
effect on retention, the effect of project characteristics on retention, the impact of the project governance on
motivation, development of strategies to maintain loyalty, the impact of leadership styles on developer turnover, and
impact of corporate sponsorship on FLOSS communities. Moreover, there is a need to design tools to assess the
health of OSS projects and develop strategies to retain new developers.
Researchers studied the effectiveness and accuracy of several metric suites using data from one or more OSS
projects. Despite of their esteemed contribution in predicting OSS projects, they suffer from lack of generalizability
due to diverse nature of OSS projects and the project specific nature of the metric suites. Also it is quite difficult to
ensure the availability and quality of metric data, which makes the results incomparable [10]. Thus, a future research
agenda would be to perform an indeepth analysis on (a) cross project validity of the studied metric suits, and (b) to
propose methods to ensure the quality of metric data.
In terms of using the architectural rules violations solutions, we observed that many violations were not restricted by
the architectural principles of the system, which may not have defined the necessary rules to reduce the severity of
major violations. This means that violations still appear over again and over again despite the good violation
solutions that were introduced and the approaches that have a significant role in capturing violations with thorough
accuracy. Therefore, it is important to highlight the identification of the necessary rules and identification of critical
cores through a broad study on architectural conformance using multiple frameworks.
After examining more than 69 different research papers and taking the survey findings into account, we constructed a
roadmap that can be used by any educational institutedespecially high schoolsdto strategically integrate OSS into
the educational system. The roadmap is designed to target three main aspects of implementation: using OSS in the
curriculum itself throughout the semester, using OSS in final projects at the end of the semester, and using OSS in
extra-curricular activities. The future direction of this research is to implement this roadmap and measure its
effectiveness. The roadmap can be applied via three phases: changes to curriculum, implementation in final course
project and establishment of extra-curricular activities.
Based on the findings of this research, we have come to the conclusion that the existing software security practices
have limitations in supporting secure open source development. Secure architecture, code review and security testing
do help secure OSS products. However, as there is less research on socio-technical security aspects and no discussion
of security knowledge management in the context of OSS development, these practicies, and software security
knowledge cannot be effectively spread within the open source community. Since OSS parcticipants are not experts
on security in general and the domain knowledge of software security is vast and extensive, it is suggested that future
research should explore soio-technical approaches in helping OSS developers learn the necessary security knowledge
to fulfill the need of their work, further, to reinforce their behaviors towards OSS security.
The results from classifying topics showed that studies aim to predict effort of maintenance activity mainly
concentrated on bug fixing time prediction, while less efforts contributed to other types of activities
Considering that there are sentiments associated with positive and negative polarity that were marked as not specified
in the selected studies regarding software practices, there is still room for further investigation on the associated
sentiments to the specific impacts. Moreover, there is a tendency of a considerable set of open source software
projects to have regular release cycles and to adopted the so called frequent releases. We plan to investigate
sentiments in this context and to which extent they influence software productivity. We also want to investigate how
programmers sentiments vary between releases.
Given the large variety of OSS systems, with different sizes, domains and complexity, we believe they are an
important source of examples to teach software design, architecture and quality (Brown & Wilson, 2012). Nonethe-
less, we found very few studies to describe these issues in, say, a case-based learning approach. Only three reported
this type of experience: two of them related to software design and architecture, and one of them, to software
evolution.
No selected study focuses on learning the areas of requirements or configuration management, despite the large use
of configuration management and issue tracking tools in OSP. We believe that these specific areas may benefit from
an active engagement with OSP and their associated tools.
Meiszner et al. (2009) point out some learning features of the OSP experience: self-learning,
project-/problem-/inquiry-based learning, collaborative learning and reflective practice. However, very few studies
cite these learning approaches, and none of them provides details on how to design and implement pedagogical
practices that result in effective learning of SE skills.
There are themes still little cited that might have some future potential: OSS process (meta-) modelling, OSS security
Agile and OSS development methods, and teaching OSS in universities.
From a practical standpoint, this research provides IS managers, as well as open source BI providers and consultants
with an initial structured lens to better understand the most important
barriersthatpreventorganizationsfromadoptingopensourceBItools.Thesebarriersrequirefurther
considerationbyallstakeholdersinterestedintheadoptionordeploymentofopensourceBI.
Fromamethodologicalstandpoint,followingPoba-Nzaouetal.(2016),thisresearchprovides
onemaincontributionthatisarigorousanalysisofQualitativeSurveydata,basedontwoprinciples
ofinterpretiveresearch(Klein&Myers,1999):thefundamentalprincipleofHermeneuticCircle,
andtheprincipleofAbstractionandGeneralization.
Toconclude,theauthorsacknowledgesomeareasoflimitations,andcallforfurtherstudiesof
opensourcebusinessintelligencetobeconducted.First,thoughitisadequateforaqualitativesurvey
andmethodologicallysufficient,thesizeofthepanelwassmall.Second,itcouldbeinterestingto
wheretheyareverypopular(suchaswebserver,operatingsystems,etc.).Third,sincethisexploratory
studyfocusesonBIexpertsinonlyonecountry,Canada,theauthorsalsorecommendfuturestudies
investigatingtheviewsofBIexpertsinothercountriesandinvolvingotherBIstakeholders(e.g.users,
executives,etc.),assuchstudiescanincreasethevalidityofthefindingsfromthisstudy.Fourth,asthis
initialstudyfocusessolelyonbarriers,theauthorsalsorecommendthatfuturestudies,includingthe
identificationofstrategiesthatmaybeinitiatedbyrelevantstakeholdersindealingwiththeidentified
barriers.Lastly,futureresearchesmaybenefitfromadoptingotherresearchmethodsaswell,such
ascasestudy,surveysorexperiments,astheymayprovidericherinsightsthanthequalitativesurvey adoptedinthisstudy.
It is possible to see the lack of studies conducting qualitative analysis as supporting the existence of problems that
hinder newcomers’ contributions. Quantitatively analyzing historical data can bring highlights of the barriers faced
by newcomers, but conducting qualitative analysis can enrich the evidence, reveal new facts, and help in finding the
issues faced by newcomers. There is still room available for studies based on observations, interviews and
experiments that can help reveal the barriers faced in practice.
The findings showed that 17 of the most commonly occurred causes contribute to the architectural decay of the OSS
community. Essentially, architectural degradation has numerous causes, which have been discussed by several
researchers in their studies [11], [12], [23], [30]. However, these reasons have been discussed from limited aspects
such as aging because of changes over time [11], identifying the reasons through only one case study [12] or based
on their investigation of industrial case studies [27]. Consequently, these causes need to be further identified in terms
of the frequency of their occurrence, especially in the scope of the OSS projects. Moreover, identifying the most
important reasons will indeed contribute to the erosion according to the chosen primary studies, which contain severa
case studies in different domains for the OSS community. These causes differ in their impacts with regard to the
actual contribution to the architectural decay prominence.
To summarize, the review results indicate that required OSS functionality is more frequently integrated in the safety
critical systems rather than the entire OSS solution is used. The complete OSS solutions found in the review are
rather small and have low activity. On the contrary, OSS solutions integrated in safety critical systems have over five
years of history and high or medium activity. Hence, we can conclude that long history of OSS projects facilitates
their use in as integral parts of safety critical systems. However, the relationship between the activity of the OSS
project and their adaption in safety-critical systems is less clear from the studied papers and therefore requires further
investigation.
Our research agenda on this topic involves investigating software development process tailoring (Pedreira et al.,
2007) as a way to achieve a balance between characteristics of software development models. Process tailoring is
the act of particularizing a general process description to derive a new process applicable to a specific situation
(Ginsberg and Quinn, 1995).
We claim that it is necessary to tailor these processes to suit specific project and organization contexts. This tailoring
should also consider the key distinctive features of plan-driven, agile and FOSS models: collaboration and discipline
(Magdaleno, 2010a).
Collaboration can be defined as a group of two or more people working to achieve a common goal (Vreede and
Briggs, 2005). Collaboration is an important factor for software organizations to achieve their productivity, quality
and knowledge-sharing goals. In particular, software development is a complex process that involves collaboration
among several people over time to achieve a common goal (Cugola and Ghezzi, 1998). Therefore, software devel-
opment is a typical example of collaborative work (DeMarco and Lister, 1999). Discipline, meanwhile, relates to the
planning level adopted in software process definition and the rigidity of control employed in process execution
(Boehm and Turner, 2003b).
Both are complementary and essential in any project, but in differing proportions, depending on project
characteristics. For a balanced mix between collaboration and discipline, it is necessary to understand how these
aspects vary and distinguish the software development models (Magdaleno et al., 2010).In order to facilitate process
tailoring, it is possible to support the project manager by automating some of the steps, possibly reducing the effort
required to conduct this activity and improving the quality and appropriateness of the resulting process (Park et al.,
2006). Thus, we intend to develop an infrastructure using an optimization- based approach (Magdaleno,
2010b).Finally, it is also important to investigate whether the results of this quasi-systematic review are consistent
with what is observed in practice, i.e., in organizational routine. To accomplish this, we intend to plan and conduct
an experimental study. This experimental study will involve a survey of industry and academic experts to validate the
results and discuss the conclusions. A similar strategy was applied by (Bekkers et al., 2008) to determine the factors
that most influence the selection of software development model.
Using OSS CASE tools: The research on OSS CASE tools has been very limited. However, Wicks and Dewar
propose a new agenda for research on tool integration, requesting a more business-oriented approach to future
research [207]. The use and development of OSS CASE tools and research on such tools could easily fit into this new
agenda. Robbins provides an extensive overview of OSS tools for development, and claims that CASE research has a
lot to learn from OSS [157]. OSS should furthermore be particularly interesting to academia since they have access to
professional state-of-the-art tools and the tools’ source code. This enables them to extend existing tools and test new
ideas in collaboration with each other.
Increased participation in OSS projects, increased collaboration between organizations, and increased use of OSS
practices will most likely require improved collaborative development tools. Hence, there is a potential for research
on:
􏰅 What kinds of tools are needed for collaborative software development across organizational and community
borders?
􏰅 Howdoorganizationscollaborateusingsuchsoftwaredevelopmenttools?
Four of five studies with the highest level of evidence (2b) examined clinical changes in orthodontics (influence of
head orientation on directional changes in 3D space; orbital volume and aperture width changes after rapid maxillary
(expansion) (18;21) and endodontics (canal transportation) (22;23), but none of the identified studies included a
comparison between OSS and any other software.The only level-2b study that included a comparative analysis of a
commercial and open-source software was assessing software precision in airway volume measurement (27). The
study was based on 33 participants and lead to the conclusion that the use of Osirix and ITK-Snap OSS presented
clinical value.
Comprehension of these results suggest that the laws and theory appear to be breaking down through non-
conforming data and findings (Table IV). Thus Lehman’s laws of software evolution which is primarily based on the
study of the large close source systems, is not sufficient to justify or account for the evolutionary pattern and behavio
of the open source software. As none-the-less these laws did not consider the community dimension of the OSS
projects which is an integral part of sustainable evolution of the open source softwareTo deal with this problem, a
viable route would be to examine the underlying ontologies for software evolution [5] considering the OSS specific
characteristics, and then re-assess the laws of software evolution to fit in OSS domain.
As a follow-up to this research, we expect to deepen the study on the aspects that may influence the participation of
women in open source software projects and software development projects, as well as to propose ways of addressing
the identified problems regarding issues of gender inequality in open source communities and software factories.
It is more and more difficult to talk about “OSS practices” as the practices used in OSS communities are
heterogeneous, and as organizations are increasingly getting involved in, and influenced by, the development of OSS.
Nevertheless, there are opportunities for further research on the use of development practices for distributed software
development. OSS development in large communities and in and between organizations, are areas where
researchers could have an impact on practice. OSS research has so far focused mainly on processes in communities o
volunteers [167], but some of this research could turn its focus towards the application of their findings within
organizations and questions like:
􏰅 How may organizations successfully collaborate through community- or consortium-based software development?
It reflects that most authors performed experiments on different open source projects and comparison of results of
different studies become arduous. There is a need of common corpus regarding evolution prediction of open source
projects that can be used by the researchers for comparing results.
Year SMS/SLR Extracted From Extracted by Classification Verfication
Future
2020 SLR Discussion Ms. Saima Imtiaz
Direction
Yes
Future
Future Research Ms. Saima Imtiaz
Direction
Yes
Discussion and Future Future

Ms. Saima Imtiaz
Work Direction
Yes
Yes
Author Future
2017 SLR Conclusion Ms. Saima Imtiaz
Intent
Yes
Future
Discussion Ms. Saima Imtiaz
Direction
Yes
Future
Direction
Yes
Conclusion and Future Author Future
Ms. Saima Imtiaz
Work Intent Yes
Future
Direction
Yes
Future
Direction
Yes
Author Future
Conclusion Ms. Saima Imtiaz
Intent
Yes
Future
2018 SMS Conclusion Ms. Saima Imtiaz
Direction
Yes
Author Future
Intent
Yes
Future
Direction
Yes
Future
Direction
Yes
Yes
Future
Direction
Yes
Future
Direction
Yes
Future
Direction
Yes

Yes
Future
Direction
Yes
SLR Overview of studies Ms. Saima Imtiaz Gap
Yes
Yes
Author Future
Intent
Yes
Summary and
Conclusion
Yes
Yes
Conclusion and Future Future

Ms. Saima Imtiaz
Work Direction

Ms. Saima Imtiaz
Work Intent
Yes
Implications for OSS Future

2009 SLR Ms. Saima Imtiaz
Research and Practice Direction
Yes
Summary of Finding Yes

Future
Direction
Yes
Future
Direction
Yes

Ms. Saima Imtiaz
Work Intent
Yes
Future
Direction
No
Future
2016 SMS Discussion Ms. Saima Imtiaz
Direction
Yes
Results and Discussion Ms. Saima Imtiaz Gap
Yes
No
Open Questions and Future

Ms. Saima Imtiaz
Research Agenda Direction
Yes
Future
Direction
Yes
Future
Direction
Yes
Future
Direction
Yes
Future
Direction
No
Discussion and Future

Conclusion Direction
No
Summary and Future

Ms. Saima Imtiaz
No
Summary and Future
Ms. Saima Imtiaz
Discussion Direction
Yes
2014 SMS Results and Discussion Ms. Saima Imtiaz Gap
Yes
Future
Direction
Yes

Yes
Future
Direction
Yes
Ms. Saima Imtiaz
Yes
Yes
Author Future
Intent
Yes
Author Future
Intent
Yes
Ms. Saima Imtiaz
Work Direction
Yes
Future
2020 SLR Future Research Ms. Saima Imtiaz
Direction
No
Future
Direction
Yes
Results from Mapping
2013 SMS Study Ms. Saima Imtiaz Gap
Yes
Yes

Ms. Saima Imtiaz
Work Intent
Yes
Main Finding and

Discussion
Yes
Future
Direction
Yes
Future
Direction
Yes
Future
Direction
Yes
Summary and Future

Yes
Main Finding and Future

Ms. Saima Imtiaz
Yes
Future
Direction
No
Future
Direction
Yes
2014 SLR Paper Analysis Ms. Saima Imtiaz Gap
Yes
Future
Direction
No
Yes
Future
Direction
Yes
Future
Direction
Future
Direction
Yes
Future
Future Work Ms. Saima Imtiaz
Direction
Yes
Future
Direction
Yes
Future
Direction
Yes
Future
Direction
Yes
Revisiting Research
2014 SLR Questions Ms. Saima Imtiaz Gap
Yes
Author Future
Intent
Yes
Future
Direction
Yes

Ms. Saima Imtiaz
Work Direction
Yes
Summary and Future

Ms. Saima Imtiaz
Yes
Future
Direction
Yes

Ms. Saima Imtiaz
Work Direction
Yes

Yes
Future
Direction
Yes
Work Intent
Yes
Future
Direction
Yes
Author Future
Intent Yes
Future
2015 SMS Findings Ms. Saima Imtiaz
Direction
Yes
Author Future
Intent
Future
Direction
Future
Direction
Future
Direction
Yes
Future
Direction
Yes
Future
Direction
Yes
Directions for Future Future
Ms. Saima Imtiaz
Research Direction
Yes
Future
Direction
Yes
Future
Direction
Yes
Future
2017 SMS Future Research Ms. Saima Imtiaz
Direction
Yes
Research Direction
Yes
Future
Direction
Yes
Future
Direction
Yes
Yes
Future
Direction
Yes
Summary and Future

Ms. Saima Imtiaz
Yes
Future
Ms. Saima Imtiaz
Direction
Implications of Findings Yes

Future
Direction
Yes
Future
Direction
Yes
Future
Direction
Yes
Future
Direction
Yes
Yes
Ms. Saima Imtiaz
Work Direction
Yes
Author Future
Intent
Yes
Yes
Yes
2020 SLR Ms. Saima Imtiaz Gap
Research Scope in Floss

Adoption Yes
Conclusion and Future

Work

Work Direction
No

Ms. Saima Imtiaz
Work Direction
Yes
Research Direction
Yes

Yes
Conclusion and Future
2019 SMS Ms. Saima Imtiaz
Research Direction Direction
Yes
Work Direction
Yes
Future
Direction
Yes
Future
Results and Discussion Ms. Saima Imtiaz
Direction
Yes
Future
Direction
Yes
Future
Direction
Yes
Yes
Future
Saima imtiaz Ms. Saima Imtiaz
Direction
Yes
Future
Results Ms. Saima Imtiaz
Direction
No
Future
Direction
Yes
Future
Direction
Yes
Future
Direction
Yes
Directions for Future Author Future

Ms. Saima Imtiaz
Research Intent
Yes
Ms. Saima Imtiaz
Research Direction
Yes
Future
Direction
Yes
Author Future
Intent Yes
Implications for OSS Future

Ms. Saima Imtiaz
Research and Practice Direction
Yes
Future
Direction
Yes

Work Direction
Year Future
missing? Direction
Again, the
year of
publication
missing - Open Questions and Future
Ms. Saima Imtiaz
for many Research Agenda Direction
other
entries as
well?
Yes
Future
Direction
Yes
Future
Direction
Yes
Future
Direction
Yes
Future
Direction
Yes
Future
Direction
Yes
Results from Mapping
Study Ms. Saima Imtiaz Gap
Yes

Ms. Saima Imtiaz
Work Direction
Yes
Future
Direction
Yes
Future
Direction
Yes
Yes
Yes
Yes
Author Future
Intent
Yes
Future
Direction
Yes
Future
Direction
Yes
Future
Direction
Yes
Results and Discussion Ms. Saima Imtiaz Gap
No
Author Future
Intent Yes
Adoption of BI tools by
2019 SLR organizations Ms. Saima Imtiaz Gap
Future
Direction
Future
Direction
Author Future
Intent
Future
Direction
Future
Direction
Summary and Future

Ms. Saima Imtiaz

Ms. Saima Imtiaz
Work Direction
Future
Direction
Author Future
Intent
Author Future
Intent
2018 SMS Conclusion Ms. Saima Imtiaz Gap
Future
Direction

Future
Saima imtiaz Ms. Saima Imtiaz
Direction

Ms. Saima Imtiaz
Future
Direction
Author Future
Intent
Future
Direction

Author Future
Intent
Future
Direction
Future
Direction
Research Contributions Future
Ms. Saima Imtiaz
and Limitations Direction
2014 SLR Overview of studies Ms. Saima Imtiaz Gap
Future
Direction
Future
Results and Discussion Ms. Saima Imtiaz
Direction
Author Future
Intent
Future
Direction
Future
2013 SLR Future Research Ms. Saima Imtiaz
Direction
Author Future
Intent
Future
Direction
2017 SMS Results and Discussion Ms. Saima Imtiaz Gap

Comments
in case of
disagreeme
nts
I think first
sentence is
future work
Split in two
first line is
gap where
as second is
future
direction
It is gap
again I think
it should be
split in gap
and future
direction
The last line

is gap. So I
think first is
future
direction
but need to
split an
identify gap
too
I think again
gap in initial
line is
different
than future
work so it
should be
split
again split
I again think
as you have
highlighted
in different
color it is
both author
intent an
Future
direction
split in gap
and future
direction
split in gap
and futrue
direction
Salma will verify until this row from top
Not sure -
these are
common
general gaps
that you will
notice in
almost all
SLRs - need
more
studies on
this/that.
Yes - but a
general RM
related
direction is
proposed.
Is this one
direction
only?
Repetition
with the
previous
row?
Repetition
with an
earlier
entry?
Is it only one
direction?
Yes - but a
very general
direction -
what can we
learn or
extract from
it?
1) Quite a
general one
again, 2) Is it
only a single
future
direction?
Again - is it
only a single
gap?
This is a
huge text
segment -
not
approporiat
e at all -
there are
only a few
sentences
which
appear
relevant for
a gap.
Is it only one
direction?
A general
direction
about more
studies
The gap is
not
mentioned
in this text.
They said
"rest of the
aspects are
addressed
considerably
very low" -
but those
rest of the
aspects are
not listed
here. What
would you
say about
the exact
gap in this
case then?
Future Direction S
Researcher: Sa
Serial Number Categorization Sub-Codes
To identify approaches, methods, techniques,

1 Author Future Intent and tools for participatory interaction design in
distributed software.
Model a participatory interatction design
process model for distributed software
2 Author Future Intent
development environments and develop a
process model.
Extend the Open Source Maturity Model with
the proposed interaction design model.
No method, tool, technique for interaction

4 Gap
design in FLOSS
Support of HCI community is required for
FLOSS projects and communities for
5 Future Direction
interaction design
To expand searching and identify new trends

and future directions of OSS research.
Security areas in governance and deployment

7 Gap are not much researched
The socio-technical perspective in a security

8 Gap management framework has not not gained
much attention
Lack of research in knowledge management

9 Gap
aspect of open source security
discuss the need to explore socio - technical

10 Future Direction approaches in helping OSS developers learn
the necessary security knowledge.
Less explored areas are OSS for safety critical
automation system, maritime systems, rail
11 Gap
industry, process industry, oil and gass, off
highway equipement and mining industry
Areas of rail traffice, oil and gas have no
12 Gap
research w.r.t safety although it is important
Need to research adaptation of OSS in safety-

13 Future Direction critical and its relation with OSS project
activity.
Need to research adaptation of OSS in safety-
14 Future Direction critical and its relation with OSS project
activity.
Research on whole systems is lacking, rather

15 Gap
components are investigaged more
Change profile analysis is done on limited

16 Gap
level as OSS systems evolve
Need of utilization of data analytics in OSS

17 Future Direction domain (understand and manage software
evolution of OSS)
Limited contribution to OSS communities by

18 Gap
most of the organizations
Lack of explicit research questions in

19 Gap
experience reports
Unclear cost analysis for introducting OSS

20 Gap into organizations and product operation over
long period of time
No empirical research on the use of OSS

21 Gap
CASE tools in organizations
Details of research methods and findings

22 Gap lacking in studies thus lack precise focus and
clear contribution
No research on deploying OSS in private
sector.
23 Gap
Less focus on describing individual
organization
Less empirical work on OSS in organizations
and that also not of good quality and lacks
24 Gap focus
Less work on OSS processes and practices

25 Gap
inside organizations
Need to research maintenance challenges

26 Future Direction while integrating OSS components into a
project
Need to help organizations participate in
27 Future Direction communities and other organizations while
working on OSS projects
Need to conduct more longitutional and
28 Future Direction indepth studies to understand context of
organizations necessary for OSS usage
29 Future Direction Align research of OSS with other related fields
Need to understand integration of OSS

30 Future Direction
components
Need to research OSS collaboration in inter-

31 Future Direction
organization and organization community
Need to see see actual problems faced by

32 Future Direction
practitioners in using OSS
33 Future Direction Research how organizations approach OSS
Increase rigor of empirical studies in the area

34 Future Direction
of OSS adoption
Need to conduct more longitutional and

35 Future Direction indepth studies to understand context of
organizations necessary for OSS usage
Align research of OSS adoption with other

36 Future Direction
related fields
Need to see OSS research in other areas then
37 Future Direction
USA and Europe
Need to see how organizations actually

38 Future Direction
approach OSS
Need to see issues that could benefit

39 Future Direction
practitioners while adopting OSS
Need to research on community collaboration

40 Future Direction
while adopting OSS
The use and development of OSS CASE tools

41 Future Direction
needs to be researched
Need to research community and organization

42 Future Direction
collaboration for OSS adoption
Need to research community practices to be

43 Future Direction
adopting by organizations
Need to research more the metric used for

44 Future Direction
prediction
Need to research prediction of OSS with

45 Future Direction
respect to community evolution
Replication of OSS projects for cross project

46 Future Direction
validity of the prediction model
Need to research generalizability of the

47 Future Direction prediction models across different OSS
projects
Need to resesarch which type of metric are
48 Future Direction good for OSS prediction accuracyor
generalizability or reproducibility
49 Future Direction Need to develop a tool for OSS prediction

Need to research validity of metric suits on
50 Future Direction
cross projects
Need to propose methods to ensure the quality

51 Future Direction
of metric data.
Conduct qualitative studies to confirm

52 Author Future Intent problems actually faced by OSS developers
and newcomers.
Refine the classification model and propose
53 Author Future Intent awareness mechanisms and tools for better
support for newcomers.
Less qualitative analysis of barriers faced by

54 Gap
newcommers.
Need to check accuracy of prediction model

55 Future Direction
defined
Need to incorporate social aspects to OSS

56 Future Direction
prediction models
Need to see how communities precieve

57 Future Direction
barriers
Need to research the effect of barriers faced

58 Future Direction
by newcommers on quality of contribution
Need to understand barriers faced by different
type of newcommers (newcommers in
59 Future Direction
development, newcommers in OSS,
newcommers to project)
Need to conduct observational and
60 Future Direction ethnographic studies to understand barriers
faced by newcommers
Need to research small projects with respect to

61 Future Direction
barriers faced by newcommers
Research needed to measure influence of

62 Future Direction
Research needed to define strategies that lower
63 Future Direction
Need to define a metric that grades support

64 Future Direction
offered by barriers faced by newcommers
Qualitative research required (to understand

65 Future Direction complex environment of understanding
Need indepth studies on technical issues faced

66 Future Direction
by newcommers in communites
Use of rigorous prediction methods for OSS

67 Future Direction
prediction should be explored in future
Validate Lehman Law for software evolution

68 Future Direction for OSS based on the change related
information. Using SCM system
Big data analytics to be used manage and
69 Future Direction understand the complex web of software
evolution.
Research area of OSSECO to explore are
70 Future Direction modelling, analysis, measuring, quality
evaluation, etc.
To use sophisticated techniques of string

similarity and data cleansing for finer results.
Need further research in OSS major research

72 Future Direction
streams.
Need research in areas of OSS process (meta-)
modelling, OSS security, Agile and OSS
73 Future Direction
development methods, and teaching OSS in
universities.
Emergent issues in OSS evolution are
prediction of properties, aggregate metrics,
74 Future Direction
change evolution analysis, OSS integrability
and licensing
Researc community doesn’t perform empirical
75 Gap
studies or replication of studies
Need to see how to use open source practices
within a closed company
76 Future Direction
Further research is needed to see how

companies can transform their proprietary
77 Future Direction
software to open source and build a
community on it.
More case studies on implementation of
78 Future Direction specific methodologies for dealing with
different aspects of open source in industry.
Need more case studies for dealing with

79 Future Direction
different aspects of open source in industry.
Empiricial foundation is lacking in studies on
OSBI tools.
80 Gap
No integrated framework is there for OSBI
adoption by organizations
Research needed to understand more about the
cost and advantages of different approaches
81 Future Direction
and criteria of choice for using OSS by
commercial organizations.
More research required in OSBI in
82 Future Direction
organizations.
compare the barriers with those
preventing organizations from adopting
83 Future Direction open source software in areas
where they are very popular (suchaswebserver
,operatingsystems,etc.).
BI experts and stakeholders from other
84 Future Direction countries than canada should also be involved
in the survey
We also need to identify strategies that need to
be initiatied by relevant stakeholders in
85 Future Direction dealing with identified barriers that prevent
organizationsfrom adopting open
source BI tools..
Introduce other resarch methods like case
86 Future Direction study, experiments in the research for OSBI in
organizations
Need further research on the area of effects of
members’ cultural background or their
87 Future Direction
geographic proximity on their development
intensity
Understand the relationships between
88 Future Direction individual or team factors and project
characteristics
Need for research on motivational factors of
OSS community members, project
89 Future Direction
characteristics, along with the impact of these
attributes on retention of community members.
Need to see how project governance can
90 Future Direction It also highlights theparticipation
need to assess the
stimulate individuals’ motives.
relationship between structural network
properties, and project characteristics of OSS
research.
91 Future Direction
future studies should analyze the ways in

which FLOSS projects can actively utilize the
92 Future Direction
interaction network of their members to foster
their retention.
further research is necessary to understand the
93 Future Direction ways in which project characteristics influence
individuals’ retention.
Unclear how relational factors influence

94 Future Direction
developers’ attraction.
feedback of other need to be seen for

95 Future Direction extending our understanding of developer
attraction.
Similarly, there is very little literature which

96 Future Direction
combines team and project characteristics.
future studies are necessary to understand how
FLOSS projects can utilize the social contacts
97 Future Direction
of their members to reach out or new
members.
further studies are essential to understand
fully how FLOSS initiatives can stimulate
98 Future Direction
individuals’ extrinsic and intrinsic motives to
join the project.
Need to look at from multiple perspective to
99 Future Direction understand the effects oof developers
attraction and retention
To include OSS article hubs like pascal and

MIT repository for this analysis.
To see how well the level of effectiveness can
be achieved by implementing fuzzy logic
101 Author Future Intent conditional probabilities characteristics into
the BN network for OSS Web application
projects.
how OSS can affect the accuracy of effort
102 Gap
estimation.
No focus on early effort estimation for OSS

103 Gap web projects
Need for early effort estimation model for

104 Future Direction
OSS web project
Need for early effort estimation model for

OSS web project
Measuring other aspect of the effort

106 Future Direction measurement attribute toward OSS effort
estimation need to be seen
To apply the estimation models to one typical
107 Author Future Intent OSS database and identify benchmarks for
estimation acuraccy.
Need to see time and effort needed to fix new
108 Future Direction introduced bug and its effect on effort already
estimted
Studies that can quantitatively infer OSS
maintenance effort from size-related metrics
are needed.
New evaluation methods are needed to

110 Future Direction validate the correctness of the estimation
methods.
Explore the capability model for OSS
developers (considering the developer-related
111 Future Direction metrics to be one of the sources of predictors
to improve prediction results of OSS
maintenance effort estiamtion)
Contribution measurement and performance
measurement need focus in future
Need to genralize OS use in computer science

education (using practical examples)
develop effective strategies for proper

alignment of OS projects and courses
Efficient evaluation approaches fitting into the
115 Future Direction specifc context of using OS in computer
science education are needed.
Mixed methodology research needed in the

field of OSS success
117 Future Direction User related success factors need to be studies
Contextual and Longitutional studies needed

in the area of OSS success
119 Future Direction Develop a general model for OSS success
Research success in OSS (via parameters such

120 Future Direction as social, cultural and economical state of
development community)
Mixed methodology research needed in the

field of OSS success
The authors intend to develop infrastructure

using an optimization-based approach.
To conduct survey of academic and industry

123 Author Future Intent experts to validate results of quasi systematic
review
further research needed in the reconciliation
of plan - driven, agile, and FLOSS in the areas
124 Future Direction of configuration management, code
integration, process diversity w.r.t knowledge
management
The authors practices.
also identify the need to map
agile development with distributed
development, especially the use of test-driven
development in a plan - driven context.
Extend study (add, recent papers, more
database and manual search)
Reconciliation between (FOSS, plan-driven

and agile model)
Reconciliation of (FOSS and plan driven
128 Future Direction configuration management practices to agile
model)
Reconcialiation of (practice of continuous
129 Future Direction code integration between agile and FOSS
models and plan driven)
Knowledge management practices(can agile

knowledge management applied to FOSS)
Use test driven development in plan driven

context
Disruptive COSS revenue models,
organizational aspects of COSS, localization
of COSS, and creation and maintenance of the
user community are among them.
Management of the creation and maintenance
of the user community.
134 Future Direction Needof new revenue models for OSS
To generalize set of practices of knowledge

management for OSS projects.
Localization of open source software as a

business model
Structural and content dimensions of the

COSS, along with the organization
Need to research on proactive measures in
OSS to reduce knowledge loss
Need to explore knowledge retention practices
We suggest thatforthere
OSS isprojects
insufficient attention
paid to knowledge management in general in
OSS, in particular, there would appear to be an
139 Future Direction absence of proactive measures to reduce the
potential impact of knowledge loss.
We
Needalso
to propose
work on the need for management
knowledge a knowledge
140 Future Direction management
metrics in OSS projects. in OSS
evaluation metric
projects similar to the ones that evaluate the
health of online communities
141 Future Direction how to keep design open needs to be seen
Can open design framing remain bound to

business
Need to evaluate and improve the components

143 Future Direction of the OSS architecture and tools, methods,
and algorithms for GIS web architecture.
Developing a meta model for mining open
source database that will facilitate in quality
assessment of projects adopting frequent
release approach
OSS Evolution process support (less work on
aspects of configuration management,
145 Gap
complexity and control, and evolutionary
paths)
Evolution Prediction (less used methods are
146 Gap choas theory and software reliability growth
models)
Evolution Prediction (need of common corpus

147 Gap
of OSS projects for comparing results)
Need to empirically evaluate predictive power

of different metrics.
Need to generalize evolution prediction

Need to predict themodels
OSS evolution with help
of requirement, design, and architecture - level
metrics
OSS Dentistry (Low level of evidence and for

151 Gap
poor quality)
OSS Dentistry (no research on use of OSS in

152 Gap
prosthodontics)
OSS Dentistry (no comparision between OSS

153 Gap
an other software)
Research is required in the areas that are
“prediction of requirement change evolution
and change propagation, contribution
evolution, clone evolution, size, refactoring,
maintenance effort, and Self - Organized
Criticality (SOC)”.
Need for external validation of tools and
approaches of evolution are required.
Need to perform experimental studies in

medicine research
Need to perform experimental studies and

algorithm evaluations in medicine research.
Model quality assessment as a multi-criteria

158 Author Future Intent decision making problem for evaluating
quality across multiple domains.
Need to develop quality models based on
159 Future Direction essential qual ity characteristics in the area of
OSS research.
Need to build models with only essential

quality characteristics.
Need to build models with only essential

quality characteristics.
Validate the quality model on actual OSS

162 Author Future Intent ecosystem and define complete quality
assessment processes
lack of qualitative measures for measuring
163 Gap process maturity for accessing the quality of a
community.
Need to work on Framework, tool-based and
164 Future Direction data mining selection methods for quality
assessment models
Need to define quality assessment process for

software ecosystem
Need to work more on quality assessment

characteristics with low measures
To perform exploratory mixed-methods
research (like a combination of inside control
167 Author Future Intent and a predefined project in a software
evolution course) using open source projects
in an undergraduate course in SE.
Not many scientific methods used to SE in
168 Gap education and methods from research areas
other than software engineering can benefit.
Need more work in the area of student
169 Future Direction assessment (on metric identification for
assessment)
Need to measure student learning based on
170 Future Direction outcomes or developed skills and, assessment
of students by peers.
Use of open source projects within SE course

(further research area)
No selected study focuses on learning the

172 Gap areas of requirements or configuration
management in Open Source Projects
New and better methods to assess student's

learning in this context would be seen
Developing a Method for requirement

174 Author Future Intent extraction from internet and for automatic
requirement prioritization
To use proposed sorting algorithm for fully
automatic and non intrustive Open innovation
approach to Requirement extraction and
prioritization.
Need for future research in Open Innovation
176 Future Direction (OI) strategies within RE that are non -
intrusive and of low cost
Tool support for automated solutions in RE
177 Future Direction specifically to extract and. prioritize
requirements
Need for future research in Open Innovation
178 Future Direction (OI) strategies within RE that are non -
intrusive and of low cost
Define process that improves the selection,
management, and maintenance of OSS
179 Author Future Intent components and understand causal
replationship between risk, failures, risk
metrics and risk mitigation.
Evaluate the OSS project community
180 Author Future Intent ecosystem and the adopting company and
evaluate data of OSS project repositories.
However, very few studies cite these learning
approaches, and none of them provides details
181 Gap on how to design and implement pedagogical
practices that result in effective learning of SE
skills
Work on quantitative and qualitative measures
182 Gap of risks is lacking in OSS specific community
and respository measures
No work on specific mitigation activities of
183 Gap risks
No work on casual relationship between risks,

184 Gap concrete risks and effectivness of mitigation
activities.
No work on risk based on quantitative data of
project failure or correlating project failure
185 Gap
and losses with quanitiative data.
186 Gap No work on mitigation activities of risks
OSS Platforms (no research on mobile

187 Gap
platforms)
Tool support for automated solutions in RE

188 Future Direction specifically to extract and. prioritize
requirements
Need to capitalize on open innovationIto
189 Future Direction automatically extract and prioritize
requirements.
Convergence between industries on OSS

platforms need to be explored
To review the relationship between OSS

191 Author Future Intent success and quality in detail.
OSS Success and Quality (Less studies to

192 Gap introduce tools for evaluating quality and
success of OSS)
Need to research on the relationship between
OSS success and quality
Need to research on the relationship between

OSS success and quality
Need to clarify terminology, identify metrics,

195 Future Direction and develop tools that are capable of
measuring quality and success.
Relationship of evolution of sub-projects with
196 Future Direction evolution of associated community needs to be
seen in detail
To study socio technical congurance since
contributions by community members derieve
software evolution as well as re define role of
community.
Thus there is a need to validate the Conway's
law in OSS projects. Need to determine
198 methods and repositories to use for validation
and impact of conways law on quality and
sustainability of software.
Empirically validated community related
metrics are required for software evolution
Framework needed for uniform data collection

200 Future Direction and representation in OSS repositories to
support comparability and cohesiveness.
To validate the existance of SOC (Self
201 Future Direction Organzied Criticality in OSS projects and its
impact on OSS evolution
There is a need to study migration of
responsibility and sustainability in OSS
projects since developers join and leave
To improve
projects on impact
and its the generalizability threat in
on software evolution
evolution studies, there is a need to categorize
the findings (current and future) on (project
domain, similar organizational structure and
practices, product complexity and size) to
Reliable prediction
propose generalmodels and methods
evolutionary pattern are
needed to support error prediction,
measurement of maintenance effort and
determining cost of OSS project
To study the community building in terms of
attracting contributors during nabula stage,
along with what is a good balanced
community.
To re-assess the laws of evolution and for OSS
206 Future Direction by examining the ontologies for software
evolution.
Need of more empirical work and theoretical
research to assess the effectiveness of Linus
law applicability and its effectiveness in the
context of certification.
Need of more empirical work and theoretical
research to assess the effectiveness of Linus
law applicability and its effectiveness in the
context of certification.
OSS Success and Quality (Less empirical
209 Gap
evidence)
Health Industry (Low level of adoption of

210 Gap
modern ICT tools and infrastructure)
Public Administration (OSSadoption not

211 Gap
mature)
Integration of ASD and OSSD (Less empirical

212 Gap
evidence to support the idea)
213 Author Future Intent Establish a research agenda on Inner Source
Conduct survey on of practitioners to identify

214 Author Future Intent key challenges involved in ISS development
and propose some resolution strategies.
Need of an in - depth analysis of empirical
215 Future Direction work on Inner Source Software (ISS)
development.
Need to customize OSS practices according to

organization
Research is required in drug discovery
involving experts from the industry for a
217 Future Direction business model, incentive development, and
identification of the impact of the use of a
publicfactors
To find motivation domain.(economic and
social) for selecting open source licenses both
for the software community and the local
(Pakistani) community.
More architecture metrics are needed to
identify code anomalies without size bias and
219 Future Direction that can detect inconsistent classes. Metrics
that impact quality relationship of other
software
There is a need to identify diverse
prioritization criteria to handle diverse
architectural problems and seeing their
effectiveness to determine code anomalies.
Code level analysis via ARCADE tool can
utilized for architectural recovery, along with
experiments on wide scope to increase
accuracy of approach
To study architectural smells agglomeration
222 Future Direction and its correla tion with architectural problems
such as architectural
Future research should focus decay
on Identifying
the problems of architectural erosion within
the OSS projects such as identifying,
addressing, avoiding, and predicting erosion.
Moreover to identify reasons of architecture
decay.
Need to research architectural eroison in OSS
or industrial systems.
Further investigation need on the topic of

architectural bad smells in combination.
Need to work on metrics to detect

architecturally related smells.
Need to work on ARCADE’s tool for

architectural recovery
To identify new causes of architectural erosion

in OSS and industrial systems
Need to conduct an architectural conformance
study using different frameworks to identify
necessary architectural rules and critical cores
to avoid architecture rule voilations
the causes of architectural decay of the OSS
commuity need to be further identified in
terms of the frequency of their occurrence,
especially in the scope of the OSS projects
Need for a validated common model for the
selection, evaluation, and adoption of OSS
Validate the proposed OSS macro process in
232 Author Future Intent light of process activities, their characteristics,
and roles.
Investigation of OSS process analysis done by
researchers in OSS process including
approaches, techniques and tools they have
used to retrieve OSS process information.
Need of tools support (for supporting and
234 Future Direction simplifying the applicability of the proposed
models for evaluation of OSS Products)
Understand OSS process in practical settings

to mitigate OSS project failure
Experience reports are lacking in the area of

236 Gap
FLOSS adoption studies
Creation and publication of guidelines (for

adopting FLOSS).
238 Future Direction Application of FLOSS in new domains
Need more research in joining process and

abandonment
Acquire indepth knowledge about community

participation and engagement.
Include software ecosystem, community
structure, mentoring, project governance,
difficulties associated with finding a task, and
project characteristics.
242 Future Direction More work needed on community dynamics
243 Future Direction Need work on mentoring of newcomers
Need to develop tools to support newcomers’

244 Future Direction onboarding and easy migration of the
developers among different projects.
Need of tools, practices, and processes for
245 Future Direction better community participation and
engagement in OSS projects
More SLRs and SMS needed in the area of

community dynamics
Expand data sources and search strings to give

insights into the community dynamics.
Include more terms in the search strings and

248 Author Future Intent add more data
Research sources
to focus on to find moreorietation
newcomers studies on
community dynamics
and reception, provide guidance towards
barriers, problems of intial contribtutions. At
the same time identify means to support initial
participation, enhancing motivation to increase
particpation, remove hinderance to improve
retention of developers and OTC(one time
Economic factors of FLOSS adoption are not
contributors)
250 Gap
well covered
Research is lacking in joining and
abondonment process in OSScommunity
251 Gap
To study abandonment of projects by

community along with focus on factors that
impact the developers decision to stay or leave
the project.
To study contributers role in ecosystem,
orgaize onboarding information that can help
newcomers and impact of resolved issues on
onboarding success.
To explore the field of mentoring and tools,
254 Future Direction practices, and processes to improve
community participation
Machine learning can be used to solve issues
of task distribution, selection task for
contribution
Discovery of toolsand that
management of migration
will help in project
information
of developers to other projects, determine
socialization patterns of community, to
256 Future Direction identify impact of joining process on retention
and relationsip between joining process and
role transition along with studying the
onboarding process
Need to invent in providing
tools detail and onboarding
its impact.
support.
Study impact of mentoring on the success of
258 Future Direction Research needed to develop strategies related
onboarding process
to choosing a task to start initial contribution
design tools to enhance retention of One Time

Contributors (OTC)
Research
explore needed in the
the motivation areas of joining
of developers to work
process, abandonment orientation, reception,
as a mentor.
and mentoring of newcomers in OSS
communities
Research on the use of AI techniques for task
selection and distribution of tasks and
management of project information should be
carried.
Need to identify tools, practices, and processes
for community participation as tools are
262 Future Direction required to support newcomer’s on - boarding
and migration between projects.
Research required to engage developers in
263 Future Direction communtiy and providing good community
support.
Implement roadmap to strategically integrate
264 Author Future Intent OSS into educational system and measure its
effectiveness.
Tools are required to facilitate good reception

of new comers.
To study impact of community dynamics on

developer turnover and in turn its impact on
performance
The influence of user experience in predicting
outcomes of bug reports is overlooked
To study aspects influencing the participation
of women in open source projects and identify
268 Author Future Intent problems regarding issues of gender inequality
in open source communities and the software
industry.
Identify key challenges in implementing inner
269 Author Future Intent source along with resolution strategies via
survey of practitioners.
The effort in researching on novel approaches
to handle default type bug reports should be
considered to improve the state-of-the-art of
severity prediction algorithms.
To see machine learning algorithms
271 Future Direction outperform traditional algorithms for bug
report severity prediction or not.
Investigate more recent techniques to provide
272 Future Direction an approach for bug report prediction be
employing in real-world scenarios.
To check when the severity prediction is more
273 Future Direction appropriate in bug report lifecycle is of critical
importance.
Need to extend the study of bug level severity

prediction in FLOSS to commercial systems
Need to investigating the impact of the
temporal evolution of a bug report information
275 Future Direction in severity level prediction accuracy, as well
as investigating data structures to store the
representation of this temporal evolution
Need to assess best practices of severity level
prediction in FLOSS in the context
277 Future Direction Need to get more knowledge on innersource
278 Future Direction Add more database (innersource phenomenon)
Research needed to effectively manage inner

source community in an organization.
To identify practices used for identifying
success factors and potential risk factors in
adapting open-source software development
Tofrom
develop open-source
vendors’ software
perspectives.
development maturity model (OSSDMM) to
measure the maturity level of vendor
organizations in implementing an open-source
development strategy and validate it via
multipe case studies.
Clear and deeper understanding of what is
open IoT
Need to research penness dimensions of IoT

platforms
Need to look into openness types of IoT
platforms (from stakeholder perspective)
Need to look into openness types of IoT

platforms (from stakeholder perspective)
Develop a prediction model (for fork

effectiveness)
To analyze sentiments and their negative and

287 Author Future Intent positive impacts on productivity, and how
these sentiments vary between relases.
Need to cover more universities and
288 Future Direction practitioners ( understand how students
perceive OSS literature)
Need to determine the influencing factors(for
289 Future Direction productivity of organizations and open source
communities)
289 Total Extracted
code of Author Intent,
Gaps, Future Direction
Future Direction Synthesis and Thematic Analysis
Researcher: Saima Imtiaz
Verbatim Text
As future work, we intend to review some of the concepts related to ISO 9241-210 [14] and participatory design to reflect on the main prin
how they were addressed or not in the papers surveyed. Moreover, with the result obtained in this systematic mapping, the gaps and the lack
involving the areas of interaction design and development of FLOSS, we intend to advance in the research on this subject. Therefore, the ne
our research will be to expand this systematic mapping to identify approaches, methods, techniques, and tools for participatory interaction
distributed software development environments.
With this, we intend to develop a participatory interaction design process model for distributed software developm
environments.
Finally, we must extend the Open Source Maturity Model [54] with the proposed interaction design model. Conside
inher- ent there are few studies
advantages of softwareon MTTSA of interaction
development following design proposed
a software or used
process model,for/in FLOSS that
we believe development;
interaction de
activities will be considered during the different stages of the FLOSS development process, particularly the
• methods of interaction design proposed specifically for the development of FLOSS were not found; studies
in the earlyf
• techniques of interaction design, proposed specifically for the development of FLOSS, were not found; one of the s
papers, Lichtner et al. [32], used pre-existing techniques and did not consider the distributed development environm
FLOSS;
• the principal interest of the selected studies is in the activities of prototyping and evaluating; few studies have ad
the activities
The results of this systematic mapping suggest of
theestablishing
need for broadrequirements and projects
support for FLOSS designing alternatives;by the HCI community, t
and communities
research efforts in the area of interaction design for the availability of MTTSA of interaction
• the majority of the selected studies do not present any type of validation through design considering the characteristics
empirical studies. of F
development. Therefore, it is necessary to develop and publish research on interaction design in the context of FLOSS.
We intend to extend our SR to include more studies by searching the well-known digital literature databases. We w
extend our data analysis as we plan to discover the trends and future directions of OSS research.
The results shows that security areas in construction and verification (secure architecture, code review, and security
are followed by researchers with more interests than other areas in Governance and deployment. Next based on our r
The socio-technical perspective has not
the security gainedinmuch
studies attention in this
OSS development areresearch area (2 out
mostly technical of 42 papers). According
driven.
result of socio-technical analysis on the selected papers, the discussions between technical and social aspects seem
unbalanced, either (Coverage rate:98% versus 16% in average). The socio-technical perspective has as the main ta
blend both the technical and the social systems in an organization. This can be viewed as a necessary condition wi
security management framework as both aspects are of equal importance. Technical security practice considering di
social aspects (e.g., culture and structure) of open source develoopment will assure the effectiveness and efficency
Furthermore, the result of this SLR study also shows the gap thatofthere
implementation is a lack of knowledge management aspect o
the tool.
source security. Several researchers did mention the knowledge problems in securing OSS development, however, w
identify
Based on the findings any study
of this research, we tackle thistosecurity
have come issuethat
the conclusion from
theknowledge management
existing software perspectives.
security practices have limitations in sup
secure open source development. Secure architecture, code review and security testing do help secure OSS products. However, as there is les
on socio-technical security aspects and no discussion of security knowledge management in the context of OSS development, these practic
software security knowledge cannot be effectively spread within the open source community. Since OSS parcticipants are not experts on se
general and the domain knowledge of software security is vast and extensive, it is suggested that future research should explore soio-tec
approaches in helping OSS developers learn the necessary security knowledge to fulfill the need of their work, further, to reinforce their b
towards OSS security.
Interestingly, the aerospace domain represented in our study in only 3 papers was the top domain in both related surv
[11]. At the same time, the most represented domain (automotive) in our study was not the most represented in the o
surveys (ranked the 4th and the 3rd) [10], [11].
Less explored, however still represented by primary studies, is work on OSS for safety-critical automation system
maritime systems. Process industries and rail industry (mentioned in the top five domains in the evidence provided
[10] and [11]) are not represented among the primary studies4. Finally, oil and gas, off-highway equipment and m
Concerning
industries the domains
represented (RQ1)
in the where
previous OSS systems
survey have beenwith
about compliance used,safety
we found 22 studies
standards originating
[11] are from researc
not represented amon
different areas. Examples of areas include medical devices, the nuclear industry, and the aerospace industry.
primary studies. Hence, it could conceivably be hypothesized that these industries have yet not been intensified Howeve
by s
also found that there were some areas systems
where safety is important but where no published
or explored by open source solutions. research was found, such a
areas of rail traffic, and the oil and gas industry.
Most OSS projects used in the primary studies are still active however, and often contain codebases with moderate to very high activity. In
therefore, it seems that the relationship between the adaptation of OSS in safety-critical and their long history is rather clear. At the same tim
research should be done to investigate the relationship between the adaptation of OSS in safety-critical and OSS project activity. Moreover, i
interesting to investigate the number of downloads of this project or its adoption within industry
To summarize, the review results indicate that required OSS functionality is more frequently integrated in the safety critical systems rather
entire OSS solution is used. The complete OSS solutions found in the review are rather small and have low activity. On the contrary, OSS
Concerning
integrated inthe functionality
safety (RQ2)
critical systems have provided by open
over five years source
of history software,
and high it was
or medium found
activity. that more
Hence, we canresearch report
conclude that longon syst
history
are subsystems
projects of larger
facilitates their systems
use in as integral than complete
parts of systems.
safety critical That
systems. is, onlytherather
However, few papers
relationship betweenwere foundofthat
the activity the reported
OSS project o
source software making up a complete system. Instead most published research concerns open source components th
adaption in safety-critical systems is less clear from the studied papers and therefore requires further investigation.
included in larger systems.
Concerning the open source communities
Software size has been the most common attribute to (RQ3) that are investigated
analyze evolution in of research it wasSeveral
OSS projects. found that most
types were activ
of metrics ha
employed to measure maturesoftware
(> 5 years).
size. However,
These metrics thererange
were from
examples
coarseofgrained
communities that aresuch
level metrics no longer active.of files, m
as number
Asand a final conclusion
functions, to fineit grained
can be seen
levelthat opensuch
metrics sourceas software
number oftodayLOC, is methods,
used as part
andofclasses.
safety critical
Severalsystems.
approaches,Thisothe
is
in both research that investigate complete systems and open source software components.
source code analysis using metrics, to analyze OSS evolution have also been employed in the research literatur It is reported from a num
• Lately, metrics related to change different domains,
activity have also andbeen
communities
included to areunderstand
active and OSSmature.
evolution. These metrics me
But that is restricted to a few of the change categories e.g. adaptive v/s non-adaptive
changes in source code such as number of program elements (functions/ /classes/methods) changes, or corrective v/schanged
non-corrective changes. A fin
in consecutive v
view of the changes can help to answer amount of progressive/ regressive work performed in a software system as it evolves. It can also be
Change activity as recorded in SCM systems is also used in a few cases. Most of the work deals with finding
validate Lehman’s 2nd law as Gonzalez-Barahona (2013) points to the lack of information available in this regard in their study of the glib chang
• Techniques and tools change
haveeffort distributions.
been devised A few
to tackle large studies
amounts do change
of data profile
generated analysis
in software as OSS
evolution systems
analysis evolve. Software
and prediction.
visualization helps in understanding the transitions in complex and large systems in an easy way. Big data analytics can also help to analyze
of data generated during software evolution. Data analytics can be used to manage and understand the complex web of software evolution as
Most organizations seem to have rather limited contributions
in source torelated
code and other the OSS communities [33, 43, 91, 98]. The most co
repositories.
way of participation is being an active user that reports occasional bugs to the community [43, 98, 99]. Only one
respondents from a sample of tertiary education institutions had participated actively by writing code, while 14
contributed to an OSS community through reporting bugs [91].
We classified 53 of the 112 empirical papers identified in this review as experience reports. Hence, the most common
ofThe
studying thecosts
general OSSrelated
phenomenon
to such in organizations
a migration is through
are unclear [62,experience
187], and reports.
there areThese experience
very few studies reports
showinglack ex
comp
calculations of the true costs research questions,
and savings and most also
of (1) introducing OSS lack a method
products intodescription.
organizations, and (2) keeping the
products operational over a longer period of time. One paper reports cost savings from an OSS migration project at B
Hospital [81], but it is published just after the initial stage of the project is finished.
Despite this lack of clarity, many organizations seem to be blinded by the perceived advantages of OSS and have th
adopted it without per- forming any cost-benefit evaluations in their own context [91, 187, 190]. The adoption of O
furthermore
Despite having frequently bottom-up,
a close relation in the sense
to the CASE thatfield,
research it is introduced
only sevenby engineersreports
experience rather than strategic
discuss the usetop-level
of OSS dec
CA
[188].
in the context of organizations. Given the number of OSS CASE tools available, it is surprising that the use of such t
not been studied in any empirical research papers.
In fact, most papers do little more than mention such issues as research questions, limitations, data analysis, and so
furthermore found that many of the publications lack details about the research methods and findings. As a conseq
several papers have limitations when it comes to how they describe these issues. Moreover, many of the research pa
A few other issues areexplorative and they are
worth mentioning. therefore
First, lacking
all of the eightaempirical
precise focus and clear
research paperscontributions.
from the public sector foc
deployment of OSS products. Besides [37], which has a mixed sample, no paper focuses on deploying OSS in the
sector. Second, 27 of the 59 empirical research papers report findings from samples of several organizations from the
sector. However, as few as eight papers report findings from one single private organization. Hence, most research
dedicate relatively little space to describing the individual organizations.
There are relatively few empirical publications on OSS in organizations, and the quality of published work is not
enough. Much of the published
research lacks relevance and a clear focus, and does not draw enough support from related literature
While there is quite a lot of research on specific development processes in OSS communities e.g. [167], there is little
on using these processes and practices inside organizations.
While there are a few studies outside the scope of this review focusing on software selection [46, 56, 105, 184] and knowledge sharing wit
communities [119, 173, 195], none of these are directed towards studying actual practice in organizations. A few studies have started to loo
of the challenges in the borderlands between integrating an OSS component and contributing to the development of it [106, 130, 186], bu
research is needed to solve the maintenance challenges facing developers who integrate a large number of components into their produ
To enable organizations to reap benefits from their participation in OSS communities, the research community should dedicate much more
to questions concerning this [48, 165]. While there are a few examples [50, 101, 176, 186], more work is needed to aid organizations in parti
communities and collaborating with other organizations through collaboratively working on OSS products [7] and to solve questions l
􏰅 When, how, and with what should organizations participate in the development of OSS products controlled by others (including in
The overall rigor of the studies performed on OSS, both within
organizational
organizations
collaborations)?
and in general, is furthermore not good enough. Conseque
should strive to do better􏰅work
Howand
cantocompanies
present this work in more
(effectively) detail
allow [180].toInbeparticular,
products partly OSS weand
agree withcommercial
partly Kitchenhamproducts?
et al. [118] in that the
the organizations being studied should be given much more attention.
We observed that few of the studies were longitudinal and that few publications focused on providing in-depth details from one or a
organizations. To really understand the profound consequences of approaching OSS, we believe there is a need for both more longitudina
depth case studies.
Finally, we found evidence that OSS is not that different from other information technologies.
OSS researchers should therefore increasingly rely on research and theories from related fields (see Section 2.4). Software engineering
information systems researchers should see OSS as an opportunity to investigate general software engineering and information systems r
We would in particular recommend investigating two issues: (1) challenges.
topics related to integration of OSS components and (2) topics con- ce
participation in organization-community or inter-organizational OSS collaborations. We find these issues important because integration o
components concerns most software-intensive organizations [98] and because participation in collaborative software development is incre
185]. The research could focus on identifying the characteristics of successful approaches to OSS, the challenges these organizations met,
they solved them.
We would in particular recommend investigating two issues: (1) topics related to integration of OSS components and (2) topics con- ce
Deploying OSS: Many claim that reducing costs is one of the advantages of deploying OSS server software, infrastructure, and applications.
participation in organization-community or inter-organizational OSS collaborations. We find these issues important because integration o
a recent study by Fitzgerald [80] is one of few studies with a longitudinal view on deployed OSS products. This highlights a need for more s
components concerns most software-intensive organizations [98] and because participation in collaborative software development is incre
􏰅 What are long-term costs and consequences of deploying and keeping OSS products operational?
185]. The research could focus on identifying the characteristics of successful approaches to OSS, the challenges these organizations met,
they solved them.
Deploying OSS: Many claim that reducing costs is one of the advantages of deploying OSS server software, infrastructure, and applications.
a recent study by Fitzgerald [80] is one of few studies with a longitudinal view on deployed OSS products. This highlights a need for more s
We furthermore advise researchers
􏰅 What to put emphasis
are long-term costs andonconsequences
how the studied organizations
of deploying actually OSS
and keeping use OSS, and operational?
products on problems that really mat
practitioners. Practitioners should be open to OSS and see that it offers several opportunities. However, they must first evaluate the implic
adopting OSS in their own context.
Maturing the research field on OSS in organizations and dealing with some of its limitations may be done through four main steps
. These observations are not particular to research on OSS. For instance, Kitchenham et al. [117], Vessey et al. [192], and Zelkowitz and Wa
observe a lack of relevant empirical research of high quality within both the software engineering and information systems fields. Finally, w
also like to see more research from outside Europe and the USA.
researchers need to pay more attention to issues that are interesting to practitioners. Hence, we recommend focusing on topics related to th
which organizations actually approach OSS, and issues that could benefit practitioners, rather than general “adoption issues”.
Researchers and practitioners should increasingly collaborate to define a common research agenda and study research questions that ma
practitioners. These research questions should be answered through several related studies from different contexts.
Succeeding at providing an OSS product is not necessarily easy as there are challenges related to collaborating with a community, like attr
relating to contributors, requirements engineering from a community, balancing focus on community and paying customers, and so on [109,
hope to see more research on these topics like e.g. [7, 205] on the following topics:
Using OSS CASE tools: The research on􏰅OSS How CASE
are OSS
toolsproviders
has beenable
verytolimited.
attract and
However,
sustainWicks
a community?
and Dewar propose a new agenda for re
tool integration, requesting a more business-oriented
􏰅 What are theapproach
success to
criteria
futurefor
research
incorporating
[207]. The
contributions
use and development
(require- of OSS CASE tools and re
such tools could easily fit into this new agenda. ments,
Robbins provides
code, an extensive etc.)
bug reports/fixes, overview
from of OSS tools for development, and claims that CAS
a community?
has a lot to learn from OSS [157]. OSS should furthermore be particularly interesting to academia since they have access to professional sta
art tools and the tools’ source code. This enables them to extend existing tools and test new ideas in collaboration with each other.
Increased participation in OSS projects, increased collaboration between organizations, and increased use of OSS practices will most likely
improved collaborative development tools. Hence, there is a potential for research on:
􏰅 What kinds of tools are needed for collaborative software development across organizational and community borders?
It is more and more difficult to talk about “OSS practices” as the practices used in OSS communities are heterogeneous, and as organizati
􏰅 Howdoorganizationscollaborateusingsuchsoftwaredevelopmenttools?
increasingly getting involved in, and influenced by, the development of OSS. Nevertheless, there are opportunities for further research on t
development practices for distributed software development.
OSS development in large communities and in and between organizations, are areas where researchers could have an impact on practic
research has so far focused mainly on processes in communities of volunteers [167], but some of this research could turn its focus towar
application of their findings within organizations and questions like:
Each metric used for prediction, either being positively or negatively associated with prediction results. For example, in case of fault pred
􏰅 How may organizations successfully collaborate through community- or consortium-based software development?
metric signifies a module as either being faulty or not faulty. In either case, the metric’s prediction recital is judged as a best, significant
predictor. In this regard, our review results show inconsistency on some metric’s performance. For instance, the metric LOC (Line of Cod
evaluated as a best or good predictor in [1][9], whereas in [11] it was noted as a bad predictor. Moreover, DIT (Depth of Inheritance Tree) w
a significant
What sets open
predictor
source
in [6],
development
but classified
apartasfrom
a badthepredictor
traditional
in [1][4].
proprietary
Possible
soft-causes
ware isbehind
the developer
these differences
communityin results
behindmight
it. Although
be the vari
the
OSS
structure
systems
and[9],
communication
differences inofimplementations
OSS communities of the
havemet-
gained
rics significant
[9], or different
research
prediction
interest,models
the research
used. However,
efforts to the
an indeepth
community investiga
in rela
prediction appear quite the opposite. Evolution
resolutionof communities
of such conflicting
is of interest
issues would
startingbefrom
a future
the paper
research
intro-
agenda.
ducing the community structure [1
search did not find much focus on community evolution tied to prediction. In [14] the authors investigate the impact of social structures b
developers and end-users on software quality and their results give support to thinking that social structures in the community do hold pr
power in addition to the source code centric approaches. It is also suggested that combining metrics focusing on code and social aspects w
better prediction model than either alone. This gives support that the question has research value and is worth looking into further: what d
This research question evolved due to the fact that most of the reviewed articles (67%) admitted the necessity of external validity of the p
community and the community structure predict for the software?
models studied. To be specific, in [4] generalization of the findings was not done because the study is subjective and is dependent on how t
are classified in the project. Again in [9], it is acknowledged that further replication across many OSS projects is required to establish the cro
validity of the prediction model.
It is also noted that the prediction models are not general and are not applicable to different software systems [10]. Specially for defect predi
els there exists very little evidence on their cross project applicability [5]. Thus a comprehensive study on the generalizability issue of the p
models across the domain of OSS projects is an area of future research.
Traditionally defect prediction models rely on metrics that represent the state of the software system at a specific moment in time [11].Thes
are used to capture a particular snapshot or release of a project to predict the next one. But metrics capturing changes over time in projects a
significant role in prediction. For example, metrics presenting the software evolution were used to predict the need of refactoring [12] and q
OSS projects with significant accuracy. Thus a future research direction would be to explore a comparative study for identifying either (a) w
of metrics are more suitable for prediction models in terms of accuracy, reproducibility, and generalizability, or (b) are these metrics comp
SLR concerning software fault prediction
to each other
was first
and conducted
should be used
by [3]inand
combina-
was extended
tion to get
withbetter
new prediction
results in [7].
results.
However these works were l
fault prediction of closed source projects and fall short of exploring OSS domain.
This SLR will help researchers to investigate prediction studies from the perspective of metrics, methods, datasets, tool sets in an effective
Future research should focus on establishing external validity and consistent accuracy of prediction models, incorporation of social aspects
projects, and building tool support to automate the prediction process.
Researchers studied the effectiveness and accuracy of several metric suites using data from one or more OSS projects. Despite of their es
contribution in predicting OSS projects, they suffer from lack of generalizability due to diverse nature of OSS projects and the project speci
of the metric suites. Also it is quite difficult to ensure the availability and quality of metric data, which makes the results incomparable [10
future research agenda would be to perform an indeepth analysis on (a) cross project validity of the studied metric suits, and (b) to propose m
ensure the quality of metric data.
Researchers studied the effectiveness and accuracy of several metric suites using data from one or more OSS projects. Despite of their es
contribution in predicting OSS projects, they suffer from lack of generalizability due to diverse nature of OSS projects and the project speci
of the metric suites. Also it is quite difficult to ensure the availability and quality of metric data, which makes the results incomparable [10
future research agenda would be to perform an indeepth analysis on (a) cross project validity of the studied metric suits, and (b) to propose m
ensure the quality of metric data.
In the future, we aim to conduct some qualitative studies to confirm the problems evidenced by the literature. We
conducting some interviews with experienced OSS developers and newcomers to verify the main problems face
newcomers from their perspective.
We also plan to refine the classification model based on the results of the interview analysis. Addition- ally, based
model, it is possible to propose awareness mechanisms and tools to offer better support for newcomers
It is possible to see the lack of studies conducting qualitative analysis as supporting the existence of problems that
newcomers’ contributions. Quantitatively analyzing historical data can bring highlights of the barriers faced by new
but conducting qualitative analysis can enrich the evidence, reveal new facts, and help in finding the issues faced
newcomers. There is still room available for studies based on observations, interviews and experiments that can help
the barriers
SLR concerning software fault prediction was first conducted faced
by [3] and in practice.
was extended with new results in [7]. However these works were l
SLR concerning software fault prediction was first conducted by [3] and was extended with new results in [7]. However these works were l
Although we considered the barriers as something that can hinder newcomers’ contributions, some barriers can be used as filters by the p
Findings from a Halfaker et al. [19] study on Wikipedia newcomers revealed that some entry barriers led to improved contributions in the
More- over, research conducted in the OSS domain [33, 13] demonstrated that socialization barriers are useful for maintaining community
and the quality of the community’s product. A clear direction for future work is to explore how the communities perceive these barriers and
impact the quality of contributions from newcomers.
Although we considered the barriers as something that can hinder newcomers’ contributions, some barriers can be used as filters by the p
Findings from a Halfaker et al. [19] study on Wikipedia newcomers revealed that some entry barriers led to improved contributions in the
More- over, research conducted in the OSS domain [33, 13] demonstrated that socialization barriers are useful for maintaining community
and
Wethe analyzed
quality the
of the
characteristics
community’s and
product.
goals of
A the
clear
newcomers.
direction for
However,
future work
manyisof tothe
explore
papershow
didthe
notcommunities
explicitly profile
perceive
the newcomers
these barriers
theyanda
This is probably related to the type of data analyzed impactandthethe
quality
type of
of contributions
study conducted,fromasnewcomers.
most of the studies only used data coming from s
repositories and did not go deeper in the analysis of the subjects. The problem is that the term newcomer can be used in a loose way, whic
the results. Newcomers can be novice developers who are starting their career, people who are experienced developers from industry but are
to OSS projects, or people who are migrating from other OSS projects. These three profiles are different and can face different barriers or e
Onbarriers
the other
differently.
hand, studies
Therefore,
such asitthose
wouldconducted
be a betterbyapproach
Steinmacher
to assess
et al. how
[PS14]
these
anddifferent
Jensen ettypes
al. [PS9]
of developers
presentedseesimplistic
the barriers
viewsandofwhat
the
whenimpressions
they drew conclusions
of them are.from
For only analyzing
example, does athe first messages
novice developerfrom new- issues
find more comerstoand their retention.
contribute than an The context isdeveloper
experienced important: Why did
without an
the messages? What motivated them? Did they really want to contribute or just clarify some doubt? Did they contribute at the end but never
background?
to the mailing list? To answer such questions, we need to merge information from different sources (issue tracker, mailing lists, documenta
It is worth noting that the main focus of analysis were large projects with a high number of developers and more than five years of exis
repository) and verify the context by talking to practitioners. Another possibility is to conduct observational and ethnographic studies by ana
Moreover, projects that focused on products used during the development cycle and developed in Java and C were preferred. Such project
barriers and effects for newcomers in real settings.
classified as clearly successful projects which, combined with the historical data available, provide an easy target to search for newcome
observed that, although projects gain several newcomers, just a small percentage are successful in contributing some source code. Becau
identification of barriers faced and surpassed by such newcomers is important, projects with a high number of developers (and newcomers)
to analyze to find evidence of such barriers. However, a high number of OSS projects present different characteristics, such as small teams
OSS researchers can also benefit from these results by using them to conceive strategies for newcomer support. To achieve this, it is necess
lifetime, and were not considered for evaluation. Naturally, such projects provide less data and are less attractive than large successful proj
more effort on specific research topics, such as understanding and creating ways to measure the influence of the barriers in newcomers’ ex
when considering newcomers, they can account for different problems than those identified by our model or modify their importance. F
identifying and creating different strategies to lower each barrier, and proposing metrics to grade the support offered for each barrier. To ga
investigation is required regarding such projects to improve the model of barriers described in this paper.
understanding of the barriers and to what extent they need to be lowered, it is important to conduct more qualitative studies because this phe
occurs in a complex, social environment in which the context of its occurrence is important. Moreover, a qualitative view complements the
literature, which relies mostly on quantitative evidence.
We also noticed a lack of in-depth studies on technical issues faced by newcomers. The reason can be attributed to the small number of qu
studies found because it cannot be quantitatively extracted from mailing lists. For example, technical hurdles are evidenced by only five
Due to the rising dominance of the OSS in the software industry; not only practitioners, but researchers as well as academicians are also
analyzed. Issues related to workspace setup is reported in only one study, by one subject in a debrief session. These kinds of issue deserv
understand the OSS software development process. Several studies have been conducted in the past in this regard. This paper presents a sy
attention, from both practitioners and researchers.
literature review of the studies performed to understand OSS evolution. A set of 190 primary studies are identified for analysis and discuss
studies are characterized on the basis of the research questions they address. The main findings are as follows:
• OSS evolution prediction studies use ARIMA modeling of time series analysis for prediction of software evolution attributes such as size
change requests etc. However, as software evolution in general and OSS evolution in particular is a discontinuous phenomenon, the use of p
techniques that just extrapolate the historic trends in to the future should be a conscious task. Furthermore, Herraiz et al. (2007c) observed
•Lehman’s laws of software evolution for OSS systems have been validated in several studies. Only two laws (I and VI) have been confirme
are no long term correlations in the time series representing OSS activity. The idea of fuzzy time series to deal with the uncertain evolut
different studies on OSS evolution. There is need to look into the change activity of these projects and validate the laws using the change
behavior of OSS systems has also been explored. The results show that a fuzzy based method for time series analysis is rather a promising
information available in the SCM systems
More rigorous prediction methods may be explored in future;
A shift in the programming languages, from procedural to object oriented, has been noticed asOSS systems, as subject systems in the corre
studies, evolved over the period of time;
• Techniques and tools have been devised to tackle large amounts of data generated in software evolution analysis and prediction. Software
automation offers to collect volumes of data in a consistent manner. Software evolution visualization helps in understanding the transitions i
and large systems in an easy way. Big data analytics can also help to analyze large sets of data generated during software evolution. Data an
be used to manage and understand the complex web of software evolution as it happens in source code and other related repositories (Go
Barahona, 2017).
The analysis of the results allows us to state that OS- SECO is a growing research area in software engineering [R16, R49, R50]. Due to t
are several new research opportunities in the empirical examination, modelling, analysis, measuring, quality evaluation, etc. of OSSECOs. A
this argumentation, in this section we provide two initial proposals to improve the current structure of the knowledge on OSSECOs: a defin
OSSECOs and a taxonomy of OSSECO related terms.
Our future work will extend this analysis to OSS article hubs like for example http://flosshub.org or http://pascal.case
or the MIT repository.
We plan to use more sophisticated techniques of string similarity and a better data cleansing to get finer resul
Since 2008, synthesizers of res'. earch have introduced frameworks and platforms to perform OSS research paving the way for future wo
analysis of non-cited papers indicates that significant research has not been exploited, yet. Therefore, we recommend the OSS community t
further the potential provided by the OSS conference series while maintaining the interest in its major research streams.
There are themes still little cited that might have some future potential: OSS process (meta-) modelling, OSS security, Agile and OSS deve
methods, and teaching OSS in universities.
Results shows that “prediction of properties”, “aggregate metrics” and “changevevolution analysis” are the most emergent open issues i
evolution,together with OSS integrability and licensing.
As far as research community concerns, the results from the systematic mapping study prove that many researchers
perform empirical studies or replications for solving the open issues.
The areas are important for research and it is interesting to see that research is available in all these areas. The question of how to use open
practices within a closed company (iv) is for example an interesting area for further research.
Based on this review we also propose that further research is conducted on how companies can transform their proprietary software to ope
and build a community on it. Further research related to all four research questions in Section 3.1 could involve more case studies on imple
of specific methodologies for dealing with different aspects of open source in industry.
Based on this review we also propose that further research is conducted on how companies can transform their proprietary software to ope
and build a community on it. Further research related to all four research questions in Section 3.1 could involve more case studies on imple
of specific methodologies for dealing with different aspects of open source in industry.
Based on this review we also propose that further research is conducted on how companies can transform their proprietary software to open
build a community on it. Further research related to all four research questions in Section 3.1 could involve more case studies on implemen
specific methodologies for dealing with different aspects of open source in industry.
Many of the studies are in the form of surveys, which gives a broad and necessary understanding. Based on this it would probably be pos
conduct more studies investigating specific cases of implementation of methodologies for dealing with different aspects of open source in
More case studies could probably be conducted on all aspects of the research questions.
From a practical standpoint, this research provides IS managers, as well as open source BI providers and consultants with an initial structur
Many of the studies are in the form of surveys, which gives a broad and necessary understanding. Based on this it would probably be po
better understand the most important
conduct more studies investigating specific cases of implementation of methodologies for dealing with different aspects of open source in
barriersthatpreventorganizationsfromadoptingopensourceBItools.Thesebarriersrequirefurtherconsiderationbyall stakeholders intereste
More case studies could probably be conducted on all aspects of the research questions. More case studies could probably also provide
the adoption or deployment of open source BI. From a methodological standpoint, following Poba Nzaouetal. (2016), this research provides
knowledge of research question 3 and research question 4. That is, research could be carried out to understand more about the cost and adv
contribution that is a rigorous analysis of Qualitative Survey data, based on two principles of interpretive
From
dif- a practical
ferent standpoint,
approaches, and why this researchapproaches
different provides ISaremanagers,
chosen. Itasiswell
alsoas opennoticing
worth source BIthatproviders
there areand consultantsexperiments
no controlled with an initial structur
at all in the
research (Klein&Myers,1999): the fundamental principle of Hermeneutic Circle, and the principle of Abstraction and Generalizatio
articles.
To conclude, the authors acknowledge some areas of limitations, and call for further studies of open source business intelligence to
barriersthatpreventorganizationsfromadoptingopensourceBItools.Thesebarriersrequirefurtherconsiderationbyall stakeholders intereste
conducted. First, though it is adequate for a qualitative survey andmethodologicallysufficient,thesizeofthepanelwassmall.Second,itcouldbein
From a practical standpoint,wheretheyareverypopular(suchaswebserver,operatingsystems,etc.).Third,sincethisexploratory
this research provides IS managers, as well as open source BI providers and consultants with an initial structur
barriersthatpreventorganizationsfromadoptingopensourceBItools.Thesebarriersrequirefurtherconsiderationbyall
investigatingtheviewsofBIexpertsinothercountriesandinvolvingotherBIstakeholders(e.g.users, stakeholders intereste
contribution
that is a rigorous analysis of Qualitative Survey data, based on two principles of interpretive
research (Klein&Myers,1999):
the fundamental principle of Hermeneutic Circle, and the principle of Abstraction and Generalizatio
better understand the most important barriersthatpreventorganizationsfrom adopting
To conclude, the authors barriers.Lastly,futureresearchesmaybenefitfromadoptingotherresearchmethodsaswell,such
acknowledge some areas of limitations, and call for further studies of open source business intelligence to
opensourceBItools.Thesebarriersrequirefurtherconsiderationbyall stakeholders interested in the adoption or deployment of open source
conducted. First, though
ascasestudy,surveysorexperiments,astheymayprovidericherinsightsthanthequalitativesurvey
it is adequate for a qualitative survey andmethodologicallysufficient,thesizeofthepanelwassmall.Second,itcouldbein
adoptedinthisstudy.
Fromexecutives,etc.),assuchstudiescanincreasethevalidityofthefindingsfromthisstudy.Fourth,asthis
a methodological standpoint, following Poba Nzaouetal. (2016), this research provides one main
To conclude, the authors barriers.Lastly,futureresearchesmaybenefitfromadoptingotherresearchmethodsaswell,such
acknowledge some areas of limitations, and call for further studies of open source business intelligence to
barriersthatpreventorganizationsfromadoptingopensourceBItools.Thesebarriersrequirefurtherconsiderationbyall
investigatingtheviewsofBIexpertsinothercountriesandinvolvingotherBIstakeholders(e.g.users, stakeholders intereste
conducted. First, though
it is adequate for a qualitative survey andmethodologicallysufficient,thesizeofthepanelwassmall.Second,itcouldbein
adoptedinthisstudy.
the adoption or deployment of executives,etc.),assuchstudiescanincreasethevalidityofthefindingsfromthisstudy.Fourth,asthis
open source BI. From a methodological standpoint, following Poba Nzaouetal. (2016), this research provides
The reviewed articles show that self-determined participation motives are most relevant for FLOSS developers’ commitment. Moreover
relational (e.g. trust) and structural (e.g. network centrality) group aspects foster FLOSS developers’ commitment. Also, the chosen code
affects developers’ commitment. Beyond these aspects, the reviewed articles suggest other factors which yet need dedicated analysis. For ex
effects of members’ cultural investigatingtheviewsofBIexpertsinothercountriesandinvolvingotherBIstakeholders(e.g.users,
background [57] or their geographic proximity [29] on their development intensity. Further, the interrelations b
research perspectives need further
scrutiny. In particular, the relationships between individual or team factors and project characteristics (se
Such cross-perspective analysis initialstudyfocusessolelyonbarriers,theauthorsalsorecommendthatfuturestudies,includingthe
is necessary to understand fully how FLOSS projects can incentivize individual and group factors which
developers’ participation. Future identificationofstrategiesthatmaybeinitiatedbyrelevantstakeholdersindealingwiththeidentified
research may draw on research results by Gallivan [21] and examine if and how governance processes fost
developers’ personal relationships.
Also, further research is needed to fully understand if and how project governance can stimulate indiv
effects of members’ cultural background [57] or their geographic proximity [29] on their development intensity. Further, the interrelations b
participation motives. adoptedinthisstudy.
research perspectives need further scrutiny. In particular, the relationships between individual or team factors and project characteristics (se
Such cross-perspective analysis is necessary to understand fully how FLOSS projects can incentivize individual and group factors which
developers’ participation. Future research may draw on research results by Gallivan [21] and examine if and how governance processes fost
developers’ personal relationships. Also, further research is needed to fully understand if and how project governance can stimulate indiv
participation motives.
For a high retention rate, it is important that FLOSS developers perceive their project work as self- determined. In addition, members’ p
continuance is influenced by relational and structural characteristics of the team. Also, less restrictive code licenses and the modularity of
affect members’ project retention. However as shown in table 3, there is little dedicated research on how team level aspects affect devel
retention. With regard to the key role of group aspects for developers’ commitment, future studies should examine this aspect closely. In add
few articles use more than one research perspective. This calls for further research. In particular, future studies should draw on research by S
Gosain [55] which suggests that members’ retention is a product of motivational and relational factors. Further, future research is necess
understand the interaction of structural network properties and project characteristics. To do so, future studies should draw on research by O
[40] and analyze the ways in which FLOSS projects can actively utilize the interaction network of their members to foster their retention.
further research is necessary to understand the ways in which project characteristics influence individuals’ retention.
few articles use more than one research perspective. This calls for further research. In particular, future studies should draw on research by S
Gosain [55] which suggests that members’ retention is a product of motivational and relational factors. Further, future research is necess
understand the interaction of structural network properties and project characteristics. To do so, future studies should draw on research by O
[40] and analyze the ways in which FLOSS projects can actively utilize the interaction network of their members to foster their retention.
further research is necessary to understand the ways in which project characteristics influence individuals’ retention.
fewInarticles
contrast,
usedevelopers
more than are
onecommonly
research perspective.
driven by extrinsic
This calls motives
for further
to join
research.
a FLOSS In particular,
project. Also,
future
structural
studies characteristics
should draw onofresearch
FLOSS by deve
S
contact [55]
Gosain network
whichinfluence
suggeststheir
thatjoining
members’behavior. Moreover,
retention the application
is a product domain
of motivational andand the development
relational phase are
factors. Further, relevant
future characterist
research is necess
developer
understand theattraction.
interactionHowever, it is network
of structural unclear how relational
properties and factors
projectinfluence developers’
characteristics. To do attraction. While Stewart
so, future studies and Gosain
should draw [55] view
on research by Ot
essential,
[40] and analyze
there is no
theattraction
ways in which
centricFLOSS
research
projects
on thiscan
aspect.
actively
Hence,
utilize
future
the evaluations
interaction network
should focus
of their
on members
this aspect.
to Moreover,
foster theiras
retention.
shown i
there is very further
few attraction
researchspecific
is necessary
research
to understand
which combines
the ways
individual
in which andproject
team characteristics
factors. Considering
influence
thatindividuals’
ideological and
retention.
status motiv
In contrast, developers are commonly driven by extrinsic motives to join a FLOSS project. Also, structural characteristics of FLOSS deve
dependent on the feedback of others, future studies should examine closely this interaction for extending our understanding of developer a
contact network influence their joining behavior. Moreover, the application domain and the development phase are relevant characterist
Similarly, there is very little literature which combines team and project characteristics. Considering the key role of structural properties [
developer attraction. However, it is unclear how relational factors influence developers’ attraction. While Stewart and Gosain [55] view t
future studies are necessary to understand how FLOSS projects can utilize the social contacts of their members to reach out or new member
essential, there is no attraction centric research on this aspect. Hence, future evaluations should focus on this aspect. Moreover, as shown i
drawing on research by Shah [48], further studies are essential to understand fully how FLOSS initiatives can stimulate individuals’ extrin
there is very few attraction specific research which combines individual and team factors. Considering that ideological and status motiv
In contrast, developers are commonly driven by extrinsic intrinsic
motives
motives
to jointoa join
FLOSS
the project.
project. Also, structural characteristics of FLOSS deve
In contrast, developers are commonly driven by extrinsic motives
intrinsic to jointoa join
motives FLOSS project. Also, structural characteristics of FLOSS deve
the project.
In contrast, developers are commonly driven by extrinsic motives
intrinsic to jointoa join
motives FLOSS project. Also, structural characteristics of FLOSS deve
the project.
intrinsic motives to join the project.
factors for FLOSS developers’ attraction and retention. Moreover, there is even less dedicated research on these two management aspects. O
because of the use of ambiguous measures such as “team size”. With such measure, it is not possible to tease out distinct lessons for the attr
retention of FLOSS developers. Thus, future research should use specific measures for this particular aspect, for example the number of
respectively retained developers. Finally, as visualized in table 1 – 3, our literature review shows that there is few dedicated research on the
intrinsic motives to join the project.
(especially on developer attraction and retention) which combines more than one research perspective. Considering the interrelations betwee
research perspectives, however, an isolated research perspective seems too narrow. For example, FLOSS developers’ extrinsic motives are s
by receiving appreciation and by particular project characteristics (e.g. corporate sponsors) [32]. With respect to these interrelations, future
FLOSS management should adopt more than one research perspective in order to fully understand the effects of the examined aspec
Our future work will extend this analysis to OSS article hubs like for example http://flosshub.org or http://pascal.case
or the MIT repository.
Most used sources development tools of the data are not properly documented. None of the study data documented w
there was any involvement of OSS features in the development.From the result we know that the most used sour
development tools mentioned in the data are not properly documented, as can be seen from the result, with the hig
percentage being unidentified development sources. Due to the poor documentation regarding the involvement of the
development tools, we have not been able to determine whether in each of the projects, was there any involvement o
Since this SLR is focused onthe
None of OSS, therefore
studies we can
discussed sayeffort
early that toestimation
the best offorour
OSSknowledge, there has been very li
Web projects.
documentation of OSSprojects
• Most of the source usage inused
the as
datasets, whichare
the datasets ledinusCMS
to further
form,investigate
and none ofhow the OSS can affectprojects
development the accuracy of
in the da
estimation.
involved any open source framework as there are no recorded details.From the findings it is shown that there has b
proper documentation detailing the involvement of OSS in the development, hence we can conclude that, none of the
have discussed early effort estimation for OSS Web projects.
Since we were not able to identify any existing studies that indicated the early effort model for OSS Web project, we therefore believe that
need for researchers to further explore this field. This is particularly relevant as OSS is being increasingly used nowadays by software pr
organizations. This can be supported by the paper of [78], even though the author in this paper only focuses on effort estimation for sof
development. However, the author strongly believes that there is a need to develop an effort estimation model especially for OSS proj
None of the studies discussed the need to develop an early effort estimation for OSS Web projects.From the findings, it is shown that there
no studies that discussed the need to develop early effort estimation for OSS Web projects. As can per seen in several papers recommendin
work, the authors only mention the need to improve the model or to conduct more detailed comparisons or propose the involvement of diffe
measurement attributes, such as [28], in which the future work is to conduct more detailed comparisons by using different estimation to
Since this study only focuses on the involvement of OSS tools towards the development, another future research area that can be investig
measuring the other aspect of the effort measurement attribute such as the year of experience of development toward the OSS as well the er
time.
In the future work, we plan to address the in-depth investigation of estimation accuracy by applying the estimation m
one typical OSS database, and identify benchmarks.
As is well known, a year of experience of the expert’s programming skills that are involved in the projectdevelopment can contribute to a
effort, therefore how about a year of experience of the expert toward the tools. Does this affect the effort estimation? Since OSS is an open
Studies
which everyone
that can quantitatively
can access freely,
infertherefore
OSS maintenance
a bug that effort
can occur
fromduring
size-related
the implementation
metrics are needed.
cannotTo bemitigate
avoided.the Asdifficulty
such, whatof would
acquiring
be the
ac
data from incomplete development effort needed
records,
to fix
Yu the
et.albug
[40][41][23]
and how this
focused
can affect
on predicting
the effortthe
cansize-related
also be further
metrics
investigated.
to indirectly estimate the m
effort. The strong correlation between size-related metrics and the actual effort has been confirmed in closed-source projects [28]. However
exists a gap between size-related metrics and time-aware effort for maintaining OSS projects. There is a need for studies that can quantitati
OSS maintenance effort from size-related metrics. Furthermore, the effort drivers used in general maintenance effort estimation models can
example to improve OSS maintenance effort estimation. For example, Nguyen [28] developed an extension to COCOMO II [9] size and
New evaluation methods are needed to validate the correctness of these estimation methods. With the growth of more companies develop
estimation models to capture various characteristics of software maintenance in general through a number of enhancements to the COCOMO
collaborating with OSS projects, estimating maintenance effort has become a major interest. More researchers have been focusing on impro
to support the cost estimation of software maintenance. Some effort drivers such as DATA (Database size), CPLX (Product Complexity), a
estimation towards the direct effort of OSS projects from both people and activity aspects by developing maintenance effort estimation m
(Platform Volatility) in his study might contribute greatly to OSS maintenance effort estimation.
However,
It will be worthwhile
since most OSSto explore
projects
thelack
capability
of complete
modeldevelopment
for OSS developers.
records and
Sinceactual
mosteffort
OSS data,
projects
it isrely
veryondifficult
task or to
issue
evaluate
tracking
andsystems
validatetot
the projects,
of these methodsrecognizing
by comparing the the
timeestimated
of specific maintenance
results with the tasks
actualcan provide
effort. Thisbetter
can bedecision support
a significant for task
threat assignment
to these as well
estimation as OSS
methods andp
management.
risks A large
to effectively amount
validate of of studies
their areThis
results. devoted
is antoissue
predicting
where bug fixing
we need time
new while a small
evaluation methodsamount
that focused on other
can validate activities such
the correctness as co
of these
and duplication identification. These studies commonly used metrics from methods.
source code changes and issue reports as predictors, which indica
prediction results are basically related to the characteristics of the tasks. There are two kinds of targets among these predictions. One is the n
days of an activity, evaluated by PRED(25) or PRED(50). Another is the bins of categorical time evaluated by accuracy, precision, recall, or
The prediction results for numerical days are not very satisfying, while the results of categorical bins are relatively high. In OSS projects,
recorded on issue tracking system and repository may not correlate with the actual effort because the developers are voluntary and self-det
whenIndividual
implementing
contribution
the tasks.
and performance
It will be worthwhile
measurement to explore
also has
thebeen
capability
receiving
model
attention.
for OSSGousios
developers
[15] and
defined
consider
a contribution
the developer-related
ratio by con
various type of parameters frombethe oneOSS
of the
community,
sources ofand predictors,
Rastogi as
et al.
an defined
opportunity
the contribution
to improve thein terms
prediction
of four
results.
different roles of stakehold
the importance of contribution measurement has been realized, it might be a promising research topic in the coming years.
As shown in Table V and Table VI, the SLR found relatively a few studies providing sufficient details regarding the advantages and chall
using OS in computer science education. Moreover, some of the advantages and challenges are supported by a few of the papers found. For
only 4 papers (19%) included specific evidence regarding the wider range of skills acquired through the use of OS, which suggests that ther
of data regarding the advantages and challenges of OS use in the practical situations. Consequently, the results of this study may not be
generalised and further investigations must be planned in practical situations where OS is currently used in computer science educati
The advantages and potential challenges of OS in computer science education have been explored using SLR methodology. In particular, the
OS in computer science education curriculum has been explored focusing on the four RASE areas of course design and delivery. The fin
represent a starting point in evaluating the use of OS in computer science from the perspective of education providers.
In addition, possible directions for future research have been suggested such as developing effective strategies for proper alignment of OS pr
courses as well as efficient evaluation approaches fitting into the specifc context of using OS in computer science education.
As Kirk and Miller stated in [22], “although no one defends a positivistic ontology, but scholars in social science has find out that much r
makes sense only in terms of a set of unexamined positivist assumptions.” Research in the field of OSS success has the same problem. We
precisely point to variables like:“general viewpoint of audience society” and “actual use of software3” as measurements of success and co
parameters such as “availability of knowledgeable developers”, “legal support and level of IT development in the development environm
affecting factors. That’s why we recommend mixed-methodology research in the field of OSS success.
As could be observed in fig 3, most of factors which affect the success of OSS are related to developers and product and most of success ind
related to product. Our study shows that user related factors have been studied less than other factors and researchers limited these factors to
downloads in both success factors and indicators. On top of this research gap, we highlight some other gaps that may help future researche
define and conduct their study in the field:
Although all research in the field of OSS success have tried to study previous work, but we observe little connection between them. One ex
reference [1] which has mentioned four previous works in the model and studied them in a longitudinal study and found some inconsistencie
original and current study.We believe that study of other work and comparing the results may lead to considering new factors (such as cont
longitudinal factors) in study of OSS success.
Except initial research by Crowston et al. [23], and Crowston et al. work on the definition of OSS success [21] , we do not find any general
OSS success. In fact many researches in the field have just tried to validate their partial model of OSS success. We believe that according
Ourrange of social,
research agendacultural
onandthistechnical factors thatinvestigating
topic involves may have an effect
soft-onware
success of OSS, developing
development processa general
tailoringmodel is not reasonable
(Pedreira et al., 20b
recommend contingency practicesin this regard. In other words we suggest researchers to develop general models for specific contexts and b
way to achieve
OSS development takes aplace
balance between characteristics
in an environment which is highly
these models of software
affected
would develop-
by socio-cultural
be more helpful ment models.
parameters
in practice. and Process tailoring
specifications is and
of users thedeve
act
particularizing
teams of OSS maya affect
general process
or alter description
the success parametersto derive
of OSS.aThat’s
new process applicable
while context to a specific
of development situation
is usually (Ginsberg
ignored while studyingandsu
OSS. Except [20] that studies specific kind of software and [4] that verifies 1995). the model in Korean software context, we do not find any other
We that was based
claim that itonisa specific
necessary context. Even these
to tailor these two papers have
processes triedspecific
to suit to generalize their and
project findings and the latercontexts.
organization one mentioned
Thisthe contex
tailoring
research as a limitation. So it seems that localizing the issue of success and paying attention to parameters
also consider the key distinctive features of plan-driven, agile and FOSS models: collaboration and discipline (Mag such as: social, cultural and econo
of development community would be beneficial point of view in future research.
Reviewing the related work in the field of OSS success, we observe different 2010a).measures and factors for success and noticed that different me
used in researchcan
Collaboration in the
befield but source
defined of data of
as a group is mainly
two orrepositories
more people of OSS projects to
working such as sourceforge.net
achieve a commonand freshmeat.net.
goal (Vreede and We m
B
recommend using variety of methods for research in the field and also want to draw attention of potential research to context of OSS develo
2005). Collaboration is an important factor for softwarefuture organizations
research.
to achieve their productivity, quality and know
sharing goals. In particular, software development is a complex process that involves collaboration among several peo
time to achieve a common goal (Cugola and Ghezzi, 1998). Therefore, software development is a typical examp
collaborative work (DeMarco and Lister, 1999). Discipline, meanwhile, relates to the planning level adopted in sof
process definition and the rigidity of control employed in process execution (Boehm and Turner, 2003b).
Both.Finally, it is also important
are complementary to investigate
and essential in anywhether
project, the
butresults of thisproportions,
in differing quasi-systematic dependingreviewonare consistent
project with w
characteristic
observed in practice, i.e., in organizational routine. To accomplish this, we intend
balanced mix between collaboration and discipline, it is necessary to understand how these aspects vary and disting to plan and conduct an experim
study.
software Thisdevelopment
experimentalmodelsstudy will involve aetsurvey
(Magdaleno of industry
al., 2010).In orderand academicprocess
to facilitate expertstailoring,
to validate thepossible
it is results and dis
to supp
Another
conclusions.interesting topic
A similar that deserves
strategy was attention in future research is the emerging body of literature on agile and distributed development (A
project manager by automating someapplied
of the by (Bekkers
steps, possiblyet al., 2008) the
reducing to determine
effort requiredthe factors that most
to conduct influence
this activity
al., 2005; Ramesh et al., 2006). The coming together of these two worlds was not explored in this study, but we consider that they can also
andtheim
the quality and appropriateness of the agile of software
resulting process development
(Park et al., model.Thus, we intend to develop an infrastr
2006).
to facilitate reconciliation among plan-driven, and FOSS development models.Likewise, the idea of processes diversity (where di
using anmay
processes optimization- based approach
be running concurrently (Magdaleno,
on the same project—in2010b).Finally,
multi-team projectsit oris changing
also important
over timetoduring
investigate
the phaseswhether the re
of the develop
this quasi-systematic
maintenance cycle) (Deck, review are consistent
2001; Lindvall with Siebel
and Rus, 2000; what etis al.,
observed in should
2003) also practice, i.e., ingated
be investi- organiza- tional
in future routine.
research, Toneed
since the acco
this diversity can be an important motivation for the reconciliation among software development models.Finally,
this, we intend to plan and conduct an experimental study. This experimental study will involve a survey of industr approaches that deal with
iation considering organization and project contexts and needs, appears to be a promising path for reconciliation in the future (Jaufman and
academic experts to validate the results and discuss the conclusions. A similar strategy was applied by (Bekkers et al
2005; Xu and Ramesh, 2008). This is the focus of the next section.
to determine the factors that most influence the selection of software development model.
The first opportunity for future research lies in reexecution of the protocol, to capture references to more recent work that extends the sear
chronologically. This could also include other search engines, such as ACM (Association for Computing Machinery), in an attempt to re
.In Section 7 (SQ2), we discussed how studies that showed how to combine two development models could be extended to the third. This
documents only indexed by these machines, which would extend the search space geographically. Finally, the search can also be expand
included ideas like: (i) the reconciliation of FOSS and plan-driven configuration management practices to agile model; (ii) the need for furth
manual searches to include: books; conferences; theses and dissertations; technical reports; and other search engines, such as Google Scho
on the reconciliation of the practice of continuous code integration between agile and FOSS models and extension of this search to the cont
AISeL (Association for Information Systems Electronic Library). Although the systematic approach adopted ensures the reliability and com
plan-driven model; (iii) investigate how the knowledge management practices in agile model can contribute to improve knowledge manag
of this study, it can be amplified by these extensions
organizations or in FOSS communities; (iv) analyze if the use of explicit knowledge management practices, coming from plan-driven an
development, can be beneficial in an agile context; (v) extrapolate the use of test driven development to a plan-driven context. These idea
opportunities for future research.An area that still deserves to be explored is the search for studies to reconcile the three models of soft
development, since only one study was identified in this quasi-systematic review. First of all, it can be present in areas that were not the foc
work. Some of these understudied areas are indicated below to guide future research. In addition, as stated before, the reconciliation researc
still at an early stage, but it is expected that in future there are more studies and results on the topic to be investigated. Finally, there also re
possibility that organizations or other researchers have already achieved positive results on the reconciliation among agile, FOSS, and plan
.In
stillSection
at an early
7 (SQ2),
stage,
webut
discussed
it is expected
how studies
that in that
future
showed
there are
howmore
to com-
studies
bineand
tworesults
development
on the topic
models
to be
could
investigated.
be extended
Finally,
to thethere
third.also
This
re
included
possibility
ideasthat
like:
organizations
(i) the reconciliation
or other researchers
of FOSS and have
plan-driven
already achieved
configuration
positive
management
results on the
practices
reconciliation
to agile among
model; agile,
(ii) theFOSS,
need for
andfurth
plan
on the reconciliation of the practice of continuous code integration
models, between
but have agile and
not written FOSS
about models and extension of this search to the cont
it yet.
.In
stillSection
at an early
7 (SQ2),
stage,
webut
discussed
it is expected
how studies
that in that
future
showed
there are
howmore
to com-
studies
bineand
tworesults
development
on the topic
models
to be
could
investigated.
be extended
Finally,
to thethere
third.also
This
re
included
possibility
ideasthat
like:
organizations
(i) the reconciliation
or other researchers
of FOSS and have
plan-driven
already achieved
configuration
positive
management
results on the
practices
reconciliation
to agile among
model; agile,
(ii) theFOSS,
need for
andfurth
plan
on the reconciliation of the practice of continuous code models,
integration
but have
between
not written
agile and
about
FOSS
it yet.
models and extension of this search to the cont
This study suggests some directions for future research. Disruptive COSS revenue models, organizational aspects of COSS, localization of C
creation and maintenance of the user community are among them.
COSS is comparable to opensourcing. In opensourcing, the companies outsource to the open source community outside of the company [8
lowers the cost of development, because on the one hand, volunteer developers code for free, and on the other hand, users report flaws in th
[43]. Hence, the existence of an open source community is vital to the success of the open sourcing company [82]. In COSS also exists
community and somehow developer is effective and improving [3]. But, the problem is creating and sustaining such communities [41, 43]. T
it is imperative that the management of the creation and maintenance of the user community in future research be investigated; an issue whi
The next point is that, as Riehle noted, open source software can possess established markets, provided that it is sufficiently disruptive [1].
been seriously considered in the literature of COSS.
disruptiveness, we suggest working on the revenue model. In fact, since the business model is a system and every change in one compone
affects the rest [4], the initiation of this disruption can be a revenue model. Therefore, it is suggested that future studies in helping to descr
revenue models examine the role of new revenue models in adding to the disruptiveness of the open source software. This is because the exp
open source software revenue models has been difficult, especially because of free distribution [83]. In addition, this study suggests that the
of complementarities has been an integral part of the COSS business model as a way of earning money. So, new disruptive revenue models
The future direction for our work is to evaluate further new practices that
configuration of can
the be incorporated in the OSS project work st
The work structure of OSS projects COSSisbusiness
informal,
modelvaries in eachpossess
and probably project, and is dy-
established namic due to transient contribu
markets.
therefore, it is a challenging task to generalise a set of practices for allwith
Due to the cultural, economic, institutional, geographic and other characteristics of developing countries OSSemerging
projects.
markets [84], the
business models of developed and matured markets is often unsuccessful [58]. Therefore, the logic of the creation and capture of import v
COSS may need to be adjusted. This implies the need to localize the COSS business model [85]. In addition, countries are demanding indig
localized software, but due to expensive licenses, some of them are looking for open source software [86]; a localized open source software.
COSS business model and probably possess established markets.
literature of the COSS business model, not only the commercialization of open source software as a business model is not considered, but
Due to the cultural, economic, institutional, geographic and other characteristics of developing countries with emerging markets [84], the
also few studies on the open source software localization. Therefore, it is suggested. Finally, there is not much research about the organiza
business models of developed and matured markets is often unsuccessful [58]. Therefore, the logic of the creation and capture of import v
about what is happening inside a COSS company. Therefore, one of the implications for future research is that it is suggested that future r
COSS may need to be adjusted. This implies the need to localize the COSS business model [85]. In addition, countries are demanding indig
examine the structural dimensions and content dimensions [87] of the COSS organization.
localized software, but due to expensive licenses, some of them are looking for open source software [86]; a localized open source software.
Further
literature
research
of theisCOSS
required
business
to explore
model,
suitable
not only
KRthe
practices
commercialization
applicable inofOSS
openprojects
source as
software
indicated
as abybusiness
one of the
model
questions
is not considered,
raised on mecha
but
team
alsonorms
few studies
that areonused
the open
to store
source
knowledge
softwarecon-
localization.
tributed byTherefore,
team members.
it is suggested.
In CSS organisations
Finally, thereKRis not
mainly
muchcomes
researchintoabout
focusthe
when
organiza
an em
leaving
aboutan organisation
what (Lindvall
is happening inside& Rus, 2003).
a COSS On the
company. contrary,one
Therefore, in OSS
of theprojects the unpredictable
implications nature of
for future research is commitment from contributors
that it is suggested that future r
element of risk (Robles & Gonzalez-Barahona, 2006).
examine the structural In OSS,and
dimensions contributors can leave since
content dimensions [87] ofthey
theare
COSSnot under any contractual binding as
organization.
Organisations invest in KM activities to organise, create, share, reuse, transfer, and retain knowledge. We found knowledge relevant activiti
organisations.
projects namely knowledge creation and knowledge sharing. Knowledge sharing was found to be abundant but there was no evidence of kn
In this literature review, we found that KM relevant activities of knowledge creation and knowledge sharing are evident in OSS projects as
retention to reduce the impact of knowledge loss in OSS projects. Moreover, knowledge sharing is reactive in nature, initiated by the contrib
in Section 6.1. Furthermore, literature examination directed us to 10 mitigations to reduce the impact of knowledge loss due to contributor
looking for task relevant knowledge. We suggest that there is insufficient attention paid to KM in general in OSS, in particular, there would
OSS projects discussed in Section 6.2.
be an absence of proactive measures to reduce the potential impact of knowledge loss. We also propose the need for a KM evaluation metri
projects similar to the ones that evaluate the health of online communities. KM evaluation metrics should be based on the extent of knowled
activities observed in a project. Such a metric could help to inform potential consumers of the OSS of the KM status on a project, somethin
non-existent today. We consider it a vital ability for OSS projects to sustain a knowledge-sharing culture that will support the long-term sur
competitiveness of OSS projects.
Utization of designs and fail at espousing an alternative knowledge sharing economy. This points to a gap in open design research, where th
keeping design solutions open, accessible, replicable, and adaptable while conforming to safety regulations and standards is a challenging t
remains mostly unresolved. Co-operatives and similar models may suggest community-based ownership and responsibility, but this model
open as open design is espoused to be enacted. This affects the reliability of these design solutions, especially when they are not widely re
online. Although larger transitions towards alternative economic models are discussed on the macro level, research on how they will be en
development, iteration and dissemination of open designs is still an important area of interest.
In the reviewed literature, open design is indeed framed as a better alternative by many authors, especially on topics proposing new way
business, prototype alternative economies, and foster sustainability. From a strictly business perspective, the potential of open design is o
mainly as a value-capture strategy and a way to achieve rapid innovation cycles for further development and wide-scale testing. Howeve
Moreover, it remains to be seen if open design as a research framing remains semantically and ontologically tied to trajectories of business-a
design’s relation to enterprise is still largely considered within the current paradigm, while the potential of an open design ‘sharing economy
has been seen in software; see Morozov, 2013), and therefore not a true alternative nor necessarily democratizing; if it is increasingly emb
generally discussed as a way to transform the way businesses operate. Toward the manufacturing side, companies open up their initial proce
research on alternative post-capitalist and postcolonial practices; or if a new term becomes more appealing to the research community and r
not develop alternative models befitting the sharing economy as suggested by open design.
entirely.
in the study with a development component but oriented to a specific requirement, 16.13% adds tools to the central axis of the GIS Web arc
and 9.68% integrates new methods and algorithms to improve different aspects of architecture.
Open Source Web Software Architecture Components. Hence, the need to carry out new research aimed at evaluating and improving the co
of the Open Source software architecture of a geographic information system in a Web environment.
but have the goal to meet customers’ expectations. For this reason, many of the lessons learned from OSS projects ca
adopted in other types of software projects regarding the use of rapid releases.
In response to RQ2, we considering
As future work, we are found that theusing
OSSthe resultsfocusing
studies of this study to buildprocess
on evolution a meta-model forused
support the mining of methods,
different open sourc t
in view of gathering data that lead to assessments of the quality of projects adopting the Frequent
approaches for OSS evolution process support. OSS Evolution process support studies have usually focused on Release approa
evo
models, exoge nous factors, maintenance support, fault detection and change propagation aspects. A very little effort
paid to the other aspects such as Configuration Management, Growth, Complexity and Control, possible evolutionar
Figure 3 depictsisthat
etc. SVN/CVS around
again 62%
found of the
to be the largest
articlesexplored
used statistical
dataset.methods such as
The detailed regression,about
explanation timethese
seriesaspects
analy- is
sis,pres
co
analysis, Pearson coefficient, Spearman’s rank correlation, Sect.principal
4.4. component analysis and Bayesian belief netw
Statistical analysis methods are found to be more reliable for predicting different aspects for OSS studies as compa
other methods. The second large set of methods employed falls into the category of machine learning algorithms.
methods such as mathematical models, probability methods, Chaos theory and SRGM’s (Software Reliability Gr
Models) on
It reflects that most authors performed experiments have very low
different openexploration.
source projects and comparison of results of d
studies become arduous. There is a need of common corpus regarding evolution prediction of open source projects
be used by the researchers for comparing results.
Researchers have contradictions on the predictive power of metrics used for the evolution of OSS studies. These contractions are discussed
4.2. There is a need of further research to empirically evaluate predictive power of different metrics. Most metrics are applied on the file lev
evolution prediction of OSS studies as highlighted in Table 19. We also analyzed that class level metrics are applied by few studies but me
metrics are applied by none of the selected primary studies. Moreover, we also reveal that code level metrics are applied by most research
evolution prediction of OSS studies. Little attention is paid to requirements, design and architectural level metrics for predicting evolutio
studies.
In both cases of evolution prediction and evolution process support, the reviewed articles admitted the necessity of external validity, wherea
of articles not addressing validation process is considerably higher (56% for evolution prediction and 68% for evolution process support). W
that vast heterogeneity of evolution prediction models building data make their evaluation difficult. Generalization of evolution prediction
regardless of their applicability on project size requires the attention of researchers.
Future research should also focus on predicting change propagation, size, refactoring, maintenance effort, contribution evolution, SOC a
evolution besides defect prediction.
During our research, we found only a small number of studies on the use of OSS in dentistry. Although the conclu
drawn from studies support the use of OSS, most of the articles presented a low level of evidence (3b-5) and poor q
reporting, which makes it difficult to recommend OSS as a clinically useful software. The only study with high-q
reporting was a case-controlled study but the conclusions were based on very small research group, three teeth, in w
The usability of OSS was evaluated only
does it not in aitfew
make areas:
possible to mental
say thatforamen localization,
the results upper airway calculation in g
are representative.
patients, an experimental assessment of hard debris in the root canal system after root canal treatment, information g
(RSS), practice management, and use for educational purposes. None of the studies revealed the use of OSS
prosthodontics which with
Four of five studies is currently the most
the highest level intensively
of evidencedevel- oping areaclinical
(2b) examined of digital dentistry.
changes Virtual planning
in orthodontics and de
(influence of
prosthetic reconstructions provide many opportunities for OSS solutions.
orientation on directional changes in 3D space; orbital volume and aperture width changes after rapid maxillary (exp
(18;21) and endodontics (canal transportation) (22;23), but none of the identified studies included a comparison b
OSS and any other software.The only level-2b study that included a comparative analysis of a commercial and open
software was assessing software precision in airway volume measurement (27). The study was based on 33 particip
lead to the conclusion that the use of Osirix and ITK-Snap OSS presented clinical value.
The architectural and requirements change evolution is also undermined area that needs attention of researchers.
The tools and approaches proposed for evolution also admit the necessity of external validation. Future research should focus on the above m
issues and try to make them more generalized regardless of the domain of OSS projects.
Our review results showed that the medical literature on the topic of OSS in dentistry is limited and includes mostly expert opinion and cas
studies.The second suggestion is to include OSS as a control group in experimental studies on software validation. Such comparative ana
have positive effects for the commercial programming vendors by showing them the most advantageous OSS solutions that can be deplo
commercial software packages.
It may also be beneficial for customers who, apart from obtaining detailed information on the performance of software packages, might be
decide if the risk of using OSS without technical support and requiring greater computer skills is justified in specific cases.
The evaluation of OSS in medicine should not be overlooked. Professional medical certification (FDA and CE) often cannot be applied
because it requires a legal commercial entity for distribution liability and support availability. It is also important that FDA and CE cert
confirm that the software and its provider ensure an error-free work- flow, but not the accuracy of software actions/calculations, and pr
appropriate documentation and support. Therefore, experimental studies and algorithm evaluations should be conducted and publish
s future work, we intend to model OSS quality assessment as a MCDM problem. This will afford us the opportunity t
from a range of MCDM methods one (or more) that can be used to evaluate quality in OSS across multiple doma
This study could help researchers to identify essential quality attributes with which to develop more robust quality models that are applicab
various software domains. Also, researchers can compare the existing selection methods in order to determine the most effective
Based on the comparison of the existing quality assessment models, there is clearly no suitable model—each model has its own limitations. A
the findings of this analysis have implications especially for practitioners who work towards coming up with new assessment models. The
note the following points in line with the research questions posed in this study: Emphasis should shift from trying to build comprehensiv
(containing all the possible software characteristics) to building models that include only essential quality characteristics. This study has sh
these essential quality characteristics include: maintainability, usability and maintenance capacity of software community. By narrowing dow
three essential quality characteristics, model developers would help to reduce the burden of OSS evaluation via existing quality assessme
The most commonly used selection method is the model approach and the least considered are the tool- based and data mining approaches.
which has been referred to largely as being laborious and time consuming to conduct
interesting result is that nearly half (47%) of the selected papers do not mention an application domain for the models in their research. Mor
should be paid to building models that incorporate only essential quality characteristics. Also, framework, tool-based and data mining se
methods should be given more attention in future model proposals.
This quality model can be used as a starting point for the quality assessment of an OSS ecosystem, and it is in our p
the future work to define a complete quality assessment process (as described in Sect. 5) and to apply it in a real q
assessment. As consequence new measures may be needed for the assessment, but this is the best way to improve
complete the quality model, and a way to prove its capabilities in quality assessment.
we commented in Sect. 4.2 the lack of measures for process maturity because in this case the assessment needs to b
with qualitative evaluations of the community. Since we have focused on quantitative measures, there may be ot
characteristics of the quality model that require or that may be complemented with qualitative evaluations.
The most commonly used selection method is the model approach and the least considered are the tool- based and data mining approaches.
interesting result is that nearly half (47%) of the selected papers do not mention an application domain for the models in their research. Mor
should be paid to building models that incorporate only essential quality characteristics. Also, framework, tool-based and data mining se
methods should be given more attention in future model proposals.
Assesment Process: It is worth mentioning that to perform a complete quality assessment of a software ecosystem we first would need to d
assessment process which is out of the scope of this paper. The quality assessment process will have to deal with, e.g., How are the value
measure interpreted (i.e., defining what are the good and the bad values)?; How can the measures be merged to provide the assessment for a
sub- characteristic of the quality model?; or What are the principles to perform the assessment with missing, incorrect, and/or inconsistent
data? We are will provide the answer to these and other questions as part of our future work in this topic.
Unbalanced Distribution of Measures: just by looking into the measure tables, it is easy to observe that the amount of measures for so
characteristics is high (e.g., activeness with 17 mfeasures, visibility with 11 measures) while for other is very low (e.g., heterogeneity with 1
information consistency with 1 measure). This unbalanced situation could be an indicator that more research is needed for the characteristi
low amount of measures.
We plan to perform an exploratory mixed-methods research using OSP in an undergraduate course in SE, in order to
new insights with an experience with this approach. We will experiment with a combination of inside control an
predefined
One of our mainproject in a software
findings evolution
is that most studiescourse,
are notusing
deeplya combination
concerned with of different
researchlearning
methods:approaches.
among the We also ps
relevant
investigate better methods to assess students’ learning in this context.
papers, we may cite two case studies, one action research, but no experiments, which seems to be in sharp contrast w
recent growth of evidence-based SE. Software engineering research is slowly adopting increasing scientific rigour in
years. Moreover, SE education is an interdisciplinary area. As such, it can strongly benefit not only from SE rese
methods, but also from research methods from areas such as sociology, anthropology, pedagogy and communication.
seems relevant to identify which research methods are more appropriate in this intersection, in order to achieve bette
Despite the scarce references to learning assessment in the selected papers, some studies assessed the experiences from both the teacher and
perspective. They also used various different instruments. in
Thethis interdisciplinary
main area.perspective to assessment is the absence of clear
issues with the teacher
Ellis, Morelli, de Lanerolle, and Hislop (2007) point out that students can perform various types of contributions. Since they have differen
of criteria to assess students’ products, performed tasks, and expected skills and attitudes. Therefore, it is important to state that student ass
grounds and previous experiences, they fulfil different roles and perform Computer Science Education different contributions. Therefore, th
deserves more thorough work.
recognize that grading is not an easy task. They suggest the need to establish a set of metrics for each role a student plays.
Learning how to solve complex problems, and knowing how to work in teams are relevant skills that are typical requirements in SE educati
students should develop some skills such as communication and leadership. Peer assessment is one important instrument to approach this ne
and de Lanerolle (2009) sustain that students should assess their peers in conjunction with teacher assessment in courses on OSP, even tho
Some trends and issues emerged from a detailed analysis adoption
of the
of this
studies:
practice
(i) solution
is still aproposals
challenge.are the main research approach; (ii) very few
focus
In anyonactive
specific SE areas;
learning (iii) the
approach, traditional
students project method
are responsible is the main
to conduct their learning approach;
own learning. (iv) mostevaluation
A formative studies use previously
that includes chosen OSP
self-, peer ani
courses; (v)can
assessment there
playis an
a balance between
important role ininside and no control
this process. approaches;
According to Ellis etand (vi) verythe
al. (2012), few papers devAll
iterative use criteria
thosetoissues
evaluate students’
represent learning
research op
either
Givenoutcomes or developed
the large variety skills.
of OSS Wesystems,
also foundwith
three
to main thoroughly
be more combinations
different of OSPinuse:
sizes,explored
domains and(a)
the full control and
future.
complexity, wepredefined projects,
believe they are (b)
an no
impco
free choice projects and (c) inside control with no or almost no project choice for students. These trends and issues provide future directi
source of examples to teach software design, architecture and quality (Brown & Wilson, 2012). Nonethe- less, we fo
research.
few studies to describe these issues in, say, a case-based learning approach. Only three reported this type of experien
of them related to software design and architecture, and one of them, to software evolution.
No selected study focuses on learning the areas of requirements or configuration management, despite the large u
configuration management and issue tracking tools in OSP. We believe that these specific areas may benefit from an
We plan to perform an exploratory mixed-methods
engagement withresearch using
OSP and OSP
their in an undergraduate
associated tools. course in SE, in order to
new insights with an experience with this approach. We will experiment with a combination of inside control an
predefined project in a software evolution course, using a combination of different learning approaches. We also p
investigate better methods to assess students’ learning in this context.
As future work, we plan to develop a method that extracts users’ software requirements automatically from internet r
and to design automated processing to support requirement prioritization.
We are interested in using a nonintrusive approach to OI in the context of RE by using fully automated approaches
companies fuel their RE processes with innovative ideas created outside their company boundaries. Following this
research, we have designed an automatic sorting algorithm based on the Kano Model [35]. We believe that the algori
be used in the process of automatically prioritization users' requirement and thus eventually may become an esse
cornerstone in a fully automated, non-intrusive approach to OI used for requirements extraction and prioritizatio
Regarding RQ 2 and RQ 3, we found that there seems to a lack of research on the use of OI for requirements prioritization and requirements
as there was only one paper dealing with these topics each, i.e., papers R_11 and R_19, respectively.
In addition, we found only one paper (paper R_18, dealing with OI in the context of requirements extraction) that presented a solution appr
tool support. This indicates that there is little automation support mentioned in the literature on the use of OI strategies in RE.
Acknowledging the lack of published research on the use of OI strategies in specific RE activities, i.e., prioritization and validation, as well
of reported tool support, we see new opportunities for research on automated and thus nonintrusive and low-cost methods for applying OI
Our
in RE.aim
Moreis specifically,
to define awe process that
think this improves
mapping studythe selection,
provides us withmanagement
a motivation toand maintenance
conduct more researchof OSS components.
on capitalizing upon OIWe to wi
aut
existing COTS quality assessment processes, including theprioritize
extract and ones enacted in the project partner companies in practice.
requirements.
particular focus on the differences between OSS and closed source components identified in the SLR, we try to und
the causal relationships between risks and failures, risk metrics, and risk mitigation. To propose such measures we in
into two directions: (i) evaluating the OSS project community ecosystem and the adopting company, by organisat
modelling and analysis, and (ii) evaluating the data available in the OSS projects repositories, such as code repositor
trackers, and mailing lists, applying measurements based on a statistical analysis. Techniques and tools for this analy
prioritisation and multi-criteria decision making will be presented in the next section.
To propose such measures we investigate into two directions: (i) evaluating the OSS project community ecosystem
adopting company, by organisational modelling and analysis, and (ii) evaluating the data available in the OSS pro
repositories, such as code repositories, bug trackers, and mailing lists, applying measurements based on a statistical a
Techniques and tools for this analysis, the prioritisation and multi-criteria decision making will be presented in the
section.
Meiszner et al. (2009) point out some learning features of the OSP experience: self-learning, project-/problem-/inqui
learning, collaborative learning and reflective practice. However, very few studies cite these learning approaches, a
of them provides details on how to design and implement pedagogical practices that result in effective learning of SE
Some of the analysed approaches propose quantitative measures and analyse their effectiveness (e.g. [SLR4][SLR6][
in the SLR for API metrics and code changes, [SLR14] for code executability, [SLR25] for business values). [SL
proposes a reliability model combining qualitative and quantitative metrics, but does not consider OSS-specific com
Few publications in this field speak about riskand repository
mitigation. measures.
Mostly, mitigation of the various risks encountered in
adoption is only mentioned informally, in form of general hints, such as to train the people, i.e. to develop the existin
of human capital [SLR21], to follow general COTS adoption decision processes, to evaluate the community [SLR2
evaluate similarity to previous projects [SLR44], to evaluate the OSS project’s roadmap and possible future direc
Empirical experiments were used tooridentify
[SLR46], to makerisks and to aware
managers identify
of effective
risks andmitigation activities.
opportunities [SLR37].However, none of th
showed evidences for causal relationships between these risks, concrete measures and the effectiveness of the miti
activities. Only for more concrete risks, as e.g. for lowering the risk of introducing errors when upgrading to new ve
Notice that
concrete there are
mitigation a huge number
activities of publications,
were proposed, which report and
such as automatically interpret
checking APIresults from qualitative
compatibility [SLR4] orandexplorin
quanti
studies to identify possible risks. It stands out that only [SLR40] calculates threshold values
executability with test cases [SLR14] to ensure correct functionality. for defining bug risks
evaluates their performance, while no paper identifies risks based on quantitative data of project failure or created lo
revenues, or correlates project failures and losses with quantitative data such as the number of bug reports and bug
Few papers consider quantitative measures on community qualities (number of contributors, activity, presence of h
Moreover,
[SLR39] anwhile
etc.), empirical evaluation
no work of theevaluates
empirically effectiveness of mitigation
the existence activities
of causal by their influence
relationships between theon metrics
the retrieved me
applied,
also missing in the works, which propose identified
specific mitigation activities. Also for
and the actual faults happening. very complete works, such as the one
group [SLR12][SLR19][SLR20][SLR47], whose surveys retrieve data for risks and mitigation activities, do not sho
link between these two: typical mitigation activities adopted in software companies are very general (see Section 1.2
would reduce various risks.
we claim that emergent research addressing the current mobile-platforms is not considering, or exploiting previou
nal works on open-source platforms, as it often should.
in RE. More specifically, we think this mapping study provides us with a motivation to conduct more research on capitalizing upon OI to aut
extract and prioritize requirements.
in RE. More specifically, we think this mapping study provides us with a motivation to conduct more research on capitalizing upon OI to aut
extract and prioritize requirements.
When contrasting previous literature on older “platforms-wars”, such as the ones from the PC and game-console industries, with the current
studied mobile platforms-war, we empirically notice that many of the market players remain the same (Microsoft and Apple). There is a sc
convergence: same firms push for similar technological standards across different platforms, i.e. Microsoft Windows within X-box, Surface
PC, Netbooks and Mobile phones. This convergence between industries remains unexplored by academia. Interesting research questions de
the implications of such convergence remain unexplored, i.e “should firms concentrate on one platform-war or run several platform-wars in
As a future work, this relationship will be reviewed in-depth with respect to RQ4 in order to expand the information
by this SM study and elicit further insights.
Our answer to RQ1.1 indicates that the majority of the articles contribute experimental (case) study to deal with eva
the quality and success of OSS. There are a similar number of articles that contribute new methods / techniques an
small number of articles that introduce tools. The lack of tool development can be interpreted as a lack of agreemen
concrete method for measuring the relationship between the success and quality of OSS. Response to RQ1.2 support
RQ.1.1. a large number of articles are classified as solution proposals. T
As observed in the study, code quality is seen as the most important criterion for measuring quality. On the other hand, market success and
activity seem to be the most important criteria for measuring success. Therefore, in future, researchers have to study how code quality may
measure software success, or how market success and developer activity may be used to measure software quality. Only then, it might be p
talk about significant relationship between OSS success and quality.
As discussed in Section IV.D, there is no concrete relationship set between quality and success criteria of OSS in the reviewed papers. The
studies examining the relationship between quality and success criteria and metrics should encourage the researchers to conduct further stud
context. In addition, there is no satisfactory number of studies in the contribution types of model, metric or tool, and it is observed that the
Sub-project evolution with their community. Large open source projects often encompass many sub-projects. Such as, sub-projects in Eclip
evident necessity to fill the gap for these types.
Linux, and Apache. Often ecology of sub-communities formed around these sub-projects, which are governed by a common governance [50
the formation and evolution of sub-projects and their communities have revealed many key characteristics, which are listed in Table V and
Since measuring the success or quality is a challenging task, especially ON CO-EVOLUTION
tool contribution is quite minimal. This may lead practitioners to co
respectively. Yet the interdependency in evolution between the ON twoCO-EVOLUTION
and their impact on the overall project evolution remain untouched. The
It is turned outwith fromresearchers
our reviewinthat orderthetounderstanding
clarify terminology,
of co-evolution
identify metrics,
of the code andanddevelop
the community
tools that are in OSS
capableprojects
of meeting
has received
this need.
little at
would be worth to investigate.
•ItDoes
is turned outafrom our review
betweenthat the understanding of co-evolution
change) ofofthethe code andand thetheir
community
associated in OSS proje
literature (Figure 2). As a consequence, the community dimension and corresponding communication channels (e.g., mailing archives, bug
there exist correlation the evolution (growth, complexity, sub-projects sub-communiti
systems) are explored seldom, as can
received little attention in literature be seen from Figure
(Figure 5 and Figure 6 respectively. Study on co-evolution in OSS projects, however, is b
the commu- nity2). As awith
change consequence,
the change inthe thecommunity
sub-project? dimension and correspondin
increasingly popular. Because, in such projects the code evolution is dependent on the contribution of community members, and that a su
communication channels (e.g., mailing • How archives, bug tracking
does a community systems)
form around are added
a newly explored seldom, as can be seen from Figu
sub-project?
evolution of the code is required for the survival of the community. The following research directions can be considered relevant. Explorin
Figure • What attributes of a sub-project attract new is developers to increasingly
join?
technical congruence. In the OSS projects contributions made by the community members not only drive the system popular.
6 respectively. Study on co-evolution in OSS projects, however, becoming evolution but Because,
also red
projects the •
code evolutionWhat happens to the sub-community when a sub- project is deleted or merged to other sub-project?
role of these contributing membersisand dependent
thus change onthethesocial
contribution
dynamics of ofthecommunity
OSS community members,
[53]. Inand this that a successful
connection, it will beevolution
very inte
code is required
investigate for the survival
the phenomenon of thecongruence
socio-technical community. in OSS The following
projects.
tion? research directions can be considered relevant. Ex
Socio-technical congruence which is a conceptualization of Conw
[67] states that therecongruence.
socio-technical should exists aInmatch the OSS between the coordination
projects contributions needs established
made by the by the technical domain (i.e., the
notarchitectural
only drivedepen
• What kind and level of communication andcommunity
collabo- members the
the software) and the actual coordination activities carried out by project members (i.e., within the members of the development team) [67
evolution but also redefine the role of these contributing
ration takes place between members and thus change the social dynamics of the O
sub-communities?
concept was already explored in closed source projects, and reported a high correlation with software build success, quality, and faster r
community [53]. In this connection, it will be
• Does very
there interesting
exist a correlation to
modification [68]. Thus socio-technical congruence plays a pivotal role in conceptualizing the co-evolution investigate
between thethe phenomenon
project socio-technical
in a project. Surprisingly, congru
this n
OSSresearch
projects. evolution and the sub-project evolution?
area has not been iven much attention among open source researchers. Although it is identified and reported as a desiredshould
Socio-technical congruence which is a conceptualization of Conway’s law [67] states that there prope
match between
collaborative the coordination
development activities likeneeds OSSestablished
projects [69].by the technical
Considering the lackdomain
of focus(i.e., thedirection,
in this architectural
we propose dependency
the following in the s
to inv
and• the Doesactual
the essence of socio-technical
coordination activities congruence
carried as outa conceptualization
by project members of Conway’s law holds
(i.e., within theformembers
OSS project? Candevelopment
of the it be stated as antea im
characteristics or property of successful
This concept was already explored in closed source projects, and reported a high correlation with software build su OSS project?
Metric
• Whatsetquantitative
for software evolution. Software
approach/method can be evolution
utilizedstudies
to verify mostly utilize metrics
the existence that are empirically
of socio-technical congruence validated
in OSS inprojects?
prior studiesWhat (asrepositori
presente
quality, andmetrics
III). These fasterare rate of modification
derived for closed source [68]. Thus and
projects, socio-technical
are primarily
used congruence
used
for this purpose? to verify the plays
Lehman’sa pivotallaw ofrole in conceptualizing
software evolution. Thoug
metricsevolution in a project.
provide valuable
• What insightSurprisingly,
correlation to OSS
can beevolution,
derivedthisbetween
notion
they do notasconsider
socio- a technical
research area has not
the community
congruence been
dynamics.
and the iven
Thus an much attention
empirically
quality/sustainability among
ofvalidated
OSS set open
projects? of metriso
Framework for the data collection and representation. As discussed in RQ6, OSS projects often produce large volume of data representin
researchers. Although ofitexplicit
is identified and
representation reported as
of the communitya desired property
is required for collaborative
to complement the existing
development and evolution history. Research to date, explores the repositories that maintain these data, a list of which is provided in Fig development
metric set. activities lik
However, data projects
collection[69].
and Considering
representation in the lackrepositories
these of focus vary in this direction,
significantly weproject
from propose the following
to project. Furthermore, to investigate.
data from the sam
may• Does the essence
have different of socio-technical
formatting (e.g., emails are congruence
often free of formatas a conceptualization
even in listing the senders of Conway’s
credentials). law Dueholds
to theseforfacts,
OSSit project?
is a challeng C
collect relevant data following stated as an format
a standard implicit from characteristics
OSS repositories. orInproperty of successful
this context, researchers often OSSemploy project? their own means to coll
represent data for research.
• What quantitative This reduces the compatibility
approach/method can be utilized and comparability
to verify theofexistence
the reportedofresults even if they use
socio-technical same data sources.
congruence in OSSTak pr
issues in consideration, a framework for uniform data collection and representation can be developed to make the results cohesive and comp
Study the existence of SOC. Another direction What repositories
of research would be can be
to study
each other.
used for
the notion thisof purpose?
SOC (Self Organized Criticality) in OSS projec
dynamics• What articulate that thecan
correlation current state of a between
be derived project is determined (or at least,
socio- technical heavily influenced)
congruence and theby events that took place long
quality/sustainability time ago.
of OSS projE
exploration of SOC in the domain of OSS projects reveals contradictory results (Table V). Thus future research can take further step in vali
Migration of responsibility and sustainability. It has been reported that migration of developers from one release to the next is high and
existence of SOC and its implication on the evolution of open source software.
developers take more responsibility as they gain experience. Yet it is a common phenomenon in open source domain that developers freel
leave the project. And when a developer leaves, his responsibilities must be assigned to someone else. For instance, the codebase maintain
outgoing developer should be taken care of by others. Else it will be abandoned and discarded from subsequent releases. Thus it will be ben
ONexplore
RESEARCH METHOD
the followings,
• How A responsibility
number of issues migrates
related among
to thethe
research
developers?
approach Does canthis
be migration
improved followto increase
preferential-attachment?,
the acceptability of the i.e.,
reported
is the responsibility
results. We pointed
handedou o
developers who are in closefollowings, connection to the outgoing developer.
External validity of the results. Empirical •study Whatisimpact
the most such
popular
migration
research
has onapproach
the projectemployed
evolution?in evolution studies (Figure 4). These s
however, are horizontal in nature (as reported in RQ5) considering only flagship OSS projects. Due to this approach of studying OSS proj
reported results suffer from generalizability threat, as reported in Figure 7. Yet to make these finding applicable and hold for the extended
OSS projects, explicit measure should be taken. An interesting route to deal with this is to categorize the findings (current or future) accord
project domain, or similar organizational structure and practices, or similar product size and complexity. This will reveal the broader picture
Predicting the future. Prediction of OSS projects is one area that is least popular among the study facets (Figure 2). Yet future research shoul
then be compared and possibly merged for proposing a more general evolutionary pattern and behavior for OSS.
developing reliable prediction models and methods supporting ON COMMUNITY
error prediction, EVOLUTION
measuring maintenance effort and cost of OSS projects. Be
Study oncommercial
the community organizations,
evolution for identifies
instance, several
requires
keysuch
properties
prediction
(reported
modelsin Table
to assess
VI),anwhich
open lay
sourcethe foundation
component for for further
adoption research
[66].
direction. We propose the followings to be investigated.
Community building. Studies reported that the majority of OSS projects failed to attract members to attain the critical mass. Only few flagsh
are able to attract developers. Factors influencing the motivation to join a community has been studied (e.g., [62] [50]), and several phenom
proposed. For instance, rich gets richer phenomenon. Yet it is not identified what exclusive properties initiate the community building proc
nebula stage of the project. Following research questions can be considered relevant,
• Why some projects are able to attract contributors during the nebula stage of the project, while most of them can not?
• What formation of the community refers to a balance one, and how the community structure changes towards a balance structure dur
evolution? Can a visible pattern be identified within the domain of OSS projects for the above two cases?
Comprehension of these results suggest that the laws and theory appear to be breaking down through non- conforming data and findings (T
Thus Lehman’s laws of software evolution which is primarily based on the study of the large close source systems, is not sufficient to ju
account for the evolutionary pattern and behavior of the open source software. As none-the-less these laws did not consider the community
of the OSS projects which is an integral part of sustainable evolution of the open source softwareTo deal with this problem, a viable route w
examine the underlying ontologies for software evolution [5] considering the OSS specific characteristics, and then re-assess the laws of s
evolution to fit in OSS domain.
More empirical and theoretical research is needed on its applicability in context of OSS certification. Important research questions include
certification concerns shape and impact OSS communities?” and “How to organize open source communities for effective and economic cer
“
The availability of source code for OSS components provides opportunities for scrutiny by third party certification bodies. However, the co
size and evolving nature of many OSS projects severely limit the practicality of such efforts, unless the software is developed “with certifi
mind”. Cotroneo’s pre- certification kit [7], Comar et al.’s Open-DO continuous certification process [6], Fusani and Marchetti’s virtual cer
repository [10], Kakarontzas et al.’s OPEN-SME reuse process [11] are examples for approaches to develop “for certification”. Some of
proposals can be considered complimentary, others are alternatives. Little empirical evidence is available to-date about their effectiveness in
As an
Our increasing
answer numberindicates
to RQ1.1 of OSS systems are subject
that the majorityto certification and may
of the articles consider experimental
contribute these proposals, (case)
the community
study towill
dealneed moreeva
with em
the quality and success of OSS. There are a similar evidence on their
number of effectiveness.
articles that contribute new methods / techniques an
small number of articles that introduce tools. The lack of tool development can be interpreted as a lack of agreemen
concrete method for measuring the relationship between the success and quality of OSS. Response to RQ1.2 support
RQ.1.1. a large number of articles are classified as solution proposals. T
Consequently, the healthcare industry is “lagging behind in terms of adoption of modern ICT tools and infrastruct
compared to other sectors (Karopka et al., 2014; Munoz-Cornejo et al., 2008).
OSS has long passed the market introduction stage but has not yet reached the maturity stage”. This has been partic
relevant for public administration where FLOSS’ technological immaturity, lack of interoperability with existing so
Software engineers in last decade
and decision makinghave been interested
influenced in agileposes
by politicians methodology and barrier
a significant open source
for itssoftware development.
wider adoption
them present some new features and they seem beneficial for better and faster software development. By doing an S
were looking for relationship between ASD and OSSD. Fortunately our study shows that both ASD and OSSD can h
other and collaborate in doing software projects by sharing their practices. There are enough evidences that agile an
source practices can support each other, mainly because of some of their common concepts and principles. Also, ho
there are a few successful experiences on integration of ASD and OSSD, but, most of the studies are optimistic in po
Weof envision future work
their integration, couldisperform
but there a deeper
no empirical analysiscase
successful and synthesis on the empirical
study for supporting research
this idea on ISSproducing
in software developmen
in
on this analysis and synthesis, we will further investigate the limitations of the current research on ISS developme
establish a research agenda on inner source.
To enhance the findings of this review, we intend to conduct a comprehensive survey of practitioners to identify t
challenges involved in ISS development and propose some resolution strategies to overcome the challenges.
show a great concentration of empirical research on the study of how organisations adopt ISS development into their internal software dev
processes. Other research areas receive much less attention. Among the frameworks/methods, models and tools identified, none of them ha
empirically validated in real industry settings. One of the implications of these findings for research and practice is the need for more em
Specifically, while ISS development is highly influenced by OSS development, there is a need to translate OSS practices to suit the organ
studies on engineering practices to support ISS development. S
context to achieve the many benefits associated with OSS. Furthermore, future research is required to empirically validate the propos
frameworks/methods, models and tools. To advanced our understanding of the inner source phenomenon, researchers need to draw on th
foundations that have been used in prior research on OSS, as well as other theoretical lens that are considered relevant to ISS approa
The implication for practice also lie in the evidence of the benefits and challenges of ISS development. The findings have shown that the ad
ISS development helps organisations to improve better quality, time-to-market and in- novativeness. However, as suggested by Brown et a
Can a newshould
newcomers pharmaceutical bethe
understand developed entirely
reality of through
the method an open
through source model?
an appropriate Likely not. so
enculturation, However,
that theya can
newrecog-
drug for a neglected
nise what worksdisease
and w
shepherded up to clinical trials utilizing a hybrid
not work,
open and
source
thusmodel
be aware
combining
of changing
open source
workingwith
processes.
other development models such as fee-for
outsourcing. To assist with this development, we believe that further research is needed on business modeling, incentive development and
Inofthis
the systematic mapping,
use of the public domain.weIt isfound the that
important publication onincludes
this research the intrinsic (social)
expert input frommotivation
researchers, factors of open source
the pharmaceutical industrysoftw
and P
also found the publication on the extrinsic motivation (economic) factors of open source software. We also found sel
assess the practicality and relevance of open source drug discovery at a task level.
open source software license on the base of economic, social and commercial (managerial) perspectives. In future, w
to find out some other motivation factors both economic and social perspectives for selecting the open source license
will be our research question in future what are the motivation factors in selecting an open source software license
respect to economic and social perspectives in software community? Are the results of RQ1 are in accordance with pe
of local (Pakistani) open source software community?
Fur- thermore, the architecture-sensitive metrics for code anomalies discovery provides the majority of awareness to engineers for the exist
smells code elements that are more significant to the architecture design than the most traditional metrics that are depending on source code
on static code metrics combination. This means that the developers and engineers could detect and repair such anomalies promptly. Therefo
studies are needed in this field for other metrics to be analyzed in order to provide the most appropriate architecture without any impact of
In the anomaly’s
bias. prioritization
Furthermore, there is astrategy,
need forthe agglomeration
metrics that have flood
a greatstandard and
ability to most optimal
discover models showed
the inconsistent classesthat someby
affected agglomerations
the degradationarefro
o
with falseclasses.
consistent positives and not precise
In addition, there isenough
a need to
to identify
identify architectural inconsistencies
the effort required in classes,
for the metrics leading
strategy to the
to archi- inabilitydetect
tecturally to capture several
related anomv
also
architectural
to deriveproblem
more metrics
types.that
In contrast,
may havetheanrecommended
impact on the heuristics,
quality relationships
architectureofblueprints,
other software
and the
thatcontext-based
are closely related
smelltoprioritization
architecturaltec
p
show the ability to rank and improve in identifying the prioritization of anomalies related to architectural problems. This reveals the need fo
ing an initial enrichment of the possible results to adopt the solution with a tendency in the ideal combination to prioritize architectural an
Consequently, there is a need to provide various prioritization criteria for seizing the diverse architectural problem types and enhancing the
techniques
This reveals used
thatfor
architecture
discovering maycode
be anomalies.
recovered with
More- acceptable
over, the accuracy
integration
unlike
of two
theorprior
moreintuition
heuristics
based
would
on its
getclaimed
better ranking
hypothesis
results’
in aneffe
ap
Additionally,
difficulty to
it is
recover
possible
conceptual
to introduce
architecture.
the new strategies
However,tothere
produce
is a clear
ranking
opportunity
on numerous
for many
criteria
researchers
in order to
to provide
shed lightvisualization
on checking capabilit
more e
approaches to architecture recovery by enriching most relevant
ARCADE’s to architectural
tool to addproblems
further different
for the developers.
current code-level analyses to it. Besides, there
need to conduct more experiments on a wide scope on more systems, especially industrial systems as well as to increase approach accuracy
documentations, pull requests, commit messages, comments, tests, and much more.
In the architectural smells group, architectural bad smells (architecture anti-patterns) stood out as the most effective smells compared to arc
change (instability) and architectural hotspots smells (as shown in Fig. 10). This reveals that the architectural bad smells were studied in iso-
not combined with more than one smell, which was covered in the prior code smells. Consequently, further research in this domain shou
conducted to identify the effect of architectural smells agglomeration and its correlation with architectural problems rather than the archi
smells in isolation in order to prove its inclusion or exclusion as one of the key indicators of the architectural decay symptoms.
It can be concluded that the problems of architectural erosion within the OSS projects, including identifying, addressing, avoiding and p
are still open research issues, which need further analysis and investigation. Consequently, more efforts on this domain should be focused on
ing the other reasons that are still unclear and suggesting other solutions to provide more performance and accuracy to address architectura
An SLR study provides directions for researchers and practitioners on architectural decay within the OSS domain as follows:
1) There are reasons that could contribute excessively to increasing architectural erosion such as rapid development of software, frequent
and lack of developers’ awareness. Therefore, further studies should be conducted as a rooted-deep study to find out other causes and to
architecture erosion whether on the OSS or industrial systems. Practitioners should follow guidelines to avoid architectural contradictions
and subsequent versions of the system in terms of identifying the potential reasons within the system environment.
2) Sincearchitecturaldegradationsymptomspresentthat code smells agglomeration has a considerable correlation to architectural problems
to code smells individual, researchers should conduct further investigation on architectural bad smells in combination. Practitioners can ch
detection way depending on the code smells agglomeration to identify degradation symptoms effectively.
3)The findings of the current study serve as evident that a metrics-based strategy is the most commonly used solution as compared to other
solutions. However, more studies are needed in this field for other metrics to be analyzed to provide the most architecturally appropriate so
identify the effort required for the metrics to detect architecturally related smells. The essential techniques of ranking used should enhan
possibility to get better effectiveness results and the identification of critical cores of architectural violations. Also, there is a clear oppor-
many researchers to highlight enriching ARCADE’s tool for efficient approaches to architecture recovery. Additionally, there are solution
3)The findings of the current study serve as evident that a metrics-based strategy is the most commonly used solution as compared to other
show that it is not effective at all in the problems of deep analysis and the difficulty of address such refactoring strategy that has no consid
solutions. However, more studies are needed in this field for other metrics to be analyzed to provide the most architecturally appropriate so
positive impact to address architectural erosion.
identify the effort required for the metrics to detect architecturally related smells. The essential techniques of ranking used should enhan
possibility to get better effectiveness results and the identification of critical cores of architectural violations. Also, there is a clear oppor-
many researchers to highlight enriching ARCADE’s tool for efficient approaches to architecture recovery. Additionally, there are solution
show that it is not effective at all in the problems of deep analysis and the difficulty of address such refactoring strategy that has no consid
These reasons are considered the most contributing to degradation,
positive impact towhile thearchitectural
address rest of the reasons
erosion.demonstrated in Table 6 are less important
three stated reasons depending on what was declared in the selected primary studies. However, further studies should be conducted to find ou
causes as a rooted-deep study in digging up new causes that could have a significant contribution to identifying the architecture erosion, wh
the OSS or industrial systems.
In terms of using the architectural rules violations solutions, we observed that many violations were not restricted by the architectural princi
system, which may not have defined the necessary rules to reduce the severity of major violations. This means that violations still appear o
and over again despite the good violation solutions that were introduced and the approaches that have a significant role in capturing violati
thorough accuracy. Therefore, it is important to highlight the identification of the necessary rules and identification of critical cores throug
The findings showed that 17 of the most commonly occurred causes contribute to the architectural decay of the OSS community. Essen
study on architectural conformance using multiple frameworks.
architectural degradation has numerous causes, which have been discussed by several researchers in their studies [11], [12], [23], [30]. How
reasons have been discussed from limited aspects such as aging because of changes over time [11], identifying the reasons through only
Asstudy
a result
[12]oforour findings,
based weinvestigation
on their propose the following directions
of industrial for future
case studies research in thisthese
[27]. Consequently, area. causes
Focus on thetodefinition
need be furtherofidentified
a common in mode
terms
may
frequency
be obtained
of their
byoccurrence,
merging multiple
especially
available
in theapproaches)
scope of theandOSSfavor
projects.
its adoption
Moreover,
through
identifying
rigorous theand
most
extensive
important validation
reasonsinwill
industrial
indeed con
set
could
the erosion
increaseaccording
the validity
to the
of the
chosen
model primary
and thus
studies,
its dissemination
which contain in industry,
several case
where
studies
OSSinisdifferent
still not widely
domains adopted.
for the OSS
Several
community.
models already
Thes
according to the results of differ
our SLR,
in their
theyimpacts
have notwith
been
regard
strongly
to the
validated
actual contribution
and, as a consequence,
to the architectural
adoptiondecay
has beenprominence.
limited. Try to target the
quality factors that are of real interest for stakeholders. Most of the available models focus on the overall quality of the product, but few of
able to assess each single factor that composes the overall quality of the OSS product. This can complicate the assessment of OSS produ
stakeholder, who are interested in specific quality factors: e.g., developers are likely more interested in reliability or testability aspects, whil
people may be more interested in cost or maintenance factors, etc..Develop tools that support the research directions listed above (i.e., tool
support and simplify the applicability of the proposed models during the evaluation of OSS products). Most of the tools mentioned in the
studies are prototypes and most of them are not available or maintained anymore.
As future work, we intend to apply the findings of our study in more recent OSS projects and provide a validation
proposed OSS macro process though a practical point of view analyzing OSS projects process activities, their charac
and understanding how roles are involved in each activity, fact that still not clear yet in the literature.
Furthermore, we intend to investigate how researchers perform OSS process analysis in OSS projects, including app
As a result of our findings, we propose the following directions for future research in this area. Focus on the definition of a common mode
techniques
may be obtained by merging multiple and
available tools they
approaches) andhave
favorused to retrieve
its adoption OSS
through process
rigorous and information.
extensive validation in industrial set
could increase the validity of the model and thus its dissemination in industry, where OSS is still not widely adopted. Several models already
according to the results of our SLR, they have not been strongly validated and, as a consequence, adoption has been limited. Try to target the
quality factors that are of real interest for stakeholders. Most of the available models focus on the overall quality of the product, but few of
able to assess each single factor that composes the overall quality of the OSS product. This can complicate the assessment of OSS produ
stakeholder, who are interested in specific quality factors: e.g., developers are likely more interested in reliability or testability aspects, whil
people may be more interested in cost or maintenance factors, etc..Develop tools that support the research directions listed above (i.e., tool
The process
support map and the
and simplify its activities characteristics
applicability description
of the proposed modelspresented in evaluation
during the our study are basedprod-
of OSS on primary academic
ucts). Most of thesources. However,initthe
tools mentioned ca
as basis for the conduction of empirical studies
studies are to investigate
prototypes and mostOSS process
of them areinnot
a more practical
available view. Weanymore.
or maintained believe that our results can contr
improve the understanding of OSS process activities and consequently help to mitigate OSS project failure.
First, regarding research types, it is worth highlighting that the high number of evaluation papers and the increas
validation contributions reflects the maturity FLOSS adoption studies. However, this research area is not yet to the p
contributing experience reports. Second, regarding FLOSS adoption factors we observe that most of the studies hav
focusing on the organizational and technological factors leaving the economic factors not so well covered.
However, according to the facts detected in the research studies, IT managers are neither using any tool nor procedures that allows them to e
adoption of FLOSS solutions. For this reason, this contribution can motivate researchers to work on the creation and publication of guidel
adopting FLOSS.
The purpose of this study is to guide future research in the application of FLOSS in new domains as a guide for the correct selection of FLO
IT managers make appropriate decisions for organizations, define policies for FLOSS adoption, among others.
Few studies are conducted on the joining process and abandonment. Therefore, future research should address the issues of these two are
authors also noticed that some studies investigated more than one research topic. Additionally, our study identified the gaps in community
area that could be useful for researchers. Moreover, mapping study also provides insights of some areas that need more exploration for i
mentoring of newcomers and finding a task to start are some of the open research areas, which need more study. There is a need to develop
better support newcomers’ onboarding and easy migration of the developers among different projects. There is a lack of evidence on how
technical barriers, how gender and age of developers influence their reten tion in a project. It shows that there is still need of tools, practic
processes for better community participation and engagement in OSS projects.
We plan to extend this work in the future by concentrating on the identified research gaps and by introducing more r
questions to acquire in-depth knowledge about community particpation and engagement.
In the future, this study can be extended to include software ecosystem, community structure, mentoring, project gov
difficulties associated with finding a task,and project characteristics that will provide more insights in the commu
dynamics
Few studies are conducted on the joining process and abandonment. domain.
Therefore, future research should address the issues of these two are
better support newcomers’ onboarding and easy migration of the developers among different projects. There is a lack of evidence on how
technical barriers, how gender and age of developers influence their retention in a project. It shows that there is still need of tools, practi
Furthermore, this study can be repeated by using other research techniques such as a combination of systematic mapping study and syst
literature review can be used to obtain better results.
Moreover, the authors are planning to expand the search by including more terms in the search strings and addition
sources, not considered by this study to find more relevant studies on community dynamics. Thus, provides more i
about research topics and issues in the area that will helpful for researchers.
Moreover, the authors are planning to expand the search by including more terms in the search strings and additio
sources, not considered by this study to find more relevant studies on community dynamics. Thus, provides more in
about research topics and issues in the area that will helpful for researchers.
Furthermore, future research should focus on newcomers’ orientation and reception to identify the problems in initial contribution as well a
guidance regarding barriers in advance. Moreover, significant research is required to explore the means to support newcomer initial partic-
enhancing motivation to increase participation, and reduce the hindering factors that will improve retention of new developers and One
We suspect that this lack of research results in economic factors
Contributors is due the reluctance of companies to provide eco
(OTC).
details. Also, FLOSS experts consider that organizations are already aware of the hidden costs when adopting FLOS
therefore, they tend to focus more on researching technological and organizational factors. Additionally, we only fo
solution proposals related with economic factors and one with technological and organizational factors. We also ob
that validation research, opinion papers and philosophical papers are gaining maturity in the FLOSS adoption area be
found taxonomies, literature reviews and systematic maps.
This study also indicates the areas such as joining process and abandonment where research is lacking.
Abandonment is another area in OSS community and has many issues that need to be explored in future research. Additionally, further res
required to explore various factors that lead developers to stay or leave a project.
Moreover, mentoring is another field that needs to be explored in the future. Furthermore, significant research is required to explore the
practices, and processes that could be helpful to improve community participation. The research community may use these findings to unde
issues in the area and select the topic for further research. Additionally, we also observed from the results that in majority of the studies, com
of survey and questionnaire was used as a research methodology to conduct research. From these results, we observed that few studies used
learning methods. In future use of these techniques will be helpful to solve the problems related to task distribution, selecting a task to
contribution, and management of project information. Furthermore, findings indicate that majority of the studies and researchers who inves
community dynamics belong to USA. We also observed that most of the studies (75%) were conducted by the research group of one count
more collaboration is required among research groups of different countries to conduct more research in the area.
Some researchers have studied the joining process of OSS communities. But more research is required to discover the tools which may h
developers to alleviate the issues experienced by them when they migrate to the other projects, to investigate the changes in the degree of so
over time, to examine the association between similarities in newcomer socialization patterns and distinction in joining scripts, to analyze
of the difference in the joining process on retention. Besides, more significant research is required to investigate contributor roles in the eco
analyze the impact of resolved issues on onboarding success, to examine the relationship between joining process and role transition, and to
effective ways to organize project information that may support newcomers during the onboarding period.
In order to increase the participation of new developers, there is a need to devise tools, which may eliminate contribution barriers and pr
onboarding support. It is necessary to examine the impact of social interaction on newcomer success, the effect of documentation on task
technical and coding issues, cultural differences, and issues related to creation of local workspace.
Fur- thermore, more research is required to study the impact of mentoring on the success of onboarding process and provide suggestions
support newcomer onboarding, and contribution barriers in virtual communities.
Besides, more research is needed to develop strategies to alleviate the issues related to choosing a task to start initial contribution, to design
enhance retention of One Time Contributors (OTC), and to explore the motivation of developers to work as a mentor.
Cmmunity dynamics studies focus on the internal and external motivation of developers and factors that influence the retention of develo
projects. However, there are some areas, which need research such as retention of veteran developers, the impact of contribution barrie
abandonment, use of gamification in student engagement and its effect on retention, the effect of project characteristics on retention, the im
project governance on motivation, development of strategies to maintain loyalty, the impact of leadership styles on developer turnover, and
corporate sponsorship on FLOSS communities. Moreover, there is a need to design tools to assess the health of OSS projects and develop st
retain new developers.
In order to reduce dropouts of developers who abandon an OSS project (especially the ones who disappear after their first contribution), t
need to look for effective ways to engage such developers in the community and providing good community support when they are taking
After examining more than 69 different research papers andOSS
in an taking the survey findings into account, we constructed
project.
map that can be used by any educational institutedespecially high schoolsdto strategically integrate OSS into the edu
A mapping
system.study review
The provides
roadmap is adesigned
structure for
to atarget
research report
three type,aspects
main which enables categorizing andusing
of implementation: giving OSS
a visual
in summary of results
the curriculum it
been published in papers of a research area[8]. This map aids to identify gaps in a research area, becoming a basis to guide new research act
throughout the semester, using OSS in final projects at the end of the semester, and using OSS in extra-curricular
The current mapping review captured the current state of research on bug report severity prediction, characterized related problems and ide ac
Themain
future direction
approaches of thistoresearch
employed is to
solve them. implement
These objectivesthis
wereroadmap
reached byand measure
conducting its effectiveness.
a mapping of existingThe roadmap
literature. cantheber
In total,
via three
identified phases:
27 relevant changes
papers to curriculum,
and analyzed them along implementation in finalthese
12 dimensions. Although course proj-
papers ectmade
have andvaluable
establishment of extra-curric
contributions in bug repor
Reception of new developers is another challenge in the open sourceactivities.
prediction, the panorama presented in this mapping study review suggests that there are potential research opportunities for further improve
community. To overcome this issue, research is needed to develop to
this topic. Among them, the following research directions appear to be more promising:
assist in better reception of new developers.
• There is an apparent lack of investigation on bug report severity prediction in other relevant FLOSS such as, for example, Linux Kernel
Linux, and MySQL, and in others BTS, for example, Github.
• Often, technical users report most bugs. Thus, the influence of user experience in predicting outcomes is still overlooked.
Moreover,
• Bug reports
more labeled
significant
withresearch
default is
severity
requiredlevel
to study
(oftenthe
“normal”)
impact of were
community
prevalentdynamics
in the most(stagnant
datasetsv/s
used
dynamic
in reviewed
community)
papers.on However,
developet
considered unreliable [26], and just discarding them andalso
the does
effectnot
of seem
turnover
appropriate.
on projectThen,
performance.
efforts in researching on novel approaches to h
type of report should be considered to improve the state-of-the-art of severity prediction algorithms.
• Most approaches were based on unstructured text features (summary and description). To handle them, researchers chose to use the tra- dit
A mapping study review provides a structure for a research report type, which enables categorizing and giving a visual summary of results
of-words approach instead of more recent text mining methods (e.g., word-embedding [66]) or data-driven feature engineering methods w
been published in papers of a research area[8]. This map aids to identify gaps in a research area, becoming a basis to guide new research act
likely improve outcomes yielded so far.
The current mapping review captured the current state of research on bug report severity prediction, characterized related problems and ide
• There is a clear research opportunity to investigate whether state-of- the-art ML algorithms might outperform the traditional algorithms u
main approaches employed to solve them. These objectives were reached by conducting a mapping of existing literature. In total, the r
reviewed papers for bug report severity prediction. The investigation of the use of Deep learning algorithms which perform very well when c
identified 27 relevant papers and analyzed them along 12 dimensions. Although these papers have made valuable contributions in bug repor
As a follow-up to this research, audio, text, and image data [67] seemson to be aaspects
promising research direction. the participation of w
prediction, the panorama presented inwe thisexpect
mapping tostudy
deepen the
review study
suggests thatthe that
there are potential may influence
research opportunities for further improve
• Researchers should investigate more recent techniques (e.g., continuous learning [68]) to provide an approach for bug report prediction wh
open source software projects and
this topic. software
Among them,development projects,
the following research as well
directions
be employed in real-world scenarios. as
appeartotopropose
be more ways of addressing the ide
promising:
• There is an apparent
problems lack of investigation
regarding issues on bug
of in report
gender severity
inequality prediction
in open in other relevant
sourceseverity FLOSS
communities such as, for
and group example,
software Linux Kernel
factories.
• Many bug reports are resolved in a few days (or a few hours) [69].Efforts to predict level for these of bug reports do not
useful. Thus, an investigation to confirm this hypothesis and to determine when the severity prediction is more appropriate in bug report life
As part of our• Often, technical
future users we
research, report most
plan tobugs.
conductThus,athe influencehensive
compre- of user experience
survey ofinpractitioners
predicting outcomes is still the
to identify overlooked.
key chall
critical importance.
• Bug reports labeled with default severity level (often “normal”) were prevalent in the most datasets used in reviewed papers. However, t
•im-
Theplementing inner
[26],source
and justand propose
study wasresolution
them also does strategies
to review the research to overcome. The survey couldonbe alsoap-performed
proaches totoh
primary objective of our mapping approaches for severity level prediction in FLOSS. However, it w
considered unreliable discarding not seem appropriate. Then, efforts in researching novel
promising research venue to extend this study the to commercial
signifi- cance systems,
of our and verifyagenda.
research whether the same findings apply to these system
type of report should be considered to improve the state-of-the-art of severity prediction algorithms.
• The approaches published in selected papers for severity level prediction did not consider the temporal information changes in bug repo
• Most approaches were based on unstructured text features (summary and description). To handle them, researchers chose to use the tra- dit
Investigating the impact of the temporal evolution of a bug report information [70] in severity level prediction accuracy, as well as investig
of-words approach instead of more recent text mining methods (e.g., word-embedding [66]) or data-driven feature engineering methods w
structures to store the representation of this temporal evolution seems a relevant research venue.
likely improve outcomes yielded so far.
• Related research areas, such as severity level prediction in help desk systems for IT service management [71], may take advantage of th
• There is a clear research opportunity to investigate whether state-of- the-art ML algorithms might outperform the traditional algorithms u
practices used in severity level prediction in FLOSS. Assess- ing these best practices in this context is also a promising research ven
reviewed papers for bug report severity prediction. The investigation of the use of Deep learning algorithms which perform very well when c
audio, text, and image data [67] seems to be a promising research direction.
• Researchers should investigate more recent techniques (e.g., continuous learning [68]) to provide an approach for bug report prediction wh
be employed in real-world scenarios.
• Many bug reports are resolved in a few days (or in a few hours) [69].Efforts to predict severity level for these group of bug reports do not
• The primary objective of our mapping study was to review the research approaches for severity level prediction in FLOSS. However, it w
identified 27 relevant papers and analyzed them along 12 dimensions. Although these papers have made valuable contributions in bug repor
A mapping study review
• Often, provides
technical usersa structure
report most for abugs.
researchThus,report type, which
the influence enables
of user categorizing
experience and giving
in predicting a visualissummary
outcomes of results
still overlooked.
prediction, the panorama presented in this mapping study review suggests that there are potential research opportunities for further improve
been published
• Bug reports in paperswith
labeled of adefault
research area[8].
severity levelThis(often
map aids to identify
“normal”) weregaps in a research
prevalent in the mostarea,datasets
becoming useda basis to guide
in reviewed new research
papers. However, actt
this topic. Among them, the following research directions appear to be more promising:
The current mapping
considered unreliablereview[26], andcaptured the currentthem
just discarding statealso of research
does noton seembugappropriate.
report severity prediction,
Then, efforts incharacter-
researching izedonrelated
novel ap-problems
proachesandtoideh
• There is an apparent lack of investigation on bug report severity prediction in other relevant FLOSS such as, for example, Linux Kernel
main approaches employed type of report
to solve should
them.be These
considered
objectives
to im-were prove
reached
the state-of-the-art
by conducting of a map-
severitypingprediction
of existing algorithms.
literature. In total, the r
•identified
Most approaches
27 relevant were papers
basedand on analyzed
unstructured them text
alongfeatures
12 dimensions.
(summary and Although
description).
these papers
To handle
have them,
made researchers
valuable contributions
chose to useinthe bugtra-
repor
dit
A mapping study review
• Often, provides
technical usersa structure
report most for abugs.
researchThus,report type, which
the influence enables
of user categorizing
experience and giving
in predicting a visualissummary
outcomes of results
still overlooked.
of-words
prediction,approach
the panorama
insteadpresented
of more recentin thistext
mapping mining study
methods
review (e.g.,
suggests
word-embedding
that there are[66]) potential
or data-driven
research opportunities
feature engi- neeringfor furthermethods
improve w
been published
• Bug reports in paperswith
labeled of adefault
research area[8].
severity levelThis(often
map aids to identify
“normal”) weregaps in a research
prevalent in the mostarea,datasets
becoming useda basis to guide
in reviewed new research
papers. However, actt
this topic. Among them,likely the following
improve research
outcomesdirections
yielded so appear
far. to be more promising:
The current mapping
considered unreliablereview[26], andcaptured the currentthem
just discarding statealso of research
does noton seembugappropriate.
report severity prediction,
Then, efforts incharacter-
novel ap-problems
proachesandtoideh
• There is aanclear
apparent
researchlackopportunity
of investigation on bug report
to investigate whetherseverity pre-the-art
state-of- dictionML in other relevant
algorithms FLOSS
might such as,the
outperform for traditional
example, Linux Kernel
algorithms u
main approaches employed to solve
type of report them.be
should These objectives
considered to im-were reached
prove by conducting of
the state-of-the-art a map-
severitypingprediction
of existing literature. In total, the r
algorithms.
reviewed papers for bug report severity prediction. Linux,The and investigation
MySQL, andof in the
othersuse BTS,
of Deep for learning
example,algorithms
Github. which perform very well when c
•identified 27 relevant
Most approaches were papers
basedand on analyzed
unstructured them along
text 12 dimensions.
features (summary and Although these papers
description). have them,
To handle made researchers
chose to useinthe bugtra-
repor
dit
A mapping study • Often,
reviewtechnical
provides usersa structure
report most
audio, for and
text, abugs.
researchThus,report
image datathe[67]
influence
type, which
seems oftouser
enables
be aexperience
categorizing
promising in predicting
research anddirection.
giving
outcomes
a visualissummary
still overlooked.
of results
of-words
prediction,approach
the panorama
insteadpresented
mapping mining study
methods
review (e.g.,
suggests
word-embedding
that there are[66]) potential
or data-driven
research opportunities
feature engi- neeringfor furthermethods
improve w
•been
• Bug
published
reports
Researchers in
labeled
shouldpapers with
of adefault
investigate research
more severity
area[8].
recent levelThis(often
techniques map (e.g.,
aids
“normal”)
to identify
weregaps
continuous prevalent
in a research
learning in thetomost
[68]) area,datasets
provide becoming useda basis
an approach in reviewed
fortobug
guide papers.
newprediction
report research
However, act
wht
this topic. Among them,likely the following
improve research
outcomesdirections
yielded so appear
far. to be more promising:
The
considered
current mapping
unreliablereview[26], andcaptured
just discarding
the currentthem statealso ofberesearch
does noton
employed seembug appropriate.
report severity
in real-world Then,
prediction,
scenarios. efforts incharacter-
novel ap-problems
proachesandtoideh
• There is aanclear
apparent
of investigation to investigate
on bug report whetherseverity
state-of-
pre-the-art
dictionML in other
algorithms
relevantmight
FLOSS outperform
such as,the for traditional
example, Linux algorithms
Kernel u
• Manymainbug
approaches
reports are employed
resolved to
insolve
type of report a few them.
should
daysbe These
(or in aobjectives
consideredfew hours)
to im-were reached
[69].Efforts
prove by conducting
the state-of-the-art
to predict a map-
severity
of severity
levelping
for of existing
prediction
these group literature.
algorithms. In total,
of bug reports do the
notr
reviewed papers for bug report severity prediction. Linux,The and investigation
MySQL, andof in the
othersuse BTS,
of Deep for learning
example,algorithms
Github. which perform very well when c
•identified
useful.
Most Thus, 27 anrelevant
approaches papers
investigation
were basedand analyzed
toonconfirm
unstructured them
this along
hypothesis
text 12and
features dimensions.
(summary
to determine Although
and when these
description).
the papers
severity have them,
To handle
prediction made valuable
is researchers contributions
more appropriate chose in useinthe
to bug bugtra-
reportrepor
life
dit
"A mapping study • Often,review provides
technical users a structure
report most
audio, text,foranda research
bugs. report
Thus,data
image the[67] type,
influence which
seems enables
oftouser
be categorizing
aexperience
promising in anddirection.
predicting
research giving a visual
outcomes is summary of result
still overlooked.
prediction,
of-words the panorama
approach insteadpresented
mapping mining study review
methods suggests
(e.g.,
critical that there are[66])
word-embedding
importance. potential research opportunities
or data-driven feature engi- neeringfor further improve
methods w
•been
• Bug
published
reports
Researchers in
labeled
shouldpapers with
of adefault
investigate research
more severity
area[8].
recent levelThis(often
techniques map (e.g.,
aids
“normal”)
to identify
weregaps
continuous prevalent
in a research
learning in thetomost
[68]) area,datasets
provide becoming useda basis
an approach in reviewed
fortobug
guide papers.
newprediction
report research
However, act
wht
• The primary objective of our mapping this topic.study Among wasthem, the following
likely
to review improve
the research
re- searchoutcomes directions
yieldedforso
approaches appear
far. to level
severity be more promising:
prediction in FLOSS. However, it w
The
considered
current mapping
unreliablereview[26], andcaptured
just discarding
the currentthem statealso ofberesearch
does noton
employed seembug appropriate.
report severity
in real-world Then,
prediction,
scenarios. efforts incharacter-
novel ap-problems
proachesandtoideh
• There promising
is aanclear
apparent
research of investigation
venue to investigate
to extend on bug
this study report
whetherseverity
to com- state-of-
pre-systems,
mercial the-art
dictionML inand
other
algorithms
relevant
verify might
whetherFLOSS outperform
the such findings
same as,the
for traditional
example,
apply toLinuxalgorithms
these Kernel
system u
• Manymainbug
approaches
reports are employed
type of report
resolved to
insolve
a fewshould
them.
daysbe These
(orconsidered
in aobjectives to im-
few hours) were prove
reached
[69].Effortsthe state-of-the-art
by conducting
to predict of
a map-
severity severity
levelping prediction
for of existing
these algorithms.
group literature. In total,
of bug reports do the
notr
•reviewed papers for
The approaches bug report
published severitypapers
in selected prediction.
Linux, The
and investigation
for severity MySQL,
level pre- anddiction
of
in the
othersuse
didBTS,
of
notDeep for learning
example,
consider algorithms
Github. information
the temporal which perform changes veryinwell
bug when
re- poc
•identified
Most Thus,
useful. approaches
27 anrelevantwere papers
investigationbasedandtoonconfirm
analyzed
unstructured them
this text
along
hypothesis features
12and
dimensions.
(summary
to determine and
Although
description).
when these
the papers
To handle
severity have them,
prediction made is researchers
more appropriate chose in to bug
useinthe
bugtra-
reportrepor
dit
life
Investigating•the Often,
impacttechnical
of the users
temporal audio,
report most
text, and
evolution bugs.
ofimage
a Thus,
bug data the[67]
report influence
seemsoftouser
information be
[70]aexperience
promising
in severityin research
predicting
level direction.
predictionoutcomes
accuracy, is still overlooked.
as well as investig
prediction,
of-words the panorama
approach insteadpresented
mapping mining study review
methods suggests
(e.g.,
critical that there are[66])
word-embedding
importance. potential research opportunities
or data-driven feature engi- neeringfor further improve
methods w
• Researchers
• Bug reports should
labeledinvestigate
with default
structuresmore severity
torecent
store level
techniques
the (often (e.g.,
representation“normal”)
continuous
of thiswere prevalent
learning
temporal [68])
in thetomost
evolution provide
seems datasets
an approach
a relevantusedresearch
in reviewed
for bug
venue.report
papers. prediction
However, wht
• The primary objective of our mapping this topic.study Among wasthem, the following
to review
likely the
improve research
re- searchoutcomes directions
approaches
yieldedforsoappear
far. to level
severity be more promising:
prediction in FLOSS. However, it w
considered
• Relatedunreliable [26], and
research areas, suchjust as discarding
severity level them alsobedoes
prediction innot
employed
helpseem
desk appropriate.
in real-world
systems for Then,
scenarios. efforts
IT service in researching
management [71],onmaynoveltakeap-advantage
proaches of to hth
is aanclear
• There promisingapparent
research of investigation
venue to extend on bug
to investigate
this study report
whetherseverity
to compre-systems,
state-of-
mercial dictionML
the-art inand
other relevant
algorithms
verify whetherFLOSS
might the such findings
outperform
same as,the
for traditional
example,
apply toLinux
these Kernel
algorithms
system u
• Manypractices
bug reports used type
areinresolved of report
severity in
level
a fewshould
daysbe
prediction (orconsidered
ininFLOSS. to
a few hours) im-[69].Efforts
Assess- prove the state-of-the-art
ing these to
best
predict
practices of
in severity
severity this
level prediction
context
for these
is also algorithms.
groupa promising
of bug reports
researchdo not
ven
in selected Linux,
prediction. and investigation
The
for severity MySQL,
level pre- anddiction
in the
of others didBTS,
use of
notDeep for learning
example,
consider Github. information
algorithms
the temporal which perform changes veryinwell
bug when
re- poc
• Most Thus,
useful. approaches were basedtoonconfirm
an investigation unstructured text features
this hypothesis and(summary
to determine andwhendescription).
the severityTo handle
predictionthem, is researchers
use the tra-life
report dit
Investigating•the Often,
impacttechnical
of the users
temporal report
audio, most
text, and
evolution bugs. a Thus,
ofimagebug data the[67]
report influence
seemsoftouser
information be
[70]aexperience
in severityin
promising predicting
research
level outcomes
direction.
prediction accuracy, is still overlooked.
as well as investig
of-words approach instead of more recent text mining methods (e.g., word-embedding
critical importance. [66]) or data-driven feature engineering methods w
• Researchers
• Bug reports should
labeledinvestigate
with default
structuresmore severity
torecent
store level
techniques
the (often (e.g.,
representation“normal”)
continuous
of thiswere prevalent
learning
temporal [68])
in thetomost
evolution provide
seems datasets
an approach
a relevantusedresearch
in reviewed
for bug
venue.report
papers. prediction
However, wht
• The primary objective of our mapping study was to review likely the
improve
re- searchoutcomes yieldedforsoseverity
approaches far. level prediction in FLOSS. However, it w
considered
• Relatedunreliable [26], and
research areas, suchjust as discarding
severity level them alsobedoes
prediction employed
innot
helpseem in real-world
desk appropriate.
systems for scenarios.
Then, efforts
IT service in researching
management [71],onmaynoveltakeap-advantage
proaches of to hth
• There promising
is a clear research
researchopportunity
venue to extend to investigate
this studywhether to com-state-of-
mercial systems,
the-art ML and
algorithms
verify whether
mightthe outperform
same findings the traditional
apply to these
algorithms
system u
• Manypractices
bug reports usedareinresolved
type of report
severity in a few
level should
daysbe
prediction (orconsidered
ininFLOSS.
a few hours)
to im-[69].Efforts
Assess- prove the state-of-the-art
ing these to predict
best severity
practices of
in severity
level
this for
prediction
context these group
is also algorithms.
of bug reports
a promising researchdo not
ven
•reviewed
The approaches
papers for published
bug report in selected
severitypapers
prediction.for severity
The investigation
level prediction
of the use did of
notDeep
consider
learning
the temporal
algorithms information
which perform changes veryinwell
bug when
re- poc
• Most Thus,
useful. approaches were basedtoonconfirm
an investigation unstructured text features
this hypothesis and(summary
to determine andwhendescription).
the severityTo handle
predictionthem, is researchers
use the tra-life
report dit
Investigating the impact of the temporal audio,evolution
text, and ofimage
a bug data
report [67]
information
seems to be [70]a promising
in severityresearch
level prediction
direction. accuracy, as well as investig
of-words approach instead of more recent text mining methods (e.g., word-embedding
critical importance. [66]) or data-driven feature engineering methods w
• Researchers should investigate structuresmoretorecent
store thetechniques
representation
(e.g., continuous
of this temporal learning evolution
[68]) toseems
provide a relevant
an approachresearch for bug
venue.report prediction wh
• The primary objective of our mapping study was to review likely the
improve
re- searchoutcomes yieldedforsoseverity
approaches far. level prediction in FLOSS. However, it w
The review establishes
• Related research areas, the state
such of as the research
severity levelonprediction
ISSDbeinemployed
terms
in help of desk
focused
in real-world
systemsknowledge areas and
scenarios.
for IT service contributions.
management [71],Majority
may take ofadvantage
research centr
of th
• There promising
is a clear research
researchopportunity
venue to extend to investigate
this studywhether to com-state-of- the-art ML
mercial systems, algorithms
and mightthe
verify whether outperform
same findings the traditional algorithms
apply to these system u
adoption
• Manypractices
bug
andreports
adaptation
usedareinresolved
of inner source
severity in a few
level indays
various
prediction (orinin
context,
a few hours)
FLOSS. while
Assess-other
[69].Efforts
ingKAsthese in to
SWEBOK
predict
best severity
practices receive
in thislevel
less attention.
for these
context group
is alsoThus our
of bug
a promisingreview
reports
callsdofor
research not
venm
in selected prediction. The investigation
for severity of the use
level prediction did of
notDeep learning
consider algorithms
the temporal which perform
information changes veryinwell
bug when
re- poc
useful.
on these
Thus,
areas
an to
investigation
advance ourtoknowledgeconfirm this onhypothesis
the inner source and to phenomenon.
determine when Furthermore,
the severitytoprediction
advance our is more
understanding
appropriate of ininner
bugsource,
report life
res
Investigating the impact of the temporal audio,evolution
text, and ofimage
a bug data
report [67]
information
seems to be [70]a promising
in severityresearch
level prediction
direction. accuracy, as well as investig
need to draw on theoretical foundations that have been used in prior critical
research
importance.
on OSS, as well as other theoretical lens that are considered
• Researchers should investigate structuresmoretorecent
store thetechniques
representation
(e.g., continuous
of this temporal learning evolution
[68]) toseems
provide a relevant
an approachresearch for bug
venue.report prediction wh
inner
• Thesource.
primaryWith the help
objective of the
of our OSS framework,
mapping study was to ourreview
study the
alsore- hassearch
identified the characteristics
approaches for severity of the prediction
level inner source phenomenon,
in FLOSS. includin
However, it w
• Related research areas, such as severity level prediction be employed
in help desk in real-world
systems for scenarios.
IT service management [71], may take advantage of th
of projects or products,
promising research the venue
motivation
to extend for this
adoptingstudyinner source,
to com- mercialits environment,
systems, and inner verifysource
whether processes
the sameand tools and
findings applythetoactors
theseinvolved
system
• Manypractices
bug reports usedareinresolved
severity in level
a fewprediction
days (orininFLOSS. a few hours)
Assess- [69].Efforts
ing these to best
predict
practices
severity
in thislevel
context
for these
is also
groupa promising
of bug reports
researchdo not
ven
• Thesource. We alsopublished
approaches highlightin theselected
challenges papersas well as the benefits
for severity level pre- fordiction
organisa- did tions that are the
not consider interested
temporal in inner source.changes
information For research,
in bugwe re-iden
po
Investigating the impact of the temporal research agenda
evolution of to further
a bug advanced
report information our knowledge in the level
[70] in severity innerprediction
source area. accuracy, as well as investig
In addition, it is possible that adding non-software
structures and information
to store the representation system
of this relatedevolution
temporal databasesseems may ayield similar
relevant or different
research venue.findings. The com
• The primary objective of our mapping study was to review the research approaches for severity level prediction in FLOSS. However, it w
findingsareas,
• Related research from such
different databases
as severity leveland the findings
prediction in helppresented
desk systems in thisforstudy can po-management
IT service tentially be considered
[71], may take as future work. of th
advantage
promising research venue to extend this study to commercial systems, and verify whether the same findings apply to these system
practices used in severity level prediction in FLOSS. Assess- ing these best practices in this context is also a promising research ven
• The approaches published in selected papers for severity level prediction did not consider the temporal information changes in bug repo
Investigating the impact of the temporal evolution of a bug report information [70] in severity level prediction accuracy, as well as investig
However, as described in Section structures 5.4.4, therethe
to store is arepresentation
lack of research in this
of this area. Thus,
temporal we need
evolution more
seems study toresearch
a relevant shed light on how to manage an
venue.
• Related research areas, such as severity level prediction inner sourcein helpcommunity
desk systems in anfororganisation.
IT service management [71], may take advantage of th
Based on the findings of this research, we have planned the following research objectives, to be carried out in the nea
practices used in severity level prediction in FLOSS. Assess- ing these best practices in this context is also a promising research venu
to more strengthen the existing research work, and to stress the OSS vendors’ community to meet the maximum ben
OSS paradigm.
• To identify the potential risk factors in adapting open-source software development from vendors’ perspectiv
• To identify the practices for addressing the identified success factors and risk factors
• To develop open-source software development maturity model (OSSDMM) to measure the maturity level of ve
As a future work, we plan tocase
• To conduct multiple studies
provide at software
a more formal vendor
definitionorganizations
of the opentoIoTevaluate
platformtherather
efficacy
thanofathe model
perspective
consulting different stakeholders in detail. The open IoT platform term is frequently used in public and without c
understanding could cause problems like misunderstanding or even wrong strategic decisions. Our future work inte
provide
typesainclearer
generaland deeperinfluenced
are directly understanding
from theofinterpretation
open IoT platforms
of different for the communities.
stakeholder The open
viewpoints. Below issues several
we highlight listed open
aboveis
some possible
• Since none of the analyzed papers define openness, directions
identifying of investigation.
openness dimensions of IoT platforms remains a challenge to be addre
• It is of utmost importance to find a consensus regarding openness among the different stakeholders to avoid confusion, and preferably ag
formal definition.
• Investigating openness not only from IoT platform perspective, but also considering IoT middleware and frameworks.
• To understand how much openness of IoT platforms has penetrated the field and in which domains, a mapping study of the application do
the identified open IoT platforms would be useful.
Another result of our study is that there are no clear definitions related to the openness of IoT platforms. One of the papers attempts to de
platform aspects only [33], another paper categorizes the openness dimensions of platforms [34], and a final study categorizes some open
platforms but without providing a definition [35]. Thus, none of them explicitly define what an open IoT platform is. Moreover, in our stud
also interested to identify the openness types of IoT platforms. Our results suggest that the most common openness types of most IoT platf
related to open-source. Other types identified are open standards, open APIs, open data and open layer-based platforms. However, to fu
This study also identifies some challenging areas for future work.
investigate the openness types of IoT platforms we believe it is important to look from a stakeholder view, as identified above [34,52]. Th
1. Append new findings into the body of knowledge on OS forking behaviour. Applying the combined approaches of SLR and CAM revea
Our results suggestessential
that the most common
to further openness
analyze types of most
how important theseIoT platforms
openness areare
types related
from to open-source.
different Other types
IoT stakeholder identified are open
perspectives.
forking types interpreted by academic researchers and the latest interpretation found is file language repository fork. This novel insight wi
open APIs, open data and open layer-based platforms. However, to further investigate the openness types of IoT platforms we believe it is im
researchers on how forking is presented and interpreted and industry practitioners in reviewing project forking health, especially project
look from a stakeholder view, as identified above [34,52]. Thus, it is essential to further analyze how important these openness types are from
programing language file repositories that are less adopted or forked by developers.
IoT stakeholder perspectives.
2. Understanding forking consequences. Case studies are an important way to highlight lessons learnt by researchers. This paper identified
impacts and consequences, with one of the worst impacts being a political strategy that divides a project community and forms a new com
Forming a new community results in less contributions by developers to the original file repository, bug fixes or feature enhancement. Al
accumulated bugs and feature enhancements to remain unfixed for a period of time can affect project health risk.
3. More research is required on forking sustainability. Reviewing these 21 papers revealed the importance of forking sustainability investig
Considering
top priority withthat
two there
specificare sentiments
areas associated
of interest.4. with positive
Studying forking and using
sustainability negativea SLR polarity that development
for software were marked withasGitHub.
not specifie
Valent
Izquierdo and Cabot (2017) used a SLR to show that forking is a good indicator of project longevity and the chance of forking issentiment
selected studies regarding software practices, there is still room for further investigation on the associated highly dep
specific impacts.
the project, whereMoreover, there is
developers provide a tendency
additional contactofinformation
a considerable set of personal
(e.g., emails, open sourcewebsitesoftware
URLs thatprojects to active
are clearly have orregular
aligne
popular
cyclesproject
and toowners)
adopted to increase social connections
the so called between aWe
frequent releases. project owner
plan and forker, and
to investigate increase developer
sentiments community
in this context andsize
to for med
which
projects and projects that are written in a forker’s preferred programming language. Future work could include developing a prediction mod
they influenceinclude
Future work could
software productivity.
the expansion
We also overcoming
of this research,
want to investigate
some stated
how programmers
limitations
sentiments
by increasing number
vary between
of universities
rele
effectiveness from forking motivation classifications in response to language repository files, where programming language survivalortime
by co
is
only experienced Open Source practitioners. This way, an weOSwould
projects’
be able
health
to provide
and survivability.
a more effective feedback to the researchers. Another po
to assess the survey participants motivations regarding the rates provided. With a greater amount of participants, this could be achieved w
extending the consumed time, which could affect the quality of the results.
As future work, it is necessary to investigate in real software projects, in software organizations and in open-source software projects, wha
influence factors observed by their developers. Comparing the results of this SLR with real observations may clarify the importance of inf
factors within the context of productivity within organizations and in open-source communities.
Traceability
Stud
Heading Year
y Id
Conclusion S2 2017
Conclusion S2 2017
Conclusion S2 2017
Main
Finding and S2 2017
Discussion
Main
Finding and S2 2017
Discussion
Conclusion S3 2009
Conclusion S4 2017
Conclusion S4 2017
Conclusion S4 2017
Conclusion S4 2017
Results and
S5 2014
Discussion
Conclusion S5 2014
Results and
S5 2014
Discussion
Results and
S5 2014
Discussion
Conclusion S5 2014
Conclusion
and Future S6 2016
Work
Conclusion
and Future S6 2016
Work
Results S7 2010
Results S7 2010
Results S7 2010
Results S7 2010
Results S7 2010
Results S7 2010
Results S7 2010
Results S7 2010
Future 2010
S7
Research
Future 2010
S7
Research
Future 2010
S7
Research
Future 2010
S7
Research
Future 2010
S7
Research
Future 2010
S7
Research
Conclusion S7 2010
Future 2010
S7
Research
Future 2010
S7
Research
Future 2010
S7
Research
Future 2010
S7
Research
Results S7 2010
Future 2010
S7
Research
Future 2010
S7
Research
Future 2010
S7
Research
Future 2010
S7
Research
Future 2010
S7
Research
Future 2010
S7
Research
Open
Questions
and S8 2012
Research
Open
Agenda
Questions
and S8 2012
Research
Open
Agenda
Questions
and S8 2012
Research
Open
Agenda
Questions
and S8 2012
Research
Open
Agenda
Questions
and S8 2012
Research
Agenda
Discussion S8 2012
Open
Questions
and S8 2012
Research
Open
Agenda
Questions
and S8 2012
Research
Agenda
Conclusion S9 2015
Conclusion S9 2015
Overview 2015
S9
of studies
Discussion S8 2012
Discussion S8 2012
Discussion S9 2015
Discussion S9 2015
Discussion S9 2015
Discussion S9 2015
Discussion S9 2015
Discussion S9 2015
Discussion S9 2015
Discussion S9 2015
Discussion S9 2015
Discussion S9 2015
Conclusion
and Future S10 2016
Work
Conclusion
and Future S10 2016
Work
Conclusion
and Future S10 2016
Work
Future 2017
S11
Research
Discussion S12 2011
Conclusion S12 2011
Discussion S12 2011
Results
from 2013
S14
Mapping
Study
Results
from 2013
S14
Mapping
Study
Discussion S15 2011
Discussion S15 2011
Discussion S15 2011
Discussion S15 2011
Adoption of
BI tools by
S17 2019
organization
s
Discussion S15 2011
Research
Contributio
S17 2019
ns and
Limitations
Research
Contributio
S17 2019
ns and
Limitations
Research
Contributio
S17 2019
ns and
Limitations
Research
Contributio
S17 2019
ns and
Limitations
Research
Contributio
S17 2019
ns and
Limitations
Discussion S18 2014
Discussion S18 2014

Discussion S18 2014
Discussion S18 2014
Discussion S18 2014
Discussion S18 2014
Discussion S18 2014
Discussion S18 2014
Discussion S18 2014
Discussion S18 2014
Discussion S18 2014
Discussion S18 2014
Discussion S18 2014
Discussion S12 2011
Future
S19 2016
Work
Discussion S19 2016
Discussion S19 2016
Conclusion S19 2016
Discussion S19 2016
Future
S19 2016
Work
Conclusion S20 2016
Future
S19 2016
Work
Discussion
and Future S20 2016
Work
Discussion
and Future S20 2016
Work
Discussion
and Future S20 2016
Work
Discussion
and Future S20 2016
Work
Discussion S21 2015
Conclusion S21 2015

Conclusion S21 2015
Discussion S22 2014
Discussion S22 2014
Discussion S22 2014
Discussion S22 2014
Discussion S22 2014
Conclusion S22 2014
Future
S23 2012
Work
Future
S23 2012
Work
Directions
for Future S23 2012
Research
Directions
for Future S23 2012
Research
Directions
for Future S23 2012
Research
Directions
for Future S23 2012
Research
Directions
for Future S23 2012
Research
Directions
for Future S23 2012
Research
Directions
for Future S23 2012
Research
Directions
for Future S23 2012
Research
Conclusion S24 2018
Directions
for Future S24 2018
Research
Directions
for Future S24 2018
Research
Directions
for Future S25 2019
Research
Directions
for Future S24 2018
Research
Directions
for Future S24 2018
Research
Discussion S25 2019
Directions
for Future S25 2019
Research
Directions
for Future S25 2019
Research
Discussion
and S26 2019
Conclusion
Discussion
and S26 2019
Conclusion
Conclusion S27 2018
Conclusion S28 2017
Summary
and S29 2017
Conclusion
Results and
S29 2017
Discussion
Results and
S29 2017
Discussion
Summary
and S29 2017
Conclusion
Summary
and S29 2017
Conclusion
Summary
and S29 2017
Conclusion
Discussion S30 2017
Discussion S30 2017
Discussion S30 2017

Summary
and S29 2017
Conclusion
Summary
and S29 2017
Conclusion
Conclusion S30 2017
Discussion S30 2017
Conclusion
and Future S31 2016
Work
Conclusion
and Future S31 2016
Work
Summary
and S31 2016
Discussion
Conclusion
and Future S31 2016
Work
Conclusion S32 2014
Discussion S32 2014
Conclusion
and Future S31 2016
Work
Discussion S32 2014
Discussion S32 2014

Conclusion S33 2015
Discussion S33 2015
Findings S33 2015
Findings S33 2015
Conclusion S33 2015
Discussion S33 2015
Conclusion S33 2015
Conclusion
and Future S35 2017
Work
Discussion S35 2017
Discussion S35 2017
Discussion S35 2017
Conclusion
and Future S35 2017
Work
Discussion S36 2014

Discussion S36 2014
Discussion S33 2015
Paper
S36 2014
Analysis
Paper
S36 2014
Analysis
Paper
S36 2014
Analysis
Paper
S36 2014
Analysis
Paper
S36 2014
Analysis
Revisiting
Research S37 2014
Questions
Conclusion
and Future S35 2017
Work
Conclusion
and Future S35 2017
Work
Conclusion
and Future S37 2014
Work
Conclusion S38 2019
Summary of
S38 2019
Finding
Conclusion S38 2019
Implications
S38 2019
of Findings
Implications
S38 2019
of Findings
Future
S39 2013
Research
Future
S39 2013
Research
Future
S39 2013
Research
Future
S39 2013
Research
Future
S39 2013
Research
Future
S39 2013
Research
Future
S39 2013
Research
Future
S39 2013
Research
Future
S39 2013
Research
Future
S39 2013
Research
Future
S39 2013
Research
Discussion S40 2016
Discussion S40 2016
Summary of
S38 2019
Finding
Discussion S41 2018
Discussion S41 2018
Conclusion S42 2013
Conclusion
and Future S43 2018
Work
Conclusion
and Future S43 2018
Work
Discussion S43 2018
Discussion S43 2018
Conclusion S44 2011
Conclusion S45 2011

Discussion S46 2020
Discussion S46 2020
Discussion S46 2020
Discussion S46 2020
Conclusion S46 2020
Implications
for OSS
S46 2020
Research
and Practice
Implications
for OSS
S46 2020
Research
and Practice
Implications
for OSS
S46 2020
Research
and Practice
Implications
for OSS
S46 2020
Research
and Practice
Discussion S46 2020
Discussion S46 2020
Discussion S46 2020
Future
S47 2020
Research
Conclusion S48 2020
Conclusion S48 2020
Future
S47 2020
Research
Discussion S48 2020
Research
Scope in
S49 2020
Floss
Adoption
Conclusion S49 2020
Conclusion S49 2020
Conclusion S50 2020
Conclusion S50 2020
Conclusion S50 2020
Conclusion S50 2020
Conclusion S50 2020
Conclusion S50 2020

Conclusion S50 2020
Conclusion S50 2020
Conclusion S50 2020
Conclusion S50 2020
Conclusion S50 2020
Research
Scope in 2020
S49
Floss
Adoption
Discussion S50 2020
Conclusion S50 2020
Discussion S50 2020
Discussion S50 2020
Discussion S50 2020
Discussion S50 2020
Discussion S50 2020

Discussion S50 2020
Discussion S50 2020
Discussion S50 2020
Discussion S50 2020
Discussion S50 2020
Discussion S50 2020
Conclusion S51 2019
Discussion S50 2020
Discussion S50 2020
Conclusion
and Future S52 2019
Work
Conclusion S53 2019
Conclusion
and Future S54 2020
Work
Conclusion
and Future S52 2019
Work
Conclusion
and Future S52 2019
Work
Conclusion
and Future S52 2019
Work
Conclusion
and Future S52 2019
Work
Conclusion
and Future S52 2019
Work
Conclusion
and Future S52 2019
Work
Conclusion
and Future S52 2019
Work
Conclusion
and Future S54 2020
Work
Conclusion
and Future S54 2020
Work
Discussion S54 2020
Conclusion
and Future S55 2020
Work
Conclusion
and Future S55 2020
Work
Conclusion S56 2020
Discussion S56 2020

Discussion S56 2020
Discussion S56 2020
Conclusion
and Future S57 2020
Work
Conclusion S58 2020
Conclusion S59 2019
Conclusion S60 2019

Theme ID Theme Name
Thematic 1 OSS Contributors
Thematic 2 OSS Evolution and Prediction
Thematic 3 Use of OSS in Different Domains
Thematic 4 OSS Development Process
Thematic 5 OSS Adaptation, OSS Adoption, OSS Integration

Thematic 6 OSS Quality and Success
Thematic 7 General OSS Research Issues

Thematic 8 OSS Selection and Evaluation
Thematic 9 Reconciliation between Different Models and OSS

Thematic 10 OSS Platforms and Open IoT Platform
Thematic 11 Use of OSS CASE and OSBI Tools

Thematic 12 OSS Licensing and OSS Certification
Thematic 13 OSS Interaction Design and Open Design
Thematic 14 Open Innovation and Requirement Engineering

Thematic 15 OSS and Risk
Thematic 16 Security in OSS
Thematic 17 OSS Ecosystem
Thematic 18 OSS Vs Proprietary

Theme Description
OSS contributions have all studies that talk about the

OSS developers, their motivation, participation in
project, newcommers issues, the community
perspective of OSS and the organizations that are
involved in developing or using OSS
OSS Evolution w.r.t evolution of code, architecture,
and prediction of severity level prediction in FLOSS
Use of OSS in different domains in education and in
medicine.
OSS Development process, development of OSS for

inside organization, OSS effort and maintenance effort
estimation which are part of the process and
knowledge management also part of the development
process
OSS adoption as it is, its adaptation and its

integration with other software all are discussed in
one theme
OSS Quality and its relation with success
General OSS research issues like lack of empirical
evidence etc
The selection and evaluation methods etc for OSS
Reconciliation of OSS with agile, plan driven and agile,
simmilarities and differences
the use of OSS tools either they be CASE of Business

Intelligence Tools
OSS interaction design, methods, tools, techniques etc

and open design area
Open innovation in the area of requirement
engineering
The link of OSS and risk
Security Issues in OSS
The complete OSS Ecosystem
Comparision of OSS with Proprietary Software from
Different Perspective
Codes in the Theme
OSS Developers, OSS Communities, Organizations Involved with OSS
OSS Evolution, OSS Prediction, Architectural Degradation, Severity Level Prediction in

FLOSS
In Education, In Medicine
OSS Process, Inner source, OSS Effort Estimation/OSS Maintenance Effort Estimation,
OSS Knowledge Management
OSS (Adoption, Adaptation, Integration)
OSS Quality, OSS Success
General OSS Issues
OSS Selection and OSS Evaluation
Reconciliation of FOSS, Agile and Plan Driven
OSS platform and open IoT platform
use of OSS CASE and OSBI tools
OSS Licensing and OSS Certification
OSS Integration Design and Open Design
Open Innovation and Requirement Engineering
OSS and Risk

Security in OSS
OSS Ecosystem
OSS Vs Proprietary and agile

Codes and Themes
Code
Identifier Themes Sub-Codes
OSS Interaction Participatory Interaction Design Model (Model
and Develop for Distributed Software
CA1 Design Development Environment
OSS Interaction OSS Interaction Design (Extend Open Source
CA2 Design Maturity Model )
General OSS OSS Research (Expand Searching and Identify New
CA3 Resesarch Trends and Future Directions
OSS Ecosystem (OSS OSS developers and newcomers (Conduct
Developers and Qualitative Studies to Confirm their Actual
CA4
OSSNewCommers)
Ecosystem (OSS Better Support forProblems)
Newcommers (Refine the
Developers and Classification Model and Propose Awareness
CA5 NewCommers) Mechanisms and Tools )
General OSS OSS Research (Expand by using OSS Article Hub
CA6 Resesarch like pascal and MIT repository for this analysis)
Estimation Models (Apply to one typical OSS
OSS Process (OSS database and identify benchmarks for estimation
CA7 Estimation)
Reconciliation of OSS acuraccy)
Infrastructure for Reconciliation of FLOSS, Agile
with other and Plan Driven
CA8 Approaches
Reconciliation of OSS Experimental Study and Industrial Survey
with other (Conduct to Validate Results of this SLR about
CA9 Approaches Reconciliation of FOSS, Agile and Plan Driven)
OSS Process (OSS
Knowledge OSS Projects (Generalize Set of Practices)
CA10 Management) Adoption of Frequent Releases by Projects
OSS and Quality (Develop Meta Model for Facilitating Quality
CA11 Assessments of these Projects)
Model Quality Assessment (Multi Criteria Decision
OSS and Quality
CA12 Making(Validate
OSS Quality Model Problem)on Actual OSS
Ecosystem)
OSS and Quality
CA13
Use of OSS in Quality Assessment Process (Define Completely)
Different Domains Better methods to Assess Student's Learning
CA14 (Education)
Open Innovation and
Open Innovation in RE ( Develop a Method for
Requirement Requirement Extraction and Prioritization)
CA15
OpenEngineering
Innovation and Open Innovation in RE ( Approach to
OSS
Requirement
Selection,
CA16 Evaluation,
Engineering
Adoption, OSS
Requirement
Selection, Management
Extraction andand
Prioritization)
Maintenance
(ADefine Process)
OSS adaptation, and
CA17 OSS Integration
OSSOSS
Risk (Understand
Community Relationship
Ecosystem Between Risk
(Evaluate)AOSS
OSS Ecosystem (OSS Failure, Risk Mitigation etc)
OSS and Risk)
Community Adoption via Company (Evaluate data of OSS
CA18 Project Reprositories)
OSS Success and Quality (Review Relationship in
OSS and Quality
CA19 detail)
Inner Source Inner Source (Further Research)
CA20
Inner Source Inner Source (Identify Key Challenges with
CA21 Resolution Strategies via Survey of Practitioners)
OSS Licensing, and OSS License (Motivation Factors for Selecting
CA22 OSS Certification License for Community)
OSS Macro Process (Validate it in light of activities,
OSS Process
CA23 their Characteristics and Roles)
OSS Process OSS Process (Investigate Analysis Done by
CA24 Researchers)
OSS Ecosystem( OSS Community Participation and Engagement
CA25 Community Community(Acquire Indepth
Dynamics Knowledge)
(Software Ecosystem,
OSS Ecosystem (OSS Structure, Mentoring, Project Governance,
CA26 Community ) Difficulties Associated with Finding a Task, and
Project Characteristic.
Community Dynamics (Expand Data Sources and
OSS Community Search Strings to give Insight)
CA27
Use of OSS in Integrate OSS into Educational System (Measure
Different Domains Participation of Women OSS Developers in OSS
CA28 (Education) its Effectiveness)
Projects (Study Aspects Influencing Participation)
OSS Ecosystem (OSS
CA29 Developers) Participation of Women OSS Developers in OSS
OSS Adaptation OSS
Projects
Adaptation
(Identify
from Problems
Vendor Regarding
Perspective
Issues
(Identify
of
Open Source
Practicies Software
for Development
Identifying
Gender Success
Inequality) Maturity
Fators and
CA30 OSS Ecosystem(OSS
Model (SSDMM) Potential
(To Measure
Risk Factors
Maturity Level of
Organization) Vendor Organization)
OSS Process
CA31
Open Source Software Development Strategy
OSS Platforms, and Open(Validate
IoT (Clear
ViaAnd
Multiple
DeeperCase
Understanding)
Studies)
CA32 Open IoT Platforms
Developer Productivity (Impact of negative and
OSS Developers positive impact)
CA33
33 Auther Intent Finally
salma to do 20
Codes of Author Intent
nd Themes
Sub-Sub- Codes
Model a participatory interatction design process model for
distributed software development environments and develop a
process model.
Extend Open Source Maturity Model with Proposed OSS Interaction
Design
To expand searching and identify new trends and future directions
of OSS research.
Conduct qualitative studies to confirm problems actually faced by
OSS developers and newcomers.
Refine the classification model and propose awareness mechanisms
and tools for better support for newcomers.
To include OSS article hubs like pascal and MIT repository for this
analysis.
To apply the estimation models to one typical OSS database and
identify benchmarks for estimation acuraccy.
To investigate software development process tailoring w.r.t agile,
FOSS and plan driven. Automate some steps of process some steps
to reduce effort and improve quality The authors intend to develop
infrastructure
To conduct using
survey of an optimization-based
academic approach.
and industry experts to validate
results of quasi systematic review
To generalize set of practices for OSS projects.
Developing a meta model for mining open source database that will
facilitate in quality assessment of projects adopting frequent release
approach
Model quality assessment as a multi-criteria decision making
problem for evaluating quality across multiple domains.
Validate the quality model on actual OSS ecosystem and define
perform an exploratory mixed-methods
complete quality assessmentresearch using OSP in an
processes
undergraduate course in SE
New and better methods to assess student's learning in this context

Developing a Method for requirement
would be seen extraction from internet and
for automatic requirement prioritization
To use proposed sorting algorithm for fully automatic and non
intrustive Open innovation approach to Requirement extraction and
prioritization.
Define process that improves the selection, management, and
maintenance of OSS components and understand causal
replationship between risk, failures, risk metrics and risk mitigation.
Evaluate the OSS project community ecosystem and the adopting
company and evaluate data of OSS project repositories.
To review the relationship between OSS success and quality in
detail.
Establish a research agenda on Inner Source
Conduct survey on of practitioners to identify key challenges
involved in ISS development and propose some resolution
strategies.and social) for selecting open
To find motivation factors (economic
source licenses both for the software community and the local
(Pakistani) community.
Validate the proposed OSS macro process in light of process
activities, their characteristics, and roles.
Investigation of OSS process analysis done by researchers
Acquire indepth knowledge about community participation and
engagement.
They also would like to include software ecosystem, community
structure, mentoring, project governance, difficulties associated
with finding a task, and project characteristic.
Expand data sources and search strings to give insights into the
community dynamics.
Implement roadmap to strategically integrate OSS into educational
system and measure its effectiveness.
To study aspects influencing the participation of women in open
source projects and identify problems regarding issues of gender
inequality in open
To identify source
practices communities
used and the
for identifying software
success industry.
factors and
potential risk factors in adapting open-source software
To developdevelopment
open-sourcefrom
software
vendors’
development
perspectives.
maturity model
(OSSDMM) to measure the maturity level of vendor organizations in
implementing an open-source development strategy and validate it
via multipe case studies.
Clear and deeper understanding of what is open IoT
To analyze sentiments and their negative and positive impacts on

productivity, and how these sentiments vary between relases.
software development models.
Process tailoring is the act of
particularizing a general process
description to derive a new process
Asapplicable
future work, to awe specific
intendsituation to review
some(Ginsberg of the concepts and Quinn, related 1995). to ISO
We claim[14]
9241-210 thatand it isparticipatory
necessary to design tailor
these
to reflect processes on thetomain suit specific
principles project
and
and
howorganization
they were addressed contexts. or This Traceable
not in the to Study
tailoring
should
papersalso consider
surveyed. the key distinctive
Moreover, with the
features
result of plan-driven,
obtained Paragraph in this agile and
systematic FOSS Heading Study ID Study Year
Finally,collaboration
models:
mapping, wethe must gaps extend
and and thethe Open
discipline
lack of
Source Maturity (Magdaleno, Model [54] with the
2010a).
studies
proposed interaction involving the areas
designasmodel. of
Collaboration can be defined a group Conclusion S2 2017
interaction
Considering design
the and
inher- development
ent advantages of
of two we
FLOSS, or moreintendpeople to advance working in the to
of
We
research software
achieve intend a common
on to development
this extendsubject. goal SRfollowing
(Vreede
ourTherefore, to include and a Conclusion
the S2 2017
software
Briggs,
more studiesprocess
2005). model,
byCollaboration
searching we believe
theisbe anto
well-
Innext
that
known the step
future,
interaction
important
digital
of we our research
aim
design
factor
literature to conduct
activities
for
will
software
databases. some
will Webe
expand
qualitative
considered thisduringsystem-
studies the atic
to mapping
confirm
different the to
will organizations
also
identifyextend ourtodata
approaches, achieve analysis
methods, theiras we Conclusion
stages S3 2009
problems
of the
productivity, FLOSSevidenced development
quality by
and the literature.
process,
knowledge-
plan
techniques,
Wealso to
arediscover
particularly and
conducting thethetrends
intools somefor
early and future
participatory
interviews
stages.
We sharing goals.
plan
directionsdesign
interaction to In particular,
refine the
of OSSinresearch. software Conclusion
classification
distributed S9 2015
with
development
model experienced
based is a
on OSS
complex
the developers
results process ofthisthe and
that
software Our
newcomers development
future to workverify willtheenvironments.
extend
main prob-
involves collaboration
interview analysis. Addition- among ally, several
based
people
on analysis
lems thefaced over
model, totime
byOSS article
itnewcomers
isto achieve
possible hubs a like
from
to common for
their
propose Conclusion S9 2015
example
goal (Cugola http://flosshub.org
perspective.
and Ghezzi, 1998). toor
awareness
http://pascal.case.unibz.itmechanisms and or tools
the MIT
Therefore,
offer software
better work, support devel-
for opment
newcomers is a Discussion S12 2011
In the
typical future
.Finally,
example repository.
it is also we importantaddress
plan
of collaborative to towork
the We
(DeMarco in-depth
investigate plan and investigation
towhether
useLister,more the of Discipline,
estimation
sophisticated
1999). results of this
accuracy
techniques
quasi-systematic by applying
of string the
similarityestimation andlevel a Conclusion S20 2016
meanwhile, relatesreview to theare consistent
planning
with models
better
adopted data to
whatinissoftware one
cleansing typical
observedprocess to OSS
get
in practice, database,
finer
definitionresult
i.e., in
andWe
organiza- theandhave identify
tional
rigidity in fact benchmarks.
routine.
of identified
control Toemployed these in Future Work
accomplish S23 2012
motivations,
The future
process
this, we execution
intend direction
strategies,
to plan forand
(Boehm advantages
ourand work
conduct Turner,isand
to
an
evaluate
difficulties
experimental further
in study.
other practices
2003b). software that
This experimental projects
can be
incorporated
that are
study
Both not OSS
willcomplementary
involve in the but OSShave
a survey project
and the goal
of essential work
industry to Future Work S23 2012
meet
and
structure. customers’
inacademic The experts
any project, expectations.
work butstructure
to validate
in differing ForOSS
of this Directions
the
reason,
results manyis discuss of
informal, the
proportions, depending on projectA for Future
projects and the lessons
varies
conclusions. learned
in each S25 2019
from
similar OSS
project,
characteristics. projects
strategy andwas isForcan
dy- also
anamic
appliedbalanced be
bydue adopted
to
(Bekkers
mix Research
s future
This
transient
betweenin quality
other
et al., 2008) work, model
we
contributors;
types
collaboration of
to determineintend
can
software beto
therefore,
and the used
model
projects
discipline, asitOSS
a
is a
factorsit Conclusion
starting quality point assessment
for the quality
asselection
a MCDM
assessment S28 2017
ischallenging
regarding
that
neces- most sary the
task
touse
influence to of generalise
understand rapid
the releases.
how a set ofof
these
of
As
Our
We problem.
an OSS
future
software
practices
aspectsaim to
plan ecosystem,
work,
isvary
toThis
development
fordefine
perform will
we
all OSS afford
are
and distinguishand it
projects.
a process
an us
is
considering
model.
exploratory the
in
the our
that Conclusion
plans
using
opportunity
improves thefor
software
mixed-methods the
resultsto future
choose
thedevelopment of
selection, thisworkfrom
study
researchmanagement to a define
models
using torange a
OSP in and Future
build of a S31 2016
MCDMcomplete
meta-model methods quality for oneassessment
the (orminingmore) of process
that
an(as
(Magdaleno
and maintenance
undergraduate
We
described are
et al.,
interested
of2010).In
Sect.
OSS in
course 5) in
and SE,
using
order
components.
to inopen
a
applyordercan
to
it
Work
be
source
facilitate
Wetowill usedbases to
process
review
obtain evaluate
in
new view of
tailoring,
existing
insights quality
gathering
COTS in
it is quality OSS
possibledata
that nonintrusive
to inleadaacross
supportreal
to quality approach
multiple
assessments
the assessment.
project towith
domains.
of
managerthe
an
OI inqualitythe
As by
Conclusion S32 2014
assessment
experience
context of RE with processes,
by this
using approach.including
fully automatedWe the
will
consequence
of projects
automating
ones
experiment enacted some new
adopting
in of
with theameasures
the thesteps,
project
combination may
Frequent possibly
partner ofbe
Asapproaches
needed
reducing future for work,
the
Release
the effort helping we
assessment, companies
plan
approach. to develop
but fuel
a ais Conclusion
this S33 2015
inside
their
method
companies
RE control
processes
that and
extracts arequired
in practice.
withpredefined
users’
innovative
to conduct
Giving project
software ideas
the
this best
activity
particular way andto
focus improve,
improving
on thecourse, and complete
the qual-
differences ity
in
the arequirements
software
created
quality evolution
outside
model, automatically
their
and a company
way to using
from
prove a Conclusion
and appropriateness
between OSSofand
combination of the
closedlearning
different resulting
source and Future S35 2017
its boundaries.
internet
processcapabilities
componentsTo propose(Park resources
Following
in
et
identified
suchquality
al., and
2006).inthis
measures toThus,
design
line
assessment.
the weof
SLR, we
we
approaches. research, We we also
have plan to
designed investigate
an(i) Work
automated
intend
investigate
try to to understand
developprocessing
into twoan thetocausalsupport
infrastructure
directions:
better
automatic methods
requirement sorting to assess
algorithm
prioritization. students’
based on Discussion S35 2017
using
evaluating
relationships an optimization-
learningthe OSS
between in[35].project
this based
risks
context. approach
community
and failures,
the Kano
(Magdaleno,
ecosystem Model
and2010b).Finally,
therisk We
adopting believe it is also
company, that
risk metrics, and mitigation. To
propose the
important algorithm
by organisational to can
investigate
such measures be used
modelling whether inand
we investigate thethe Discussion S36 2014
process
results
into
analysis, two ofofdirections:
automatically
this
and quasi-systematic
(ii) evaluating prioritization
(i) evaluating thereviewdata the
users'
are
OSS project requirement
consistent
available with and
what
in the OSSecosystem
community thus
is eventually
observed
projects and in
may
practice, become
repositories,
As a future i.e., an
in
work, essential
organiza-
such this ascompany,
code cornerstone
tional
relationship routine.
repositories, in Discussion S36 2014
a the automated,
fully adopting non-intrusive by will
bugTo
be accomplish
reviewed
trackers, and
organisational this,
in-depth wewith
mailing
modelling intend
lists,
and respect to plan
applying
analysis, to
RQ4approach
andand in conduct
(ii)order
measurements to
evaluating OI an used
to expand based for
experimental
the data requirements
theainformation
on available study.in Conclusion
statistical S38 2019
This
gathered
analysis.
the extraction
experimental
OSS by this
projects
Techniques and SM prioritization.
study study
repositories,
and will
tools involve
and such
forelicit asa
this
survey
analysis, of
code repositories,industry
the further and academic
insights.
prioritisation bug trackers, and multi- experts
and
to validate
mailing criteria lists, the
decision results
applying and discuss
making
measurementswill bethe
conclusions.
based on aAinstatistical
presented similar
the next strategy
section.
analysis. was
applied by (Bekkers
Techniques and tools for this analysis, et al., 2008) to
determine the
the prioritisation and multi-criteria factors that most
influence
decision making the selection
will be presented of software in the
In this systematic mapping, we found
the publication on the intrinsic (social)
Wemotivation
envision future factors workof open couldsource perform
software.
a deeper We analysisalso foundand synthesisthe publicationon the
on the extrinsic
empirical research motivation (economic)
on ISS development.
To
Basedfactors
enhance
on of this open
the findings
source
analysis and software.
ofsynthesis,
this review,Wewe Conclusion
and Future S43 2018
willalso
we intend
furtherfound selectingaof
toinvestigate
conduct the open
compre- source
limitationshensive of Work
surveysoftware
the ofcurrent license
practitioners on the
research base
to identify
on ISSofthe Conclusion
economic,
development
key challenges social and commercial
and establish
involved ainresearch
ISS and Future S43 2018
As future
(managerial)
development
agenda work, we
perspectives.
on and intend
inner propose to future,
In
source. apply
somethe we Work
findings willof
resolution tryour to study
strategies
find out intomore
some
overcome recent
other the OSS
projects
motivation andfactorsprovide
challenges. a validation
both economicofand the Conclusion S45 2011
proposed
social perspectives
OSS macrofor process
selecting though the a
Furthermore,
open
practical source point weofintend
license. view These to will
investigate
analyzing beOSSour Conclusion S48 2020
how
research researchers
projects question
process perform
inactivities,
future OSS whatprocess
their are
We
the plan to
characteristics
analysis
motivation in OSSextend
and thisinwork
projects,
understanding
factors in the
including
selecting how
an
After examining
future more
bytechniques
concentrating than 69on different
the Conclusion S48 2020
approaches,
roles
open aresource
involved in each
software and activity,
license tools withthey
fact
research
that identified
have In the
still
respect usedpapers
notfuture,
toclear
to and
research yettaking
this
retrieve
economic study
ingapsandthe
OSS
the and
can survey
be
by
literature.
process
social
findings
introducing
perspectives intomore
extended account,
intosoftware
include
information.
researchwe software
constructed
questions to
community? a Conclusion S50 2020
Are road-
Moreover,
ecosystem,
acquire map
the results thethat
in-depth of can
community
authors
RQ1 be are
knowledge
are used by
instructure,
plan- any to
ning
about
accordance
with educational
expandmentoring, the search
community
perception institutedespecially
project by including
ofparticpation
local governance,
(Pakistani) high
andmore open
schoolsdto Conclusion S50 2020
sourceinstra-
terms
difficulties the tegically
search
associated
engagement.
software integrate
strings
with
community? and OSS
finding a
additional
task,and into the data
educational
project sources,
characteristicssystem.
not considered The will
that
Based
roadmap
by on the findings inof this research,
provideAs athis morestudy
follow-upis designed
totofind
insights this more
toresearch,
thetargetrelevant
three
community we Conclusion S50 2020
wemain have
studies planned
aspects
on dynamicsof
community the following
implementation:
expect to deepen the study on the domain.
dynamics. research
using
Thus,
objectives,
OSS vides
pro- in themore to beinsights
curriculum carried outthroughout
itself
about in research
the near
future, aspectsto more thatstrengthen
may influence thethat the
existing Conclusion S51 2019
the semester,
topics
participation and using
issues
of in
women OSSthe in
in final
area
open projects
will
research
at the end work,
helpful of the and
for to stressand
semester,
researchers. thesource
OSS
using
softwarecommunity
vendors’ projects and tosoftware
meet the Conclusion S53 2019
OSS in extra-curricular
development projects, activities.
astowell asThe
to
As a future
maximum
future work,of
benefits
direction wethis
of plan
OSS provide
paradigm.
research is toa
• To
propose
Considering develop ways
that open-source
of
there addressing
areand software
the
sentiments Conclusion
more
implement •formal
To identify
development thisdefinition
the
roadmap of
practices
maturity the open
for IoT
measure
identified
associated
platform
addressing
its effectiveness.
problems
with
rather
the than
identified
The
regarding
positive andmodel
aroadmap
perspective
success issues
negative by
factors
can be
of and Future S55 2020
(OSSDMM)
gender
consulting
polarity thatto
inequality measure
different
were in
marked the
open
stakeholders maturity
source
as notin Work
• To identify
applied
level via
of the
three
vendor potential
phases:
organization risk factors
changes in to in Conclusion
communities
detail.
specifiedThe open in and
theIoTsoftware
selected
platform factories.
studies
term is and Future S55 2020
adapting
curriculum,
implementing open-source
implementation
open-source software in final
regarding
frequently
development
course proj- software
usedfrom
ect and practices,
invendors’
public and
establishment there of
without
perspective is Work
stillclear
room development
for further strategy
investigation on
• To • To understanding
identify
extra-curricular the could
practices
activities. cause
for
problems theconduct
addressing associated
like
the
multiple
sentiments
misunderstanding
identified
case studies
success to orthe at
even
factors
Conclusion S56 2020
software
specific
wrong strategic impacts.vendor organizations
decisions.
Moreover, to
and risk factors Our there future
is a
evaluate
tendency
work intendsofthe toefficacy
provideofathe
a considerable setmodel
clearer of open
and Conclusion S58 2020
source deeper softwareunderstandingprojects to ofhaveopenregular
IoT
release
platforms cyclesfor the andcommunities.
to adopted the The so
called
open frequent
issues listedreleases.
above are Wealso plan someto
investigate
possible directions sentiments of investigation.
in this context
and to which extent they influence
software productivity. We also want to
investigate how programmers
sentiments vary between releases.
Traceable to Complete
Future Direction
Synthesis Sheet
1
52
53
71
107
122
123
135
144
158
162
173
174
OSS
175 Selection,
Evaluation,
179 Two Themes Adoption,
OSS
adaptation,
180 and OSS
Integration
191
213
214
218
232
233
240
241
247
264
268
280 two themes
281
282
287
Codes and Themes
Code Identifier Themes
OSS Interaction Design

CG1
Security In OSS
CG2
Security in OSS
CG3
Security in OSS
CG4
OSS Adaptation (Safety Critical Systems)
CG5
OSS Evolution
CG6
(OSS Ecosystem) OSS Community and
CG7 Organization
(OSS Ecosystem) OSS Community and
CG8 Organization
(OSS Ecosystem)Organizations Involved in
CG9 OSS
Use of OSS CASE and Open-Source Business
CG10 Intelligence (OSBI) Tools
General OSS Research

CG11
(OSS Ecosystem) Deploying OSS
CG12
OSS in Organization
CG13
OSS Ecosystem (OSS Process and practices
CG14 inside Organization)
(OSS Ecosystem) Newcommers of OSS
CG15 Communities (Barriers)
General OSS Research

CG16
Use of OSS CASE and Open-Source Business
CG17 Intelligence (OSBI) Tools
(OSS Process) OSS Effort Estimation

CG18
(OSS Process) OSS Effort Estimation
CG19
OSS Evolution
CG20
OSS Evolutionand Prediction
CG21
OSS Evolution and Prediction
CG22
Use of OSS in Different Domains (OSS in
CG23 Medicine)
CG24 Medicine)
CG25 Medicine)
OSS and Quality
CG26
Use of OSS in Different Domains
CG27 (Education)
CG28 (Education)
CG29 (Education)
OSS and Risk

CG30
OSS and Risk
CG31
(OSS Ecosystem) OSS Platform
CG32
OSS Success and Quality
CG33
OSS Success and Quality
CG34
CG35 Medicine)
CG36 Medicine)
Reconciliation of OSS with other
CG37 Approaches
(OSS Selection, Evaluation, Adoption, OSS
adaptation, and OSS Integration) OSS
CG38 Adoption
(OSS Selection, Evaluation, Adoption, OSS
adaptation, and OSS Integration) OSS
CG39 Adoption
(OSS Ecosystem) OSSS Community
CG40 OSS Developers
Codes of Gaps
Codes and Themes
Sub-Codes Sub-Sub- Codes

OSS Interaction Design (Method, Tool, No method, tool, technique for interaction
Technique) design in FLOSS
Security areas in governance and
Security in OSS (Governce and deployment are not much researched
Deployment)
Security Management Framework (Socio- The socio-technical perspective in a
Technical Perspective) security management framework has not
notof
Lack gained much
research in attention
knowledge
Security in OSS (Knowledge Management) management aspect of open source
Less explored areas are OSS for safety
OSS Safety Critical Systems (to explore security
critical automation system, maritime
areas such as maritime systems, rail
industry, process industry, oil and gass, off systems, rail industry, process industry, oil
highway equipement andProfile
miningAnalysis
industry) and gass, off highway equipement and
OSS Evolution (Change Change profile analysis
mining is done on limited
industry
Done on Limited Level) level as OSS systems evolve
Organization Contribution to OSS Limited contribution to OSS communities
Community (Limited Level) by most of the organizations
Experience Reports in OSS Area (Lack of Lack of explicit research questions in
Introducing
Explicit Research
OSS into Questions)
Organziations experience reports
(Unclear Cost Analysis) Unclear cost analysis for introducting OSS
into organizations and product operation
Product Operation Over Long Period of over long period of time
Use of
Time
CASE
(Unclear
Tools inCost
Organization
Analysis) (No No empirical research on the use of OSS
Empirical Research) CASE tools in organizations
Details of research methods and findings
OSS Research (Lak of Focus, Details of
Deploying
Research OSS in
Method andPrivate
Clear Sector (No
Contribution) lacking in studies thus lack precise focus
Research) No research
andon deploying
clear OSS in private
contribution
sector.
Less
Less empirical
focus work onindividual
on describing OSS in
OSS Organization (Less Focus on organizations and that also not of good
OSS in Organizations (Less Work of Poor organization
Describing) quality and lacks focus
Quality and Lack of Focus)
OSS Processes and Practices (Less Work Less work on OSS processes and practices
Inside Organizations) inside organizations
Barriers Faced by Newcommers (Less Less qualitative analysis of barriers faced
Qualitative Analysis) by newcommers.
OSS Research (Research Community Research community doesn’t perform
Doesn’t Perform Empirical Studies or
OSBI ToolsReplication
(Empricial of Studies) Lacking) Empiricialstudies
Foundation empirical or replication
foundation is lacking of studies
in studies
on OSBI tools.
OSBI Adoption by Organization (No No integrated framework is there for OSBI
Integrated Framework) adoption by organizations
Effort Estimation ( How OSS can Affect how OSS can affect the accuracy of effort
Accuracy of Effort Estimation) estimation.
Early OSS Effort Estimation (No work for No focus on early effort estimation for
OSS Web Projects) OSS web projects
OSS Evolution process support (less work OSS Evolution process support (less work
on aspects of configuration management, on aspects of configuration management,
complexity and control, and evolutionary complexity and control, and evolutionary
Evolution Prediction
paths)
(less used methods Evolution Predictionpaths)
(less used methods
are choas theory and software reliability are choas theory and software reliability
growth models)
Evolution Prediction (need of common growth models)
Evolution Prediction (need of common
corpus of OSS projects for comparing corpus of OSS projects for comparing
results) results)
OSS in Dentistry (Low level of evidence and OSS Dentistry (Low level of evidence and
for poor quality) for poor quality)
OSS in Dentistry (No Research on use of OSS Dentistry (No Research on use of OSS
OSS in prosthodontics) in prosthodontics)
OSS in Dentistry (No Comparision between OSS Dentistry (No Comparision between
OSS an other Software) OSS an other Software)
lack of qualitative measures for
Community Quality ( Lack of Qualitative measuring process maturity for accessing
Measures for Measuring Process Maturity) Not many thescientific
quality ofmethods used to SE in
a community.
USE of OSS in SE Education (Not many
education and methods from research
scientific methods used from other areas other than software engineering can
Open Source Research
ProjectsAreas)
in SE Education No selected studybenefit.
focuses on learning the
(Requirements or Configuration areas of requirements or configuration
Effective Learning
Management) However, veryinfew
of SE Skills ( None of the management Openstudies cite
Source these
Projects
learning approaches, and none of them
approaches provides details on how to provides details on how to design and
design and implement pedagogical practice
OSS Risk (Work on quantitative
for them) and implement pedagogicaland
Work on quantitative practices that
qualitative
qualitative measures of risks is lacking in result in effective learning of SE skills
OSS specific community and respository measures of risks is lacking in OSS specific
community and respository measures
OSS Risk (No work
measures)
on casual relationship OSS Risk (No work on casual relationship
between risks, concrete risks and between risks, concrete risks and
effectivness of mitigation activities) effectivness of mitigation activities)
OSS Platforms (no research on mobile OSS Platforms (no research on mobile
platforms) platforms)
OSS Success and Quality (Less studies to
OSS Success and Quality (Introduce tools
for evaluating quality and success of OSS) introduce tools for evaluating quality and
success of OSS)
OSS Success and Quality (Less empirical OSS Success and Quality (Less empirical
evidence) evidence)
OSS Adoption (Low Level of Adoption of OSS Adoption (Low Level of Adoption of
Modern ICT Tools and Infrastructure in Modern ICT Tools and Infrastructure in
Health Industry) Health Industry)
OSS Adoption ( Not Mature in Public OSS adoption ( Not Mature in Public
Administration) Administration)
Integration of ASD and OSSD (Less Integration of ASD and OSSD (Less
empirical evidence to support the idea) empirical evidence to support the idea)
FLOSS adoption (Experience reports are FLOSS adoption (Experience reports are
lacking) lacking)
FLOSS adoption (Economic Factors are not FLOSS adoption (Economic Factors are not
well Covered) well Covered)
OSS community (Research is lacking in OSScommunity (Research is lacking in
Joining and Abandonment Process of Joining and Abondonment Process of
Developers) Developers)
40 Gaps Finally
there are few studies on MTTSA of interaction
design proposed or used for/in FLOSS
development;
es of Gaps • methods of interaction design proposed
specifically for the development of FLOSS were
not found; the studies found
used existing methods of interaction design in Traceable to Study
The socio-technical the context perspectiveof FLOSS; has not gained
much
• techniques attention of in this
interaction
Paragraph research design,areaproposed
(2 out of Heading Study ID
42 papers).
specifically forAccording
the development to the result of FLOSS,of socio- were
The
technical results
Interestingly,
analysis shows that
theselected
on aerospace
the securitydomain
selected areas inthe
papers,
not found;
constructionone of the and verification papers,(secure Lichtner Main Finding and
represented
discussions in our
between study in
technicalonly 3 papers
and social was D+H3:J42iscussion S2
et al. [32], used
architecture, codepre-existing
review, andtech-securityniques and
testing)
the
did top
aspects domain
not consider
Software seem
size has inquite
thebeenboth related
unbalanced,
distributedthe most surveys either
development
common [10],
[11]. are
(Coverage At followed
the same
rate:98%
environmentbytime, researchers
the
versus of most
16%
FLOSS; with more
represented
in average). Conclusion S4
attribute
interests to
than analyze
other evolution
areas in of OSS
Governance projects.and
domain
The
•Several
the principal(automotive)
socio-technical
typesNext ofinterest
metrics in
perspective our
ofof
havethe study has
selected
been was
as not
the the
main
studies
employed
Furthermore,
deployment.
most
target represented
to blend the result
based
both in thethe on this
our
other
technicalSLR
two study
research,surveys
and also
the the
security is
to measure
shows in
the thegap
studies ac-
software
that
intivities
OSSthere of
size. is prototyping
These
developmenta lack metrics
of and
knowledge
are range
mostly Conclusion S4
social(ranked the 4th anand the 3rd) [10], [11].
fromsystems
evaluating;
management coarse few in studies
grained
aspect
technical
organization.
oflevelhave
open
driven. metrics
source This
addressed such canthe
security. asbe
Less
number viewed explored,
activities as
of files, however condition
ofaestablishing
necessary
modules, still represented
requirements
and functions, within and
to aby
fine
Severalprimary researchers
studies, did
is work mentionon OSS the for knowledge
safety- Conclusion S4
security
grained levelmanagement
designing
metrics such framework
alternatives;
asdevelopment,
number asof bothLOC,
problems
critical
aspects are in
automation
of securing
equal OSS
systems
importance. and maritime
Technical
•methods,
the majority
however, we and of
cannot theidentify
classes. selected
Several studies
any approaches,
study do not
tackle
systems.
security
present Process
practice
any type industries
considering
of validationand rail
different industry
through social Results and
other
(men- than
this tionedsource
security in code
issue
the top analysis
from
five using metrics,
knowledge
domains in the Discussion S5
Most aspects
The togeneral (e.g.,
organizations
analyze culture
empirical
costs
OSS
management related
seem
evolution and structure)
studies.
toto suchrather
have
have
perspectives. ofbeen
a migration
also open
limited
evidence
source
are unclear
contributions
employed provided
develoopment
[62,
tointhe 187],
theOSSin both
and
research [10]
there
communities and
will literature;
assure [11])
are very the are
[33,few 43, Conclusion and
not
studiesrepresented
effectiveness
showing among and
complete the
efficencyprimary
calculations of studies4.
the of have
the S6
•91, Lately,
98]. The metrics mostrelated common to change
way of participation
activity Future Work
Finally,
istrue
being oilan
costs andand
activegas,
implementationsavingsoff-highway
user of
that of
(1) the equipment
tool.
introducing
reports occasional OSS and
also
We classified
mining been included
53 of to understand
the 112 empirical OSS
papers
products
bugs
evolution. toindustries
theinto community
These
represented
organizations,
metricsas [43, 98,
measure
in99].
and the
(2) previous
keeping
Only
changes one in Results S7
identified
survey
the about
OSS in this
products review
compliance operational withexperience
safety
over reports.
astandards
longer
of
Hence, 32
source respondents
the code
most such
common from
as a
number sample
methodour of of tertiary
program
of primary
studying
[11]
period
education are
elements of not
time. represented
One
institutions
(functions/ paper among
reports
had/classes/methods)
participated cost savings
actively
fromthe OSS
studies.
an OSS phenomenon
Hence,
migration it could in
projectorganizations
conceivably
at Beaumont betois Results S7
by writing
changed
through code,
in
experience consecutive
while 14
reports. had versions.
contributed
These Change
experience an
hypothesized
Hospital [81], that these
butthrough industries have yet not
activity
OSSDespite
reports
been
community
aslack
recorded
having explicit
intensified a itclose
inisresearch
by
published
SCM reporting
systems
relation
software
just
questions,
systems
isafter
to bugs
the also the
[91].
CASE used
and
or
in In
research fact,field,
ainitial
few
most
mostonly
stage
cases.
also lack
papers
of the
Most seven
aclarity,
method
do
ofproject
the little
workismore
experience finished.
deals
description.
thanwith
reports Results S7
mention
Despite explored
this suchlack by of open
issues assource
research
many solutions.
questions,
organizations
finding
A few
discuss change
other
the use size,
issues
of OSS are changeworthtools
CASE effort distributions.
mentioning.
in First,
Aallfew limitations,
seem
studies to be dodata blinded
change analysis, by
profile and
the sothe
perceived
analysis on. context
We
as OSS
ofoforganizations.
the eight
furthermore
empirical Given
found
research
the
that number
many
papers
of of
the
from
OSS Results S7
advantages
the public
CASE tools of OSS
systems
sector focus
available, and it have
is evolve.
on therefore
deployment
surprising that adopted
of
the OSS use
publications
it without
products. lack forming
per-
Besides details[37], about
anystudied
which the
hasresearch
cost-benefitainmixed
of such tools has not been any
methods
evaluations
sample, no and
in their findings.
paper focuses
empirical own
research As
context a consequence,
on papers. [91,
deploying 187, OSS 190].in Results S7
several
The adoptionpapers of have
OSS limitations
is
the private sector. Second, 27 of the 59 furthermore when it comes
frequently
There
to howare
bottom-up, they relatively
indescribe
the sense few empirical
these
that issues. publications
Moreover,
it is introduced by
empirical
on OSS research
in organizations, papers report
andare thefindings
qualityfrom of Results S7
manyengineers
samples of theof research
rather
several than papers
strategic
organizations explorative
top-level
from the
published
and Itthey areworkdecisions
istonot seegood enough. Much of
private sec-therefore
is possible tor.the However, lacking
the
published
[188]. asafew
lack precise
of studies
as eight focus
Results S7
conducting and clear contributions.
papers
While
research
report
there lacks isqualitative
findings
quite
relevance a lotfromanalysis
of
and
oneas
research
a clear
supporting
single on private
focus, specific
and
the existence
organization. Hence, of problemsmost that hinder
research papers
development
does
newcomers’not drawprocesses enough support
contributions. in OSS communities
from related
Quantitatively Results S7
dedicate
e.g.Most [167], usedrelatively
sources
there little
is little space
development
literature research to describing
ontoolsusing ofthese
thethe
analyzing
data are notand historical
individual
properly data can
organizations.
documented. bring highlights
None of the of
processesthe practices inside organizations.
data documentedby
barriers faced
Broadlyspeaking,thesystematicreviewrevealsth
study newcomers,
whether there but was any Overview of studies S9
conducting
Asat:(1)themajorityofstudiesarenormative
far as research
involvement qualitative of community
OSS analysis
features concerns,
caninenrich the the the
results
evidence, from
reveal the new systematic
andlackempiricalortheoreticalfoundations(2)no
development.From facts, and mapping
help in study
finding
the result we know that the Results from Mapping
prove
the issues
most thatused many
faced byresearchers
neofthestudiesfocusontheperspectiveofBI
sources newcomers.
development don’tThere perform
toolsis still S14
empirical Study
mentioned in the data are not properly the
room studies
available or
experts(3)therelatedbodyofknowledgeisscatter replications
for studies for
based solving
on
None of theinterviews
observations, studies discussed early effort that Adoption of BI tools
documented, estimation as open
ed;thus,itislackinganall-
can be
forbarriers
OSS
issues.
and experiments
seen
Web from the result,
projects. S17
can help reveal the faced in practice. by organizations
with the highest
• Most the percentage
encompassed,integrated
of source projects being used unidentified
as the
development sources. Due to the poor
framework.Itisimportanttorememberthatthelas
datasets are inregarding
documentation CMS form,the and none of theof
involvement Discussion S19
tobservationisconsistentwiththeresultsofa
development projects in the dataset involved
the
anysources development
recentreviewofresearchon“gettingvaluefromBI”
open source framework tools, we have
as there are not no
been able to determine
conductedbyTrieu(2017,p.1). whether in each of the Discussion S19
recorded details.From the findings it is shown
projects, was there
that there has been no proper documentation any involvement of OSS.
Since this SLRthe
detailing is focused
involvement on OSS, of OSStherefore
in the we
can say that to the best of our knowledge,
development, hence we can conclude that,
there
none ofhasthebeen studies very have littlediscussed
documentation early effort of
OSS estimation
usage in thefor datasets,
OSS Web projects.which led us to
further investigate how OSS can affect the
accuracy of effort estimation.
used different
Figure 3 depictsmethods, that around tools 62% andofapproaches
the articles
for OSS evolution process support. OSS
used statistical methods
Evolution process support studies have usually such as regression,
time series analysis, correlation analysis,
focused
Pearson oncoefficient,
evolution models, Spearman’s exoge ranknous
factors, maintenance
correlation, principal component support, fault detection
analysis and Summary and
and change propagation aspects. A very little S29
Bayesian
effort hasour belief
been network.
paid towe Statistical
thefoundotheronly aspects analysis such Conclusion
During
methods areresearch,
found to be more reliable a smallfor
It reflects
as Configuration that most authors performed
Management,
number
predicting
experiments of
different
on studies aspects
different on openthe foruse OSS
sourceofGrowth,
OSS
stud- in
ies as
projects
Results and S29
Complexity
dentistry. and
Although Control,
thewas possible
conclusions evolutionary
drawn from Discussion
andcompared
The usability
comparison withof of other
OSS results methods.
evaluated
of differentThe second
only in
studies a
paths
studies
large etc.
set SVN/CVS
support
ofmental
methods thewith is
use again
employedof found
OSS, to
mostinto
falls be
of thethe
the Results and
few Four
become ar- ofeas:
largest five
arduous. studies
explored foramen
There dataset. the
islevel
a need highest
localization,
The of level
com-
detailed upperof
mon S29
articles
category
evidence
airway presented
of
(2b) machine
calculation a
examined low learn-
in growing clinicalof
ing evidence
algorithms.
changes
patients, an (3b-
in in Discussion
corpus
explanation regarding about evolution
these aspectspredictioniswhich
presentedof open
5)
Otherand
orthodontics
experimental poor
methods
source projects quality such
(influence
assessment of asreport-
that of mathematical
can head
of ing,
hard
be orientation
used debris by the makes
models,
in the on
it difficult to recommend Sect. 4.4. OSS as aandclinically Discussion S30
probability
directional
root canal methods,
changes
researcherssystem after
in
for Chaos
3D root theory
space;
comparing canal orbital SRGM’s
treatment,
results. volume
useful
(Software
One and software.
Reliability
of aperture
informa- our main TheGrowth
findings
tionwidthgathering only study
Models)
is (RSS),
changes thatafter with
most have
practice high-
studies
rapid very
quality
are
maxillary reporting
not
management, deeply
(expansion)lowandwas a case-controlled
exploration.
concerned
(18;21)
use for withand
educa- research
endodontics
tional study Discussion S30
but
methods:
Given
(canal the conclusions
the among
large
transportation) variety
the were
relevant
of
(22;23),based
OSS on
selected
systems,
but very
none small
papers,
with
of the
purposes. wemay None of
commented the studies
in teeth,
Sect. revealed the use
research
of different
we
identified
OSS in
group,
sizes,
cite
studies
prosthodontics
three
two
domainscase
included and4.2
studies,
which
in the
what
acomplexity,
is one
compar- lack
way
currentlyactionofdoes
ison wethe
measures
itmost
not
research,
believe make for
butthey process
itand
nopossible
are anmaturity
experiments, tooping because
saysoftware.The
important that
which the
source inofthis
results Discussion S30
between intensively
caseexamples
the OSS
assessment any
devel- other
needs to area
be ofseems
done digital
withonlyto
belevel-2b
in sharpstudy
dentistry. are
contrast
Virtual representative.
tothatteach
with
planningsoftware
included the recent
and design,
a comparative
designgrowth of of
qualitative
architecture
evidence-based evaluations
and quality
SE. of the
Software community.
(Brown &
engineering Wilson, Since Discussion S32
analysis
prosthetic of areconstructions
commercial andprovide open-source many
we
research
software haveisNonethe-
2012). focused
slowly
was onless,
adopting
assessing quantitative
we found
increasing
software measures,
very
precision few
scientific in
there opportunities
may be latest
other for OSS solutions.
characteristics ofathe
studies rigour
waytovolume
air-quality in
describe
the these
measurement years.issues Moreover,
in, say,
(27). The SEcase-
study
education
based model
learningis an that
approach. require
interdisciplinary Only or that
three
area. may
reported
As be
such, Discussion S33
was
Meiszner based
complemented eton al. 33 participants
(2009)
with point outand
qualitative some leadlearning
evaluations. to the
this ittype
conclusion
features canof of experience:
strongly
that
the OSP thebenefit
use two
ofnot
experience: ofonly
Osirix themand fromrelatedSE to
ITK-Snap
self-learning,
software
research OSSdesign
methods, and
presented architecture,
butclinical
also from value. and
researchof
one Discussion S33
project-/problem-/inquiry-based
Some of
them,the analysed
to software approaches
evolution. learning,
propose
collab- methods
Empirical orative from
experiments
learningareasand such
were as sociology,
used
reflective to identify
practice.
No quantitative
anthropology,
selected study measures
pedagogy
focuses and
and
on analyse
learning
communication. their
the areas
risks and
However, veryto identify
few studies effective
cite these mitigation learning Discussion S33
of effectiveness
requirements
Thus, itHowever,
activities. seems (e.g.
or [SLR4][SLR6][SLR15]
relevant
configuration
none oftotheidentifymanagement,
works whichin
showed the
approaches,
SLR for API and
metrics none and of them
code provides
changes, details
[SLR14]
research despite
evidences methods thecausal
for large
are use
more of appropriate
relationshipsconfiguration between in this
on for how
managementcode to design
executability,
and and
issue implement
[SLR25]
tracking forpedagogical
tools business
in OSP. Paper Analysis S36
intersection,
these risks,
practices in
that result order
concrete to achieve
measures
in effective better
learningand the results
of SE
values).
WeOurbelieve [SLR17]
in thisof
answer that proposes
these
interdisciplinary
to RQ1.1 a reliability
specific
indicates areas
area. that maymodel
the
effectiveness
combining qualitative the mitigation
skills.
and activities.
quantitative Only
metrics,
forbenefit
majoritymore from
of the
concrete an active
articles
risks, engagement
contribute
as e.g. for with
experimental
lowering OSP the Paper Analysis S36
we does
but claimnot that
and emergent
consider
their research
OSS-specific
associated tools. addressing
community
(case)
riskOur ofstudy to deal errors
introducing with evaluatingwhen the quality
upgrading to
and theanswer
success current
and tomobile-platforms
of repository
OSS. RQ1.1
Thereindicates
measures.
are a similar that
is not the
number
new
majority
considering,versions,
of theorarticlesconcrete
exploiting mitigation
contributeprevious activities
experimental
semi- nal Revisiting Research S37
were
(case)of articles
proposed, that contribute
such as new
automatically methods checking / Questions
worksstudy
techniques
to deal withplatforms,
on open-source
and very small
evaluatingas
number
the
of
quality
itarticles
often
API compatibility
and success of OSS. should. [SLR4] or exploring
There are a similar number code
of that
executability
Software articles introduce
engineers
that tools.
withcontribute
test lastThe
in cases lackmethods
[SLR14]
decade
new of tool
to
have ensure
been / Summary of Finding S38
development
interestedand
techniques can
correct
in agile be interpreted
functionality.
verymethodology
small numberand as a lack
open
of articles of
source agreement
software ondevelopment.
a concrete method Both of for
them Summary of Finding
that
Consequently, introduce the tools. The
healthcare lack of
industry tool is S38
measuring
present
development some the new
can relationship
be features
interpreted between
and they
as a the
seem
lack of
“lagging
OSS
success has
beneficialbehind
long
and passed
in
quality
foron terms
betterof the
OSS.ofmarket
adoption
Response
and faster introduction
ofto
software modern
RQ1.2
stage
ICT agreement
butand
tools a concrete
has infrastructure”,
not yet reached method
the
compared for
maturity to Discussion S41
supports
development.
measuring that By
the ofdoing
RQ.1.1.
relationshipan SLR a large we
betweennumber
were theof
looking
stage”.
other
articles This
sectors
are has been
(Karopka
classified particularly
as et al.,
solution 2014; relevant
Munoz-
proposals. for
T
successfor relationship qualitybetween
and administration of OSS. where ASD and OSSD.
Response to RQ1.2
We First,
Fortunately public
suspect regarding Cornejo
that
our this
study et
research
lack
showsal., 2008).
ofatypes,
research
that both itFLOSS’
isresults
worth
ASDof andin Discussion S41
supports that
technological of RQ.1.1.
immaturity, large number
lack of of
economic
highlighting
OSSD
articles can helpfactors
that
are classified each
the is
high due
other
as the
number
and
solution reluctance
collaborate
of evaluation
proposals. in
interoperability
companies
doing
papers to
andprovide
software thewith
projects existing
economic
increase software,
byofsharing details.
validation theirandT
Also,
decision
FLOSS
practices. making
experts
contributions There influenced
consider
reflects
are enough theby
that politicians
organizations
maturity
evidences FLOSS poses
that are Conclusion S42
a
adoption significant
already barrier for its wider adoption
agile andstudies.awaresource
open of
However,
thepractices
hidden thiscostsresearch when
can support area Research Scope in
adopting
each is other,
not FLOSS,
yetmainly
to the andbecause
point
therefore,of of contributing
they of
some tend their to S49
focus
experience more reports.
on researching
Second, technological
regardingAlso, FLOSSand Floss Adoption
common concepts and principles.
orga-however,
adoption nizational
factors factors.
there are Additionally,
we observe a fewthat successful
most weofonly the Research Scope in S49
found
experiences two solution
studies on
have integration
been proposals
focusing
of ASD related
onand with
theOSSD, Floss Adoption
economic
This
but,study
organizational most factors
also
ofand theindicates
and one with
technological
studies the
are areastech-
factors
optimistic such
nological as
leaving
in
andjoining
the
possibility process
organizational
economic of their and
factors abandon-
factors. not so
integration, Wewell ment
also
but where
covered.
observed
there is no Discussion S50
that validation research
research,
empirical successful case study for supporting is lacking.
opinion papers and
philosophical
this idea inpapers software are producing
gaining maturity industry. in the
FLOSS adoption area because we found
taxonomies, literature reviews and systematic
maps.
Traceable to
Complete Future
Study Year Direction Synthesis
Sheet
2017 4
2017 7
2017 8
2017 9
2014 11
2016 16
2010 18
2010 19
2010 20
2010 21
2010 22
2010 23
2010 24
2010 25
2015 54
2013 75
2019 80
2016 102
2016 103
2017 145
2017 146
2017 147
2017 151
2017 152
2017 153
2014 163
2015 168
2015 172
2015 181
2014 182
2014 184
2014 187
2019 192
2019 209
2018 210
2018 211
2013 212
2020 236
2020 250
2020 251
Codes and Themes
Code
Identifier Themes Sub-Codes
OSS Interaction Design (HCI

CF1 OSS Interaction Design Community Support in Research on the
Topic)
Security in OSS (Socio-Technical

CF2 Security in OSS
Aspect)
OSS Adaptation (safety Critical OSS Adaptation (Safety Critical Systems

CF3 System) via Project Activity)
OSS Evolution (Management via Data

CF4 OSS Evolution
Analytics)
OSS Integration (Maintenace

CF5 OSS Integration Challenges while Integrating
Components)
Context of Organization for OSS Usage

CF6 OSS and Organizations (Conduct more longitutional and
Indepth studies)
OSS Integration (Understand)
Participation
Align Researchin of
Organization-OSS
OSS with other
CF7 General OSS Research
OSS Integration Communities
Related Fields
Community and Organization Inter-Organization OSS Collaboration
Collaboration
CF8 Characteristics of successful
Inter-Organization OSS approaches to OSS
Collaboration
Challenges Organizations meet while
CF9 Deployment and Operation
Using OSS (Practioners of OSS
Problem) Developing OSS and their
Using OSS (Practioners Solution
Problem)
Deployment and Operation of OSS
(Long Term Cost)
OSS and Organizations Organizations and their approach to
CF10 OSS
OSS Research (Different Countries to

CF11 General OSS Research
see other than USA and Euprope)
CF12 OSS Adoption OSS Adoption (Issues)
Organization and Community

Community and Organization Collaboration (Development and Use
CF13 Collaboration (Development and of CASE Tools)
Use of CASE Tools)
Community and Organization Community and Organization

CF14 Collaboration (Development Collaboration (Development Practicies)
Practicies)
Community and Organization Community and Organization
Collaboration (Development Collaboration (Development Practices
CF15 Practices of Community to be used of Community to be used in
in Organizations) Organizations)
OSS Prediction (W.r.t Community
CF16 OSS Prediction Strucutre and Evolution)
CF18 OSS Prediction OSS Prediction Models (Generalize)
Best Metrics for OSS Prediction (In

CF19 OSS Prediction Terms of Accuracy, Generalizability,
Reproducibility)
CF20 OSS Prediction OSS Prediction Process (Tool Support)
CF21 OSS Prediction OSS Prediction Models (Accuracy)
CF22 OSS Prediction OSS Prediction Models (Social Aspect)
OSS Community (Perception of

OSS Community (Barriers Faced by Barriers)
CF23 Newcommers)
OSS Community (Effect of Barriers on
Quality of Contribution)
OSS Community (Barriers Faced by OSS Community (Categorization of
CF24 Barriers According to Newcommers
Newcommers) Profile)
OSS Community (Barriers Faced by OSS Community (Understand Barriers

CF25 Via Observational and Ethnographic
Newcommers) Studies)
OSS Community (Barriers Faced by OSS Community (Small Project
CF26 Newcommers) Barriers)
OSS Community (Barriers Faced by OSS Community (Measure Influence of

CF27 Newcommers) Barriers in Newcommers Experience)
OSS Community (Barriers Faced by OSS Community (Strategies to

CF28 Minimize Barriers Faced by
Newcommers) Newcommers)
OSS Community (Barriers Faced by OSS Community (Metric to Grade

CF29 Newcommers) Barriers Faced by Newcommers)
OSS Community (Barriers Faced by OSS Community (Barriers Faced by

CF30 Newcommers) Newcommers)
OSS Community (Barriers Faced by OSS Community (Technical barriers

CF31 Newcommers) faced by newcommers)
OSS Prediction (Explore Rigorous

CF32 OSS Prediction
Prediction Methods)
OSS Evolution (Validate Lehman Law

CF33 OSS Evolution OSS Ecosystem (Analysis)
using SCM)
OSS Ecosystem (Modeling)
CF34 OSS Ecosystem OSS Ecosystem (Measuring)
OSS Ecosystem (Evaluation)

OSS Major Streams (Need Research in
OSS Process, OSS security, Agile and
CF35 General OSS Research OSS development methods, Teaching
OSS in universities)
CF36 Innersource OSS within Company
OSS Vs Proprietary (Transformation)

CF37 OSS Vs Proprietary
OSS Vs Proprietary (Building
Community for it)
Methodologies for Dealing with
CF38 OSS and Organizations Different Aspects of OSS in Industry
(Conduct Case Studies)
Using OSS by Commerical
Organizations (Cost of Different
Approaches)
CF39 OSS and Organizations
Using OSS by commerical organization
(Advantages of Different approaches)
Open Source Business Intelligence in
CF40 Adoption (OSBI in Organizations) Organizations (Research Required)
Adoption of OSS in Popular Areas

CF41 Adoption (OSBI in Organizations)
(Compare Barriers)
Adotption of OSBI in Organizations

CF42 Adoption (OSBI in Organizations)
(Area Experts to be Included)
Adotption of OSBI in Organizations

CF43 Adoption (OSBI in Organizations) (Strategies to deal with barries that
prevent OSBI adoption)
Adotption of OSBI in organizations

CF44 Adoption (OSBI in Organizations) (Introduce other Resarch Methods Like
Case Study, Experiment)
Individual Developer/Team and Project

CF45 OSS Developers
Characteristics (Relationship)
CF46 OSS Developers Retention of Community Members
Role of Governance ( on Developer

CF47 OSS Developers Personal Relationship and Individual
Participation Motives)
OSS developers Retention (Influence of

CF48 OSS Developers Project characteristics)
OSS Developer Attraction (Combine

CF49 OSS Developers
team and project characteristics)
Identification of New Members (via

CF50 OSS Community (Newcommers)
Social Network of Members)
OSS developers Intrinsic and Extrinsic

CF51 OSS Developers
Motivation (via FLOSS Initiatives)
OSS Developer Attraction and
CF52 OSS Developers Retention (Multiple Perspective
Research Required)
Effort estimation/Maintenance
CF53 Effort Estimation in OSS Early Effort estimation Model in OSS
Effort estimation/Maintenance OSS Effort Estimation (Measuring

CF54 other Aspects
Effort of Effort
Estimation Meansurement
(Effect from the
Effort Estimation in OSS
Experience ofAttribute)
the Expert Toward the
Tools)
CF55 Effort estimation of new bug (Time
Effort Estimation in OSS Needed)
Effect of new bug effort estimation on

Effort estimation/Maintenance OSS Maintenance Effort effort
already estimated Estimation
CF56 Effort Estimation in OSS (from Size Related Metrics Needed)
OSS Estimation Models (Validate

CF57 Effort Estimation in OSS Correctness via New Evaluation
Methods)
Improve Prediction of OSS
Effort estimation/Maintenance Maintenance Effort Estimation (via
CF58 Effort Estimation in OSS Developer Related Metrics of
Capability Model of OSS)
Effort estimation/Maintenance Better OSS Maintenance Effort

CF59 Estimation (Via Contribution and
Effort Estimation in OSS Performance Measurement)
Use of OS in Different Domains Generalize OS Use in CS Education (Via

CF60 (Education) Practical Examples
Align Course with OS projects (Develop
Use of OS in Different Domains Strategies)
CF61 (Education)
Use OS in CS Education (Efficient
Evaluation Approaches)
OSS success (Mixed Methodology
CF62 OSS success
Research Needed)
OSS success (User Relation Success

CF63 OSS success
Factors)
OSS success (Contextual and

CF64 OSS success
Longitudinal Studies Needed)
CF65 OSS success OSS success (Develop General Model)
OSS success (Study

Reconciliation viaAgile
of FOSS, parameters
and Plan
CF66 OSS success such as (Extend
Driven social, cultural andmore
by adding economical
recent
state of development
papers ) community
Reconciliation of FOSS, Agile and Reconciliation of FOSS, Agile and Plan

CF67 Plan Driven Driven (Extend by adding more
databases)

CF68 Driven(Configuration
Driven (Extend by adding manual
Management
Plan Driven search)
Practicies)
Reconciliation of FOSS, Agile and Plan

Reconciliation of FOSS, Agile and
CF69 Plan Driven Driven (Practice of Continuous Code
Integration )
Reconciliation of FOSS, Agile and Plan

Reconciliation of FOSS, Agile and
CF70 Plan Driven Driven (Knowledge Management
Practicies )

CF71 Plan Driven Driven (Test Driven Development)
COSS User Community (Management

CF72 Commerical OSS
and Creation)
CF73 Commerical OSS COSS Revenue Models (Needed)
CF74 Commerical OSS COSS Business Model (Localize)
COSS (Structure and content

CF75 Commerical OSS
Dimension)
CF76 Knowledge Management in OSS Knowledge Retention in OSS Projects
Knowledge Loss in OSS Projects

CF77 Knowledge Management in OSS
(Proactive Measures to Reduce)
Knowledge Management in OSS
CF78 Knowledge Management in OSS Projects (Evaluation Metrics Needed)
CF79 Open Design Open Design (how to Keep Open)
Open Design Framing (Bound to

CF80 Open Design
Business)
Components of OSS for GIS web

CF81 OSS Integration
architecture (Evaluate and Improve)
Predictive Power of OSS Evolution

CF82 OSS Evolution Prediction Prediction Metric (Empirically
Evaluate)
OSS evolution prediction Models

CF83 OSS Evolution Prediction (Generalize)
OSS evolution prediction (via

CF84 OSS Evolution Prediction requirement, Design and Architecture
Level Metrics)
OSS Evolution Prediction (Requirement
Change)
CF85 OSS Evolution Prediction
Clone Evolution
OSS Evolution Tools and Approaches

CF86 OSS Evolution (Need to Generalize and Validate)
Use of OS in Different Domains OSS in Medicine (Experimental and

CF87 (Medicine) Algorithm Evaluation)
Quality models for OSS (Having

CF88 OSS and Quality
EssentialAssessment
Quality Quality Attributes)
Models
(Framework -based Selection Method)
Quality Assessment Models (Work on
Tool-based Selection Method)
Quality Assessment Models (Data

Mining
Quality SelectionCharacteristics
Assessment Methods)
(Work on ones having Low Measures)
Use of OSS in SE Education (Establish a
Set of Metrics for Each Role a Student
Plays)
Use of OS in Different Domains
CF91 (Education) Need to Measure Student Learning
(Based on Outcomes or Developed
Skills and, Assessment of Students by
Use of OS in Different Domains Peers)
CF92 (Education) Use of OSS (in SE course)
Open Innovation and Requirement Open Innovation and Requirement

CF93 Engineering (Strategies that are non
Engineering intrusive and low cost)
Open Innovation and Requirement Open Innovation and Requirement

CF94 Engineering (Tool Support to Extract
Engineering and Prioritize Requirements)
Open Innovation and Requirement

Open Innovation and Requirement
CF95 Engineering Engineering (Capitilize to Extract and
Prioritize rRquirements)
OSS Platforms (Convergance between

CF96 OSS Platforms OSS Quality and Success (Relationship)
Industries)
OSS Quality and Success (How Code
Quality May be Used to Measure
CF97 OSS Quality and Success Software Success)
OSS Quality and Success (Clarify
Terminology)
OSS Quality and Success (How Market
Success and Developer Activity May be
OSS to
Used Quality and Software
Measure Success (Identify
Quality)"
CF98 OSS Quality and Success
Metrics)
OSS Quality and Success (Develop

OSS Evolution
Tools to (Relationship
measure) of
CF99 OSS Evolution evolution of sub-projects with
evolution of associated community)
Study Social Technical congruance

CF100 OSS Evolution ( Community members Derive
Software Evolution)
OSS Evolution (Emprically Validate

CF101 OSS Evolution
Community related Metrics)
OSS Evolution Data (Framework

CF102 OSS Evolution
Needed)
OSS Evolution (Validate Existance of

CF103 OSS Evolution
SOC and Impact on Evolution)
Developer Migration of Responsibilty
CF104 OSS Evolution /Developer and Sustainability (its Imapct on
Software Evolution)
General Evolutionary Patterns

CF105 OSS Evolution (Propose)
OSS Prediction (Need Reliable
Prediction Model for meauring effort
and cost and supporitng error
CF106 OSS Evolution Prediction
prediction)
Community Building (Exclusive

OSS Community (Community Properties)
CF107 Building)
OSS Evolution (Re-Assess Laws of

CF108 OSS Evolution Evolution for Examinging Software
Evolution Ontologies)
Certification (Research for the

CF109 OSS Certification Effectiveness of Linus Law
Applicability)
Innersource (In-depth Analysis

CF110 Innersource
Needed)
Use of OS in Different Domains Drug Discovery (Business modeling,

CF111 Incentive Development and the Impact
(Medicine) of the Use of the Public Domain)
Diverse Architectural Problems
OSS Architecture (Architectural Architectural
(Identify DiverseProblems (Metrics
Prioritization to
Criteria)
CF112 Problems and Metrics) Identify)
Improve Prioritization of Anomalies
Related to Architectural Problems (via
OSS Architecture (Architectural ntegration of two or more heuristics)
CF113 Problems and their Strategies)
Architectural Problems of Developers
Architectural
(Introduce theRecovery (Work on
new strategies to
OSS Architecture (Architectural ARCADE Tools)
produce ranking on numerous criteria
CF114 Recovery) in order to provide visualization
Architectural Recovery (Increase
capabilities))
Accuracy and Conduct Experiments)
Evolution (Architectural Architectural Bad Smells in
CF115 Degradation) Combination (Need Research)
Evolution (Architectural Architectual Smellls (Detect via

CF116 Degradation) metrics)
Architectural Erosion in OSS and
Evolution (Architectural
CF117 Degradation) Industrial Systems (Identify new
Causes)
Evolution (Architectural Architectural Rules and their Voilation

CF118 Degradation) (Identify via Conformance Study Using
Different Frameworks)
OSS Architecture (Causes of
Architecture Decay) OSS Community (Causes of
CF119 Architectural Decay)
OSS Community
OSS Adoption, Selection and Evaluation

CF120 OSS Adoption
(Validated Common Model Needed)
IoT platform perspective (Model for

CF121 OSS Adoption Evaluation of OSS Products)
OSS Process (Undertstand in Practical

CF122 OSS Process Settings to Mitigate Project Failure)
CF123 OSS Adoption OSS adoption (Create Guidelines)
CF124 OSS Adoption OSS Adoption (New Domains)
Newcommers Onboarding in OSS

Community (Tool Support)
CF125 OSS Community (Newcommers) Newcommers of OSS Community
Newcommers (Orientation)
Migration between OSS
Projects (Tool Support)
Newcommers of OSS Community
OSS Community Dynamics (More
(Reception)
CF126 OSS Community (Dynamics) Evidence Based Research)
(Initial Contribution Problem and
Barriers Guideance)
Abandonment
(Initial of Project
Participation and(Community)
Motivation
OSS Community (abandonment of
Project) Support)
CF128 Abandonment of Project (Impact of
Developers
OSS Decision
Ecosystem
Newcommers to Community
Stay or Role)
(Contributors
of OSS Leave
OSS Developers Project)
OSS Ecosystem (Improve Retention and One Time
NewcommersContribution)
of OSS Community (help
CF129 OSS Community (Newcommers) via Organzing Onboarding Information)
OSS Developers Onboarding Success (Impact of

Resolved Issues)
Community Participation (Tools
Required)
OSS Community (Task Selection,

Distribution, Management via Machine
CF130 OSS Community Learning)
OSS Community Dynamics

(Newcommers
OSS Developers in
Collaboration is OSS Community
Required
Onboarding Among
(Impact
CF131 OSS Developers (Joining
Different Process)
Countries)
of Mentoring)
Newcommers in OSS Community
(Abandonment Orientation)
Reception
Newcommersof Newcommers in OSS
in OSS Community
Community (Tools
(Reception)Required)
Community Participation
Newcommers (Tools)
in OSS Community
(Mentoring)
Community Participation (Practices)
Good Community Support(Processes)
(Engage
CF134 OSS Community Developers)
Newcommers to OSS Community

CF135 OSS Community (Newcommers) (Tools Required for their Reception)
Community Dynamics (Impact on

OSS Community Dynamics Developer Turnover)
CF136
OSS Project
Project Performance (Turnover Effect)
Bug Prediction in FLOSS (Influence of

CF137 Severity Level Prediction in FLOSS
User Experiance)
Normal Bug Handling in FLOSS

CF138 Severity Level Prediction in FLOSS (Improve State of the Art of Severity
Prediction Algorithms)
Bug Report Severy Prediction in FLOSS

CF139 Severity Level Prediction in FLOSS (Apply Machine Learning Algorithms)
Bug Report Severity Prediction in

FLOSS (Recent techniques Applied Like
Contineous Integration in Real
Scenarios)
Severity Level Preiction in FLOSS

(When to Perform in Bugs Life)
Severity Level Prediction in FLOSS

(Extend to Commercial Systems)
Severity Level Prediction Accuracy in
FLOSS (Via Temporal Evolution)
CF143 Severity Level Prediction in FLOSS Severity Level Prediction Accuracy in
FLOSS (Data Sturctures to Store the
Temportal Evolution)
Severity Level Prediction in FLOSS
CF144 Severity Level Prediction in FLOSS (Learn from Severity Level Prediction
of Other Areas Best Practices)
Innersource (More Knowledge and

CF145 Innersource
theoratial Foundation Needed)
Innersource (More Databases

CF146 Innersource
Coverage for Understanding)
Innersource (Manage Community and

CF147 Innersource Organization Effectively)
CF148 IoT Platforms IoT Platforms (Openness dimensions)
IoT Platforms (Stakeholder

CF149 IoT Platforms
Perspective)
IoT Platforms (IoT platform

CF150 IoT Platforms
perspective)
OSS Projects (Research needed on Fork

CF151 OSS Projects
Sustainability and Fork Model)
Use of OS in Different Domains Students Perception of OSS (Cover

CF152 (Education) more Universities)
Productivity of OSS organizations
OSS and Organizations (influencing factors)
CF153
OSS Community Productivity of OSS Community
(influencing factors)

Codes Book of Future Direction
Sub-Sub- Codes
HCI community support required

for OSS interaction design
Consideration of of socio-technical
aspects in OSS security
Adaptation of OSS in safety critical

systems via OSS project activity
Data analytics used for manageing

software evolution of OSS
Maintenance Challenges while

integrating OSS components
Considering context of organization

for
OSSOSS usage via(Understand)
Integration indepth and
longitutional studies
Participation in Organization-OSS
Communities
Align resarch of OSS with other
related fields
Inter-Organization OSS
Collaboration
Characteristics of successful
approaches to OSS
Organizations and their approach
Challenges Organizations meet
Practioners
to OSS problems
(Increase OSSwhile
while DevelopingRigor of
and using
Studies)
their
OSS
Solution
Long Term Cost of Deploying and

Organizations and their
Operation of OSSapproach
to OSS (Conduct longitudinal, in-
depth studies)
Organizations
OSS research toand their from
be seen approach
areas
toother
OS (Align our research
than USA and europewith
related research fields)
Issues in adopting OSS
Use of CASE tools in OSS
Collaboration Venture (Between
Organization and Community)
Development of CASE tools in
OSSCollaboration Venture
(Between Organization
Development Practices and
for
Community)
Distributed Software
Development(Organization and
Community Collaboration)
OSS Community Development
Practicies Adoption in
Organizations
OSS Prediction with respect to

Community Strucutre and
Evolution
Generalize OSS prediction models

in OSS Projects
Best Metrics for OSS Prediction (In

Terms of Accuracy,
Generalizability, Reproducibility)
Tool Support for Automating OSS

Prediction Process
Accuracy of OSS Prediction models
Social Aspect of OSS Prediction

Models
Communities Perception of
Barriers
Effect of barriers faced by

newcommers on quality of
contribution
Different type of newcommers and
their Barriers
Understanding Barriers faced by

newcommers (observational and
ethnographic studies)
Barriers faced by newcommers (in
Small Projects to Improve Model of
Barriers)
Measure Influence of Barriers in

Newcommers Experience
Strategies that Minimize Barriers

Faced by Newcommers
Metric to Grade Support Offered

for Barriers Faced by
Newcommers
Understanding Barrier Faced by

Newcommers in Context
(Qualitative Research)
Technical barriers faced by

newcommers (Indepth Studies
Needed)
Explore Rigorous methods for OSS

prediction
Validate lehman law using SCM

change information for software
evolution
OSSECO, Analysis, Modeling,

Measuring and Evaluation (Need
OSS Process
Further Research)
OSS security
Agile and OSS development
methods
Teaching OSS in universities
OSS within Closed ompany
Transformation of Proprietary
Software into OSS
Builing Community for
Transformed Proprietary to OSS
Case studies for Implementation of
Specific Methodologies for Dealing
with OSS in industry
Understand Cost and Advantage of
Different Approaches and their
Criteria of Choice for Using OSS by
Commerical Organization
Research required in Open Source
Business Intelligence in
Organizations
Compare Barriers from Preventing
Organizations from Adopting Open
Source in Organizations in Popular
Areas
Different area Business Intelligent

Experts to be Included in Survey
Strategies Required for Dealing

with Barriers Preventions OSBI tool
Adoption in Organizations
Use of Different Research Methods

to be Implemented in Open Source
Business Intelligent Tools Research
The relationships between

individual or team factors and
project characteristics
Motivational Factors of OSS
Community Members, and its
Impact on Retention of Community
Members
Project
Role charactersitics
of Governance Impact on
Processes
Retention of Community Members
Dvelopers’ Personal Relationships
Role of Project Governance in
Individuals Participaton Motives
Influence of project characteristics
on individuals retention
OSS Developer Attraction (Combine

team and project characteristics)
Utilize Social contacts of

community members to Reachout
for New Members
FLOSS Initiatives Similuation in OSS

developers extrinsic and intrinsic
motives to Join Project
Effect of developer attraction and
developer retention from multiple
perspective
Early effort estimation model for

OSS web projects Needed
OSS Effort Estimation (Measuring

Effortother Aspects
Estimation of Effort
(Effect from the
Meansurement
Experience Attribute)
of the Expert Toward
the Tools)
Effort estimation of new bug (Time

Needed)
Effect of new bug effort estimation

OSSonMaintenance Effort from
already estimated effortSize
Related Metrics (Studies Needed)
Validate Correctness of Estimation

Models for OSS Effort Estimation
via New Evaluation Methods
OSS Maintenance Effort Estiamtion
(Improve Prediction via capability
model for OSS developers via
developer related metrics)
Contribution and performanance
meansurement of developers in
community for better OSS
maintenance effort estimation
Generalize OS Use in CS Education

(Via Practical Examples)
Align Course with OS projects
(Develop Strategies)
Use OS in CS Education (Efficient

Evaluation Approaches)
Mixed methodology research
needed in the field of OSS success
User related success factors in

ensuring OSS success
Contextual and Longitutional

studies needed in the area of OSS
success
Develop a general model for OSS
success
Study success in OSS via

parameters such as social, cultural
and economical state of
development community
Add more databases and manual
search in the research area of
reconciliation between FOSS, agile
and plan driven
Reconcile Configuration
management practices between
FOSS, plan driven and agile
Reconcile continuous code

integration between FOSS, plan
driven and agile
Reconcile Knowledgement
management practices of agile
applied to FOSS
Need to reconcile agile
development and distributed
development w.r.t test driven
development in plan driven
context
Management of the creation and
maintenance of the user
community.
New revenue models required for

OSS
Localize OSS as a business model
Structural and content dimensions

of the COSS, along with the
organization
Knowledge Retention Applicable to

OSS projects
Reduce Knowledge Loss (Proactive

Measures)
Knowledge Management Metrics
in OSS Projects
how to keep design open needs to

be seen
Can open design framing remain

bound to business
Need to carry out new research
aimed at evaluating and improving
the components of the Open
Source software architecture of a
geographic information system in a
Web environment.
Need to empirically evaluate
predictive power of different
metrics.
Generalize evolution prediction

models for OSS
Need to predict the OSS evolution

Prediction
with help ofofrequirement,
requirement design,
change
evolution and change propagation
and architecture - level metrics
Contribution evolution
Clone evolution
size, refactoring, maintenance

Need forand
effort, external
Self - validation
Organizedof
tools andCriticality
approaches of evolution
(SOC)”.
are required.
Experimental studies for OSS in
medicine research
Algorithm evaluations for OSS in

medicine research
Need to develop quality models
based on essential quality
attributes in the area of OSS
research
Need to work on Framework, tool-
based and data mining selection
methods for quality assessment
models
Work on Quality Assessment

Characteristics with Low Measures
Need to establish a set of metrics
for each role a student plays.
Need
Use ofto measure
open sourcestudent learning
projects within
based on outcomes or developed
SE course (further research area)
skills and, assessment of students
by peers.
Need to measure student learning
based on outcomes or developed
skills for
Need and,future
assessment of students
research in Open
Innovation (OI) strategies
by peers. within RE
that are non - intrusive and of low
cost
Tool support for automated
solutions in RE specifically to
extract and. prioritize
requirements
Need to capitalize on open
innovation to automatically extract
and prioritize requirements.
Need to research
Convergence betweenon the
industries
relationship between OSS
on OSS platforms need tosuccess
be
and quality
explored
OSS Quality and Success (How
Code Quality May be Used to
Measure Software Success)
OSS
NeedQuality and terminology,
to clarify Success (How
Market Success and Developer
identify metrics, and develop tools
Activity May be Used to Measure
that are capable of measuring
Software
quality andQuality)
success.
Relationship of evolution of sub-
projects with evolution of
associated community needs to be
seen in detail
To study socio technical
congurance since contributions by
community members derieve
software evolution
Empirically validated community
related metrics are required for
software evolution
OSS Evolution Data (Framework

Needed)
To validate the existance of SOC

(Self Organzied Criticality in OSS
projects and its impact on OSS
evolution
need to study migration of
responsibility and sustainability in
OSS projects since developers join
and leave projects and its impact
on software evolution
Need to generalize threat to
evolution studie and propsoe
general evolutionary pattern
OSS Prediction (Need Reliable
Prediction Model for meauring
effort and cost and supporitng
error prediction)
Community Building (Exclusive

Properties)
To re-assess the laws of evolution

for examining ontologies for
software evolution
Need of more empirical work and
theoretical research to assess the
effectiveness of Linus law
applicability and its effectiveness in
Research
the context is required in drug
of certification.
discovery involving experts
Need of an in - depth analysis from
of
the industry
empirical for
work a business
on Inner model,
Source
Most Appropriate
incentive Architecutre
development, and
Software
Irrespective (ISS)
of development.
Size (Metrics tothe
be
identification of the impact of
Analyed)
use of a public domain.
ItMetrics
There
(Having
is important
is athe
need
great
that thisability
to identify
to
research
diverse
discover inconsistent
includes expert input classes
prioritization
affected by the criteria to from
degradation handle
from
researchers,
diversethe the
architectural pharmaceutical
problems and
industry consistent
and PDPs classes)
to assess tothe
seeing their effectiveness
practicality
determine andcoderelevance of open
Effort
source required foranomalies
drug discovery theatmetrics
a task
strategy to architecturally
level. detect
Improve
related Prioritization
anomalies and of Anomalies
also to
Related
derive to Architectural
more metrics that Problems
may have
(viaanntegration
impact onofthe twoquality
or more
relationships heuristics)
of other
Code level analysis viasoftware
ARCADE thattool
arecan closely related
utilized for to architectural
architectural
Architectural
problemsProblems of
recovery,
Developers along with experiments
(Introduce the new
onstrategies
wide scope to increase
to produce accuracy
ranking on
of approach
numerous criteria in order to
Further
provide investigation need on the
visualization capabilities)
topic of architectural bad smells in
combination.
Need to work on metrics to detect

architecturally related smells.
To identify new causes of
architectural erosion in OSS and
industrial systems
Need to conduct an architectural
conformance study using different
frameworks to identify necessary
architectural rules and critical
cores
the to avoid
causes architecturedecay
of architectural rule
voilations need to be
of the OSS commuity
further identified in terms of the
frequency of their occurrence,
especially in the scope of the OSS
projects
Need for a validated common
model for the selection, evaluation,
and adoption of OSS
Need of tools support (for
supporting and simplifying the
applicability of the proposed
models for evaluation of OSS
Products)
Understand OSS process in
practical settings to mitigate OSS
project failure
Creating guidelines for adopting

FLOSS
Application of FLOSS in new

domains
Tools for supporting newcommers
onboarding
Focus in different
on newcomers OSS
orietation
community projects
and reception in OSS community
projects
Tools for supporting newcommers
migration
Provide in different
guidance towards OSS
barriers,
problems of intial projects
More community
evidance based research
contribtution
need in the area of
from developers in OSS Community
community
dynamics
projects
identify means to support initial

participation of developers in OSS
community projects
Focus on abandonment of projects
by community
Enhance motivation of developers
See contributors role ofinOSS
to increase particpation OSS
impact of developers
projects decision to
in ecosystem
community projects
stay or leave the OSS community
organize project information
Removeonboarding
hinderance to improve
helpful
retention of developersinand
for newcommers OSS
community projects
OTC(one time contributors) in OSS
community projects
impact of resolved issues on
onboarding success in OSS
community projects
processes that could be helpful to
improve community participation.
Machine learning to be used for
task selection, distribution and
management of OSS project
information
Collaboration is required among
research
Impact
joining groups
of mentoring
process of newofon
different
success of
commers in
countries
onboardingto process inmore
conduct
OSS communities OSS
research
community in the area.
projects
Abondonment orientation of
newcommers in OSS communities
Reception of newcommers in OSS

Toosl for good reception of
communities
newcommers in OSS community
Mentoring of newcommers in OSS
Tools, practices and processes for
communities
newcommers support and
community participation
Provide good community support
for engaging developers
Toosl for better eception of

newcommers in OSS community
Impact of community dynamics on

developer turnover and Turnover
Effect on Project Performance
Influence of User Experience in

Prediction Outcome of Bug Reports
Improve State-of-the-art of
Severity Prediction Algorithms
(Noval Approaches for Normal Bug
Handling)
Bug Report Severy Prediction
(Apply Machine Learning
Algorithms)
Bug Report Severity Prediction
(Recent techniques Applied Like
Contineous Integration in Real
Scenarios)
When to do severity level

prediction in bug report life
Extent bug level severity prediction

in FLOSS at commerical level
Temporal evolution of bug report
and its impact on severity level
prediction accuracy
Data structures for storing

temporal evolution of bug report
Best practices From Other Systems
Can Benefit Severity Level
Prediction in FLOSS
More Knowledge on innersource
Cover more databases for

understanding innersource
Manage innersource community

effectively in organizations
Openness dimensions of IoT

Platforms
Different types of IoT Platforms

from stakeholders perspective
Different types of IoT Platforms

fromIoT platform perspective
Fork Sustainability (More Research
Needed)
Prediction model for Fork
Effectiveness
For students perception of OSS
literature, cover more universities
Infuencing factors for OSS

organization and communities
productivity
Codes Book of Future Direction
Traceable to Study
Paragraph
The results
Based on theoffindings
this systematic mappingwe
of this research, suggest
have comethe need forconclusion
to the broad support that
for FLOSS projects and communities by the
the existing software security practices have limitations in supporting HCI community, through
research
secure open efforts
source in the area of interaction
development. design for thecode
Secure architecture, availability
review and of
MTTSA of interaction design considering
security testing do help secure OSS products. However, as there is lessthe characteristics of FLOSS
development. Therefore, it issecurity
research on socio-technical necessary to develop
aspects and noand publish research
discussion of security on
interaction design in the context
knowledge management in the context of OSS development, these of FLOSS.
practicies,
Most and software usedtoinsecurity knowledge studiescannot beactive
effectively spread
ButOSS projects
that is restricted a the
fewprimary
of the change are still
categories however,
e.g. adaptive and
v/s
within
often the open
contain sourcewith
codebases community.
moderate Since
to veryOSS parcticipants
high activity. Inare not
general,
non-adaptive changes, or corrective v/s non-corrective changes. A fine-
experts on
therefore, security
it seems thatinthegeneral and the domain knowledge
relationship of software
grained view of the changes can help tobetween answer the adaptation
amount of OSS in
of progressive/
security is vast
safety-critical and
and extensive,
their long it is
history suggested
is rather that future
clear. At research
the same should
time,
regressive work performed in a software system as it evolves. It can also
explore
further soio-technical
research should approaches
be done to in helpingthe
investigate OSS developersbetween
relationship learn thethe
be used to validate Lehman’s 2nd law as Gonzalez-Barahona (2013) points
necessary of
adaptation security
OSS in knowledge
safety-critical to fulfill
and the
OSS need
projectof their work,Moreover,
activity. further, toit
to the lack of information available in this regard in their study of the glibc
reinforce their
would be interesting to investigatesystem; behaviors towards OSS security.
the number of downloads of this project
While there are a few or
• Techniques and tools have been devisedits
studies adoption
outside within
the scopetoindustry
of this
tackle review
large amountsfocusing
of dataon
software selection
generated [46, 56,evolution
in software 105, 184]analysis and knowledge sharingSoftware
and prediction. within OSS
The overall rigor
communities [119, of the195],
173, studies none performed
of these on
are OSS,
directedbothtowards
within organi-
studying
evolution visualization helps in understanding the transitions in complex
zationspractice
actual and in general,
in is furthermore
organizations. A few not goodhave
studies enough.startedConsequently,
to look at somewe
and large systems in an easy way. Big data analytics can also help to
should strive to
of the challenges do better work and to present this work in more detail
analyze large sets ofindata the borderlands
generated during between integrating
software an OSS
evolution. Data
[180]. In particular,
component and we agree
contributing to withdevelopment
the Kitchenhamofetital.[106, [118] in that
130, 186],the
analytics can be used to manage and understand the complex web ofbut
context
further of the organizations
research is needed to being
solve studied
the should be challenges
maintenance given muchfacing more
software evolution as it happens in source code and other related
developers who integrate arepositories. attention.
large number of components into their
We Weobserved
would inthat few of recommend
particular the studies were longitudinal
investigating
products. twoand that (1)
issues: fewtopics
publi-
cations
related focused on of
to integration providing
OSS componentsin-depth details and (2)from topicsonecon-or acerning
few
OSS researchers
organizations.
participation Toshould
really therefore
understand
in organization-community increasingly
the profound relyconsequences
on research OSS
or inter-organizational and
of
theories
collaborations.from
approaching OSS, related
We we fields
findbe- (see
lieve
these Section
thereimportant
issues 2.4). Software
is a need because engineering
for both integration of and
more longitudinal OSS
information systems
components concerns and researchers
mostin-depth should see OSS as an opportunity
case studies.organizations [98] and
software-intensive to
investigate
Finally,
becausewe general software
found evidence
participation engineering
that OSS issoftware
in collaborative and information
not that development systems
different fromisother research
infor-
increasing
[7, 185]. The research could mation challenges.
focus technologies.
on identifying the characteristics of
successful approaches to OSS, the challenges these organizations met,
and how they solved them.
Deploying
We furthermore adviseclaim
OSS: Many that reducing
researchers costs is one
to put emphasis on ofhowthethe
advantages
studied
of deploying actually
organizations OSS server use software,
OSS, and infrastructure,
on problems that andreally
applications.
matter to
However,
practitioners. a recent study byshould
Practitioners Fitzgeraldbe open [80]tois OSSone of and fewseestudies
that it with
offersa
longitudinal
Maturing the view
research on deployed
field on OSSOSS in products.
organizations
several opportunities. However, they must first evaluate the implications This highlights
and dealing a need
with forof
some
of its limitations
adopting may more
OSS be instudies
done on: context.
theirthrough
own four main steps:
􏰅1.WhatFocus areresearch
long-term on costs
topicsand thatconsequences
are relevant toofhow deploying and keeping
organizations ap-
OSS products proach operational?
OSS
. These observations3. Conduct are more
not particular
longitudinal, to research
in-depth onstudies
OSS. For instance,
Kitchenham4.etAlign al. [117],
our research with related research fieldsand Wallace
Vessey et al. [192], and Zelkowitz
[214] observe a lack of relevant empirical research of high quality within
both the software engineering and information systems fields. Finally, we
would also like to see more research from outside Europe and the USA.
Using OSS CASE tools: The research on OSS CASE tools has been very
limited. However, Wicks and Dewar propose a new agenda for research on
tool integration, requesting a more business-oriented approach to future
Researchers
research [207].and The practitioners
use and developmentshould increasingly
of OSS CASE collaborate
tools and to define
research a
oncommon
such tools research
could agenda
easily fitand intostudy
this research
new agenda. questions
Robbins thatprovides
matter toan
practitioners. These research questions should
extensive overview of OSS tools for development, and claims that CASE be answered through several
related studies from different
research has a lot to learn from OSS [157]. OSS should furthermore be contexts.
particularly interesting to academia since they have access to professional
state-of-the-art tools and the tools’ source code. This enables them to
extend existing tools and test new ideas in collaboration with each other.
ItIncreased
is more and more difficult
participation in OSS to talk aboutincreased
projects, “OSS practices”collaboration as the between
practices
used in OSS communities are heterogeneous,
organizations, and increased use of OSS practices will most likely require and as organizations are
OSS development
increasingly getting in large communities
involved in, and and inby,
influenced andthe between organi-of
development
improved
What sets collaborative developmentapart tools.fromHence, there is a potential for
zations, areopen
OSS. Nevertheless, areassource
where there
development
researchers
are opportunities could have antheimpact
for further
traditional proprietary
on practice.
research on the OSS use
soft- warehas is the research on:
research
of development so developer
focusedcommunity
farpractices mainly
for on behind
distributed processes it. Although
software in communities
development.
the socialof
􏰅 What
structure and kinds of tools
communication are needed for collaborative software development
volunteers [167], but some of of thisOSS communities
research could turn have its gained significant
focus towards the
research across the
interest, organizational
research and community
efforts to the community borders? in relation to
application of their findings within organizations and questions like:
􏰅prediction
Howdoorganizationscollaborateusingsuchsoftwaredevelopmenttools?
􏰅 How can appear developmentquite the opposite.
practices from Evolution of communities
OSS communities be adoptedis of
interest starting from the paper within introducing the community structure [13]
organizations?
but our search did not find
􏰅 How may organizations successfully collaborate much focus on community through evolution
community- tied toor
prediction. In [14] the authors investigate
consortium-based software development? the impact of social structures
between
It is alsodevelopers
noted that and end-users on
the prediction software
models are notquality
generaland andtheirare results
not
give applicable to different software systems [10]. Specially for defecthold
support
Traditionally to thinking
defect that
prediction social
models structures
rely onin the
metrics commu-
that nity
represent do the
prediction
state of power
the in
software addition
system toatthe
a source
specific code
moment
prediction models there exists very little evidence on their cross project centric
in approaches.
time [11].These It is
also
metrics suggested
applicability are used that
[5]. tocombining
Thus capture metrics focusing
a particular
a comprehensive study ononor
snapshot thecode
releaseandofsocial aspects
a project
generalizability to
issue
work
predict as a
thebetter
next prediction
one. But model
metrics than
capturingeither
of the prediction models across the domain of OSS projects is an area of alone.
changes This
over gives
time support
in projects that
the play
also question has research
a significant role in value and
prediction.
future is worth
research. looking into
For example, metricsfurther: what
presenting
does
SLR the
the softwarecommunity
concerning evolution and
software the
were community
faultused to predict
prediction structure
wasthefirst predict
need for the
of refactoring
conducted software?
by [3][12] and
and quality of OSS projects with significant
was extended with new results in [7]. However these works were limited to accuracy. Thus a future
research directionofwould
fault prediction closedbe to explore
source projects a comparative
and fall short study for identifying
of exploring OSS
either (a) which form of metrics are domain.more suitable for prediction models
inSLRterms
This SLR of will
accuracy,
concerning help reproducibility,
software
researchersfault prediction and generalizability,
to investigate wasprediction
first conducted or (b)
studies are
[3]these
byfrom and
the
metrics
was perspective of metrics, methods, datasets, tool sets in an effectivetionto
complementary
extended with new to each
results in other
[7]. and
However should be
these used
works in combina-
were limited
fault prediction
manner. of closed
Future research to get
shouldbetter
source prediction
projects
focus andresults.
fall short
on establishing of exploring
external validityOSS and
domain.
consistent accuracy of prediction models, incorporation of social aspects of
SLR concerning
Thisprojects,
SLR willand software
help researchersfault prediction
to investigate wasprediction
first conducted studiesbyfrom [3] andthe
OSS building tool support to automate the prediction process.
was per- extended
spective of metrics, methods, datasets, tool sets in an effective to
with new results in [7]. However these works were limited
fault prediction
manner. Future research of closedshould sourcefocus projects and fall short
on establishing of exploring
external validityOSS and
domain.
consistent accuracy of prediction models, incorporation of social aspects of
OSSThis
AlthoughSLR will
projects, weand help researchers
building
considered to investigate
toolbarriers
the support prediction
astosomething
automate the studies
thatprediction
can hinder from the
process.
new-
We perspective
analyzed theof metrics,
characteristics methods, and datasets,
goals of
comers’ contributions, some barriers can be used as filters by the projects. tool
the sets
newcomers. in an effective
However,
manner.
many Findings Future
of the from research
papers a did not
Halfakershould focus
explicitly
et al. [19] onstudy
profileestablishing
the
on newcomers
Wikipediaexternal they validity
newcomersanalyzed. and
consistent
Thisrevealed accuracy
is probably
that some of entry
related prediction
to the models,
type
barriers of
leddatatoincorporation andofthe
analyzed contributions
improved social
typeaspects
of the of
in study
OSS projects,
conducted, and
as mostbuilding
of the tool
studiessupport
only to automate
used
future. More- over, research conducted in the OSS domain [33, 13] data the
coming prediction
from process.
software
repositories and
demonstrated that didso-notcialization
go deeperbarriers in the analysis
are useful of for
the maintaining
subjects. The
On the other
problem is integration hand, studies
that the termand such
newcomer as those conducted
canofbetheused by Steinmacher
in a loose way, which et can
al.
community the quality community’s product. A clear
[PS14] and
bias thefor Jensen
results. et al. [PS9] presented simplistic views of the problem
direction futureNewcomers
work is to explore can be novicehow the developers
communities whoperceive
are starting these
whencareer,
their they drew people conclusions
who are from only analyzing
experienced developers thefrom
first industry
messagesbut from
are
barriers and how they impact the quality of contributions from newcomers.
newcomers and their retention. The context
not used to OSS projects, or people who are migrating from other OSS is important: Why did they
send These
projects. the messages?
three profiles Whatare motivated
differentthem? and can Didface theydifferent
really want to or
barriers
contributebarriers
experience or just clarify some Therefore,
differently. doubt? Diditthey would contribute
be a better at the end butto
approach
never got back to the mailing list? To answer
assess how these different types of developers see the barriers and such questions, we needwhat to
merge information from different sources
their impressions of them are. For example, does a novice developer find (issue tracker, mailing lists,
more documentation,
issues to contribute code repository) and verify the
than an experienced contextwithout
developer by talking an OSSto
practitioners. Another possibility background? is to conduct observational and
ethnographic studies by analyzing the barriers and effects for newcomers
in real settings.
Moreover, projects that focused on products used during the development
cycle and developed in Java and C were preferred. Such projects can be
classified as clearly successful projects which, combined with the
historical data available, provide an easy target to search for newcomers.
WeOSS observed that, although
researchers can also projects
benefit from gain these
several newcomers,
results by usingjust them a small
to
percentage are successful in contributing some
conceive strategies for newcomer support. To achieve this, it is necessary source code. Because the
identification of barriers faced and surpassed
to put more effort on specific research topics, such as understanding and by such newcomers is
important,
creatingprojects
ways to with measurea high thenumber
influence of developers
of the barriers (andinnewcomers)
newcomers’are
easier
OSS to analyze
researchers to
can find
also evidence
benefit of
from such
experience, identifying and creating different strategies to lower these barriers.
results However,
by using a high
them to
each
number
conceive of OSS
strategies projects
for present
newcomer different
support. characteristics,
To
barrier, and proposing metrics to grade the support offered for each barrier. achieve this, such
it is as small
necessary
To teams
to gain
put more and
a better short
effort onlifetime,
specificand
understanding were
research
of the not
topics,
barriers considered
such
and to as for extent
evaluation.
understanding
what and
they need
Naturally,
creating such
ways projects
to measure provide
the less data
influence and
of
to be lowered, it is important to conduct more qualitative studies because are
the less
barriers attractive
in than
newcomers’ large
OSS researchers
successful projects, canwhen
but also benefit from these results by using them to for
experience,
this phenomenon identifying
occurs in aconsidering
and creating
complex,different newcomers,
social they
strategies
environment tocanlower
in account
which each
the
conceive
barrier, andstrategies
different problems
proposing forthan
newcomer
those
metrics support.
identified Toour
by achieve
model this,
orforit is
modify necessary
their
context of its occurrence istoimportant.
grade theMoreover,
support offered
a qualitative each barrier.
view
Toto put more
importance.
gain a bettereffort
Further on investigation
specific research
understanding of the topics,and
is barriers
required such as
regarding
to understanding
what such
extent projects and
they needto
complements the existing literature, which relies mostly on quantitative
creating
to be lowered, ways
improve to measure
it is the model of
important the influence
barriers of
conduct described
to evidence. the barriers
more qualitative in newcomers’
in this paper.
studies because
OSS researchers
experience, can also benefit
identifying from these results by using them to
this phenomenon occurs and in a creating
complex,different strategies
social environment toin lower
which each
the
conceive
barrier, strategies
and of proposing for newcomer
metricsistoimportant.support.
grade theMoreover,To achieve
support offered this, it is necessary
for eachview barrier.
context its occurrence a qualitative
Toto gain
put morea bettereffort on specific research
understanding of the barrierstopics,and suchto as understanding
what extent and
they need
complements the existing literature, which relies mostly on quantitative
creating
to betolowered, ways to
it is measure the
importantoftothe influence
conduct of
more the barriers in newcomers’
Due the rising dominance evidence.
OSS in the qualitative studies because
software industry; not only
experience,
this identifying and creating different strategies toin lower each
practitioners, but researchers as well as academicians are alsowhich
phenomenon occurs in a complex, social environment keen the
to
barrier, and of
context proposing
itsOSS metricsistoimportant.
occurrence grade theMoreover,
support offered for eachview
a qualitative barrier.
understand the software development process. Several studies have
To gain
complements
We alsoa better
noticed understanding
the existing of thestudies
literature, barriers
which and
relies to whatissues
mostly extent
on they need
quantitative
been conducted in athelackpastofinin-depth
this regard. This on technical
paper presents faced
a systematicby
to be
newcomers.lowered, The it is
reasonimportant
can be to conduct
evidence.
attributed more
to the qualitative
small number studiesof because
qualitative
literature review of the studies performed to understand OSS evolution. A
this phenomenon
studies found becauseoccurs in a complex,
it cannot social environment
be quantitatively extracted in which
from the
mailing
set of 190 primary studies are identified for analysis and discussion. The
context
lists. For are of its
example, occurrence is important.
technicalonhurdles Moreover,
areofevidenced a qualitative
by only view
five studies
studies characterized the basis the research questions they
complements
analyzed. Issues therelated
existing to literature, which
workspace setup relies
is mostly
reported in on
only quantitative
one study,
address. The main findings are as follows:
evidence.
•byOSSoneevolution
subject inprediction
a debrief studies
session. These
use ARIMA kindsmodeling
of issue deserve
of time more series
analysis attention,
for laws
prediction from both
of software practitioners
evolution and researchers.
attributes
•Lehman’s of software evolution for OSS systemssuch haveasbeen size,
defects, change requests etc. However, as
validated in several studies. Only two laws (I and VI) have been confirmed software evolution in general
and
so farOSS evolutionstudies
in different in particular
on OSS is evolution.
a discontinuous Therephenomenon,
is need to looktheinto usetheof
prediction
change techniques
activity of thesethat just extrapolate
projects and validate thethe
historic
laws usingtrendsthe in change
to the
future should be a conscious task. Furthermore,
related information available in the SCM systems Herraiz et al. (2007c)
AThe observed
in the that
analysis
shift of the there
programming are allows
results no longusterm
languages, tofromcorrelations
state that OS-inSECO
procedural the time
to objectis aseries
growing
oriented,
representing
research
has beenarea OSS
in
noticed activity.
software
asOSS The idea
engineer-
systems, asingof[R16,
fuzzysystems
subject time R50].
R49, series
in the tocorresponding
Due deal withthere
to this, the
uncertain
are several evolutionary
new researchbehavior of
opportunities OSS
studies, evolved over the period of time; systems
in the has also
empirical been explored.
examination,
The results analysis,
modelling, show thatmeasuring,
a fuzzy based qualitymethod for timeetc.
evaluation, series analysis is
of OSSECOs.
rather a promising approach. More rigorous
Along with this argumentation, in this section we provide two initial prediction methods may be
proposals to improve the current explored in future;
structure of the knowledge on OSSECOs:
There
The are
a definition
areas themes
for still littlefor
are OSSECOs
important cited
and a that might
taxonomy
research and itof have some future
isOSSECO
interesting related potential:
to see terms.
that
OSS process (meta-) modelling, OSS
research is available in all these areas. The question of how to use open security, Agile and OSS
development
source practices withinmethods,
a closed companyand teaching (iv)OSS is forinexample
universities.an interesting
area for further research.
Based Theonareas are important
this review we alsofor research
propose thatand it is interesting
further research istoconductedsee that on
research is available in all these areas.
how companies can transform their proprietary software to open The question of how to usesource
open
source practices within a closed company (iv)
and build a community on it. Further research related to all four research is for example an interesting
questions in Section area3.1 forcould
further research.
involve more case studies on
Based Theonareas
this are important
review we also for research
propose that and it is interesting
further researchwith istoconducted
see that
implementation of specific methodologies for dealing different on
research
how companiesis available in all these
canaspects
transform areas.
their The question
proprietary of how
software to to openusesource
open
of open source in industry.
source practices within a closed company (iv)
and build a community on it. Further research related to all four research is for example an interesting
questions in Section area3.1 forcould
further research.
involve more case studies on
Based on this reviewofwe
implementation also propose
specific methodologiesthat further researchwith
for dealing is conducted
different on
how companies can transform their proprietary
aspects of open source in industry. software to open source
and build a community on it. Further research related to all four research
questions in Section 3.1 could involve more case studies on
implementation of specific methodologies for dealing with different
aspects of open source in industry.
barriersthatpreventorganizationsfromadoptingopensourceBItools.Thesebarr
Nzaouetal. (2016), this research provides one main
FromMany a practical
of the standpoint,
studies
iersrequirefurtherconsiderationbyall are in the thisformresearch provides
ofofsurveys,
stakeholders IS managers,
which givesindata,
interested athe asadoption
broad well as
andon
contribution that is a rigorous analysis Qualitative Survey based
open source BI
necessary understanding. providers
or two and
Based
deployment consultants
onof thisopen with
it would an initial
sourceproba- structured
BI. bly be possible to lens to
principles of interpretive
conduct more
From a better
studies understandstandpoint,
investigating
methodological the most cases
specific importantof implementation
following Poba of
research (Klein&Myers,1999): the fundamental principle of Hermeneutic
barriersthatpreventorganizationsfromadoptingopensourceBItools.Thesebarr
methodologies for
From a Circle,
Nzaouetal.
practical and thedealing
(2016),with
principle
standpoint, this of
different
this Abstraction
research provides
research
as-and
provides
pects of open
one mainsource in
Generalization.
IS managers, asadoption
well as
iersrequirefurtherconsiderationbyall
industry.
contribution More that case
is a studies
rigorous could
analysis stakeholders
probablyof be interested
conducted
Qualitative Survey onindata,
alltheaspects
based of
on
To conclude,
open source BI providersthe authors acknowledge
and consultants some with areas of limitations,
an initial and
lenscall
the research questions.or two
deployment
More case
principles ofstudies
open
of source
could
interpretive BI. structured
probably also provide
to
for further studies
better understand mostofimportant
open source business intelligence to be
more conducted.
knowledgeFrom athe ofmethodological
research
barriersthatpreventorganizationsfrom
standpoint, following Poba
First, thoughquestion
ittheis 3 and for
fundamental
adequate
adopting
research
principlequestion
a qualitative 4. That is,
of Hermeneutic
survey
research Nzaouetal. (2016), this research provides one main
a practical standpoint, this research provides IS managers,cost
Circle, couldand be thecar- ried
principle out of to understand
Abstraction
andmethodologicallysufficient,thesizeofthepanelwassmall.Second,itcouldb
From
opensourceBItools.Thesebarriersrequirefurtherconsiderationbyall
more
and about the
Generalization. and as
asstakehol
well
contribution
To advantages
conclude, that
of
the isauthors
dif- a ferent
rigorous analysis ofand
approaches,
acknowledge someQualitative
why Survey
different
areas of data, based
approaches
limitations, and are on
call
openders source BI providers
interested the and
in two einterestingto
consultants
adoption or with an initial
deployment of open structured
source lens
BI. to
chosen. It is
for also worth
further ofprinciples
studiesnoticing openthat source
comparethesebarrierswiththosepreventingorganizationsfromadoptingopens
of interpretive
there are important
businessno controlled
intelligence experiments
to be at
research From abetter
(Klein&Myers,1999):
understand
methodological the
the most
standpoint,
fundamental following
principle Poba
of Hermeneutic
conducted. First, all in
though theit identified
is adequate
ourcesoftwareinareas
barriersthatpreventorganizationsfromadoptingopensourceBItools.Thesebarr articles.
for a qualitative survey
Circle, Nzaouetal. (2016),
and the principle this research provides
of Abstraction one main
and Generalization.
wheretheyareverypopular(suchaswebserver,operatingsystems,etc.).Third,si
iersrequirefurtherconsiderationbyall stakeholders interested indata,
the adoption
contribution that is a
To conclude, the authors acknowledgerigorous analysis of Qualitative Survey
some areas of limitations, based
and call on
or two
deployment einterestingto
ncethisexploratory
ofof open source BI.
principles
for further studies of open source business intelligence to beinterpretive
studyfocusesonBIexpertsinonlyonecountry,Canada,theauthorsalsorecomme
From a methodological standpoint, following Poba
conducted. First, though ittheis fundamental
adequate for principle a qualitative of Hermeneutic
survey
Nzaouetal. ourcesoftwareinareas
(2016), ndfuturestudies
this research provides one main
Circle, and the principle of Abstraction
andmethodologicallysufficient,thesizeofthepanelwassmall.Second,itcouldb and Generalization.
investigatingtheviewsofBIexpertsinothercountriesandinvolvingotherBIstak
contribution
To conclude,that theisauthors
a rigorous analysis ofsome
acknowledge Qualitative Survey data, based on
einterestingto areas of limitations, and call
ncethisexploratory
eholders(e.g.users,
twoofprinciples of interpretive
for further studies open source
comparethesebarrierswiththosepreventingorganizationsfromadoptingopens business intelligence to be
executives,etc.),assuchstudiescanincreasethevalidityofthefindingsfromthiss
research (Klein&Myers,1999): the fundamental
conducted. First, though it is adequate
ourcesoftwareinareas for principle
a qualitative of Hermeneutic
survey
Circle, and the principle ndfuturestudies
tudy.Fourth,asthis
of Abstraction and Generalization.
initialstudyfocusessolelyonbarriers,theauthorsalsorecommendthatfuturestu
To conclude,
The reviewed the authors
articles show acknowledge
einterestingto
that some areas
self-determined
ncethisexploratory
of limitations,
participation motives and call
are
for further studies of eholders(e.g.users,
dies,includingthe
open source business intelligence to be
most relevant for FLOSS developers’
studyfocusesonBIexpertsinonlyonecountry,Canada,theauthorsalsorecomme commitment. Moreover, both
identificationofstrategiesthatmaybeinitiatedbyrelevantstakeholdersindealin
conducted. First, though it is adequate for acentrality)
qualitativegroup survey
relational (e.g. trust) and ourcesoftwareinareas
structural (e.g. network
ndfuturestudies aspects
tudy.Fourth,asthis
gwiththeidentified
foster FLOSS developers’ commitment. Also, the chosen code license
barriers.Lastly,futureresearchesmaybenefitfromadoptingotherresearchmeth
The reviewed articles commitment. einterestingto
show ncethisexploratory
that self-determined
affects developers’ Beyond theseparticipation
eholders(e.g.users, aspects, the motives reviewedare
dies,includingthe
odsaswell,such
articles suggest other factors which yet need dedicated analysis.both
most relevant for FLOSS developers’
commitment. Moreover, For
ascasestudy,surveysorexperiments,astheymayprovidericherinsightsthanthe
relational (e.g. trust) and ourcesoftwareinareas
structural (e.g. network centrality) group aspects
example, the effects of members’ ndfuturestudies
tudy.Fourth,asthis cultural background [57] or their
qualitativesurvey
developers’ gwiththeidentified
commitment. adoptedinthisstudy.
foster FLOSS Also, theintensity.
chosen code license
geographic proximity [29] on their development
initialstudyfocusessolelyonbarriers,theauthorsalsorecommendthatfuturestu Further, the
The reviewed
affects articles commitment.
developers’ show ncethisexploratory
that self-determined
Beyond these participation
aspects, the motives
reviewed are
interrelations between the research eholders(e.g.users,
perspectives
dies,includingthe need further scrutiny. In
most
articlesrelevant
suggest for FLOSS
other factors odsaswell,such
developers’
which yet commitment.
need dedicated Moreover,
analysis. both
For
particular, the relationships between individual or team factors and project
relational
example, (e.g. thetrust)
effects andofstructural (e.g.
ndfuturestudies network centrality) group aspects
characteristics
For a high (see
retention table
rate, 1).itmembers’
tudy.Fourth,asthis
Such
gwiththeidentified
is important
cultural
cross-perspective
that
background
FLOSS analysis [57]
developers
or their
is necessary
perceive to
foster FLOSS qualitativesurvey
developers’ [29] commitment. adoptedinthisstudy.
geographic proximity on their development Also, theintensity.
chosen code Further, license
the
understand fully how FLOSS projects can
their project work as self- determined. incentivize
In addition, individual
members’ and group
project
affects
interrelations developers’
betweencommitment. Beyond
eholders(e.g.users,
the research perspectives theseneedaspects,
further the scrutiny.
reviewed In
factors
continuance whichisincrease
influenced developers’
by dies,includingthe
participation.
odsaswell,such
relational and Future
structural research
characteristics mayofdraw
articles the suggest other factors which yet needor
particular, relationships between individual dedicated
team analysis.
factors and For the
project
on research results by Gallivan code[21] and examine
team. Also,developers
less restrictive licenses andby if and how of
theextrinsic
modularity governance
In example,
contrast,
characteristics the
(see effects
tableare of
1). members’
commonly
tudy.Fourth,asthis
Such cultural
driven
cross-perspective background
analysis [57]
motives
isAlso,orthe
to code
their
necessary join toa
processes
affect foster
members’ FLOSS
qualitativesurvey
project gwiththeidentified
developers’
retention. personal
adoptedinthisstudy.
However asrelationships.
shown in table 3, further
there is
geographic
FLOSS proximity
project. Also, [29] on
structural their development
characteristics
understand ofintensity.
FLOSS Further,
developers’ the
isfully
little dedicated
how toFLOSS
research
projects can
research needed fullyonunderstand
how team iflevelincentivize
and how project
aspects
individual
governance and group
can
interrelations
contact
factors network
which between
influence
increase thetheir
research
developers’ joining perspectives
dies,includingthe behavior.
participation. needaffect
further
Moreover,
Future research
developers’
thescrutiny.
application
may In
draw
retention. stimulate
With regard to theodsaswell,such
individuals’key participation
role of group motives.
aspects for developers’
particular,
domainthe
on research andrelationships
the by
results development
Gallivan between phase
[21] individual
andare
identificationofstrategiesthatmaybeinitiatedbyrelevantstakeholdersindealin or team
relevant
examine andfactors
if characteristics and for
how governance project
In commitment,
contrast, developersfuture studies
are commonlyshould examine
driven by thisanalysis
aspect
extrinsic closely.
motives to In
join a
characteristics
developer
processes foster FLOSS (see table
attraction. 1). Such
However, cross-perspective
it
gwiththeidentified
developers’ personal relationships. Also, furtherto
is unclear how relational is necessary
factors
addition,
FLOSS very few
project. qualitativesurvey
articles
Also, use
structuralmore adoptedinthisstudy.
than
characteristicsone research
of FLOSS perspective.
developers’ This
understand
influence isfully
needed how
developers’ toFLOSS projectsWhile
attraction. can
research fully understand if and incentivize
Stewart andindividual
how project Gosain
governance andview
[55] group
can
contact
factorscallswhich
trust foressential,
network
as further
increase research.
influence their
developers’
there is In attraction
no particular,
joining participation.future studies
behavior.
centric Moreover,
Future
research should
onthe
research thisdraw on
application
may
aspect.draw
stimulate odsaswell,such
individuals’ participation motives.
on research
Hence, domain
futureand
research by Stewart
the by
results
evaluations and
development
Gallivan Gosain
should [21]
focus[55]
phase and
ascasestudy,surveysorexperiments,astheymayprovidericherinsightsthanthe on which
areexamine
this suggests
relevant
aspect. and that
howmembers’
if characteristics
Moreover, asfor
governance shown
In contrast,
retention is
developer developers
a product
attraction. of are commonly
motivational
However, it driven
and
is byhow
relational
unclear extrinsic
factors. motives
relational Further, to future
factors join a
inprocesses
table 2 there fosterisFLOSS
very few developers’
qualitativesurveyattraction personal
specific
adoptedinthisstudy.relationships.
research which Also, further
combines
FLOSS
research is project. Also, structural characteristics of FLOSS developers’
influence
research
individual is necessary
developers’
needed
and team to to understand
attraction.
fully
factors.understand the
While
Considering ifinteraction
andStewart of
andstructural
how ideological
that project Gosain
governance
and network
[55] view
can
status
contact
properties
trust network
as and
essential,influence
project
there their
is no joining behavior.
characteristics.
attraction To
centricdo Moreover,
so, future
research on the
studies
this application
should
aspect.
motives are dependent stimulateon individuals’
the feedback participation
of others,motives.
future studies should
draw domain and
on research theby development phase are relevant characteristics for
Hence,
examine future closely this Oh
evaluations and Jeon
should
interaction focus[40]extending
for on andthisanalyze ourthe
aspect. ways inas
Moreover,
understanding which
shown
of
inFLOSS developer
table 2 projects attraction.
thereattraction.
is verycan few However,
actively it
utilize
attraction is unclear how
theisinteraction
specific relational
research network factors
which combines of their
developer Similarly, there very little literature which
influenceto
members
individual developers’
foster
and team their attraction.
retention.
factors. While
Considering Finally, Stewart
further
that and Gosainisand
research
ideological [55] view
necessary
status
combines team and project characteristics. Considering the key role of
trust as essential,
to understand thethere
ways onis40],
innowhich
attraction project centric research
characteristics onstudies
this aspect.
motives
structural are dependent
properties [26, the feedback
future studiesof others, future
are necessary toinfluence
understandshould
Hence,
examine future evaluations
closely thiscan should
individuals’
interaction focus on this
retention. aspect. Moreover, as shown
how FLOSS projects utilize thefor extending
social contacts ourofunderstanding
their membersofto
in table 2
developer there is very
attraction. few attraction
Similarly, specific research which combines
reach out or new members. Finally,there drawing is very onlittle literature
research by Shah which[48],
individualteam
combines and and teamprojectfactors.characteristics.
Considering that ideological
Considering the and key status
role of
further studies are essential to understand fully how FLOSS initiatives
motives
structural are dependent
properties on the feedback
[26, 40],extrinsic
future studies of others, future
are necessary studies
to understand should
can stimulate individuals’ and intrinsic motives to join the
examine closely this
how FLOSS projects can utilize the interaction forsocial contacts of their membersofto
extending our understanding
project.
reach developer
out or new attraction.
members. Similarly,
Finally,there drawing is very onlittle literature
research by Shah which[48],
combines team and project characteristics.
further studies are essential to understand fully how FLOSS initiatives Considering the key role of
structural properties [26, 40], future studies
can stimulate individuals’ extrinsic and intrinsic motives to join the are necessary to understand
how FLOSS projects can utilize the social contacts of their members to
project.
reach out or new members. Finally, drawing on research by Shah [48],
further studies are essential to understand fully how FLOSS initiatives
is because of the use of ambiguous measures such as “team size”. With
such measure, it is not possible to tease out distinct lessons for the
attraction and retention of FLOSS developers. Thus, future research should
use specific measures for this particular aspect, for example the number of
new, respectively retained developers. Finally, as visualized in table 1 – 3,
our literature review shows that there is few dedicated research on these
aspects
Since we (especially
were not on able developer
to identify attraction
any existing and retention)
studies that which combines
indicated the
more effort
early than one model research
for OSS perspective.
Web project, Considering
we therefore the interrelations
believe that there between is a
theneed threefor research perspectives, however, an isolated
researchers to further explore this field. This is particularly research perspective
seems too
relevant as narrow.
OSS is being For example,
increasingly FLOSS used developers’
nowadays extrinsic by software motivesprovider are
stimulated
organizations. This bycanreceiving
be supportedappreciationby theand paper by of particular
[78], even project
though the
characteristics (e.g. corporate sponsors) [32]. With respect to these
Sinceauthor this studyin thisonly paper only focuses
focuses on effort estimation
on the involvement of OSS tools for software
towards the
interrelations,
development. future
However, studies
the on
author FLOSSstrongly management
believes should adopt moreto
development,
Studies that can another future
quantitatively research
infer OSS area that canthat
maintenance there from
beeffort is a need
investigated is
size-
than one research
develop an perspective
effort estimation in order
model to fully understand
especially forattribute
OSS theprojects.
effects of the
measuring
related the
metrics other
are
As is well known, a year of examined aspect
needed. of
To the effort
mitigate
experienceaspects. measurement
the difficulty of acquiring
of the expert’s programming skills such as
actualthe
year
that of
effort are experience
data ofthe
from incomplete
involved in development
project toward the
de-development
velopment OSS
records,
can asYuwell
contribute et.althe error
[40][41][23]
to fixing
a different
on predicting the size-relatedtime.
effort, therefore how about a year of experience of the expert towardthe
focused metrics to indirectly estimate the
maintenance
tools. Does this effort.
affectThe thestrong correlation between
effort estimation? Since OSS size-related
is an openmetrics source
which and the actual can
everyone effort has been
access freely, confirmed
thereforeinaclosed-source
bug that can occur projects during[28].the
However there still exists a gap between size-related metrics and time-
Itimplementation
New
will evaluation
be worthwhile cannot to be
methods avoided.
are
explore needed Astosuch,
the capability validate what
modelthewould be the
correctness
for OSS time
of these
developers. and
aware
effort effort
estimationneeded fortomaintaining
methods. fix theWithbug OSS
and
the projects.
how
growth this
of There
can
more affect is athe
companies need for can
effort studies
developing also that
be
or
Since most OSS projects rely on task or issue tracking systems to maintain
can quantitatively
collaborating with infer projects,
OSS OSS maintenance
further investigated.
estimating effort from size-related
maintenance effort has metrics.
become
the projects, recognizing the time of specific maintenance tasks can
a Furthermore,
major interest. the effort
More drivers used
researchers in general
haveassignment
been focusing maintenance
on improving effortthe
provide better decision support for task as well as OSS project
estimation
estimation models
towards can serve
the direct as an
effort example
of OSSare to improve
projects from OSS maintenance
both people and
management. A large amount of studies devoted to predicting bug
effort
activity estimation.
aspects byFor example, maintenance
developing Nguyen [28]effort developed estimationan extension
methods. to
fixing time while a small amount focused on other activities such as code
COCOMO
However, II [9]
sinceduplicationsize and effort
most OSSidentification. estimation
projects lack of models
complete to capture
development various
review and These studies commonly records used
and characteristics
actual effort of software
data, it is maintenance
very difficult to inevaluate
generaland through a number
validate thewhich of
results
metrics from source code changes and issue reports as predictors,
enhancements
of theseindicatemethods to the
bytheCOCOMO the
comparing II models
estimated to support
results with the cost the estimation
actual of
that prediction results are basically related to the effort.
software
This maintenance.
can be a significant Some threat effort drivers
to these such
estimation as DATA (Database size),
characteristics of the tasks. There are two kinds ofmethods targets among and raises these the
CPLXto(Product
risks effectively Complexity),
validate of and
their PVOL
results. (Platform
This is Volatility)
an issue where in hiswe study
need
predictions. One is the numerical days of an activity, evaluated by
might
new contribute
evaluation greatly
methods andthatto OSS
can maintenance
validate effort estimation.
Individual
PRED(25) or contribution
PRED(50). Another performance
is the ofthe
binsmeasurement correctness
categorical also of
timehas these
been
evaluated
by receiving
accuracy, attention.
precision,Gousios estimation
recall, or[15] methods.
defined The
f-measure. a contribution
prediction ratio results byfor
Asconsidering
shown in Table V and Table VI, the SLR found relatively a few studies
numerical days are not very satisfying, while the results of categoricaland
various type of parameters from the OSS community, bins
providing
Rastogi et al.sufficient
defined details regarding theterms advantagesof fourand challenges roles of
are relatively high. In the
OSScontribution
projects, theintime recorded different
on issue tracking of
using
stakeholder. OS in computer
Since themay science
importance education.
of contribution Moreover,
measurement some of the
has been
system and repository not correlate with the actual effort because the
advantages
realized, and
it challenges
might be a are supported
promising research by atopicfew of in the
the papers
coming found.
years. For
developers are voluntary and self-determined when implementing the
instance, only 4 papers (19%) included specific evidence regarding the
tasks. It will be worthwhile to explore the capability model for OSS
wider range of skills acquired through the use of OS, which suggests that
developers and consider the developer-related metrics to be one of the
there is a lack of data regarding the advantages and challenges of OS use in
sources
In practicalof predictors,
addition, as an opportunity
possible directions for future to improve
research the prediction results.
the situations. Consequently, the results of have been suggested
this study may not be
As
such Kirk and
as generalised Miller
developing and stated
effective in [22],
strategies “although
for proper no one defends a positivistic
easily further investigations mustalignment
be planned of in
OSpractical
projects
ontology,
and courses but as scholars
well as inefficient
social scienceevaluation has find out thatfitting
approaches much into research
the
situations where OS is currently used in computer science education.
makes sense only in terms of a set of unexamined
specifc context of using OS in computer science education. positivist assumptions.”
Research in the field of OSS success has the same problem. We want to
precisely point to variables like:“general viewpoint of audience society”
and
As could “actual beuse of software3”
observed in fig 3, as mostmeasurements
of factors which of successaffectand the contextual
success of
parameters such as “availability of knowledgeable
OSS are related to developers and product and most of success indicators developers”, “legal
support and level of IT development in the
are related to product. Our study shows that user related factors have been development environment” as
affecting
studied factors.
lessallthan That’s
otherwhy factorswe andrecommend
researchers mixed-methodology
limited research
Although research in the field of OSS success havethese triedfactors
to study to
number in the field of OSS success.
previous work, but we observe little connection between them. Onethis
of downloads in both success factors and indicators. On top of
exception research gap, we highlight
is reference [1] whichsome other gapsfour
has mentioned thatprevious
may helpworks futurein the
researchers
model and studiedbetterthem define inand conduct their
a longitudinal study study andinfoundthe field:
some
inconsistencies between original and current study.We believe that study
of other work and comparing the results may lead to considering new
factors (such as contextual or longitudinal factors) in study of OSS
success.
Except initial research by Crowston et al. [23], and Crowston et al. work
on the definition of OSS success [21] , we do not find any general model
OSS of OSS success. takes
development In factplace manyinresearches
an environment in the whichfield have is highlyjust triedaffectedto
validate their partial model of OSS
by socio-cultural parameters and specifications of users and development success. We believe that according to
wide
teams
.In range
of OSS
Section of social,
7 (SQ2), may affect cultural and
or alter the
we discussed technical
how success factors
studiesparameters that
that showed may have
of how an
OSS.toThat’s effect
com-
on bine
while success two development models could be extended to the third. This weof
context of of OSS, developing
development is a general
usually model
ignored is
while not reasonable
studying but
success
recommend
OSS. Except contingency
[20] thatfor practicesin
studies specific this
kindregard. In other words we suggest
The discussion
first included
opportunity ideas like:
future (i) the
research liesofinsoftware
reconciliation reexecution ofand FOSS [4]the
of that
and verifies
plan-
protocol,
researchers
the
driven model in to
Koreandevelop general
software models
context, for
we specific
do not findcontexts
any otherand believe
research
to capture references to more recent work that extends the search spacefor
configuration management practices to agile model; (ii) the need
.In
that Section
was studiesthat
based7 (SQ2), these
on onaThismodels
specific would how
wereconciliation
discussed beEvenmore
studies helpful twoin
that showedpractice. how totried
com-
further
chronologically. the couldcontext.
also include of these
theotherpractice
search papers
of continuous
engines, havesuch code asto
bine
generalize
integration two development
their findings
between agile models
andFOSS
and could
the later be extended
one mentioned
models and extension to the
the of third.
context
this This
based to
search
ACM (Association for Computing Machinery), in an attempt to retrieve
discussion
research included ideas like: (i) the reconciliation of FOSS and plan-
the documents only indexed by these machines, which would extend theand
contextas aof limitation.
the So
plan-driven it seems
model; that localizing
(iii) investigate the issue
how theof success
knowledge
driven
paying
management configuration
attentionpractices to management
parameters
in agile practices
such as:the to agile
social, model;
cultural and(ii) the
economical need for
search
.In Sectionspace geographi-
7 (SQ2), cally.model
wereconciliation
discussed Finally,how
can contribute
studies search thatcan
to improve
showed also behow
knowledge
expanded
tocode
com-
further
state
management studies
of manual
development
in orga- on the community
nizations or in would FOSSof thebe practice
beneficial
communities; of continuous
point
(iv) of view
analyze ifinthe
with
bine two development searches to include:
models books; conferences; theses and
integration
use of explicit between knowledge agile and FOSScould
future
management models
research.
be extended
practices,and extension
coming
to theofthird.
from this This to
search
plan-driven
dissertations;
discussion technical
included reports;
ideas like: and
(i) the other search engines,
reconciliation of such and
FOSS as Google
plan-
the
and context
FOSS and of the plan-driven
development, can be model;
beneficial (iii) investigate
in an agile how
context; the knowledge
(v) extrap-
drivenScholar
configuration AISeL (Association
management for
practices Information
to agile Systems
model; (ii) Electronic
the need for
management
olate theAlthough
use practices
of testthe in agile
driven model can contribute
development to adopted
a plan-driven to improve context. knowledge
Library).
.In Section
further 7 (SQ2),
studies on thewe systematic
discussed
reconciliation approach
how of studies
the ensures
that showed
practice of continuous how toThese
the reliability
com-
code
management
ideas aretwo in organizations
allpleteness
opportunities for or in FOSS
future research.An communities; areato that (iv)
still analyze
deserves if the
andbine
com-
integration development
between of this
agile and study,
models
FOSS itcould
can
models be
be amplified
extended
and extension by these
the of third.
this This to
extensions
search to
use be of explored
explicit knowledge is the search management
for studies practices,
to reconcile coming
theofthree from plan-driven
models of
discussion
the context ofincluded
the ideas like:
plan-driven (i) the(iii)
model; reconciliation
investigate how FOSS the and plan-
knowledge
and FOSSdevelopment,
software development,since can be only beneficial
one study in was
an agile context; (v) extrap-
driven
management configurationpractices management
in agile model practices to agileidentified
can contribute model;
to improve
in the
(ii) this quasi-
need
knowledge for
olate
systematicthe usereview.of test driven
First of development
all, it can be to
present a plan-driven
in areas context.
that were These
not the
further
management studies on
in orga- nizationsthe reconciliation
or in FOSS of the practice
communities; of continuous code
ideas
focusare all opportunities
of this work. Some forthese
of future research.An area are that(iv) stillanalyze
deserves if theto
integration
use of explicit between knowledge agile and FOSSunderstudied
management models
practices,and areas
extension
coming
indicated
of this
from
below
search
plan-driven to
tobe explored
guide future is the
research.search In for studies
addition, to
as reconcile
stated before, the three
the models
reconciliation of
the
and context of the plan-driven
FOSSdevelopment,
development, can be model;
beneficial (iii) investigate
in an agile how
context; the knowledge
(v) extrap-
software
research areapractices
is still at in ansince
earlyonly stage, onebut study
it is was identified
expected that in in future
this quasi-
there
management
olate the use of test driven agile model
development can contribute
to a plan-driven to improvecontext. knowledge
These
systematic
are more review.
studies and First of all,
results on itthe
cantopic be present
to be in areas thatFinally,
investigated. were notthere the
management
ideas are in organizations
all opportunities forthese or in FOSS
future research.An communities; area are that(iv) stillanalyze
deserves if theto
focus
also of this work. Some of understudied areas indicated below
use of remains
be explicit knowledge
explored
the possibility
is the search
that organizations
management
for studies practices,
to reconcile
orcoming
other researchers
the from
three models
have
plan-driven of
to
already
COSSguide isfuture
achieved
comparable research.
positive In addition,
results
to can
opensourcing. on theasreconciliation
stated
In before,among
opensourcing, thethe reconciliation
agile,
companies FOSS,
and FOSS
software development,
development, since be
only beneficial
onebut study in an agile
was identifiedcontext; (v) extrap-
in future
this quasi-
research
outsource area
andto the is
plan-drivenstill
open at an
source early
models, stage,
community but have itnotis
outside expected
written
of the that
about
company in
it yet. [82]. there
This
olate the use
systematic review.of testFirst driven of development
all, it can be to a plan-driven
present in areas context.
that were These
not the
arelowers
more studies
theopportunities and results
cost of development, on the topic
because to be investigated.
on thearea onethat hand, Finally, there
ideas
focus are
of all
this work. Some forthese
of future research.An
understudied areas are stillvolunteer
indicated deserves below to
also remains
developers code thefor possibility
free, and that organizations
onstudies
the other hand, orusers
otherreport researchers flaws have
in the
The be explored
next
to guide point is
is
future research. the
that, search
as for
Riehle
In addition, noted, to
openreconcile
source
asreconciliation
stated before,among the three
software models
can
the reconciliation possessof
already
software achieved
[43]. positive
Hence, the results
existence on the of an open source community agile,isFOSS, vital
software
established
researchCOSS development,
area markets,
is
business still at an
model since
providedearly
and only
that
stage,
probably itone study
isbut
sufficiently
it is
possess was identified
expected disruptive
established that inmarkets.
in thisFor
[1].
future quasi- this
there
to the and
success plan-driven
of the open models,
sourcing but have
company not written
[82]. In about
COSS it yet.
also exists a
systematic
disruptiveness,
are more
Due tostudiesreview.
the cultural,weand First
suggest of all,
results onitthe
working
economic, can be the
on
topic present
to be
institutional, revenue in areas
model.
investigated.
geographic thatIn were
fact,
Finally,
and othernot
since
therethe
user
focus community
of this work. and Some somehowof countries
these developer
understudied is effective
areas and improving
areresearchers
indicated [3].
the
alsobusiness
remains
characteristics model
thedeveloping
of is a system
possibility thatand every
withchange
organizations emergingor in one
other markets component [84], below also
have
the use
But,
to
affects the
guidethe problem
future
rest [4], is creating
research.
the In
initiation and
addition,
of sustaining
this as stated
disruption such communities
before,
can bethe a [41,model.
reconciliation
revenue 43].
already ofachieved
business positive models of results
developed on theand reconciliation
matured markets amongisagile, oftenFOSS,
Therefore,
research
Therefore,
COSS area isisitstill
is imperative
itplan-driven
business at
suggestedmodel andthat
anmodels,
early
that stage,
future the but
probably management
itof
studies isthe
possess expected
in helpingof the
established that creation
toand in future
describe and there
more
and
unsuccessful [58]. Therefore, butlogic
the have not written
creation about itmarkets.
yet.
capture of
maintenance
are more
revenue studies of theand user community
results on the in
topic future
to be research
investigated. be investigated;
Finally, there an
import value of COSS may need to be adjusted. This implies the needthe
Due models
to the examine
cultural, the
economic,role of new revenue
institutional, models
geographic in adding
and other to to
issue which
alsolocalize
remainsthe
disruptiveness hastheCOSS not been
possibility
the open seriously
thatmodel
source considered
organizations
software. in
or the
is other literature
researchers of COSS.
have
characteristics of developing business countries [85].This
with emerging
In addition,because
markets the[84],
countries explanation
the
are use
already achieved
ofdemanding
open source
of business positive
software
models ofand
indigenous results
revenue
developed on
localized the
models reconciliation
and has
matured
software, been among
difficult,
butmarkets
due to is agile,
especially
often
expensive FOSS,
because and
of
unsuccessful plan-driven
free distribution
[58]. models,
Therefore, [83]. but
theIn
licenses, some of them are looking for open source software [86]; have
addition,
logic not
of the written
this study
creation about suggests
and it yet.
capture thatof athe
provision
import
localized value
Further research of
open sourceof complementarities
COSS may
software.
is required need has
to bebeen
But, in suitable
to explore adjusted.an
the literature integral
This part
implies
of the COSS
KR practices of the
the COSS
need
business
applicable toin
business
OSS localize
model,
projectsmodel
notthe asCOSS
only as a way of
business earning
the commercialization
indicated by one model money.
of the[85]. of In
questions So,
open new
addition, disruptive
raisedcountries
source software
on revenue
mechanisms are
as a
demanding
Organisations
business
and team model norms is models
indigenous
invest
thatnotare can
in KM help
and
considered, new
localized
activities
used to store configuration
software,
to
butknowledge organise,
there are also butof the
due
create,
con- few to expensive
share,
studiesbyonteam
tributed reuse,the
open licenses,
transfer,
source
members. some
andsoftware
Inretain CSS ofknowledge.
them are looking
localization.
organisations WeKR found
Therefore,for knowledge
mainly open source
itcomes
is suggested.
intosoftware
relevantfocus [86]; there
activities
Finally,
when a in
an
localized
is notOSS much
employee open
projects issource
research namely
leaving software.
anknowledge
about But, increation
the organization
organisation the literature
(Lindvall andandabout of what
&knowledge
Rus, the2003).
COSSissharing.business
happening
On the
model,
Knowledge
inside a COSS
contrary, not
sharing only
in OSScompany. the
was commercialization
found
projects the to
Therefore, be
unpredictableabundant of
one ofnature open
but source
there
the implications was
of commitmentsoftware
no evidenceas
for future a of
from
business
knowledge
research
contributors model
is thatcreates is
retention not
it is suggestedconsidered,
to reduce
an element that the but
offuture there
impact are
of
research
risk (Robles also
knowledge
&examine few studies
loss
Gonzalez-Barahona, in on
OSS
the structural the
open source
projects.
dimensions
2006). Insoftware
Moreover,
OSS, localization.
knowledge
and contributors
content dimensionscan Therefore,
sharing leave issince
[87] ofit the
reactive is
theysuggested.
inare
COSS nature, Finally,
initiated
organization.
not under anythereby
is not
the much research
contributor while about
looking the
for organization
task
contractual binding as in CSS organisations. relevant and about
knowledge. what We is happening
suggest that
inside
there is a COSS
insufficient company.
attention Therefore,
paid to
In this literature review, we found that KM relevant activities of KMone in of the
general implications
in OSS, in for future
particular,
research
there would
knowledge is creation
thatappear it is andsuggested
to beknowledge that future
an absence research
of proactive
sharing are evident examine
measures in OSS the structural
to reduce
projects theas
dimensions
potential impact and of content
knowledge dimensionsloss.
discussed in Section 6.1. Furthermore, literature examination directed us We [87]also of the
propose COSS the organization.
need for a KM
to 10evaluation
mitigations metric in OSSthe
to reduce projects
impactsimilar of knowledgeto the ones lossthat dueevaluate
to contributorthe
health of online turnover communities.
in OSS projects KM evaluation discussedmetrics in Section should 6.2.be based on
the extent of knowledge sharing activities observed in a project. Such a
metric could help to inform potential consumers of the OSS of the KM
status on a project, something that is non-existent today. We consider it a
vital ability for OSS projects to sustain a knowledge-sharing culture that
will support the long-term survival and competitiveness of OSS projects.
keeping design solutions open, accessible, replicable, and adaptable while
conforming to safety regulations and standards is a challenging topic and
remains mostly unresolved. Co-operatives and similar models may suggest
community-based ownership and responsibility, but this model is not as
open as open design is espoused to be enacted. This affects the reliability
of these design solutions, especially when they are not widely reviewed
online. Although larger transitions towards alternative economic models
are discussed on the macro level, research on how they will be enacted as
development, iteration and dissemination of open designs is still an
important area of interest.
In the reviewed literature,
Moreover, it remains to be seen openifdesign is indeed
open design as framed
a research as aframing
better
alternative
remains by many authors,
semantically especially tied
and ontologically on topics proposing
to trajectories ofnew ways to
business-as-
do business, prototype alternative economies, and
usual (as has been seen in software; see Morozov, 2013), and therefore not foster sustainability.
From aalternative
strictly business perspective, the potentialifof open design is
ina the
truestudy with anor necessarily
development democratizing;
component it is
but oriented increasingly
to a specific
observed
embraced mainly as a value-capture
by research on alternative strategy and a way to postcolonial
post-capitalist achieve rapid
requirement, 16.13% adds tools to the central axis andof the GIS Web
innovation
practices;and cycles
or 9.68% for further
if a newintegrates development
term becomes and
more appealing wide-scale testing.
to the research
architecture
Researchers have contradictions newthemethods
on predictiveandpower
algorithms to improve
of metrics used
However, open design’s
community relation to enterprise
and replaces is still
it entirely. largely considered
for the evolution of different
OSS aspects
studies. These of architecture.
contractions are discussed in Sect.
within the current paradigm, while the potential of an open design ‘sharing
Open
4.2. Source
There is Web
a needSoftware
of furtherArchitecture
research toComponents.
empirically Hence, the
evaluate need to
predictive
economy’ is not yet generally discussed as a way to transform the way
carry
power outofnew research aimedMost
different at evaluating andapplied
improving thefile
components
businesses operate.metrics. metrics are
Toward the manufacturing on the
side, companies level
open forup
of
the the Open
evolution Source
pre- software
diction of OSSarchitecture
studies as of a geographic
highlighted in information
Table 19. We
their initial processes but do not develop alternative models befitting the
system in ametrics
Web environment.
Inalso
bothanalyzed of that
casessharing class level
economy
evolution areevolution
as suggested
prediction and applied
by open by few studies
design.
process support, butthe
method-level
reviewed articles metrics are applied
admitted by noneofofexternal
the necessity the selected primary
validity, studies.
whereas the
Moreover, we also reveal that code level metrics are
ratio of articles not addressing validation process is considerably higher applied by most
researchers
(56% for evolution
for evolution prediction prediction of
and 68% forOSS studies.
evolution Little support).
process attention is We
paid
real to izedrequirements, design and of
that vast heterogeneity architectural level metrics
evolution prediction for predicting
models building
data make their evaluation evolution
difficult.ofGeneralization
OSS studies. of evolution prediction
models
Future regardless
research should ofalso
theirfocus
applicability on project
on predicting changesize requires the
propagation, size,
attention of researchers.
refactoring, maintenance effort, contribution evolution, SOC and clone
evolution besides defect prediction.
The architectural and requirements change evolution is also undermined

area that needs attention of researchers.
The tools and approaches proposed for evolution also admit the necessity
of external validation. Future research should focus on the above
The evaluation
mentioned issues andoftryOSS in medicine
to make them moreshould not be over-
generalized looked.of the
regardless
Professional medical certification (FDA and
domain of OSS projects. CE) often cannot be applied
to OSS because it requires a legal commercial entity for distribution
liability and support availability. It is also important that FDA and CE
certification confirm that the software and its provider ensure an error-free
work- flow, but not the accuracy of software actions/calculations, and
This study
provide could help
appropriate researchers to
documentation andidentify
support. essential quality
Therefore, ex- attributes
perimental
with
The which
studies
most and to develop
algorithm
commonly more robustshould
evaluations
used selection qualitybe
method models
is that are
conducted
the model andapplicable
and in
published.
approach the
the various software domains. Also, researchers can compare
least considered are the tool- based and data mining approaches. Another the exist-
ing selection
interesting result methods in order
is that nearly halfto determine
(47%) of thethe most effective.
selected papers do not
mention an application domain for the models in their research. More
attention should be paid to building models that incorporate only essential
Unbalanced DistributionAlso,
quality characteristics. of Measures:
framework, just tool-based
by lookingandintodata
the mining
measure
tables, it is
selection easy toshould
methods observebethat themore
given amount of measures
attention formodel
in future some
characteristics is high (e.g., activeness
proposals.with 17 mfeasures, visibility with
11 measures) while for other is very low (e.g., heterogeneity with 1
measure, information consistency with 1 measure). This unbalanced
situation could be an indicator that more research is needed for the
characteristics with a low amount of measures.
Computer Science Education different contributions. Therefore, the
authors recognize that grading is not an easy task. They suggest the need to
establish a set of metrics for each role a student plays.
Learning how to solve complex problems, and knowing how to work in
Someteams trends and issues
are relevant skillsemerged
that arefrom typicala detailed
requirementsanalysisinofSEtheeducation. studies: (i)
solution
Thus, students proposals shouldare thedevelopmainsome research skills approach; (ii) very few papers
such as communication and
focus on specific SE areas; (iii)
leadership. Peer assessment is one important instrument the traditional project method is the main
to approach this
learning
need. Morelli approach; and de (iv)Lanerolle
most studies (2009) usesustain
previously chosen OSP
that students should in regular
assess
courses;
their peers(v) in there is a balance
conjunction with teacherbetweenassessmentinside and in nocourses
controlon approaches;
OSP, even
and (vi) very though, the adoption of this practice is still a challenge. based
few papers use criteria to evaluate students’ learning
In any on either
active outcomes
learning approach, or developed studentsskills. areWe also foundtothree
responsible conduct maintheir
combinations
own learning. A of formative
OSP use: (a) full control
evaluation that and predefined
includes self-, peer projects, (b) no
and faculty
Regarding RQ 2choice
and RQprojects 3, we found that therecontrol
seems with to a lack ofalmost
research
assessment can play an important role in this process. According to Ellisno
control and free and (c) inside no or et
on the use
project choice of OI forfor requirements
students. These prioritization
trends and issues andprovide
requirements future
al. (2012), the iterative devAll those issues represent research opportunities
validation as there was only one paper dealing with these topics each, i.e.,
to be more directionsthoroughlyfor research.
explored in the future.
papers R_11 and R_19, respectively.
In addition, we found only one paper (paper R_18, dealing with OI in the
context of requirements extraction) that presented a solution approach with
tool support.the
Acknowledging This lack indicates
of published that there researchis littleonautomation
the use of OI support
strategies
mentioned in the literature on the use
in specific RE activities, i.e., prioritization and validation, as well as the of OI strategies in RE.
When
lack ofcontrasting
reported tool previous
support, literature
we see on new older “platforms-wars”,
opportunities for research suchon as
the ones from
automated and thus non- ON the PC and game-console
intrusive industries, with
and low-cost methods for applying OI the current and
CO-EVOLUTION
under-studied
strategies mobile platforms-war, we empirically noticestudy that many of
It is turnedinout RE.from More ourspecifically,
review thatwe thethink this mapping
understanding of co-evolution provides of
the
us with market players remain the same (Microsoft and Apple). There is a
the code aand motivation
the community to conduct in OSS more research
projects hasonreceived
capitalizing little upon
attention OI toin
scenario of convergence:
automatically sameand firms push forrequirements.
similar technological
literature (Figure 2). As aextract consequence, prioritize
the community dimension and
standards
As across in
observed different
the study, platforms,
code channels i.e. Microsoft
quality is(e.g., asWindows
seen mailing the most within
important X-box,
corresponding
Sub-project evolution communication
with their community. Large open archives,
source bug
projects
Surface
criterionsystems)Tablets,
for measuring PC, Netbooks
quality.seldom, and
On theasMobileother phones.
hand, This
market convergence
tracking
often encompass are
many explored
sub-projects. Such can as,besub-projects
seen from success
Figure
in Eclipse, 5and
and
between
developer industries
activity remains
seem to unexplored
be the most by academia.
important Interesting
criteria for research
measuring
FigureGNU, 6 respectively.
Linux, and Apache. Study on co-evolution in OSS projects, however,
formed is
questions
success. dealing
Therefore, inwith
future,theOftenresearchers
ecologyofofsuch
implications sub-communities
haveintosuch convergence
study how code remain
quality
aroundbecoming increasingly
these sub-projects, popular.
whichconcentrate Because,
are governed projects
by aplatform-war the
common governance code
may unexplored,
be used to i.emeasure
“should softwarefirms success, on
or one
how market success or run and
evolution
[50]. Study is dependent
on theseveral on the contribution
formation and evolu- in of community
tion of sub-projects members,and their and
developer activity may be platform-wars
used to measure parallel?
software quality. Only then,
that a successful
communities haveevolution
revealed of
many the code is required for the
key characteristics, which survival
are of the
listed in
itSince
might
community.
measuring
beThepossible the
followingtosuccess
talkresearch or quality
about significantis a challenging
directions relationship
can be
task,
between
considered
especially
OSS
relevant.
Tabletool V and Table
contribution is quiteVI, respectively.minimal. Yet the interdependency
This may lead practitioners to in evolution
Exploringthe socio-technical success
congruence. and quality.
between
collaboratetwo with and their impact
researchers on theIn
in order
the OSS
tooverall
clarifyproject
projects
terminology,
contributions
evolution remain
identify
made byuntouched. the community The members
following not
would only be drive
worth the to system evolution but
investigate.
metrics, and develop tools that are capable of meeting this need.
• also
Doesredefine
there exist the arole of these between
correlation contributing members and
the evolution thus change
(growth, complexity, the
social
change) dynamics of the OSS community
of the sub-projects and their associated [53]. In this connection, it will
sub-communities? Does be
very interesting to investigate the phenomenon
the community change with the change in the sub-project? socio-technical congruence
in OSS • How projects.
does aSocio-technical
community form congruence
around a newly whichadded is a conceptualization
sub-project?
of Conway’s
• What attributes of a sub-project attract new developersbetween
law [67] states that there should exists a match to join? the
coordination needs established
• What happens to the sub-community when a sub- project is deleted by the technical domain (i.e., the or
Metric set for software
architectural
Framework fordependency
the data evolution.in the
collection Software
software)
and evolution
and the
representation. studies
actual As mostly utilize
coordination
discussed in
merged to other sub-project?
metrics
activities
RQ6, that are
OSS•carried empirically
projects outoften
by project validated
produce members in
large prior(i.e.,
volume studies
within
of data (asthe presented
members oftheir
representing in Table
the
What dependencies lead to inter project communica-
III).development
These metrics
development team) are evolution
and derived
[67]. This for closedResearch
concept
history. source
was projects,
already to and
explored
date, are
explores in primarily
closed
the
tion?
used
source to verify theand Lehman’s law ofa software evolution. Though these 6.
repositories •projects,
that
Whatmaintain
kind and reported
these
leveldata, a high
of correlation
list of which
communication with
is
and software
provided
collabo- in build
Figure
metrics
success,
However, provide
quality,data valuable
and insight
fasterplace
collection rate
and of to OSS evolution,
modification
representation they
in[68].
these Thusdo not consider
socio-technical
repositories varythe
ration takes between sub-communities?
community
congruence
significantly dynamics.
plays
from Thus
a project
pivotal an
role empirically
in conceptualizing validated set
the of metrics
co-evolution in favor
in a
• Does there existto aproject.
correlation Furthermore,
between the data from the same
project
of
project.explicit
source may representation
Surprisingly,
haveevolution
different of
this notion the
formatting community
as a research
(e.g., is required
area
emails has
are to
not
oftencomplement
been
free iven
of much
format the
and the sub-project evolution?
attention
even inamong
listing open
the source
senders existing metric set.
researchers.
credentials). Although
Due of it isfacts,
to research
these identifiedit is be aand
Study the existence of SOC. Another direction would to
reported
challenging as a desired
task to property
collect for
relevant collaborative
study the notion of SOC (Self Organized Criticality) in OSS projects.from
data following development
a standard activities
format like
SOC
OSS projects
repositories. [69]. InConsidering
this context, the lack
researchers
dynamics articulate that the current state of a project is determined (or at of focus
often in this
employ direction,
their we
own
meansheavily
least, to collect propose
influenced) theby
and represent following
data for
events to research.
that investigate.
took place Thislong reduces
time ago. the
• Does the
compatibility essence and of socio-technical
comparability of congruence
the
Existential exploration of SOC in the domain of OSS projects reveals reported as a
results conceptualization
even if they useof
same Conway’s
data law
sources. holds
Taking for OSS
these project?
issues
contradictory results (Table V). Thus future research can take further step in Can it be
consideration, stated as
a an
frameworkimplicit for
uniform
in validating characteristics
data collection
the existence orof
and property
representation
SOC and of successfulcan be OSS
its implication developed
onproject? to make the
the evolution of
• What quantitative resultsapproach/method
cohesive and can
comparable
open source software. be utilized
to each to verify
other. the existence
of socio-technical congruence in OSS projects? What repositories can be
used for this purpose?
• What correlation can be derived between socio- technical congruence and
the quality/sustainability of OSS projects?
Migration of responsibility and sustainability. It has been reported that
migration of developers from one release to the next is high and that the
developers take moreON responsibility
RESEARCH as METHOD
they gain experience. Yet it is a
common phenomenon in open
A number of issues related to the research approach source domain that developers
can be improvedfreely jointoor
leave the project.
increase And whenof
the acceptability a developer
the reported leaves,
results. hisWe responsibilities
pointed out the must
be assigned to someone else. For instance,
followings, the codebase maintained by a
outgoing developer should be taken care
External validity of the results. Empirical study is the most popular of by others. Else it will be
research abandonedapproach andemployed
discardedinfrom subsequent
evolution studies releases.
(FigureThus it willstudies,
4). These be
beneficial to explore
however, are horizontal in nature (as reported in RQ5) considering onlythe followings,
•flagship
How responsibility ON COMMUNITY theEVOLUTION
OSS projects.migrates Due to this among approach developers?
of studying DoesOSSthis migration
projects, the
Study on the community
follow preferential-attachment?, evolution identifies
i.e., is the threat, several
responsibility key properties
handed over to7.
reported results suffer from generalizability as reported in Figure
(reported
the to devel- in Table VI), which lay the foundation forisfurther research in this
Predicting
Yet makeopers
thethese who
future.
finding are in
Prediction close
applicable ofconnection
OSS holdtoforthe
and projects outgoing
one
the area
extended developer.
that is least
region of
popular direction.
• Whatamong impact We
the study propose
such migrationthe
facetsshould followings
hasbeon
(Figure 2). the to be
Yetproject investigated.
future evolution?
research route should
OSS projects, explicit measure taken. An interesting to
Community
focus on this building.reliable
developing Studiesprediction
reported that the andmajority of OSS projects
deal with is to categorize the findingsmodels (current ormethods supporting
future) according to
failed
error to attract
prediction, members
measuring to attain
maintenance the critical
effort mass.
and Only
cost of few
OSS flagship
projects.
the project domain, or similar organizational structure and practices, or
projects
Because, are able
the sizeto attract developers.
commercial organizations, Factors forinfluencing
instance, the motivation
similar product and complexity. This will reveal therequires
broader such picture
to join a community
prediction has been studied (e.g.,the[62] [50]), and several
which canmodels
Comprehension then of betotheseassess
compared results an open
suggest
and source
possiblythat component
merged lawsfor and fortheory
adoption
proposing appear
a more[66].to
phenomena
be breakinggeneral are
down proposed.
through For
non- instance,
conforming rich gets data richer
and phenomenon.
findings (Table Yet
IV).
evolutionary pattern and behavior for OSS.
Thus it isLehman’s
not identified laws whatof softwareexclusive properties
evolution whichinitiate is primarilythe community
based on the
building process at the nebula stage of
study of the large close source systems, is not sufficient to justify the project. Following research or
questions can be considered
account for the evolutionary pattern and behavior of the open source relevant,
• Why some As
software. projects are able these
none-the-less to attractlawscontributors
did not consider duringthe thecommunity
nebula stage
dimension of of thethe OSS project,
projects whilewhichmostisof anthemintegral canpartnot?of sustainable
•evolution
What empirical
More formation of the community refers
of the open source softwareTo deal with thisitsproblem,
and theoretical research is to a
neededbal- ance
on one, and ahow
applicability viable the
in
community
context of OSS structure
certification. changes towards
Important
route would be to examine the underlying ontologies for software a balance
research structure
questions during
include its
“How
do evolution?
certification
evolution
show a great Can
[5] a visibleshape
concerns
considering
concentration pattern
the and
OSS beimpact
of empirical identified
specific OSS
research within
characteristics,
on the thestudy
communities?” domain
and ofandof “How
then
how OSSre-
or-
toganisations
organize
assess the adoptopen projects
laws of source
ISS for
software the
communities
development above
evolution two
for cases?
effective
intototheir fit ininternal and
OSS domain. economic
software
development processes. Othercertification? research areas “ receive much less attention.
CanAmonga newthepharmaceutical
frameworks/methods, be developed models entirely
and tools through an open
identified, nonesource of
model?
themFur-have Likely not.
been empirically
thermore, However, a new drug
validated in real
the architecture-sensitive for a neglected
industry
metrics for settings.disease
code anoma- may
One of be
liesthe
shepherded
implications
discovery provides up to
of these clinical trials
find- ingsoffor
the majority utilizing
research to
awareness a hybrid
and open
practice for
engineers source
is the model
theneed for
existence
combining
of the more open
empirical
smells source with
studies on
code elements other
that development
engineering models
practices to
are more significant such
tosupport as fee-for-
ISS
the architecture
Inser-
design the vice
thanoutsourcing.
anomaly’sthe most To assist
prioritization
traditional with this
strategy,
development.
metrics thedevelopment,
that Sagglomeration
are depending weon believe
floodsource that
standardcode
further
and most research
optimal is needed
models on
showed business
that model-
some
and based on static code metrics combination. This means that the ing,
agglomerationsincentive development
are overburden
andwith
developers thefalse
impact
andpositivesof the and
engineers use of
couldnotthe public
precise
detect domain.
enough
and repairtosuch It is important
identify that
architectural
anomalies this
promptly.
research
inconsistencies includes
in expert
classes, input
leading from
to the
Therefore, more studies are needed in this field for other metrics to be researchers,
inability to the
capture pharmaceutical
several various
industry
analyzed and PDPs
architectural inproblem
order to assess
totypes.
providetheInpracticality
contrast,
the mostthe and relevancearchitecture
recommended
appropriate of heuristics,
open source
architecture
without any impact drug
blueprints, of the discovery
and sizethebias.at Furthermore,
a task level.
context-based smellthere prioritization
is a need for
techniques show the ability to rank and
metrics that have a great ability to discover the inconsistent classes improve in identifying the
prioritization
affected
This reveals bythat of anomalies
the architecture
degradationmay relatedfrom to architectural
bethe consistent
recovered problems.
withclasses.
acceptable This reveals
In addition,
accuracy
there the
unlikeistheneeda needfor provid-
priortointuition ing
identifybased an initial
the effort enrichment
on itsrequired
claimed hypothesis of the possible
for the metrics results
strategy
in an application to to
adopt the solution
architecturally
difficulty to recover with
detect a tendency in
relatedarchitecture.
conceptual the ideal
anomalies and combination
However, also tothere to
derive prioritize
is amore
clear
architectural
metrics
opportunity thatfor anomalies.
may many have anConsequently,
researchersimpacttoon there
the
shed ison
quality
light a need to provide
relationships
checking more of various
other
efficient
prioritization
software criteria
that are for seizing
closely the
related diverse
approaches to architecture recovery by enriching ARCADE’s tool to add to architectural
architectural problem
problems. types
and enhancing
further different current the essential techniques
code-level analysesused to it.for discovering
Besides, there iscode also a
anomalies.
2)
need findings
3)The More-
to conduct over,
of more the
the current integration
Sincearchitecturaldegradationsymptomspresentthat
experiments study serve of two
on a wide or
as evident more
scopethat heuristics
on morecode would get
smells
systems,
a metrics-based
better ranking
agglomeration
especially
strategy industrial
is the results’
hasmost effectiveness.
a considerable
systems
commonly correla-
as well Additionally,
totion
usedassolution increase it is possible
toasarchitectural
approach
compared to problems
accuracy
other to
introduce
compared
through
proposed the
to new
code
documentations, strategies
smells
solutions. However, to
individual,
pull requests, produce ranking
researchers
more commit
studies are on numerous
should
messages, conduct
needed comments, criteria
further
in this field tests, in
for
inves- order to
tigation provide
on visualization
architectural
other metrics to be analyzed toand bad capabilities
smells
much the
provide in
more. that
combination.are most relevant
Practitioners
most architecturally appropriate to can
change solution andarchitectural
their detection identifywaythe problems
depending on
effort requiredforthethecode developers.
for thesmells
metrics agglomeration
to detect to
identify degradation symptoms
architecturally related smells. The essential techniques of ranking used effectively.
should enhance the possibility to get better effectiveness results and the
identification of critical cores of architectural violations. Also, there is a
clear opportunity for many researchers to highlight enriching ARCADE’s
tool for efficient approaches to architecture recovery. Additionally, there
are solutions, which show that it is not effective at all in the problems of
deep analysis and the difficulty of address such refactoring strategy that
has no considerably a positive impact to address architectural erosion.
These reasons are considered the most contributing to degradation, while
the rest of the reasons demonstrated in Table 6 are less important than the
three stated
In terms of using reasons depending on
the architectural rules what was declared
violations solutions, in theweselected
observed
primary studies. However, further studies
that many violations were not restricted by the architectural principlesshould be conducted to find out of
thethe
The other
findings
system, causes
which as
showed
may a rooted-deep
thathave
not 17 ofdefined
thestudymost incommonly
the digging
necessary uprules
new to
occurred causes causes
reduce thatthe
As
could a result
haveofaof
contribute our architectural
findings,
significant
tomajor
the we decay
contribution propose the following
toofidentifying
thethatOSS thedirections
community.architecture forerosion,
future
Essentially,
severity violations. This means violations still appear over
research in this area.
whether Focus on the definition of a common model (which
architectural
again anddegradation
over againover has the
despite OSS
numerous
the good orcauses,
industrial
violationwhich systems.
have been
solutions thatdiscussed
were
bymay be obtained
several researchers by merging
in their multiple
studies available
[11], approaches) and favorthese its
introduced and the approaches that have a[12], [23],
significant [30].
roleHowever,
in capturing
adoption
reasons through
have been rigorous
discussed and extensive
from limited validation
aspects in industrial settings.
violations
As result with
a could of our thorough
findings, accuracy.
we propose Therefore,
the following it issuch as aging
important
directions to because
highlight
for future
of
This
changes increase
over time the
[11], validity
iden- of the model
tifying the and thus
reasons throughits dissemination
only one case in
the identification
research in this area. of the necessary
Focus rules and identification ofmodel
critical cores
industry,
study [12]where
or basedOSSon stillon
istheir not the definition
widely
investigation adopted. of a Several
of industrial
commonmodels
case studies
(which
already
[27].
through
may but, a
be obtained broad study
by tomerging on architectural
multiple conformance
available approaches) using multiple
and favor its
exist according
Consequently, these the results
causes need oftoour SLR, they
be further have not
identified in been
termsstrongly
of the
adoption through rigorous frameworks.
and extensive validation in industrial
validated
frequency and,
of theiras aoccurrence,
consequence, adoption
especially inhasthe been
scopelimited.
of the OSS Try settings.
to target
projects.
This could increase
the modelsidentifying the
at quality factors validity of the
thatimportant model
are of real and thus
interestwill its
forindeed dissemination
stakeholders. Mostin
Moreover, the most reasons contribute to
of industry, where OSS is still not widely adopted. Several models already
thethe available
erosion modelstofocus
according the chosenon theprimary
overall quality studies,ofwhich the product,
contain but few
several
exist but, according
of them are inable totoassess
the results of our factor
each single SLR, they composes have not been strongly
case studies different domains for the OSSthat community. These the overall
causes
validated
quality of and,
the as
OSS a consequence,
product. This adoption
can has
complicate been the limited.
assessment Try to
of target
OSS
differ in their impacts with regard to the actual contribution to the
the models at quality
products by stakeholder, factors
who that
are are of
interested real interest
in specific for stakeholders.
quality factors: Most
e.g.,
architectural decay prominence.
ofdevelopers
the available aremodels
likely more focusinterested
on the overall quality of
in reliability or the product,
testability but few
aspects,
of thembusiness
while are ablepeopleto assess mayeach singleinterested
be more factor thatincomposes the overall
cost or maintenance
The
qualityprocess
of themapOSS and its activities
product. This characteristics
can complicate description
the assessment presented
of OSS in
factors, etc..Develop tools that support the research directions listed above
our studyby
products arestakeholder,
based on primary who areacademic
interested sources. However, it can be used
(i.e., tools able to support and simplify the in specific
applicability quality
of thefactors:
proposed e.g.,
asdevelopers
basis for the are conduction
likely more of empirical
interested in studies
reliabilityto investigate
or testability OSSaspects,
process
models during the evaluation of OSS products). Most of the tools
in a more
while practical view.
business Webebelieve that our results can contribute to
mentioned in the people
primarymay studies more interested
are prototypes in cost
and most or ofmaintenance
them are not
improve
factors, the understanding
etc..Develop tools thatof OSS process
support the researchactivities and consequently
directions listed above
However, according to available or
to the factsOSSmaintained
detected in the anymore. research studies, IT
(i.e., tools able to help supportmitigate
and simplify project failure.
the applicability of the proposed
managers are neither using any tool nor procedures that allows them to
models during the evaluation of OSS products). Most of the tools
evaluate the adoption of FLOSS solutions. For this reason, this
mentioned
Few studies in theareprimary
conducted studieson thearejoining
prototypes process and andmostabandonment.
of them are not
contribution can motivate researchers to work on the creation and
Therefore,
The purposefuture available
of thisresearch toorguide
should
studyofisguidelines maintained
address
future anymore.
the issuesin
research ofthethese two areas.
application of
publication for adopting FLOSS.
The authors also noticed that some studies
FLOSS in new domains as a guide for the correct selection of FLOSS investigated more than oneto
research topic. Additionally,
help IT managers make appropriate our study identified
decisions forthe gaps in community
organizations, define
dynamics areapolicies that could be useful for researchers.
for FLOSS adoption, among others. Moreover, mapping
study also provides insights of some areas that need more exploration for
instance mentoring of newcomers and finding a task to start are some of
the open research areas, which need more study. There is a need to develop
tools to better support newcomers’ onboarding and easy migration of the
developers among
Furthermore, this studydiffer- can entbeprojects.
repeatedThere by using is a other
lack ofresearch
evidence on how
techniques
to remove technical barriers, how gender
such as a combination of systematic mapping study and systematic and age of developers influence
their reten-
Furthermore, tion inreview
future
literature a project.
research can It shows
should
be used focus tothatonthere
obtain is stillresults.
newcomers’
better need of tools,and
orientation
reception to identify the problems in initial contribution as well asand
practices, and processes for better community participation provide
guidance regarding barriers engagement in advance. in OSS projects. significant research is
Moreover,
required
Moreover,to explore
mentoring the means
is another to support
field that newcomer
needs toinitial partic- in
be explored ipation,
the
enhancing motivationsignificant
future. Furthermore, to increaseresearch participation, is required and reduce
to explore the hindering
the tools,
factors
practices, that andwillpro-improve
cesses retention
that could of
Abandonment is another area in OSS community and has many issues benew develop-
helpful to ers
improve and One
community Timethat
participation. The Contributors
research community (OTC).
need to be explored in future research. Additionally, further research may use these findings to is
understand the issues
required to explore in the factors
various area and thatselect
leadthe topic forto
developers further
stay or research.
leave a
Additionally, we also observedproject. from the results that in majority of the
studies, combination of survey and questionnaire was used as a research
methodology to conduct research. From these results, we observed that few
studies used machine learning methods. In future use of these techniques
will be helpful to solve the problems related to task distribution, selecting a
task to start contribution, and management of project information.
Furthermore, findings indicate that majority of the studies and researchers
who investigated community dynamics belong to USA. We also observed
that most of the studies (75%) were conducted by the research group of
one country. Thus more collaboration is required among research groups
of different countries to conduct more research in the area.
practices, and processes that could be helpful to improve community
participation. The research community may use these findings to
understand the issues in the area and select the topic for further research.
AAdditionally,
mapping study we review
also observedprovides from the results
a structure for that in majority
a research reportoftype, the
studies, combination of survey and questionnaire
which enables categorizing and giving a visual summary of results that was used as a research
methodology
have been to conductinresearch.
published papers of From these results,
a research area[8].we Thisobserved
map aids thattofew
studies used machine learning methods.
identify gaps in a research area, becoming a basis to guide new research In future use of these techniques
will be helpfulstudy
Aactivities[7].
mapping to
The solve
review the provides
current problems
mappingarelated structure
review to task
for adistribution,
captured research
the current reportselecting
state type,of a
Fur-
Cmmunity
which thermore,
taskenables
to start more
dynamics research
contribution,
studies is
and
focusrequired
management
on the to study
internal the
ofsummary impact
project
andized of
information.
external men-
of re-motivation toring
research on bug categorizing
report severity and giving
prediction, a visual
character- related sults
problems that
on
of
have the
Furthermore, success
developers
been of onboarding
findings
and
published indicate
factors
in that
papers process
that majority
influence
of a andthe
research provide
of the
retention
area[8]. suggestions
studies of
This and
developers
map toaids
better
researchers in
and identified
support newcomer the main
onboarding,approaches and employedtion
contribu- to solve
barriers them. Theseto
in virtual
who investi-
projects.
identify gaps gated
However,
in a community
there
research are
area,dynamics
somebecoming areas,
objectives were reached by conducting a mapping of existing literature. belong
which
a basisto USA.
need
to We
research
guide newalso observed
such
research as
Aactivities[7].
mapping study review communities.
provides areview
structure offor athe
research report type,
Inthat
Cmmunity most
retention
total, theof of the
dynamics
review studies
veteran
The studies
identified (75%)
developers,
current 27were
mapping
focus the
on
relevant conducted
impact
the papers captured
internal by
contribution
andand theresearch
current
external
analyzed group
barriersstate
motivation
them onof
of
along
which
one
abandonment,
research enables
country.
on bug usecategorizing
Thus of
report more
gamification
severity and giving
collaborationin
prediction, stu- a visual
is
dent summary
required
engagement
character- among
ized ofand
relatedre- itssults
research
effect
problems that
in on
12 of developersAlthough
dimensions. and factors thesethatpapers
influence havethe maderetention
valuable of developers
contributions in
have
groups
retention,and
projects. been
of
the published
different
effect
identified of
the in
countries
project
main papers toof a
characteristics
approaches research
conduct employed more
on area[8]. research
retention,
to solveThis the map
in
them. the
impact aids
Thesearea.
such asthe to
of
bug reportHowever, there are some
severity prediction, areas, which
the panorama need research
presented in this mapping
identifygovernance
project
objectives
retention gaps
ofwere in reached
veteran a research
on by area,
motivation,
developers, conductingbecoming
thedevelopment
a map-
impact a basisping
offor oftostrategies
ofguide
contribution existing new to researchon
maintain
literature.
barriers
study
A mappingreview suggests
study review that there are
provides a potential
structure researcha opportunities
research report for
type,
In activities[7].
loyalty, the impact
total, improvements
abandonment, The
reviewuse current
of leadership
identified
of mapping
27
gamification styles
relevantreview
inonstu- captured
developer
papers dent and the current
turnover,
analyzed
engagement and
them state
and impactof
along
its
further
which enables categorizing in this topic.
and giving Among a them,
visual the
summary following
of re- research
sults that
research
ofdimensions.
12effect ononretention,
corporate bugsponsorship
reportthe
Although severity on
these
effect prediction,
FLOSS
papers
ofoftoprojecthave character-
communities.
made ized related
Moreover,
valuable
characteristics contributions
on problems
there
retention, is ain
have been directions
published in appear
papers a be more
research promising:
area[8]. This map aids to
bug
the and
need identified
to an
report
impact design
severity
of the the
tools main
tolack
prediction,
project approaches
assess the health
governance employed
panorama onofamotivation,
OSS to
presented solve
projects them.
in and
this
development These
develop
mapping of
• There
identify isgaps apparent
in a research of investigation
area, becoming on
basis bug to report
guide newseverityresearch pre-
objectives
study review
strategies were reached
suggests
toThe strategies
maintain by
that conducting
to
there
loyalty, retain
arethe newa map-
potential
impact ping
developers.
research
ofabandon of
leadershipexisting
opportunities literature.
styles for
onof
Adiction
In mapping
orderthe
activities[7]. in other
study
to review
reduce relevant
review
dropouts
current FLOSS
provides
of
mapping asuch
developersreview as,who
structure forfor
captured example,
a research anLinux
the current report
OSS Kernel,
type,
project
state
In total,
further
developer improvements
turnover, identified
in this
and 27
topic.
impact relevant
of Among papers
corporate them, and the analyzed
sponsorshipfollowing them
on along
research
FLOSS
Ubuntu
which
(especially
research Linux,
enables
onthe bugones and
reportwho MySQL,
categorizing and
disappear
severity and in
giving others
afterhave
prediction, a visual
their BTS, for
summary
first valuable
character- contri- example,
of
izedbution),
relatedre- Github.
sults
therethat
problems isina
12communities.
dimensions. Although
directions these
appearpapers made contributions
•have
Often,
andneedbeen technical
to look
identified Moreover,
published
for
theusers in report
effective
main there
papers ways
approaches ofistoaatobe
most need
bugs.more
research
engage
employed to promising:
design
Thus,
area[8].
such tothe tools
influence
Thisthem.
developers
solve to
map assess
in of
aids
the
These userthe
to
bug health
• There report anseverity
isexperience
apparent
of prediction,
lackarea, ofand thedevelop
panorama
investigation presented
on bug reportin this mapping
severity pre-
identify
community
objectives gaps and
were inOSS projects
in
a research
providing
reached predicting
bygood outcomes
becoming
community
conducting a strategies
map- isping
asup-
basisstill
port toexisting
when retain
overlooked.
toof guide new
they new
research
are taking
literature.
study
Adiction
mappingreview
in othersuggests
study relevant
review thatprovides
there
FLOSS are
developers.a potential
such as,
structure forresearch
for example,
a opportunities
research Linux report Kernel, for
type,
•InBug total,reports
activities[7].
the review labeled
The current withmapping
baby
identified default
steps
27 in an severity
relevantreview
OSS leveland
captured
project.
papers (often
analyzed“normal”)
the current them were
statealong of
further
Ubuntu
which improvements
Linux,
enables and
categorizing in this
MySQL, topic.
and and in
giving Among
others
a them,
visualBTS, the
for
summary following
example,
of re- research
Github.
sults that
research Reception
prevalent
12 dimensions. in of
the
on bugAlthough new
most developers
datasets
report severity these is
used another
prediction,
papers in
have challenge
reviewed
character-
made in
papers. the
ized related
valuable open
However,
contributions source
problems theyin
•have
Often, been technical directions
published users appear
in report
papers oftoa be
most bugs.more
research promising:
Thus,
area[8]. the influence
This map of
aids usertonot
commu-
are considered
and
bug report nity.
identified To overcome
unreliable
the main this
[26], issue,
approaches and research
just is
discarding
employed needed
to them
solve to develop
them.also does
These tools,
• There
identify anseverity
isexperience
gaps apparent
in reached
prediction,
lackarea,
in predicting
aassist
research
the panoramaon
of investigation
outcomes
becoming
presented
isping
a basis bug
still reportin this
overlooked.
toof guide new
mapping
severityresearch pre-
seem
objectives
study review which
appropriate.
were suggests Then, inbybetter
that efforts
conducting
there reception
in
are researchingof
a map-
potential new
researchdevelopers.
on novel
existing ap- literature.
opportunities proaches for
diction
•InBug
activities[7]. in other
reports labeled
The relevant
with
current FLOSS
default
mapping such
severity
review as,captured
for
level example,
(often Linux state
“normal”)
thetocurrent Kernel, were
of
to
further handle
total, this
the review
improvements type ofinreport
identified this 27 should
topic. relevant
Among bepapers
considered
them, and the analyzed improve
following them the
along
research
Ubuntu
Moreover,
prevalent
research on Linux,
in more
the
bug and
most
report MySQL,
significant
datasets
severity and
research
used in
prediction, others
in is reviewedBTS,
required
character- tofor
papers.example,
study
ized the
However,
related Github.
impactproblems of in
they
12 dimensions. state-of-the-art
Although
directions of severity
these
appearpapers to be prediction
have more made algorithms.
valuable
promising: contributions
are • Often,
community
considered
and technical
identifieddynamics users
unreliable
the main report
(stagnant
[26],
approachesmost
v/s
and bugs.
dynamic
just Thus,
discarding
employed tothe
community) influence
them
solve on
them.also of
developer
does
These user not
•• There
Most
bug reportapproaches
anseverity
isexperience
apparent were
in
basedofon
prediction,
lack
predicting
theunstructured
panoramaon
investigation
outcomes is
text
presented
bug
still
features
reportin this
overlooked.
(summary
mapping
severity and
pre-
seem
objectives
description).
study turnover
appropriate.
review were
To and
reached
handle
suggests the
Then, effect
them,by efforts
that FLOSS of
conducting turnover
researchers
there aresuchin on
researching
a map-
chose
potential project
ping
to on
use
research performance.
ofnovel
theexisting
tra- ap-
opportunities proaches
literature.
ditional bag-
for
diction
•InBug in other
reports labeled relevant
with default severity as, for
level example,
(often Linux Kernel,
tototal,
of-words
further handle the this
review
approach
improvements type of
identified
insteadin report
of
this 27
more should
topic. relevant
recent
Among bepapers
considered
text and
mining
them, the to “normal”)
analyzed
methods improve
following them
(e.g.,
were
the
along
researchword-
Ubuntu in
prevalent Linux, and MySQL,
theAlthough
most datasets and
used inin others
reviewedBTS, papers.for example, However, Github. they
12 dimensions.
embedding state-of-the-art
[66]) or users
data-driven
directions of severity
these
appearpapers
feature
to be prediction
haveengi-
more madeneering
promising:algorithms.
valuablemethods contributions
which may in
are • Often,
considered technical unreliable report
[26], most
and bugs.
just Thus,
discarding the influence
them also of
does user not
•• There
Most
bug reportapproaches
anseverity
isexperience were
likely
apparent basedofon
prediction,
in improve
lack
predicting
theunstructured
outcomespanorama
investigation
outcomes yielded
ontext
presented
bug
is still sofeatures
far.
reportin this(summary
mapping
severity and
pre-
seem
description).
study appropriate.
review To handle
suggests Then,them,
that efforts
researchers
there in
are researching
chose
potential to onoverlooked.
use
research novel
the tra- ap-
opportunities proaches
ditional bag-
for
•• There
diction
Bug is in
a clear
reports other research
labeled relevant
with opportunity
FLOSS
default to investigate
such
severity as, for
level whether
example,
(often Linux state-of-Kernel, the-
to handle
of-words
further this
approach
improvements type of
insteadinreport should
ofoutperform
this more
topic. recent
Among betext
considered
mining
them, the to “normal”)
methods improve
following (e.g.,
research
were
the
word-
art ML
Ubuntu
prevalent in algorithms
Linux, and might
MySQL,
the most datasets and in the
others
used inprediction traditional
BTS,
reviewed papers. for algorithms
example, Github.
However, they used
[66]) directions
or users
data-driven of severity
appear feature
to report
be engi-
more neering
promising:algorithms.
methods which may
are in all reviewed
• Often,
considered technical papers
unreliable for bug
report
[26], most
and bugs.
just severity
Thus,
discarding theprediction.
influence
them also of The
does user not
•• There
Most approaches
investigationis an apparent
experience of
wereimprove
likely
the inuse
basedofon
lack of
predicting Deep
unstructured
outcomes
investigation
learning
outcomes yielded
ontext bug
algorithms
is stillsofeatures
far.
reportwhich
overlooked.
(summary
severity
perform
and
pre-
seem appropriate.
description). To handle Then,them, efforts
researchersin researchingchose to on novel
use the tra- ap- proaches
ditional bag-
•very
• There
diction
Bug wellis in
awhen
reports clear
other research
relevant
classifying
labeled with opportunity
FLOSS
audio,
default text, to and
such
severity investigate
as, image
for
level whether
example,
data
(often Linux
[67] state-of-
seems Kernel, to the-
be
to handle
of-words this
approach type of
instead report
of more shouldrecent betext
considered
mining to “normal”)
methods improve (e.g.,
were
the
word-
artUbuntu
prevalentML algorithmsLinux,
in the andamight
most MySQL,
promising
datasets outperform
and inin
research
used thedirection.
others traditional
reviewedBTS, papers. algorithms
for example, However, Github. used
they
[66]) or users data-driven of severity
feature prediction
engi-Thus,neering algorithms.
methods which may
are •inResearchers
• Often, all reviewed
considered technical papers
should
unreliable for bug
report
investigate
[26], most
and report
bugs.
more
just severity
recent
discarding prediction.
thetechniques
influence
them (summaryalso of The
(e.g.,
does user not
• Most approacheslikely wereimprove
based onoutcomes unstructured yielded textso features
far. and
investigation
seem continuous experience
appropriate. of the
learning inuse
Then, of
predicting
[68]) Deep
to
efforts learning
outcomes
provide
in an
researching algorithms
is still
approach on for
novel which
overlooked. bug
ap- perform
report
proaches
description).
•very To handle them, researchers chose to use the traditional bag-
• There
Bug well
to handle
is awhen
reports
prediction
clear
this
research
classifying
labeled
which
type with
of could
opportunity
reportaudio,
default
beshould text, to and
severity
employed
investigate
inimage
level whether
data
(often
real-world [67]
to “normal”)
state-of-
seems the
scenarios. to the-
were be
of-words
art ML approach
algorithms instead
might ofoutperform
more recentbe the
considered
text mining
traditional methods im-
algorithms
prove
(e.g., word-used
prevalent
• Many in bugthereports
most
state-of-the-arta promising
datasets
are of resolved research
used
severity in direction.
inprediction
areviewed
few dayspapers. (or in aHowever,
algorithms. few hours) they
embeddinginResearchers[66])
all reviewed or data-driven
papersinvestigate
for bugfeature report engineering
severity methods
prediction. which Themay
are •
considered
•[69].Efforts
Most approaches to predict should
unreliable [26],
severity
wereimprove
based onoutcomes and
level more
just
for these
unstructured recent
discarding group
text techniques
them
of bug
features also
(summary (e.g.,
reportsnot
does do
and
investigation of likely
the use of Deep learning yielded
algorithms so far. which perform
seem
not continuous
seem
description). appropriate.
very To learning
useful.
handle Then,
Thus,[68])
them, anto
efforts provide
in researching
investigation
researchers an
chose approach
toto on novel
confirm
use thefor bug
ap-
this
tra- reportbag-
proaches
hypothesis
ditional
•very
There wellis awhenclearclassifying
research opportunity audio, text, to and
investigate whether state-of- tothe-
and
of-words prediction
to handle
to determinethis which
approach typewhen ofcould
instead report
the beshould
severity
ofoutperform
more employed
recent betext
prediction inimage
real-world
considered
mining
data
is more to [67]
methods
seems the
scenarios.
improve
appropriate
(e.g., word-
be
in
art ML algorithms amight
promising research the traditional
direction. algorithms used
• Many bug
embedding bug
[66]) reports
state-of-the-art
report
or are resolved
of
lifecycle
data-driven severity inprediction
is report
feature of a fewneering
critical
engi- days (or
importance. in a few
algorithms.
methods hours)
which
•inResearchers
all reviewed papers
should for bug
investigate more severity
recent prediction.
techniques Themay
(e.g.,
[69].Efforts
•• Most
The approaches
primary to predict
objective
likely severity
wereimprove
based
of our on level
mapping
outcomes for
unstructured these
study
yieldedwas group
textso to of
features
review
far. bug reports
(summary
the research do
and
investigation
continuous of the use[68])
learning of Deepto learning
provide an algorithms
approach forwhich bug perform
report
•notThere
very
seem
description).
approaches wellis very
awhenTouseful.
for
clear handle
severity
research
classifying
Thus,
them,
level an investigation
researchers
prediction
opportunity
audio, text, toin chose
FLOSS.
investigate
and
totoconfirm
use the tra-
However,
whether this ithypothesis
ditional
would
state-of- bag-
be
tothe-
and
of-words prediction
to determine
approach which
when
instead could
the
of be employed
severity
more recent prediction
textinimage
real-world
mining is
data
more
methods
[67] seems
scenarios.
appropriate
(e.g., word-
be
in
artaML promising
algorithms researchmight
a promisingvenue
outperform to extend
research the this study
traditional
direction. to com-
algorithms mercial used
• Many bug
embedding bug
[66]) reports
report
or are resolved
lifecycle
data-driven in critical
is report
feature of a few
engi- days (or
importance.
neering in a few
methods hours)
which
systems,• alland
inResearchers verify
reviewed whether
papers
should forthe bug
investigate same findings
more severity
recent apply to these
prediction.
techniques Themay
systems.
(e.g.,
[69].Efforts
• The primary to predict
objective
likely severity
of our
improve level
mapping
outcomes for these
study
yieldedwas groupto
so of
review
far. bug the reports
research do
• The approaches
investigation
continuous of thepublished
learning use[68]) of Deeptoininvestigation
selected
learning
provide papers
an algorithms
approach for severity
forwhich bug level
perform
reportpre-
not seem
approaches veryfor useful. Thus, an to confirm this hypothesis
•very
There
diction wellisdid
prediction not severity
awhen
clear research
consider
classifying
which
level
could
prediction
opportunity
the temporal
audio,
be employedtext, toin FLOSS.
investigate
information
and inimage
However,
whether
changes
real-world data [67] it would
instate-of-
bug
seems
scenarios. re- to portbe
the-
be
and
art a to determine
promising
ML algorithms when
researchmight the
venue severity
outperform to extendprediction
this
thetemporal is
study
traditional moreto appropriate
com-
algorithms mercial used in
features.
• Many bug Investigating
reports a the
promising
are impact
resolved of
research the direction.
in critical
a few days evolution
(or in a few hours) of a bug
systems, alland bug
verify report
whether lifecycle
forthe is report
same of findings importance.
apply to these as systems.
report •inResearchers
[69].Efforts
reviewed
informationto predict
papers
[70]
should ininvestigate
severity
severity
bug
level levelmore
for
severity
prediction
theserecentgroup
prediction.
accuracy,
techniques
of bug
The
(e.g.,
reports welldo as
• The primary
•continuous
The
investigation approachesobjective
of the of of
published
use ourDeep mapping
in selected
learning study
papers wasfor
algorithms to severity
review which the
level re-pre-
perform search
not investigating
seem very dataThus,
learning
useful. structures
[68])an to storeanthe
to investigation
provide representation
approach
to confirm forthis of
bughypothesis this
report
approaches
diction
very welldid for
not severity
when consider
classifying level prediction
theaudio,
temporal text, in FLOSS.
information
and image However,
changes
data [67] in bug it would
seems re- to portbe
be
and to temporal
prediction
determine whichevolution
when thecould seems
be
severityemployeda relevant
prediction in research
real-world
is more venue.
scenarios.
appropriate in
a promising research venue to extend this study to commercial
• features.
Related
• Many researchInvestigating
bug reports areas,arethesuch
a promising impact
resolved asresearchofinthe
isseverity a fewtemporal
direction.
level
days evolution
prediction
(or in a few of
in help a bug
hours) desk
systems,
report • andbug
information
Researchers verify report
whether
[70]
should
lifecycle
in the same
severity
investigate
of
level
critical
findings
more prediction
importance.
recent apply to these as
accuracy,
techniques systems.
(e.g.,well as
systems
•[69].Efforts
The primary forto IT serviceseverity
predict
objective ofmanagement
our mapping level for [71], may
these
study was take
group advantage
to reviewof bug re-ofsearch
thereports thedo
• Bug to handlereports labeled
this type of withreportdefault should severity level (often
be considered to “normal”)
improve the were
prevalent in the most datasets
state-of-the-art used inprediction
of severity reviewed papers. algorithms. However, they
are considered unreliable [26],
• Most approaches were based on unstructured text features (summary and just discarding them also does not and
seem appropriate. Then, efforts
description). To handle them, researchers chose to use the traditional in researching on novel approaches bag-
of-words to handle this type
approach insteadof report
of more shouldrecentbetext considered
mining methods to improve the
(e.g., word-
embedding
The review state-of-the-art
[66]) or data-driven
establishes of state
the severity
featureof the prediction
engi-
researchneering algorithms.
on methods
ISSD in terms whichof may
•focused
Most approachesknowledge were
likelyareas based
improve on unstructured
outcomes yielded
and contributions. text
Majority sofeatures
far. (summary
of research centred and
description).
• Thereon the isadoption To handle
a clear research them,
and adaptation researchers
opportunity of inner chose
to investigate to
source in various use the
whether traditional
state-of-
context, whilebag-
the-
of-words
artother ML KAs approach
algorithms
in SWEBOK instead
might of more
outperform
receive recent text mining
the traditional
less attention. methods
Thus ouralgorithms (e.g.,
review callsused word-
for
embedding
more instudies [66])
all reviewed
on these or data-driven
papers
areas to feature
foradvance
bug reportengi-
our neering prediction.
severity
knowledge methods
on the inner which
The may
source
investigationFurthermore,
phenomenon. oflikely
the use improve
oftoDeep outcomes
advance learningouryieldedalgorithms
understanding so far.which of inner perform
source,
•very There
researchers is
well whena clear research
needclassifying
to draw on audio, opportunity
theoretical to investigate
text,foundations
and image that whether
datahave state-of-
[67]been seems usedtothe-be
in
art ML algorithms
prior research on OSS, might
a promising outperform
as well as research the traditional algorithms
direction.lens that are consid-
other theoretical used
ered •inResearchers
all reviewed
relevant to inner papers
should source. forWith bug the
investigate report helpseverity
more recent
of the OSS prediction.
techniques
framework, Theour
(e.g.,
investigation
continuous ofhas
study alsolearning theidentified
use[68])of Deep
to learning
theprovide an algorithms
characteristics approachof the inner which
for bug perform
sourcereport
very well when
prediction
phenomenon, classifying
which could audio,
be employedtext, and image
inorreal-world data [67] seems
scenarios. to be
In addition, it isincluding
possible its
thattypes
adding of projects
non-software products, the
and information motivation
system
• Many bug inner
reports a promising
are resolved research direction.
for
related adopting
databases may source,
yield its orindifferent
environment,
similar a fewinner days (or inThe
source
findings. a few
processes hours)
comparison and
•
[69].Efforts Researchers to predict should investigate
severity level more
for these recent group techniques
of bug (e.g.,
reports do
oftools
findings and from
the actors involved
different in innerand
databases source.
the findingsWe alsopresented
highlightinthe this
not continuous
seem very learning
useful. [68])
Thus, to provide
an investigation an approach
organisa-totions for bug report
challenges as well
study canaspo- the benefits
tentially forconsidered
be asconfirmthat work.
future this
are hypothesis
interested in
types
and innerprediction
indetermine
to general For
source. which
are when could
directly
research, be employed
theinfluenced
severity
we fromainthe
prediction
identified real-world
interpretation
research is more agenda scenarios.
tooffurther
appropriate different in
However,
• Manystakeholderas described
bug reports
viewpoints. in Section
are resolved
Below 5.4.4,
wein a there
few
highlight is aseveral
days lack
(or of
in research
a
open few in this
hours)
issues:
This study bug
advanced report
also identifies lifecycle
our knowledge is of
some challenging critical
in the innerareas importance.
source for area.
future work.
area.none Thus, we need morepapersstudy to shedthese lightgroup on how tobugmanage an do
••[69].Efforts
Since
The Appendof
1. primary toobjective
newpredict
the analyzed
findings severity
of our level
intomapping
the bodyfor
define ofopenness,
study was to review
knowledge ofon
identifyingOS thereports
openness
re-
forkingsearch
not Anotherseem effective
result
very of
ofour innerstudy source
isanthat community
there in are into
no an organisation.
clear definitions related to
approaches
behaviour. dimensions for useful.
severity
Applying IoT theThus,
platforms
level
combined investigation
remains
prediction approaches aFLOSS. of confirm
challenge SLR toand
However, bethis
CAM ithypothesis
addressed.would
revealed be
•and theaistoopenness
ofdetermine of IoT platforms. One of the papers attempts to define the
Itseven forking types interpreted by academic researchers and the latestin
utmost
promising when
importance
research the to
venue severity
find to a prediction
consensus
extend this is more
regarding
study to appropriate
openness
commercial among
systems, andaspects
the platform
interpretation different bug
verify
found
only lifecycle
report
stakeholdersiswhether
[33],
to avoid
file language
another
the same of paper
isrepository
critical
confusion,
findings categorizes
importance.
and
fork. preferably
apply
Thisto
the openness
these
novel agree onwill
systems.
insight a
•assistdimensions
The• The primary of
approaches platforms
objective of
published [34],
our and
mapping
formalin is a finalstudy
definition.
selected study
papers was categorizes
to review some
the re- open-
search
researchers on how forking presented andfor severity level
interpreted pre-
and industry
source
approaches
•diction
Investigatingplatforms
did for
not severitybut without
openness
consider level
not
the providing
prediction
only
temporal from in
IoTa FLOSS.
definition
information platform [35]. Thus,
However,
perspective,
changes in it
bug none
would
but
re- ofbe
also
port
practitioners in reviewing project forking health, especially projects with
them explicitly
a promising define
research what
venue an open
to extend IoT platform
this study is. Moreover,
to com- in our
features.
programing considering
Investigating
language filetheIoT middleware
impact
repositories the and
of that are frameworks.
temporal less evolution
adopted or mercial
of a bug
forked by
systems,
• To study we
and
understand were
verify
report information [70] in severity howalso interested
whether
much the
openness to
sameidentify
level prediction accuracy, of
of findings
IoT the openness
platformsapply to
has types
these
penetrated IoT
systems.
as well theas
developers.
field platforms.
•investigating
Our The
and Our
approaches
in which
results results
suggest domains,
data thatsuggest
published thea most
structures that
mapping the
in selected
common
to most
study
store papers common
of
openness
the the openness
forapplication
severity
types
representation oflevel types
pre-
domains
most
of thisIoT ofof
2. Understanding forking consequences. Case studies are an important way
most
diction IoTdid platforms
not consider are related
the to
temporal open-source. Other types identified are
toplatforms
highlight the
are
temporal identified
lessonsrelated
evolution to
learnt open by seems
researchers.ainformation
IoT platforms
open-source. Other
relevant This would
types changes
research
paper beidentified
identified in bug
useful.
venue. re-
areforking
open port
open standards,
features.
standards, Investigating
open APIs, openopen APIs,
the data open
impact and data
of
open and
the open layer-based
temporal
layer-based evolution
platforms. platforms.
of a bug
However,
•impacts
Relatedand research
consequences, areas, such with as oneseverity
of the worst level impacts
prediction being in help desk
a political
reporttoHowever,
further
systems toIT
information further
investigate
for [70]
service investigate
the in severity
openness
management thetypes
openness
level of
[71], IoT
may types
prediction ofaccuracy,
platforms
take IoTwe platforms
advantage as
believe well
of we
it isas
the
strategy that divides a project community and forms a new community.
believe
important
best it is
investigating
practices important
to look data
from to look
structures
a from
stakeholder a
to stakeholder
store
view, the as view,
representation
identified as identified
above of above
this
[34,52].
Forming a newused community in severity results level in prediction
less contributions in FLOSS. Assess- ing
by developers to
Thus,
these [34,52].
best Thus, itevolution
it istemporal
essential
practices toisinfurther
essential to further
seems
analyze
this context ahow analyze
relevantimportant howthese
research important
venue.
openness these types
the original file repository, bug fixesis also
or a promising
feature enhancement. research venue.
Allowing
• Related openness aretypes
research are
areas, fromsuch different
as severity IoT stakeholder
level prediction perspectives.
accumulated bugs from
and different
feature IoT
enhancementsstakeholder unfixedinfor
perspectives.
to remain help desk
a period
systems
Future work for IT couldservice timemanagement
of include canthe expansion
affect [71],of
project maythis take
health risk.advantage
research, overcoming of the
best practices
some stated used in
limitations severity
by level
increasing
3. More research is required on forking sustainability. Reviewing these prediction
number in
of FLOSS.
universities Assess-
or by ing 21
these best
considering practices
only in this
experienced contextOpen is
papers revealed the importance of forking sustainability investigation as a also
Source a promising
practitioners. research
This way, venue.
we
would be ablewith
top priority to providetwo specifica moreareas effective feedbackStudying
of interest.4. to the researchers.
forking
Another possibility is to assess
sustainability using a SLR for software development with the survey participants motivations
GitHub.
As future work,
regarding it isprovided.
necessaryWith to investigate in real software projects, in
Valentio, the rates
Javier, Izquierdo and Cabota greater
(2017) amount
used aofSLR participants,
to show that this
software organizations
couldisbea achieved and
without in open-source software projects, what are the
forking good indicator of extending
project longevity the consumed and thetime, chance which could is
of forking
influence factors observed affect by their
the quality developers.
of the Comparing the results of
results.
highly dependent on the project, where developers provide additional
this SLR with real observations may clarify the importance of influencing
contact information (e.g., emails, personal website URLs that are clearly
factors within the context of productivity within organizations and in open-
active or aligned with popular project owners) to increase social
source communities.
connections between a project owner and forker, and increase developer
community size for medium-size projects and projects that are written in a
forker’s preferred programming language. Future work could include
developing a prediction model for fork effectiveness from forking
motivation classifications in response to language repository files, where
programming language survival time is critical to an OS projects’ health
and survivability.
aceable to Study
Traceable to Complete
Heading Study ID Study Year Future Direction
Synthesis Sheet
Main Finding and Discussion S2 2017 5
Conclusion S4 2017 10
Results and Discussion S5 2014 13
Conclusion and Future Work S6 2016 17
Future Research S7 2010 26
Results S7 2010 37
Open Questions and Research Agenda S8 2012 45
Discussion S8 2012 49

Research Contributions and Limitations S17 2019 78

Future Work S19 2016 99
Future Work S19 2016 104
Discussion and Future Work S20 2016 105

Directions for Future Research S23 2012 121

Discussion and Conclusion S26 2019 138
Discussion and Conclusion S26 2019 139
Summary and Conclusion S29 2017 141

Findings S33 2015 165
Implications of Findings S38 2019 190

Implications for OSS Research and

S46 2020 223
Practice
Implications for OSS Research and

S46 2020 224
Practice


288
289
adoption of usage
multiple themes
Its not complete
Researchers
studied the
effectiveness
and
Researchers
accuracy of
studied
severalthe
effectiveness
metric suites
using anddata
from one of
accuracy or
several
more OSS
metric suites
projects.
in what OSS Adoption using
Despite dataof
fromtheirone or
more
esteemed OSS
projects.
contribution
inDespite
predicting of
their
OSS
esteemed
projects,
contribution
they suffer
in
from predicting
lack of
OSS
generalizabil
projects,
ity due to
they suffer
diverse
fromnaturelackofof
generalizabil
OSS
ity due and
projects to
thediverse
project
Need to nature
specific of
Results
OSS Open
research nature of the
shows that Questions
validity of projectsmetricand
Emergent “prediction
the project and
metric suits suites. Also
issues
Need in
to of Research
on cross itspecific
is quite Open
OSS
propose properties”, Agenda
projects nature
difficult of the
to Questions
evolutionto “aggregate
methods metricthe
ensure and
are the metrics”
ensure suites. and
Also
availability Research
prediction
quality ofof “changevevo
it isquality
quite
and Agenda
properties, lution to
metric data. difficult
of metric Results from
aggregate analysis”
ensure are
the
data, which Mapping
metrics, the most
availability
makes the Study
change emergent
and quality
results
evolution open issues
of metric
incomparabl
analysis, in OSS
C134 edata,
[10].which
Thus,
OSS evolution,to
makes the
a future
integrability gether resultswith
research
and OSS
incomparabl
agenda
licensing eintegrability
[10]. Thus,
would be to
and
a future
perform an
licensing.
research
indeepth
agendaon
analysis
would
(a) crossbe to
perform
projectan
indeepth
validity of
analysis
the studied on
is not written in text
Didn’t Add size,
refactoring,
maintenance effort, and
Self - Organized
Criticality (SOC)”.
Two Themes
We are assuming
Evaluation and
Selection as sub parts
of OSS adoption
Two Themes
Three Themes
Two Themes
Two Themes
S8 2012
Thus there is Thus there is

a need to a need
S8 the
validate 2012to
validate the
Conway's Conway's
law in OSS
projects law in OSS
S14 projects.
2013
Need to Need to
determine determine
methods methods and
and repositories
repositories to use for
to use for validation
validation and impact
and impact
of conways of conways
law on law on
quality and quality and
sustainabilit sustainabilit
y of y of
software. software.
In Initial Analysis from Main She
88 Future Direction
81 Future Direction The authors

of S31, also
feel the need
100 Future Direction for research
on quality
144 Future Direction framework,
tool - based,
and data
132 Future Direction mining
selection
methods.

The results
from
113 Future Direction classifying
topics
showed that
studies aim
61 Gap tosame as
predict
above
effort of
Emergent
maintenance
issues
activity in
69 Gap OSS
mainly
OSS Safety evolution
concentrated
Critical
OSS onarebug
Systems (to prediction of
Evolution From Gap Sheet
fixing time
explore
(Change properties,
prediction,
OSS areas such
Profile as Areas of rail
aggregate
C42 while
OSS less
Risk
Evolution maritime
Analysis traffice,
metrics, oil
Use of OSS (Noefforts
work on
systems, rail and
Done on contributed gas
change have
in Safety risk based
C6 industry,
Limited No no research
evolution
work on
Critical OSS to
Risk w.r.t safetyother
on
process
Level) analysis,
specific
System types of
Mitigation quantitative
C34 OSS Risk industry, oil although OSSofit is
mitigation
activities
data
Activities
and gass, off integrability
importantof
activities
project
(No Work)
highway
C36 and or
risks
failure
equipement OSS Risk
licensing
correlating
(No work on
and mining
C37 OSS Risk project
mitigation
industry)
failure and
activities of
losses with
risks) From Author Intent Sheet
quanitiative
data)
(like a
combination
of inside
Include
control and a
Community Identifymore terms
predefined key
C14 Dynamics inproject the search
challenges in a
Inner Source strings
(Expand in and
software
Community (Identify Data Key implementin
add more
evolution
C29 Challenges
Dynamics Sources and data sources
course)
g inner
Search
with to find more
using open
Strings to source along
C32 Inner Source Resolution studies
Strategies
give with on
source
Insight) community
via Survey of projects
resolution in
OSS dynamics
Adaptation
an
strategies
Organization From Future Direction Sheet
Practitioners
Adaptation
) viaofsurvey
OSS
s inof
undergradua
Organization
(Safety participation
safety
OSS Participation te course in
practitioners
C4 Adaptation Critical critical
inSE.
.OSS
in OSSvia communitie
Systems systems via
Communitie
Project OSS project
s s for OSS
Organization Activity) activity
projects
C11 Particiaption Context of
in OSS Organization Organization
Organization
s
for OSS Considering
Participation s
Organization with Usage participation
context of
other with other
C15 Particiaption Organization
(Conduct organization
in OSS more
s organization
for OSS
longitutional
Align s for
usage
AlignOSS
and Indepth
resarch projects
with resarch with
General OSS studies) other other
C16 Research
related related
fields fields
Organization Organization Organization
C18 Particiaption Particiaption s and their
in OSS in OSS approach to
(Approach) OSS
OSS
Adoption Community
OSS
C20 Adoption (Community and OSS
Collaboratio adoption
n)
OSS Big data
Evolution Reconcilein
OSS analytics
(Manageme Configuratio
C44 Evolution software
nt via Data evolution n
Analytics) managemen
OSS Major OSS major
General OSS Streams t practices
C46 research
between
Research (Need
Research) FOSS, areasplan
OSS success driven
Reconciliatio Mixed and
n (Mixed
of FOSS, research
agile
C87 OSS success Methodolog
Agile and methodolog
y Research
Plan y in OSS
Driven Reconcile
Needed) continuous
(Configurati success
Need to
Reconciliatio on code
n of FOSS, reconcile
C88 managemen integration agile
Agile and t, code developmen
between
Plan Driven Reconciliatio
integration
of FOSS, FOSS, plan
Reconciliatio nprocess, driven
t and
and
Agile and distributed
n of FOSS, Plan
knowledge agile
Driven developmen
C89 Agile and mangement
(Test
Plan Driven practices) Driven t w.r.t test
Developmen Reconcile driven
t) Knowledge
developmen
ment
t in plan
managemen
driven
t practices
context
of agile
models,
COSS
(Business
model, organization
Revenue al aspects of
model, user COSS,
community
C96 Commerical OSSmanagemen localization
t and of COSS,
Experimenta
Use of OS in creation,
OSS in
Different Structure l studies
Medicine and for
C112 and content creation
Domains (Experiment OSS inand
medicine
Need to
(Medicine) Dimension)
al Studies) maintenanc
research
work on
Knowledge e of Needthefor user
Knowledge managemen
Open knowledge
community
C120 Managemen Innovation managemen future
t metrics in are among
research in
t in OSS t metrics in
Open OSS andProjects them.
Open
OSS
Requiremen
Innovation Open Innovation
projects.
and t Tool support
(OI)
C125 Innovation
Engineering
Requiremen (Strategies for
strategies
and automated
t
Open Requiremen within RE
that are non solutions in
Engineering
Innovation intrusive t that are non
- ToolsRE
intrusive
and Engineering
and low to
C126 Requiremen (Tool specifically
and of low
cost) tosupport
Need of
extract
t Need
support to migration cost to of
more
and. on
Engineering OSS
extract and research
Quality developers
empirical
prioritize
OSS Quality prioritize the
between
and work
requirement and
C130 relationship
requirement theoretical
and Success Success(Rela OSS
Certification betweens
s)
tionship) community
research to
(Research OSS success
projects
assess
Community and quality the
OSS Effectivenes effectivenes
Participation
C144 Certification s of Linus determine
(Tools s of Linus
Tools
Developmen
Law socialization
Required) t practices,
Applicability law
of strategy
patterns
applicability
process of
for
) for choosing
OSS
OSS Community task and
better its
in OSS
C169 community
Community Participation effectivenes
community
community
s in the
(Practices) participation
projects
Identify
context
in OSSof
impact
Community certification.
projects of
C176 Design
joiningtools
Participation to enhance
process on
(Processes) retention
retentionof
Oneand Time
Contributors
relationsip
CF161 (OTC) in OSS
between
community
joining
projects
process and
Need to
CF163 role
develop
explore
transition the
quality in
motivation
Quality OSS
models
models for community of
OSS and OSS (Having projects based
Need
developers on
to
CF102 essential
Quality Essential todevelopwork as a
qualityin
mentor
Quality characteristi
Impact of
models
community
Attributes) onboarding
Quality cs
basedin the
projects on
OSS and process
area of OSSon
CF103 models for community
essential
Quality OSS research.
quality
characteristi
cs in the
area of OSS
research.
Define
OSS Quality
Ecosystem Assessment Quality
Assessment
CF105 Process for Process for
OSS Quality Individual
Software Undertand
Software
Assessment Ecosystem relationship
Developer/T
eam and Ecosystem
between
OSS Project individual/te
CF51 Developers Characteristi am and
cs project
Aspects for characteristi
(Relationshi Case studies
OSS In OSSp) in for Different
cs
Organization
CF43 Industry Industry Aspects
s and their for
(Case OSS
approach to in
OSS OSS
Studies) industry
Community Community OSS
(Barriers (Barriers (Increase
CF27 Faced by Faced by Rigor of
Newcomme Newcomme OSS Studies)
rs) rs)
Prediction Metric for
OSS (Check OSS
CF18 Prediction Efficiency of prediction
Different Organization
Good or Bad
Metrics) s andstheir
Organization
Participation approach to
OSS in s andintheir participation
CF12 Organization Organization
approach to OSS
in OSS
(Conduct
communitie
Future
OSS
-OSS longitudinal,
Inter- s for OSS
Communitie research To study
in-depth
Organization s projects
should focus
CF6 OSS architectural
studies)
on
smells
Collaboratio Organization
Identifying
n Inter- agglomerati
Organization
s
Organization son the
and its
and their
participation
problems of
OSS correla
approach tion
to
CF135 with other
Collaboratio architectural
OS with
(Align
organization
Architectura
n erosion
architectural
ours
l Erosion in within research
for OSS
the
problems
with related
projects
OSS and OSS projects
such as
research
Industrial architectural
such as
CF136 fields)
Systems identifying,
Architectura Need
decayto
(Identifyin addressing,
l Erosion research
new OSSCauses) avoiding,
and architectural
CF137 Industrial eroison and in
Systems predicting
OSS or
(Identify
Architectura Need to
erosion.
industrial
new work
Causes) systems on
l Recovery ARCADE’s
CF140 (Work on Identify
tool forof
reasons
ARCADE
Tools) architectural
architecture
OSS recovery
New decay.
Commers of Community OSS
CF150 Dynamics Community
OSS (More dynamics
Community Work) Mentoring
New Newcomme
Commers of rs in OSS of
CF151 newcommer
OSS Community s in OSS
Community (Mentoring)
Community
interaction
on
newcomer
success
OSS need to
Developers the translate
OSS effect
OSS of
CF159 Developers Onboarding documentati
(Tool practices to
Effort onsuit onthetask
Support) Early accuracy effort
estimation/ Early Effort organisation
estimation
Maintenanc estimation al context to
CF58 e Effort Model in achieve model for the
technical
OSS web
Estimation Joining
OSS Reconciliatio
and many
coding
Reconciliatio Reconciliatio
Process of nBenefitsprojects
between
n of FOSS, Newcommer
in OSS n of FOSS,
Developers See
FOSS,
issues
joining
plan-
CF73 Agile and s in OSS
Agile and associated
(in driven
with and and
OSS
PlanOSS Community
Driven Community)
Plan Driven abondonem
Innersource cultural
agile
and seemodel
Community (Mentoring)
(Need to ent explore
differences,
process
challenges
the
CF132 Translate
(Newcomme Abandonme and mentoring
OSS OSS andas of issues
well
tools,
rs) Community
nt Process developers
related to
Practices
of to practices,
in a OSS
(Newcomme Developers creationand of
Furthermore
Suit the
(Tools community
CF138 rs) Organisation , future
local
processes to
(in
Required) research
al Context )
Community) required improveis
workspace.
Community Community community to
Participation Participation empirically
Innersource participation
validate
CF117 Innersource (Empirically
(Practices) proposed in OSSthe
validate community
OSS the frameworks
proposed
Developers
Community Influence
/methods, of
frameworks
Attraction
Participation relatioal
models
OSS /methods,
CF51 (How factors on
Developers (Processes)
Developer and
models developers
Relational
OSS
tools.
Developer Developer
and
Retention
tools.to Attraction
Factors
Attraction attractionto
Retention
researchers
(Effect from OSS
Influence)
OSS projects (Effect projects
from
OSS Innersource
Team level Team need to
CF52 (How (How
draw level
on
Developers (Theoratical
aspects)
Relational Relational
aspects)
theoretical
OSS Foundation)
Factors Factors
foundations
Developers Structural
Influence) Influence)
thatSee have
CF48 Network been relationship
used in
OSS OSS
Properities OSS
between
prior
Projects Developers Developers
and OSS research structural on
Retention
Project Retention
network
OSS, as well
OSS (Foster via properities
(Foster
as other via
CF49 Developers Characteristi
OSS
Active Active
Cross
and OSS
theoretical
cs of Utilization
Prediction
Utilization of
lens project
that are
Models
(Relationshi
Interaction Interaction
validity
characteristi
considered of
OSS (Cross
p)
Network) Network)
prediction
CF17 Project relevant cs to
Prediction Validity via ISS models
approach in
OSS projects
Replication via
Across OSS
Projects) replication
a vital need to improve the quality of reporting empirical
studies of OSS. We assert that an improvement in the
empirical
Newer models studiesshould of OSS will help the
incorporate community
selection meth-ods to better
that
understandtothe
are amenable results and
automation aslimitations
this is not the of the case reported
in most of
the existing OSS quality assessment research. In Initial models Analysis reviewed from in Main thisSheet
Implications
We have
Another
study. The presented
aspect
selection of the amethods
set
useofofguidelines
OSS
mostly that areare
in dentistry
adopted isexpected
education:
the model to
for OSS
help improve
10,000
Furthermore,
(32%), process the
our quality
participants
study and
(21%) hasof
from reported
revealed
other 138 that
(16%) studies
countries
thesuchquality inas OSS-related
used
of the
reported
guidelines, S3
empirical research on OSS has significant Research
research.
Supercourse
whichFrom are We
Fig.not5,doeasily
not
canclaim
e-module amenable that
entitled, the setroom
of
to “Sterilization
automation for improvement.
guidelines we
andquality
of(Fahmy cross- To
ethave
al.
that end, we have it proposed be aobserved
set of guidelines that 47%for reporting the empirical and Practice
proposed
2012).infection
assessment Model is
research on OSS. exhaustive
control
developers
models in a or
dental complete.
should
considered
We claim that thesepractice”,
thus However,
turn
doguidelines which
not mention their
can help we
makes
focus
the the believe
to
domainthis
OSSdata Conclusion S3
that
research
of significant
software
mining one of
techniques
community
application. improvements
the(Leopairote
to
This most popular
improve
implies the can
that be
etmost
quality al. made
e-lectures
2013),
of designing
of the inframework
theand
on
models quality
this topic
reporting
were orof
(31). reporting
Supercourse empirical empirical
research
is a methods,
network studies.
of if56,000
theare future papers on 174
tool-based
designed selection
to be domain-independent. which Asscientists
currently
such, from
among
domain- Discussion S30
countries
the empirical
least who studies
share
considered aoffreeOSS
options. provide
library
independence should be the focus of model developers The of all
5,802
advantage the information
lectures
this in
offers thirty-
is
that three
(Wagner languages,
it will ethelp suggested
but A
quicken
al.future
2015). by evaluation
it the
contains
domain the guidelines.
only eleven
independent processlectures resulting ison in Summary
Furthermore, research should focus onmodel newcomers’ one
and S31
dental
faster
that topics.
isdecision-
able These
to assessmaking. numbers
qual- Following arevarious
impressive,
this advice but
could several
also
orientation and reception toity in
identify category
the problems of
in OSS
initial Discussion
issues
bring
Navigating
Moreover,
including regarding
about increased
through
those thethe
mentoring
that use ofanother
adoption
amounts
is openofof intellectual
the
OSS
field models
that prop-
components
needs erty,
insoftware,
practice
to beand a Summary
contribution as well asare data-dominant,
provide guidance system regarding barriers
unified
(Wang related
explored et
control-dom- rating
al.
infor-
in the system
2013). mation
future. In and
addition,a
availablestandard
Furthermore,
inant and significant model
across
computation-dominant. citation
developers
the
significant system
Internet can
researchis
It shouldmay
also
a is and S31
in advance. Moreover, research is required to
consider
significant
required
also require
betomodelingchallenge
explore
able to this proper
quality
the [143].
tools,
with regulation
assessment
littleHowever,
practices,
or (19;32).
asthe
and
no customization.a pro-
multi-
information criteria
cesses Bythat Discussion
explore the means to support newcomer initial partic-
offered
coulddecision-making
following over
be the
helpful Internet
to
this particular (MCDM)
improve through problem
OSS
community
consideration, so as
communities, to
participation.
theparticipation, facilitate
model proposed web The fo-
ipation,
Moreover,enhancing mentoring motivation isuse to increase
another field that needs to beand Conclusion
automation
rums,
research
can tend and soas
community
tohindering seen
on,
be widelyfactors inmayrecent
constitutes
adopted studies
at
these
andthe (Fakir
same
findings
possibly time and
to a Canbolat
valuable
understand
standardized. S50
reduce
explored thein the that will improve retention of
2008;
resource.
the issuesCavus infuture.
Due 2010,
to the
the area Furthermore,
2011).
easy A MCDM
and access
select tosignificant
the problem
reusable
topic research
in this is
forsoftware
further
new
required developers and One Time Contributors (OTC).
context to
components,
research. canexplore
beweregarded
Additionally, see thethattools,
we as apractices,
software
alsoprocess ofand
systems
observed pro-
choosing
from cesses
are constantly
the among
resultsthat
Abandonment
could be helpful istoanother
improve area in OSS community
community participation. and The has Future
available
that growing.
in alternatives
majority Software (i.e.
of the studies, differ-
developers ent
areOSS
combination integratingalternatives)
of survey an and S7
many
research issues that need to be explored in future research. Research
basedcommunity
questionnaire
increasinglyon a number was may
larger usedof use
number as athese
attrib- ofutes
research
OSS findings
(quality
andmethodologytocriteria).
understand
commercial to
Additionally,
the issues further
in the research
area and is required
select the topic to explore
for furthervarious
Considering
conduct
components research. intothisFromoption
their opens
products.
these results,the model
In doing we observedsodeveloper
they thattofew
have to
factors that
research. lead developers
Additionally, alsoto
wecontribute stay or leave
observed from atheproject.
results
several
relate,
studies adapt,well-known
used and
machinepossibly MCDM
learning methods
methods. to athat large
In amenable
future
num-use berto
ofof Discussion S50
that
providers.
thesein majority
automation such
Therefore,
techniques of as:the
will studies,
weDEA, believe
be helpful combination
Analytic
research Hierarchy
to solve could of problems
the survey
Process
focus onand the
(AHP), questionnaire
andofTechnique
related
Interests was
toresearchers used
following
task distribution, as
fortoward
Order a research
of Preference
questions:
selecting
predicting methodology
taskby
a and to
to Similarity
start
supporting
􏰅 conduct
to Idealmay
How research.
Solu-
contribution,
evolution tion From
organizations
process and (TOPSIS)
of these
management
OSS most results,
to
projects mention
efficiently
ofare we
project
shownaobserved
few
navigate
information. that
in(Zavadskas
Tables few
through11
studies
andavailable
Furthermore, used
12. Research in-machine
formation
findings interest learning
et al. 2014).
and select
indicate
toward methods. In
OSS components?
thatpredicting
majority future use
studies Discussion
of theprojects
OSS of S50
has these
anddominantly techniques
researchersfocused will
who investi- be helpful
defect gated to
prediction. solve
community the
Change problems
dynamics
propaga-
belong related to task
tion,to,maintenance
USA. Wedistribution,
also observed
effort and selecting
SOC a task
that (self-organized
most of tothestart
studies
contribution,were and management
criticality) have got slightly better attention. The restone
(75%) conducted by the of project
research information.
group of of the Results and
Furthermore, findings indicate thatvery majority S29
aspects country. Thus
are addressed moreconsiderably
collaboration low. of
is required In the among
the studies
context Discussion
and
of OSSresearchers
research groupswho
evolution of
processinvesti-
different gated
support,countries community
to conduct
researchers dynamics
paid more
major
belong to USA. We also
research observed
in
attention to evolution models and exogenous factors the that
area. most of the studies
(75%) were
contributing toconducted
support Results by the research
evolution process. group of one
Maintenance S20
country. Thus more collaboration
support is the second largest aspect, and fault detection and is required among
research
change groups of are
propagation different
the third countries
largestto conductaspects.
explored more
research in the area. From Gap Sheet
its not Research gap
Duplicate
Duplicate
Duplicate
Duplicate
From Author Intent Sheet

We plan to perform an exploratory mixed-methods research
using OSP in an undergraduate course in SE, in order to
obtain new insights with an experience with this approach.
We will experiment
Moreover, the authors withareaplan-combination
ning to expand of inside thecontrol
search Conclusion S33
and a predefined project in a software
by including more terms in the search strings and additional evolution course,
using asources,
combination of different bylearning approaches. We
As datapart of our future not considered
research, we this study
plan to conduct to find more
a compre- Conclusion S50
also
relevant plan to
studiesinvestigate
on ofcommunitybetter methods
dynamics. to assess
Thus, students’
hensive survey practitioners to identify theprovides
key
more insightsinabout learning
research in this context.
topics and issues the area Conclusion
inpropose
Tochallenges
summarize,
We wouldthat
im-review
the plementing
in particular results
recommend
inner
indicatesource and
investigating twoOSS and Future
that required S54
resolution strategieswill helpful over-for
tofrequently researchers.
come. The survey could be
functionality is more
issues: (1) topics related to integration of OSS integrated incomponents
the safety Work
also performed torather
critical validate thanthe signifi- cancesolution
of our research
and (2)systemstopics concerning the entire
participation OSS in is
organization-
From Future Direction Sheet used.
The complete agenda.
community or OSS solutions
inter-organizational foundOSS in thecollaborations.
review are rather We
small and have low activity.
find these issues important because integration On the contrary, OSSofsolutions
OSS
integrated inconcerns
components safety critical systems have over organizations
most software-intensive five years of Results and
S5
[98] and because participation in collaborative we
history and high or medium activity. Hence, can
software Discussion
conclude that long history of OSS
development is increasing [7, 185]. The research could projects facilitates their
use in asonintegral
focus identifying partsthe of safety critical systems.
characteristics of successful However, ap-
the relationship between the activity of the OSS project and Future
proaches
MaturingtotheOSS, the challenges
research field on OSS these in organizations
organizations met, and S7
their adaption in and safety-critical systemsthem. is less clear from the Research
dealing with some of how its they solved
limitations may be done through
studied
Deploying papers OSS: andManythereforeclaim requires furthercosts investigation.
four mainthat reducing
steps: is one of
1.the advantages
Focus research ofon deploying
topics that OSS areserver
relevant software,
to how Future
infrastructure,
Maturing the and applications.
research field on However,
OSS in a recent
organizations studyandby Research S7
organizations approach OSS
Fitzgerald
dealing
2. Strive with [80]
to some is one
increase of its oflimitations
the few
rigorstudies
of the maywith bea done
empirical longitudinal
through
studies
view on 3. deployed
Conduct more OSS fourproducts.
main steps:
longitudinal, Thisin-depth
highlights a need for
studies
1.4.Focus research more
on studies
topics that
Align our research with related research fields on:
are relevant to how Future
S7
􏰅 What are long-term organizations costs and ap- consequences
proach OSS of deploying Research
2. Strive
researchers andtoneedkeeping
increase
to pay OSS
themore products
rigor of the
attention operational?
empirical studies
Succeeding at providing an OSS producttois issues that are
not necessarily
3. Conduct
interesting to more
prac- longitudinal,
titioners. Hence, in-depth
we studies
recommend
easy as there are challenges related to collaborating with a Future
com-focusing
munity, on liketopics related
attracting andtorelating
the ways to in which
contributors, Research
S7
organizations
requirementsactually engineering approach fromOSS, and issuesbalancing
a community, that could
benefit
focus practitioners,
on community rather
and have than
paying general
customers, “adoption
and so issues”.
onlarge
[109,
• Techniques and tools been devised to tackle
200]. We hope to see more research
amounts of data generated in software evolution analysis on these topics like e.g. Future
S7
[7, 205] on the following
and prediction. Software evolution automation offers to topics: Research
􏰅 How
collect are OSSofproviders
volumes able to attract
data in a consistent manner.and sustain
Software a
community?
evolution visualization helps in understanding the Conclusion
􏰅 Whatinare
transitions the success
complex and criteria
large for incorporating
systems in an easy way. and Future S10
Since 2008, synthesizers of res'. earch have introduced
Big data analytics contributions
can also help (require-
to analyze
frameworks and platforms to perform OSSlarge research sets paving
of data Work
ments, code,
Another
generated interesting
during bugsoftware
reports/fixes,
topic that etc.) Data
deserves
evolution. from a community?
attention
analytics in future
can be
the way for future work. The analysis of non-cited papers
research
used tothat ismanage
the emergingand understand body of literature
the not complex on agile
web of and
indicates significant research has been exploited,
distributed
software development
evolution (Angioni etsource
al.,of2005; Ramesh et Conclusion S12
Reviewing
yet. Therefore, the weasrecommend
related it happens
work in thein field
the OSS OSScodesuccess,
community and other towe
al.,observe
2006). The coming together of these two worlds was not
exploitrelatedfurther repositories
different measures
the potential (Gonzalez-Barahona,
and factors
provided by the forOSS 2017).
success and
conference
explored
noticed in this
that study,methods
different but we consider
are used that
in they caninalso
research the
series while maintaining the interest in its major research
contribute to facilitate
field but source of datastreams. reconciliation among
is mainly repositories of OSS plan-driven,
agile and FOSS development models.Likewise, the idea Conclusion S22
projects such as sourceforge.net and freshmeat.net. We of
mainlyprocesses
recommend diversity using(where variety different
of methods processes may be in
for research
running
the field concurrently
and also want on the same attention
to draw project—in multi-team
of potential Directions
projects or changing over time
research to context of OSS development in future research. during the phases of the
for Future S23
development and maintenance cycle) (Deck, 2001; Lindvall
Research
and Rus, 2000; Siebel et al., 2003) also should be investi-
gated in future research, since the need to manage this Directions
diversity can be an important motivation for the for Future S23
reconciliation among software development models.Finally, Research
approaches that deal with reconciliation considering
organization and project contexts and needs, appears to be a
promising path for reconciliation in the future (Jaufman and
Munch, 2005; Xu and Ramesh, 2008). This is the focus of
the next section.
Our review results showed that the medical literature on the
topicstudy
This of OSS in dentistry
suggests some isdirections
limited and for includes
future research. mostly
Disruptiveexpert COSS opinionrevenue and case-control studies.The second
models, organizational aspects of
suggestion is to include OSS as a control Conclusion S24
COSS, localization of COSS, and creation andgroupmaintenancein
experimental of the studies on softwareare
user community validation.
among them. Such compara-
tive analysis can have positive effects for the commercial
Despite pro- thegramming vendors bytoshowing
scarce references learningthem the most
assessment in the Conclusion S30
advantageous OSS solutions
selected papers, some studies assessed the experiences that can be deployed intofrom
commercial software
both the teacher and the student perspective. They also used packages.
It may also
various be beneficial
different instruments. for customers
The mainwho, issues apart
withfrom the
obtaining
Acknowledging detailed
the information
teacher perspective to assessment is the absence the
lack of published on the performance
research on of of Findings
use
of clear S33
OIsoftware
strategies
definitionspackages,
in specific might
of criteria beassess
able to
REtoactivities, decide
i.e.,
students’ if the risk of
prioritization
products, and
using OSS
validation, with-
as wellout astechnical
the lack support
of
performed tasks, and expected skills and attitudes.reported and requiring
tool support, greater
we
see new
Therefore, computer
opportunities skills for
it is important is justified
to state in
research on
thatspecific
studentcases.
automated and thus Conclusion
assessment
Acknowledging
nonintrusive the lack
and low-costof published
methodswork. research on
for applying OI the use of and Future S35
deserves more thorough
OI strategies in specific RE activities,
strategies in RE. More specifically, we think this mapping i.e., prioritization and Work
validation,
study provides as wellus aswith
the lack of reported
a motivation tool support,
to conduct morewe
see new opportunities
research on capitalizing foruponresearch
OI to on automated and thus Conclusion
The availability of source code forautomatically
OSS components extract
non-
As intrusive
discussed and low-cost
andinprioritize
Sectionfor methods
requirements.
IV.D, there by for
is no applying
concrete OI and Future S35
provides opportunities scrutiny third party
strategies inset
relationship RE. More specifically,
between quality and we thinkcriteria this mapping Work
certification bodies. However, the success
complexity, sizeofand OSS
study
inevolving provides
the reviewed us with a motivation to conduct more
naturepapers.
of many The OSS lack of studies
projects examining
severely limit the the
research onbetween
capitalizing upon OI to joining
automatically extract
practicality of such efforts, unless the software is developed Implications
relationship
Few studies are quality
conducted andon success
the criteria
processand metrics
and
S38
andtheprioritize requirements.
shouldcertification
abandonment.
“with encourage Therefore, researchers
in mind”. future to conduct
research
Cotroneo’s should
pre- furtheraddress
certification studies kit of Findings
the
in this
issues [7],ofcontext.
these two
Comar etInal.’s
addition,
areas. Thethere
Open-DO authors is noalso
continuous satisfactory
noticed number
that
certification some
of process
studies
studies in the contribution
investigated more types
than of
one
[6], Fusani and Marchetti’s virtual certificationmodel,
research metric topic.or tool,
and repository [10], Kakarontzas et al.’s OPEN-SME reusethe Discussion
it is observed
Additionally, our that
studythere is an
identified evident
the necessity
gaps in to
community fill S40
Some researchers
dynamics have
areaexamples
that studied
gap could
for forthese the joining
types.
beapproaches
useful process of OSS
for researchers.
process [11] are to develop “for
com-
Moreover, munities. map- But more
pingof research
study is required to discover the
certification”. Some thesealso provides
proposals can insights
be consideredof some
tools which
areas that need maymore helpexploration
the developers to alleviate
for instance the issues
complimentary, others are alternatives. Littlementoring
empiricalof
experienced
newcomers by them when they migrate to the other
evidence isand findingto-date
available a task about
to starttheir are some of the
effectiveness open
in Conclusion S50
projects,
research areas,to investigate
which need the
morechangesstudy. in the
There degree
is a of to
need
practice. As an increasing number of OSS systems are
socialization
developtotools over time, to
to better examine
support the association between
subject certification and maynewcom-
consider these ers’ onboarding
proposals,
andsimilarities
easy migration in newcomer
of the socialization
developers pat-
among terns
differ- and
the community will need more empirical evidence on ent
their
distinction in joining
projects. There is a lack scripts,
of to analyze
evidence on the
how effect
to of
remove the Discussion S50
effectiveness.
difference
Cmmunity
technical in the joining
dynamics
barriers, how process
studies
gender onand
focus retention.
on
agethe Besides,and
of internal
developers more
Besides,
significant
external more
motivation research
research ofis is needed
required
developers to to develop
investigate
and factors strategies
contributor
that influence to
influence their retention in a project. It shows that there is
alleviate
roles the
inneed
the issues related to choosing a task to start initial
the
Based retention
stillon the ecosystem,
oftools,
of developers
comparison toofanalyze
practices,in
theprojects.
and the
existing impact
However,
processesqualityfor ofassessment
resolved
there
betterare
contribution,
issues
someon to
onboarding
areas, design tools
success, to enhance
to examine retention
the of One
relationship ofhas Discussion S50
community
models, there iswhich
clearlyneed
participation noand research
engagement
suitable such as
model—each in retention
OSS projects.
model
Time
between Contributors
joining (OTC),
process and and to
rolethe explore
oftransition, the motivation onof
veteran
its developers,
own limitations. thea impact
As result, findings and
contribution of this to discover
barriers
analysis
effective
abandonment, developers
waysuse to orga- to work
nize project
ofespecially
gamification as a mentor.
in information
student engagement thatworkmay
have implications for practitioners who
support
towardsand coming newcomers
its effectup onwith during
retention, the onboarding
the effect of
new assessment period.
projectThey
models. Discussion S50
should note the following points in line with theproject
characteristics on retention, the impact of the research
governance
questions posed on inmotivation,
this study:development
Emphasis should of strategies
shift from to
tryingmaintain
to build loyalty, the
comprehensive impact of
modelsleadership
(containing styles on
all the Summary
The most commonly used selection method is the model
developer
possible softwareturnover, and impact
characteristics) of corporate sponsorship on
approach and the least considered to arebuilding
the tool-models based that and and S31
FLOSSonly
include communities.
essential Moreover,
quality there is a need
characteristics. This tostudy
design has Discussion
data mining approaches. Another interesting result is that
tools to
shown that assess
thesethe health quality
essential of OSScharacteristics
projects and develop include:
nearly half (47%) of the selected papers do not mention an
maintainability, strategies to
usabilityretain and new developers.
maintenance capacity of Conclusion
application domain for the models in their research. More
softwareshould community. and Future S31
attention be paidBy narrowing
to building down that
models to these three
incorporate
essential quality characteristics, model developers would Work
only essential quality characteristics. Also, framework, tool-
help
basedtoand reduce datathe burden
mining of OSSmethods
selection evaluation should viabe existing
given
qualitymore assessment models, which
attention in future model proposals. has been referred to
largely as being laborious and time consuming to conduct
The reviewed articles show that self-determined
Assesment
participation Process: motivesIt is worthare most mentioning
relevant for thatFLOSSto perform
adevelopers’
complete quality assessment of a
commitment. Moreover, both relational (e.g. software ecosystem we
first
trust)would need to define
and structural the assess-
(e.g. network ment process
centrality) groupwhich aspectsis
out of the scope of this paper.
foster FLOSS developers’ commitment. Also, the chosen The quality assessment
process will have to deal with,
code license affects developers’ commitment. Beyond these e.g., How are the values of
each
aspects, measure interpreted
the reviewed articles(i.e.,suggest
definingother whatfactors
are thewhich good
Discussion S32
andyet need dedicated analysis. For example, the effects ofto
the bad values)?; How can the measures be merged
provide
members’ the assessment
cultural background for a particular [57] or sub-their characteristic
geographic of
the quality[29]
proximity model?;on their or What
development are the principles
intensity. Further, to perform the
the assessment with missing, incorrect,
interrelations between the research perspectives need further Discussion and/or inconsistent S18
measure
Many
scrutiny. data?
of the We are
studies
In particular, arewill in the
the provide
form the
relationships answer
of surveys,
between to
which thesegives
individual and
other
a broad
or team questions
and necessary
factors as part
and of our
understanding.
project futureBased
characteristics work on in this
(see thistabletopic.
it would
1).
Althoughbe
probably wepossible
considered to the barriers
conduct more asstudies
something that can
investigating
Such
Each cross-perspective
metric used analysis iseither
for prediction, necessarybeing to understand
positively or
hinder
specific new- cases comers’ contributions,
of implementation ofsome barriers
methodologies canfor be
fully
negatively how FLOSS
associated projects
with can incentivize
prediction results. individual
For example, and Discussion S15
used
dealing as filters by the projects.
with different aspects Findingsof open source from ainHalfaker industry.et
ingroup
al. [19]
factors
casestudyof fault on
which
prediction,
Wikipedia
increase developers’
anewcomers
metric signifies
revealed
participation.
a modulethat as
Moreresearch
Future case studies may could
draw probably
on research be results
conducted by on some
Gallivan all
either
entry being
barriers faulty
led to or improved
not faulty.contributions
In either case, inthe themetric’s
future.
[21] andtion aspects
examine ifofand thehow research
governance questions. processesorfoster
predic- recital is judged
More- over, research conducted in the OSS domain [33, as a best, significant bad13] Discussion S9
FLOSS developers’
predictor. In this personal
regard, relationships.
our review resultsAlso, show further
demonstrated that socialization barriers are useful for
research is needed
inconsistency on some to fully
metric’s understand
performance. if and how project
For instance, Open
maintaining community integration and the quality of the
governance
the metric can
LOC stimulate
(Line of individuals’
Code) was participation
evaluated as a motives.
best or Questions
community’s product. A clear direction for future work is to
good
explore
Maturing predictor
howthethe incommunities
research[1][9],field whereasonperceive
OSS in [11] it was
these
in organizations noted and
barriers as a
and and S8
bad To
dealingpredictor.
how enable
with Moreover,
they organizations
impact
some ofthe DIT
its quality
limitations(Depth
to reap of ben- ofefits
Inheritance
contributions
may be done fromfrom Tree)
their
through Research
was noted as ainsignificant
participation OSS communities,
four predictor
newcomers.
main steps: in
the [6], but
research classified
community as a Agenda
bad predictor
should dedicate in [1][4].
much Possible
more
1. Focus research on topics that are relevant to how causes
attention tobehind
questions these Future
differences in[48,
results might S7
concerning this organizations165]. ap-beproach
While the variations
there are
OSS a fewin OSS
examples Research
systems
In2. [50,[9],
theStrive 101,differences
to
architectural 176, 186],
increase smells inmore
the implementations
rigor
group, work theisempirical
ofarchitecturalneeded of the met-
tostudies
bad aidsmells rics
[9], or different
organizations
3. Conduct
(architecture prediction
in participating
more longitudinal,
anti-patterns) models in
stood outin-depth used.
communities
as the most However,
studies and an
effective
indeepth
smells Aligninvestigation
4.collaborating
compared towith
our research andother
architectural resolution
with organizations
related
change of such
research conflicting
through
(instability)fields and Future
S7
collaborativelyissues would
working be a
on future
OSS
architectural hotspots smells (as shown in Fig. 10). This research
products agenda.
[7] and to solve Research
reveals that the architectural questions bad like:
smells were studied in iso-
lation 􏰅 When,
and nothow, combined and with with whatmore should
than one organizations
smell, which
participate
was covered in the development
in the prior of OSS products controlled
It can be concluded that code smells. Consequently,
the problems of architectural further
ero- Discussion S46
by
researchothers in (including
this domain inter-should organizational
be conducted collaborations)?
to identify the
sion
An within the provides
OSS projects, including identifying,and address-
􏰅 SLR
effect How of
study
can companies
architectural
directions
smells(effectively) for allow
agglomeration
researchers products
and its toprac-
be
correla-
ing,
titionersavoiding and predicting
on architectural decay arewithin
still open research
theproducts?
OSS domain issues, as
tion with partly OSS andproblems
architectural partly commercial
rather than the architectural
3)Thewhich findings need of further
the current analysis
follows: studyand serveinvestigation.
as evident that a
smells in isolation in order to prove its inclusion or Conclusion S46
1)Consequently,
metrics-based
There are reasons morethat
strategy isefforts
thecould on
most this domain
commonly
contribute should
used
excessively beto
solution
asexclusion
focused
compared as to
on one
identify-
otherof the keythe
ing
proposed indicators
other
solu-reasons of the
tions. architectural
that
However, are stillmore
increasing architectural erosion such as rapid development
unclear
studies and suggesting
are needed decay
inchanges,
this symp-
other field toms.
solutions
forlack
other to provide more Implications
of software, frequent and ofmetrics
developers’ to be
Few
performance studies
analyzed toTherefore, are conducted
and accuracy
provide the most on
to addressthe joining
architecturally process
architectural appropriate and
decay. for OSS
awareness. further studies should be conducted S46
abandonment.
solution Therefore,
and identify future
the effort research should address the Research
as a rooted-deep study to findrequired
out otherforcauses the metrics and to to
issuesdetect of these two areas. The authors also noticed that some and Practice
identifyarchitecturally
architecture erosion relatedwhether smells. on Thethe essential
OSS or Implications
techniques studies investigated
of ranking more
used shouldshould than one
enhance research
the topic.
possibility to
industrial systems.
Few studies our Practitioners
are study
conducted on thethe joiningfol- low
process guidelines
and for OSS
Additionally,
gettobetter nessidentified gaps in community
abandonment. avoideffective-
architectural
Therefore,
results and
contradictions
future research
the inidentification
the new
should addressand ofthe Research S46
dynamics
critical cores area that could beviolations.
of architectural useful forAlso, researchers.there is a
subsequent
issues of these versions
twoping of the
areas. Thesystemauthors inalso
terms of identifying
noticed that some and Practice
Moreover,
clear map-
opportunity study
for many also provides
researchers insights
to highlightof some
the potential
studies reasons
investigated within
more than the system
one environment.
research topic. of
areas that need
enriching more exploration
ARCADE’s tool for for instance
efficient mentoring
approaches to
Additionally,
newcomers our study identified the gaps in community
architecture recovery. Additionally, there are solutions, Conclusion
and finding a task to start are some of the open S50
dynamics
research areas, area thatneed couldmore be useful for researchers.
which show thatwhichit is not effective study.
at all in There is a need to
the problems of
Moreover,
develop map-
tools to ping
better study
support alsonewcom-
provides ers’ insightsonboardingof some
deep analysis and the difficulty of address such refactoring
areas
and that
easyneed more exploration
migration for instance
amongmentoring of
strategy that has noofconsiderably the developers a positive different
impact to
newcomers
projects. There and finding
is a lack a task to
of evidence start are some
on how to remove of the open Conclusion S50
address architectural erosion.
research areas, which need more
technical barriers, how gender and age of developers study. There is a need to
influence tools
develop to better
their retention support newcom-
in a project. It showsers’ onboarding
that there is
and easy migration of the
still need of tools, practices, and processes fordevelopers among differ- better ent
projects. There is a lack of
community participation and engagement in OSS projects. evidence on how to remove
technical barriers, how gender and age of developers
influence their retention in a project. It shows that there is
still need of tools, practices, and processes for better
community participation and engagement in OSS projects.
how to combine two development models could be
extended to the third. This discussion included ideas like: (i)
the reconciliation of FOSS and plan-driven configuration
Inmanagement
order to increase practices to agile model;
the participation of (ii)
newthe need for
developers,
further
there is a need to devise tools, which may eliminateof
studies on the reconciliation of the practice
None of the studies
continuous
contribution discussed
code integration
barriers and provide the need toagile
between
onboarding develop andan
support. FOSSearly
It is
effort
models estimation
and for
extension OSS of Web
this
necessary to examine the impact of social interaction on projects.From
search to the the
context findings,
of theit Discussion S50
isnewcomer
shown thatsuccess,
plan-driven there
model; have (iii)
the been no
of studies
investigate
effect doc-how thatthediscussed
umentation knowledge
on task the
need
management
Few to
studiesdevelop practices
are early
conducted effort
in agile
accuracy, technical and coding issues, cultural differences,on estimation
model
the joiningcan for OSS
contribute
process Web
and to
projects.
improve
abandonment. As
and issues can
knowledge per seen
Therefore,
related in several
management
to future
creation ofinpapers
research orga-
local recommending
nizations
should
workspace. address or inthe
future
issues Moreover,
FOSS ofwork, mentoring
thetwoauthors
communities;
these areas.(iv) is
only another
Theanalyzemention
authors field
ifalso
the that
the use need needs
noticed to
oftoexplicit be
improve
that some Discussion S19
explored
thestudies
knowledge modelin management
theorfuture.
to conduct
investigated Furthermore,
moremore
practices, than signifi-
detailed
coming
one cantplan-driven
comparisons
from
research research
topic. or is
required
propose
and FOSS
Additionally, totheexplore
involvement
development,
our study the tools, ofcan practices,
different
identified and pro-
the effort
be beneficial gaps cesses
measurement
in incommunity
an agilethat Directions
could
context; be
attributes, helpfulsuch
(v) extrap-
dynamics areaasto improve
[28],
olate
that the
could community
in use
which
be of the for
test
useful participation.
future
driven work
development
researchers. is toThe for Future S23
research
toconduct community
a plan-driven
Moreover, more
map- detailed
context.
ping may
study use
These these
comparisons
also ideas findings
provides by using
are all to understand
different
opportunities
insights of some Research
Specifically,
thethat
issues while
inmoretheISS area development
and select is highly
the topic for influenced
further by
for
areas future
In contrast, research.An
need developers estimation
area
exploration
are that
commonlytools.
still
for deserves
instance
drivenOSS to be
mentoring explored
by extrinsic of
OSS development,
research. Additionally, there is a need to translate practices
is the
newcomers search andtofor studies
finding awe to
taskalso to observed
reconcile start are the some from models
three thethe
of resultsof
open Conclusion S50
thatto motives
suit
in majority
join
the organisational
of
a FLOSS
the studies,
project.
context to Also,
combination achieve structural
of the
survey many and
software
research development,
areas,
characteristics which
ofwithFLOSS since
need only
more one
study.
developers’ contact studyThere was is identified
a need
network to
benefits associated
questionnaire was used OSS. as aFurthermore,
research future
methodology research
toand
in this
develop
influence quasi-systematic
tools to
their joiningbetter review.
support
behavior.validate First
newcomers’
Moreover,of all, it can
onboarding be
the application present
conduct is required
research. toFrom
empirically
these results, we the proposed
observed that few
in Inareas
easy that
migration
contrast, were of
developers not
the
domain and the development the focus
developers
are commonlyof this
amongphase work.
differ-
driven Some
by of these
entextrinsic
projects.
frameworks/methods,
studies used machine mod-
learning els and methods.tools. are In
relevant
Tofuture
advanced use our
of Discussion S50
There
understudied
is a lack
motives
characteristics to areasof
joinfor evidence
are
adeveloper
FLOSS indicated
on how
project. below
attraction. toAlso,
remove
to guide technical
structural
However, future
it is
understanding
these techniques of thewill inner source phenomenon, researchers
barriers,
research. how
characteristics
unclear In addition,
how gender of FLOSS
relational andasbeage helpful
stated of before,
developers’
factors
to solve
developers
influence the the
contact problems
reconciliation
influencenetwork
developers’ their
need to draw
related to on tasktheoretical
distribution, foundations
selecting thata have
task to been
start used
research
reten tion
influence
attraction. area
their isajoining
inWhile still
project. at an Itearly
shows
behavior.
Stewart and stage, but[55]
that there
Moreover,
Gosain it isthe expected
still
view need
application
trustlensthat
of
as
in prior research
contribution, and on OSS,
management as well ofas other
project theoretical
information.
in futuretools,
essential,
For thatthere
domain
a high are
practices,
and
there more
the
is
retention and
no studies
processes
development
attraction
rate, it and
is results
for
phase
centric
important better on
are
research the
community
relevant
that topic
on
FLOSS to
this be
Furthermore, arefindings
considered indicaterelevant that tomajority
ISS approach. of the studies Discussion S43
investigated.
participation
characteristics
aspect.
developers Hence,Finally,
perceive forand
future there
developer also
engagement
theirevaluations
project remains in
attraction.
work OSS
should the possibility
projects.
However,
focus on ofit that
is
thisthe
The
and implication
researchers for
who practice
investi- also
gated in as
liecommunitytheself-evidencedetermined.
dynamics
organizations
unclear
aspect. how
Moreover, or other
relational researchers
as project
shown factors have
influence already achieved
developers’
In addition,
benefits
belong toand
members’
USA. challenges
Wethe also ISSin
ofobserved table 2 there
continuance
development.
thatamongmost
is very
is influenced
The
of thefindings
fewby
studies
positive
attraction.
attraction
relational results While
specific
and on
structuralStewartreconciliation
research andwhichGosain
characteristics [55]
combines
of the agile,
view FOSS,
trust
individual
team. Also,as
have(75%) shown werethatconducted
the adoption of
byithave
the ISS development
research group ofhelps
one or-
and
lessand plan-driven
essential,
For a
teamhigh
restrictive there models,
is
retention
factors. no
code licenses but
attraction
rate,
Considering and is not
centric
important
thethat written
research
that
ideological
modularity about on
FLOSS
of the it
and yet.
this
code Discussion S18
ganisations
country. to improve
Thus future more better quality,
collaboration time-to-market
isas required among and in-
aspect.
developers
status
affect Hence,
motives
members’perceiveareproject their
dependentevaluations
project
retention. onworkthe should
feedback
However focus
self- onothers,
asdetermined.
of
shown thisin
novativeness.
research groups However,ofasdifferent as suggested
countries by Brown
to conduct et al.
more [4],
Inaspect.
future
tableaddition, Moreover,
3,studies
there members’
isshould
little shown in
project
examine
dedicated table 2this
continuance
closely
research there
on is is very
influenced
interaction
how team few by
for
level
that newcomers should researchunderstand in the the reality
area. of the method
attraction
relational
extending
aspects specific
and structural
ourdevelopers’ research
understanding which
characteristics combines
of developer of the individual
team.
attraction. Also,
through anaffect
appropriate retention.
enculturation, so With
that they regard cantorecog-
the
less and team
restrictive
Similarly, factors.
ofthere code Considering
licenses
is aspects
very and the that ideological
modularity of theand code Discussion S18
nise key whatroleworks group and whatlittle doesforliterature
not work,which
developers’ and combines
commitment,
thus be aware
status
team affect and
future motives
members’
project
studies are dependent
project
characteristics.
should retention.
examine onthisthe feedback
However
Considering
aspect asthe of key
shown
closely. others,
Inin
role
of changing working processes.
future
table
ofaddition, 3,studies
there
structuralvery is should
little
properties
few articles examine
dedicated
[26, 40], closely
research
use future
more than this
on
studies interaction
how
one team
areresearch for
level
necessary
extending
aspects
to understand
perspective. affectour how
This understanding
developers’
FLOSS
calls projects
for further of research.
retention. developer
canWith utilize In attraction.
regard to the
the social
particular,
Similarly,
key role of there
group is very
aspects little
for literature
developers’ which combines
commitment, Discussion S18
contacts
future of theirshould
studies members drawtoon reachresearch out or bynew Stewart members.and
team and[55]
future
Finally,
Gosain project
studies
drawing
which characteristics.
should examine
on research
suggests that by Considering
this aspect
Shah
members’ [48], the
closely.
further
retention keyIn
isrole
a
ofaddition,
structural
studies are properties
very few
essential [26,
articles
to 40],
use
understand
product of motivational and relational factors. Further, future
more studies
than
fully one
how are necessary
research
FLOSS
toinitiatives
future
This understand
perspective.
researchcan
research how
This
question FLOSS
calls
stimulate
is necessary for to
evolved projects
further
individuals’
understand tocan
dueresearch. theutilize
factIn
extrinsic
the themost social
particular,
interaction
that and of of Discussion S18
contacts
future of their
studies
intrinsic
structural
the reviewed members
should
arti- motives
network draw
cles (67%) to onreach
to join
propertiesresearch
admitted out
theand or
by
project.new
Stewart members.
necessityand
theproject of
Finally,
Gosain [55]
characteristics.
external drawing
validitywhich To ondoresearch
of suggests
the so, that by
future
prediction Shahshould
members’
studies
models [48],
studied. further
retention drawToisonbea Open
studies
product
researchinby
specific, areof essential
motivational
[4]Oh to understand
and Jeon [40]ofand
generalization and relational
the analyze fully
factors.
findings the how
wasways FLOSS
Further,
not done in Questions
future
becauseinitiatives
research
which
the study can
FLOSS isstimulate
is necessary
projects
subjective toindividuals’
understand
can
andactively
is dependent extrinsic
the interaction
utilize onthe and the
how of and S8
errors intrinsic
structural
interaction are network
classified motives
network of
in their
the toproject.
join
properties
members theAgainproject.
and toproject
foster
in [9], ittheiris Research
characteristics.
retention. Finally,
acknowledged To do
that further so,
further future studies
research across
replication should
is necessary many draw to
OSS on Agenda
research
understand
projects by
is required Oh
the waysand Jeon [40]
in whichthe
to establish and
project analyze
cross project the
characteristics ways
validity of in
which FLOSS influence theprojects
individuals’
prediction canmodel. actively
retention. utilize the
interaction network of their members to foster their
retention. Finally, further research is necessary to
understand the ways in which project characteristics
influence individuals’ retention.
not future direction
Duplicate
Duplicate
Duplicate
already covered
not mentioned here but mentioned

somewhere
S14
S5
S36
S36
S36
2015 167 duplicate
2020 248 duplicate
2020 269 duplicate
2014 14 Duplicate
2010 31 Duplicate
2010 35 Duplicate
2010 36 Duplicate
2010 38 Duplicate
2010 40 Duplicate
2016 67 Duplicate
2011 69 Duplicate
2014 118 Duplicate
2012 119 Duplicate
2012 120 Duplicate

2018 129 Duplicate
2017 150 Duplicate
2015 164 Duplicate
2017 171 Duplicate
2017 176 Duplicate
2019 189 Duplicate
2016 206 Duplicate
2020 243 Duplicate
2020 254 Duplicate
2020 257 Duplicate
2020 259 Duplicate
2016 156
Duplicate
2016 157
Duplicate
Not Future Direction, One it
2014 160is of general software and second maybe author intent
2014 85 Duplicate Emergent

issues in
OSS
evolution
2011 76 Duplicate are
prediction of
properties,
2015 aggregate
56 Duplicate
metrics,
change
evolution
2012 44 analysis,
OSS
Duplicate
integrability
and
2010 34 Duplicate licensing
2010 27 Duplicate
2020 220
Duplicate
2020 221
Duplicate
2020 222 Duplicate
2020 225
Duplicate
2020 238
Duplicate
2020 239
Duplicate
2020 255
Duplicate
2016 98 duplicate
2012 124 to do
duplicate
2020 237 Duplicate
2020 252 Duplicate
2018 208 duplicate
2014 91 duplicate
2014 92 duplicate
Two
Themes,
2014 88
one
duplicate
2014 89 duplicate
2012 46 duplicate
Results
shows that
“prediction
of
properties”,
“aggregate
metrics” and
“changevevo
lution
analysis” are
the most
emergent
open issues
in OSS
evolution,to
gether with
OSS
integrability
and
licensing.
Study ID Extracted From Data Extracted by
S1 Conclusion Ms. Saima Imtiaz
S2 Main Finding and Discussion Ms. Salma Imtiaz
S3 Implications for OSS Research and Practice Mr. Muhammed Usman
S4 Results and Discussion Mr. Naveed Ikram
S5 Conclusion and Future Work
S6 Results
S7 Future Research
S8 Open Questions and Research Agenda
S9 Discussion
S10 Overview of studies
S11 Results from Mapping Study
S12 Adoption of BI tools by organizations
S13 Research Contributions and Limitations
S14 Future Work
S15 Discussion and Future Work
S16 Directions for Future Research
S17 Discussion and Conclusion
S18 Summary and Conclusion
S19 Summary and Discussion
S20 Findings
S21 Paper Analysis
S22 Revisiting Research Questions
S23 Summary of Finding
S24 Implications of Findings
S25 Research Scope in Floss Adoption
S26 Conclusion and Research Direction
S28
S29
S30
S31
S32
S33
S34
S35
S36
S37
S38
S39
S40
S41
S42
S43
S44
S45
S46
S47
S48
S49
S50
S51
S52
S53
S54
S55
S56
S57
S58
S59
S60
Year of Publication Categorization Decision
2009 Future Direction SLR Yes
2010 Gap SMS No
2012
2013
2014
2015
2016
2017
2018
2019
2020
Author
Gap: Gaps Future
are Intent: The
the potentialauthors of the
research study clearly
directions statebyinthe
identified
the conclusion
Definition
authors or at
based on thethe end ofwhere
findings the paper
theywhat
statethey
that will
lessdo
or in
nothe Code Identifi
future
work or what they aim to in
dothis
in the future. CA1
Futureis Direction: Future
available, however case
Directions areno explicit identified
explicitly future direction
future
is identified.
areas and directions of research by the authors of the study CA2
based on findings. CA3
CA4
CA5
CA6
CA7
CA8
CA9
CA10
CA11
CA12
CA13
CA14
CA15
CA16
CA17
CA18
CA19
CA20
CA21
CA22
CA23
CA24
CA25
CA26
CA27
CA28
CA29
CA30
CA31
CA32
CA33
CA34
CA35
CA36
CA37
CA38
CA39
CA40
CA41
CA42
CA43
CA44
CA45
CA46
CA47
CA48
CA49
CA50
CA51
CA52
CA53
CA54
CA55
CA56
CA57
CA58
CA59
CA60
CA61
CA62
CA63
CA64
CA65
CA66
CA67
CA68
CA69
CA70
CA71
CA72
CA73
CA74
CA75
CA76
CA77
CA78
CA79
CA80
CA81
CA82
CA83
CA84
CA85
CA86
CA87
CA88
CA89
CA90
CA91
CA92
CA93
CA94
CA95
CA96
CA97
CA98
CA99
CA100
CA101
CA102
CA103
CA104
CA105
CA106
CA107
CA108
CA109
CA110
CA111
CA112
CA113
CA114
CA115
CA116
CA117
CA118
CA119
CA120
CA121
CA122
CA123
CA124
CA125
CA126
CA127
CA128
CA129
CA130
CA131
CA132
CA133
CA134
CA135
CA136
CA137
CA138
CA139
CA140
CA141
CA142
CA143
CA144
CA145
CA146
CA147
CA148
CA149
CA150
CA151
CA152
CA153
CA154
CA155
CA156
CA157
CA158
CA159
CA160
CA161
CA162
CA163
CA164
CA165
CA166
CA167
CA168
CA169
CA170
CA171
CA172
CA173
CA174
CA175
CA176
CA177
CA178
CA179
CA180
CA181
CA182
CA183
CA184
Code IdentifiCode Identifier of Future Direction
CG1 CF1
CG2 CF2
CG3 CF3
CG4 CF4
CG5 CF5
CG6 CF6
CG7 CF7
CG8 CF8
CG9 CF9
CG10 CF10
CG11 CF11
CG12 CF12
CG13 CF13
CG14 CF14
CG15 CF15
CG16 CF16
CG17 CF17
CG18 CF18
CG19 CF19
CG20 CF20
CG21 CF21
CG22 CF22
CG23 CF23
CG24 CF24
CG25 CF25
CG26 CF26
CG27 CF27
CG28 CF28
CG29 CF29
CG30 CF30
CG31 CF31
CG32 CF32
CG33 CF33
CG34 CF34
CG35 CF35
CG36 CF36
CG37 CF37
CG38 CF38
CG39 CF39
CG40 CF40
CG41 CF41
CG42 CF42
CG43 CF43
CG44 CF44
CG45 CF45
CG46 CF46
CG47 CF47
CG48 CF48
CG49 CF49
CG50 CF50
CG51 CF51
CG52 CF52
CG53 CF53
CG54 CF54
CG55 CF55
CG56 CF56
CG57 CF57
CG58 CF58
CG59 CF59
CG60 CF60
CG61 CF61
CG62 CF62
CG63 CF63
CG64 CF64
CG65 CF65
CG66 CF66
CG67 CF67
CG68 CF68
CG69 CF69
CG70 CF70
CG71 CF71
CG72 CF72
CG73 CF73
CG74 CF74
CG75 CF75
CG76 CF76
CG77 CF77
CG78 CF78
CG79 CF79
CG80 CF80
CG81 CF81
CG82 CF82
CG83 CF83
CG84 CF84
CG85 CF85
CG86 CF86
CG87 CF87
CG88 CF88
CG89 CF89
CG90 CF90
CG91 CF91
CG92 CF92
CG93 CF93
CG94 CF94
CG95 CF95
CG96 CF96
CG97 CF97
CG98 CF98
CG99 CF99
CG100 CF100
CG101 CF101
CG102 CF102
CG103 CF103
CG104 CF104
CG105 CF105
CG106 CF106
CG107 CF107
CG108 CF108
CG109 CF109
CG110 CF110
CG111 CF111
CG112 CF112
CG113 CF113
CG114 CF114
CG115 CF115
CG116 CF116
CG117 CF117
CG118 CF118
CG119 CF119
CG120 CF120
CG121 CF121
CG122 CF122
CG123 CF123
CG124 CF124
CG125 CF125
CG126 CF126
CG127 CF127
CG128 CF128
CG129 CF129
CG130 CF130
CG131 CF131
CG132 CF132
CG133 CF133
CG134 CF134
CG135 CF135
CG136 CF136
CG137 CF137
CG138 CF138
CG139 CF139
CG140 CF140
CG141 CF141
CG142 CF142
CG143 CF143
CG144 CF144
CG145 CF145
CG146 CF146
CG147 CF147
CG148 CF148
CG149 CF149
CG150 CF150
CG151 CF151
CG152 CF152
CG153 CF153
CG154 CF154
CG155 CF155
CG156 CF156
CG157 CF157
CG158 CF158
CG159 CF159
CG160 CF160
CG161 CF161
CG162 CF162
CG163 CF163
CG164 CF164
CG165 CF165
CG166 CF166
CG167 CF167
CG168 CF168
CG169 CF169
CG170 CF170
CG171 CF171
CG172 CF172
CG173 CF173
CG174 CF174
CG175 CF175
CG176 CF176
CG177 CF177
CG178 CF178
CG179 CF179
CG180 CF180
CG181 CF181
CG182 CF182
CG183 CF183
CG184 CF184
OSS Effort Estimation/Maintenance Effort
Estimation
OSS for Requirement Engineering
OSS and Risk
OSS Licensing
OSS Community and Ecosystem
Security in OSS
Safety Critical Systems and OSS Adaptation

OSS CASE and Open-Source Business
Intelligence OSBI Tools
Open IoT Platform
OSS Platform
OSS Quality
OSS Developers/Women Developers

OSS Developers or OSS
Developers/Communities
OSS User Interface Design
OSS Integration
OSS Evolution
OSS Evolution Prediction/ Severity Level

Prediction in FLOSS
General OSS Research:
Reconciliation between different models and
OSS
OSS Knowledge Management

Application of OSS in Different Domains
OSS Process and OSS Inner Source
OSS Communities and Organizations
OSS Research Problems

OSS Development Integration with other
development methodologies
OSS Adoption
OSS Vs Propriety
Barriers Faced by Newcomers
OSS and Success
OSS Development Process

SWEBOK Areas
in SWEBOK its cost/effort estimation and test process measurement under the main heading of
Practical Considerations inside test process
in SWEBOK its software Elicitation under the main heading of Software Requirements
in SWEBOK in risks and uncertainty in the main heading of software engineering economics
in SWEBOK its Accreditation, Certification, and Licensing under Professionalism Under the
main heading of Software Engineering Professional Practice
In SWEBOK Ecosystems in practical considerations in the main heading of Software
Engineering Economics
in SWEBOK its under Security in key issues in software design under the main heading of
Software Design
in SWEBOK it's Software Safety under the main heading of Software Quality Fundamentals
in SWEBOK CASE Tools under the main heading of Software Engineering Models and
Methods
in SWEBOK its Platform Standards under Construction Technologies
in SWEBOK its Platform Standards under Construction Technologies
in SWEBOK its Software quality tools are under the main heading of Software Quality
in SWEBOK it's under the main heading of Basic Developer Human Factor
in SWEBOK it's under the main heading of Basic Developer Human Factor
in SWEBOK User Interface Design Inside the main heading of Software Design Fundamentals
in SWEBOK its Integration under the main heading of Practical Considerations in the main
heading Software Construction
in SWEBOK its Evolution of Software inside Software Maintenance fundamentals under the
main heading Software Maintenance
Author Intent Gap Future Direction
√ √ √
√ √
√ √
√ √
√ √
√ √
√ √
√ √
√ √
√ √
√ √
√ √
√ √
√ √
√ √
√ √ √
√ √ √
√ √
√
only ecosystem in second
also oss prediction in second

Future Direction Categorization (Final)

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Future Direction Categorization (Final)

Uploaded by

Copyright:

Available Formats

Study ID

2020 SLR Discussion Ms. Saima Imtiaz Future Direction

Future Research Ms. Saima Imtiaz Future Direction

2014 SLR Discussion Ms. Saima Imtiaz Gap

2017 SLR Conclusion Ms. Saima Imtiaz Author Future Intent

Discussion Ms. Saima Imtiaz Future Direction

Discussion Ms. Saima Imtiaz Future Direction

Future Research Ms. Saima Imtiaz Future Direction

Conclusion Ms. Saima Imtiaz Author Future Intent

2018 SMS Conclusion Ms. Saima Imtiaz Future Direction

Conclusion Ms. Saima Imtiaz Author Future Intent

Discussion Ms. Saima Imtiaz Future Direction

Future Research Ms. Saima Imtiaz Future Direction

Results Ms. Saima Imtiaz Gap

Conclusion Ms. Saima Imtiaz Future Direction

Discussion Ms. Saima Imtiaz Future Direction

Results Ms. Saima Imtiaz Gap

Discussion Ms. Saima Imtiaz Future Direction

SLR Overview of studies Ms. Saima Imtiaz Gap

Results Ms. Saima Imtiaz Gap

Conclusion Ms. Saima Imtiaz Author Future Intent

2017 SLR Discussion Ms. Saima Imtiaz Gap

2019 SMS Ms. Saima Imtiaz Gap

Conclusion Ms. Saima Imtiaz Future Direction

Discussion Ms. Saima Imtiaz Future Direction

2014 SLR Discussion Ms. Saima Imtiaz Future Direction

2016 SMS Discussion Ms. Saima Imtiaz Future Direction

Open Questions and

Future Research Ms. Saima Imtiaz Future Direction

Future Research Ms. Saima Imtiaz Future Direction

Discussion Ms. Saima Imtiaz Future Direction

2017 SMS Discussion Ms. Saima Imtiaz Future Direction

Discussion Ms. Saima Imtiaz Future Direction

Results Ms. Saima Imtiaz Gap

Discussion Ms. Saima Imtiaz Future Direction

Open Questions and

Discussion Ms. Saima Imtiaz Gap

Conclusion Ms. Saima Imtiaz Author Future Intent

Conclusion Ms. Saima Imtiaz Author Future Intent

Future Research Ms. Saima Imtiaz Future Direction

Open Questions and

Main Finding and

Results Ms. Saima Imtiaz Gap

2011 SLR Conclusion Ms. Saima Imtiaz Future Direction

Discussion Ms. Saima Imtiaz Future Direction

Main Finding and

2020 SMS Discussion Ms. Saima Imtiaz Future Direction

Future Research Ms. Saima Imtiaz Future Direction

2014 SLR Paper Analysis Ms. Saima Imtiaz Gap

Discussion Ms. Saima Imtiaz Future Direction

Paper Analysis Ms. Saima Imtiaz Gap

Conclusion Ms. Saima Imtiaz Gap

Discussion Ms. Saima Imtiaz Future Direction

Future Work Ms. Saima Imtiaz Future Direction

Conclusion Ms. Saima Imtiaz Future Direction

Discussion Ms. Saima Imtiaz Future Direction

Discussion Ms. Saima Imtiaz Future Direction

Future Work Ms. Saima Imtiaz Author Future Intent

Conclusion Ms. Saima Imtiaz Future Direction

Results Ms. Saima Imtiaz Gap

Discussion Ms. Saima Imtiaz Future Direction

Discussion Ms. Saima Imtiaz Future Direction