Requirements Eng
DOI 10.1007/s00766-013-0192-5
ORIGINAL ARTICLE
Requirements engineering education: a systematic mapping study
Sofia Ouhbi • Ali Idri • José Luis Fernández-Alemán
Ambrosio Toval
Received: 1 February 2013 / Accepted: 7 November 2013
 Springer-Verlag London 2013
Abstract Requirements engineering (RE) has attracted a
great deal of attention from researchers and practitioners in
recent years. Requirements engineering education (REE) is
therefore an important undertaking if the field is to have
professionals who are capable of successfully accom-
plishing software projects. This increasing interest
demands that academia should provide software engineer-
ing students with a solid foundation in the subject matter.
This paper aims to identify and to present the current
research on REE that is available at present, and to select
useful approaches and needs for future research. A sys-
tematic mapping study was therefore performed to classify
the selected studies into five classification criteria: research
type, empirical type, contribution type, RE activity, and
curricula. A total of 79 papers were selected and classified
according to these criteria. The results of this systematic
mapping study are discussed, and a list of advice obtained
from the REE literature for instructors is provided.
Keywords Software requirements
Requirements
engineering
Education
Systematic mapping study
S. Ouhbi (&)
A. Idri
Software Project Management research team, ENSIAS,
University Mohammed V Souissi, Rabat, Morocco
e-mail: ouhbisofia@gmail.com
A. Idri
e-mail: idri@ensias.ma
J. L. Fernández-Alemán
A. Toval
Department Informatica y Sistemas, University of Murcia,
Murcia, Spain
e-mail: aleman@um.es
A. Toval
e-mail: atoval@um.es
•
1 Introduction
Education research is becoming increasingly important not
only for educators but also for practitioners and researchers
in the domain. Top software engineering journals publish
papers on software engineering education. Important
Software and Requirements engineering (RE) Conferences,
such as the International Conference on Software Engi-
neering and the International Conference on RE, include
tracks or workshops on Software or requirements engi-
neering education (REE), respectively.
RE is concerned with the real-world goals for functions
(functional RE) and constraints (nonfunctional RE, e.g.,
constraints on quality and costs) on software systems. It is
also concerned with the relationships that these factors
have with precise specifications of software behavior and
with their evolution over time and across software families
[1]. RE is a multidisciplinary activity that deploys a variety
of techniques and tools at different stages of development
and for different kinds of application domains [2].
Research endeavors in software development have found
that the failures and deficiencies of software systems are
often rooted in the requirements activities undertaken [3–
6]. One of the main causes of this is the lack of appropriate
skills and knowledge of those engaged in RE activities [7].
The correct teaching of RE at university level has therefore
become an important mission for the profession. This
education should ideally be provided as an integrated part
of developing the requisite of RE and software engineering
technical skills, shortly before students become engineers
and enter the workforce [8]. There has thus been an
increasing emphasis on incorporating RE into the univer-
sity curricula of both undergraduate and postgraduate stu-
dents in the past decade [9–13], and a significant number of
studies have been carried out in the area of REE [14–17].
123
 Requirements Eng

In order to present academics and practitioners with Some examples of the three categories are presented in the
effective methodologies that can be adopted to provide following subsections in order to provide an overview of
high quality and relevant RE courses, a preliminary survey the results presented in Sect. 4.8, although this is not an
of REE was carried out beforehand [18]. The survey in exhaustive description:
question identified the contribution types of 31 articles,
while the paper presented here provides a new and com-
2.1 Model curricula
plete systematic mapping study. In addition to the formal
protocol established by systematic mapping studies, the
• The Joint IEEE/ACM Computing Curriculum—Soft-
scope has been broadened, now including 79 papers in
ware Engineering (CCSE) [10]. This is a basic under-
addition to new classification criteria, i.e., RE activities and
graduate curriculum in software engineering that
RE curricula are now considered. To the best of our
contains a bibliography of information that every soft-
knowledge, no systematic mapping study has been pub-
ware engineer should know. One of the programs in this
lished in the field of REE to date. A systematic mapping
curriculum aims to help students understanding the
study [19] is a defined method with which a classification
process of determining client needs and translating
scheme can be built and a field of interest structured. It
them to software requirements.
provides a structure of the type of research reports and
• The Joint IEEE/ACM Computing Curriculum—Com-
results that have been published by categorizing them. In
puter Science (CCCS) [11]. This curriculum contains
this paper, a systematic mapping study has been preformed,
undergraduate programs in each of the major comput-
which has allowed us to summarize the approaches pro-
ing disciplines. It highlights their commonalities and
posed for the development of the RE skills and to address
differences and describes the characteristics of gradu-
REE deficiencies. This mapping study has additionally
ates from each kind of undergraduate degree program.
enabled us to discover which RE activities are most fre-
Some of the units in this curriculum address software
quently addressed in the REE literature and whether the
requirements, specifications, and analysis.
authors of the selected papers have based their solutions on
• ACM/AIS Curriculum Guidelines for Undergraduate
model curricula. The primary studies were identified by
Degree Programs in Information Systems (IS) [21].
using search strings in digital libraries.
One of the learning objectives of this curriculum is to
The structure of this paper is as follows: Sect. 2 lists
teach undergraduate students how to ‘‘apply informa-
main RE curricula. Section 3 presents the research method
tion requirements specification processes in the broader
of our study. Section 4 reports the results obtained from the
systems analysis & design context’’ and how to ‘‘write
systematic mapping study. Section 5 discusses the main
clear and concise business requirements documents and
findings of this mapping study, presents a list of hints for
convert them into technical specifications.’’
RE instructors, and outlines threats to validity. Finally, our
• Graduate Software Engineering (GSwE2009)—Curric-
conclusions and future work are shown in Sect. 6.
ulum Guidelines for Graduate Degree Programs in
Software Engineering [22]. This curriculum is the first
product of Integrated Software & Systems Engineering
2 RE in model curricula, bodies of knowledge,
Curriculum (iSSEc) Project. It primarily addresses the
and standards
education of students for a professional masters degree
in software engineering. Among its guidelines is a
This section presents an analysis of the current situation in
Knowledge Area (KA) that focuses on the comprehen-
REE, as regards the most important software engineering
sion of software requirements fundamentals, require-
model curricula, bodies of knowledge (BoKs), and stan-
ments elicitation, requirements analysis, requirements
dards. Various organizations have made great efforts to
specification, and requirements validation.
identify RE knowledge [15]. Many curricula, BoKs, and
standards, including RE content, have been provided to
assist REE scholars to successfully undertake RE activities. 2.2 Body of knowledge
A BoK represents the complete set of concepts, terms, and
activities that a professional domain institutes, while a • The Software Engineering Body Of Knowledge
model curriculum is a set of guidelines with which to (SWEBOK) 12. is a comprehensive description of the
address education issues [15]. A standard is a technical knowledge needed to practice software engineering. It
specification that has been approved by a recognized contains ten KAs such as requirements, design, testing,
standardization organization with the objective of achiev- construction, and configuration management. It breaks
ing the optimum degree of order in a given context [20]. down each KA into topic areas. SWEBOK refers to RE

123
Requirements Eng
as Software Requirements (SR), and it breaks the RE
KA into SR fundamentals, process, elicitation, analysis,
specification, validation, and practical considerations.
• Software Engineering Education Knowledge (SEEK) [9]
draws on the knowledge areas of the SWEBOK. It
defines SE education BoKs that are appropriate to guide
the development of undergraduate SE curricula. It
contains recommended course sequences, guidelines
for curriculum development, and recommendations for
alternative teaching environments. The KAs of SEEK are
as follows: foundations, requirements, design, construc-
tion, maintenance, process, quality, and management.
• The Requirements Engineering Body of Knowledge
(REBOK). Aoyama et al. [14] has developed the model
and architecture of the body of knowledge of REBOK
and its proof of concept.
2.3 Standards
• IEEE Std 1233-98 [23] for system requirements spec-
ification. This standard was replaced by ISO/IEC/IEEE
29148:2011, which provides additional guidance in the
application of RE and requirements management
processes.
• IEEE Std 1465-98 [24] is a guideline for quality
requirements in software packages.
• IEEE Std 830-98 [25] is a guideline for the production
and content of the software requirements specification.
• ISO/IEC 25030:2007 [26] is a guideline for software
product quality requirements and evaluation. It applies
the quality model defined in ISO/IEC 9126-1 [27], and
it complies with the requirement processes defined in
ISO/IEC 15288 [28], which is a system engineering
standard.
• ISO/IEC TR 24766:2009 [29] is a guideline or RE tool
capabilities.
3 Research methodology
According to Petersen et al. [19], the main goal of a sys-
tematic mapping study is to provide an overview of a
Fig. 1 Systematic mapping
process [19]
research area and identify the quantity and type of research
and results available within it. A researcher often wishes to
map the frequencies of publication over time to see trends.
A secondary goal may be to identify the fora in which
research in the area has been published.
Figure 1 shows the mapping process, which covers the
search for relevant publications, the definition of a classi-
fication scheme, and the mapping of publications. A
mapping study differs from a systematic literature review
[30], which aims to establish the state of evidence, focus on
identifying best practices, and shows where particular
evidence is missing or is insufficiently reported in existing
studies. This is not the goal of a systematic mapping study,
since the articles are not studied in sufficient detail. Its
main focus is rather on classification, conducting a the-
matic analysis and identifying publication fora [19].
3.1 Research questions
The overall objective of our study is to gain insight into the
solutions designed to address the problems in REE, such as the
problems identified by Memon et al. [31]. In order to obtain a
detailed view on this topic, the systematic mapping study
addresses seven research questions (RQ). Table 1 presents the
seven RQs with their corresponding motivations. These
questions will allow us to categorize the current research in
REE and to identify future areas of research in the field.
3.2 Search strategy
The articles were identified by consulting the following
sources: IEEE Digital Library, ACM Digital Library, and
Science Direct. Google Scholar was also used to seek gray
literature in the field such as white papers and technical
reports. The efficient use of Google Scholar to perform
bibliometric studies has been demonstrated [32]. The fol-
lowing search string was used in order to perform the
automatic search in the digital libraries selected:
‘‘Requirements’’ AND (‘‘Engineering’’ OR ‘‘Elicita-
tion’’ OR ‘‘Specification’’ OR ‘‘Analysis’’ OR ‘‘Valida-
tion’’ OR ‘‘Process’’) AND (‘‘Educat*’’ OR ‘‘Train*’’ OR
‘‘Teach*’’ OR ‘‘Course’’ OR ‘‘Learn*’’ OR ‘‘Curricul*’’
OR ‘‘Guide’’ OR ‘‘Coach*’’ OR ‘‘Explain*’’).
123
 Requirements Eng

Table 1 Research questions Figure 2 shows the search process result. 79 studies
No. Research question Main motivation
were selected of 1,359 identified studies.

RQ1 Which publication channels To identify where REE 3.4 Quality assessment
are the main targets for research can be found as
REE research? well as the good targets for
publication of future studies The quality assessment (QA) is usually carried out in
RQ2 How has the frequency of To identify the publication systematic literature reviews and less in systematic map-
approaches related to REE trends over time of REE ping studies. However, in order to enhance our study, a
changed over time? research questionnaire was designed to assess the quality of the
RQ3 What are the research types To explore the different types papers selected. The QA was carried out by the two authors
of REE studies? of research reported in the who retrieved the studies. The way in which this ques-
literature concerning REE
tionnaire was written was inspired by a previous mapping
RQ4 Are REE studies empirically To discover whether research
validated? on REE has been validated
study [34].
through empirical studies (a) The study provides a contribution toward how REE
RQ5 What are the approaches that To discover the existing REE can be conducted. The possible answers were ‘‘Yes
were reported in REE approaches reported in the
research that address REE existing REE literature (?1)’’ and ‘‘No (?0).’’
problems? (b) The study presents clear solutions to the problems in
RQ6 What are the RE activities To identify in which RE REE. The possible answers were ‘‘Yes (?1),’’
that were addressed in REE activities REE researchers ‘‘Partially (?0.5),’’ and ‘‘No (?0).’’
literature? are interested in (c) The study presents empirical results. The possible
RQ7 Were REE approaches To discover if researchers answers were: ‘‘Yes (?1)’’ and ‘‘No (?0).’’
reported in literature based take into consideration RE
on RE curricula? curricula in REE
(d) The study has been published in a recognized and
approaches design stable publication source. This question was rated by
considering the computer science conference rankings
(CORE) [35] (A, B, and C), and the Journal Citation
3.3 Study selection Reports (JCR) lists. The possible answers to this
question were
The aim of the selection process was to identify the articles
that are the most relevant to the objective of this mapping • for conferences, workshops, and symposia:
study. When the same paper appeared in more than one • (?1.5) if it is ranked CORE A,
source, it was considered only once according to our search • (?1) if it is ranked CORE B,
order. Each paper was retrieved by one author and evaluated • (?0.5) if it is ranked CORE C,
by two authors, in order to decide whether if it should be • (?0) If it is not in CORE ranking.
included, by considering its title, abstract, and keywords.
Articles that were judged differently were discussed by the
two authors who carried out the evaluation of articles until
an agreement was found. The remaining authors reviewed
the final selection. The Cohen’s Kappa coefficient was used
to calculate the interrater agreement between the two
authors in their evaluation. The Kappa coefficient was 0.95
which, according to Landis and Koch [33], indicates an
almost perfect agreement between the two assessments.
The first step after the articles had been identified was to
eliminate duplicate titles, and titles clearly not related to
the review. The inclusion criteria were limited to the search
string, and the studies that met at least one of the following
exclusion criteria (EC) were excluded:
EC1 Papers that are not focused on education.
EC2 Papers presenting a general focus on software
education.
EC3 Papers about requirements for engineering
education. Fig. 2 The primary studies selection process

123
Requirements Eng
• for journals:
• (?2) if it is ranked Q1,
• (?1.5) if it is ranked Q2,
• (?1) if it is ranked Q3 or Q4
• (?0) If it has no JCR ranking.
• for others; (?0).
The score for the quality criterion (d) refers to the fact
that journals are more advantageous than conferences,
workshops, and symposia because the authors believe that
the possibility of publishing work in Q1 or Q2 journal may
be more difficult than in other publication channels. The
sum of the four closed-question scores for each study
provided a final score (an integer between 0 and 5).
3.5 Data extraction strategy and synthesis method
The data extraction strategy was based on providing the set
of possible answers to research questions.
RQ1. To answer this question, publication source and
channel for each paper should be identified.
RQ2. To draw the publication trend, articles should be
classified per publication year.
RQ3. A research type can be classified into the
following categories [30]:
• Evaluation Research: An evaluation or an investiga-
tion of REE approaches is conducted. This also
includes the identification of problems in REE.
• Solution Proposal: A solution to REE problems is
proposed. This solution may be novel or a significant
extension of an existing approach. The potential
benefits and the applicability of the solution are
shown with a small example or a good
argumentation.
• Experience Papers: These papers must express the
author’s personal experience and explain what has
been done and how it was realized in practice.
• Other: e.g., Theoretical papers, opinion papers,
reviews.
RQ4. The empirical research type can be classified into
[36, 37]:
• Case study: An empirical inquiry that investigates a
phenomenon within its real-life context. This term
may refer to single or multiple case studies.
• Survey: A method for collecting quantitative infor-
mation concerning items in REE, e.g., a
questionnaire.
• Experiment: An empirical method applied under
controlled conditions and using students as subjects
in order to observe its effects on REE.
RQ5. An approach can be classified into the following
categories as proposed in [19, 38].
• Method: A means or manner of procedure and a
series of steps taken to acquire knowledge in REE.
• Tool: Anything used as a means to accomplish a task
or purpose in REE.
• Model: A representation of a system that allows for
investigation of the properties of REE to be
investigated.
• Guideline: An indication of policy or procedure by
which a course of action in REE can be determined.
• Framework: A real or conceptual structure intended
to serve as a support or guide for the building of
something that expands the structure into something
useful in REE.
RQ6. Upon considering the SWEBOK [12] RE KA
breakdown, RE activities can be classified into the
following:
• Requirements process. A paper is categorized into
this criterion if it discusses all RE activities in detail.
• Requirements elicitation. If the paper reports how
and where to collect RE.
• Requirements analysis. If the paper discusses
requirements classification, conceptual modeling,
architectural design, and requirements allocation.
• Requirements specification. If the paper shows the
procedure of the production of a document, or its
electronic equivalent, which can be systematically
reviewed, evaluated, and approved.
• Requirements validation. If the paper presents the
process used to examine the requirements documents
to ensure that the right system is being defined.
• Requirements engineering. If the paper does not
discuss any RE activity and if it mentions RE in
general.
RQ7. The word ‘‘curricula’’ in this question refers to
model curricula, bodies of knowledge, and standards
defined in Sect. 2 The answer to this question can be
classified into: SWEBOK, CCSE, CCCS, IS, GSwE,
REBOK, SEEK, IEEE 830, IEEE 1233, IEEE 1465,
ISO/IEC TR 24766, ISO/IEC 25030, or Other.
The synthesis method is based on counting the primary
studies that are classified in each of the RQ responses,
presenting a ranking of the primary studies based on their
QA, and finally, presenting charts for the classification
result. A variety of evaluation approaches have been used
in the analysis of the results and a narrative summary with
which to recount the principal findings of this systematic
mapping study and describe them is presented in the
discussion.
123

4 Results
This section describes the results related to the systematic
mapping questions presented in Table 1. Some studies
were chosen to illustrate examples of each RQ’s result. We
consider that they are relevant and make an important
contribution to REE.
4.1 Selection results
Of the 148 articles deeply investigated, 69 papers were
discarded and 79 were finally selected. The 79 papers
identified were analyzed in order to answer the RQs
explained above. Table 2 presents the list of the selected
papers with details of the overall classification results and
their QA.
4.2 RQ1. Which publication channels are the main
targets for REE research?
Table 3 lists all the sources, the different publication
channels, and the number of articles per publication source.
Four different publication channels were identified in
addition to one technical report and a working paper.
About 46 % of the selected papers appeared in workshops,
39 % were presented at conferences, 9 % were published
in journals, and 4 % were presented at symposia. Two
reports were also included in the systematic mapping study.
Note that 38 % of the selected papers were published in the
International Workshop on REE and training (REET).
4.3 RQ2. How has the frequency of approaches related
to REE changed over time?
Figure 3 presents the number of articles published per year
from 1995 to 2012. There are three apparent outliers in the
years 2005 and 2008. The reason for this result is that some
selected papers were published in workshops as it is shown
in Fig. 3. Workshops and especially REET workshop
impacted the trend of REE publications per year.
The REET workshop started in 2005. The next editions
of this workshop were held yearly after 2007, which
explains the drop in publications on 2006. The spike of
other sources on 2008 is explained by the fact that on this
year, 4 articles were presented in the IEEE International
Requirements Engineering Conference. The reader will also
notice that in 1996, 1998, and 2001, no articles were pub-
lished. That could be explained by the fact that REE gained
interest on the beginning of the last decades after the pub-
lication of software engineering BoK and RE standards. The
decrease in 2012 may be explained by the point in time at
which this mapping study was performed and probably does
not reflect the real number of articles in 2012.
123
Requirements Eng
4.4 RQ3. What are the research types of REE studies?
Four research types were identified in this systematic
mapping study. Evaluation research (17 articles), solution
proposal (31 papers), experience paper (30 papers), and one
review. Theoretical papers or opinion papers were not
found in the REE literature. Around 77 % of the selected
papers (solution proposal and experience paper) therefore
present a solution to an REE problem, while 22 % evaluate
REE approaches, and one paper reviews the methods
reported in the REE literature. Examples of research types
are presented in the following paragraph.
Regev et al. [8] described their experience in teaching
an RE course. The course used an active, affective, expe-
riential pedagogy giving students the opportunity to expe-
rience a simulated work environment which demonstrates
the social/design-problem complexities and richness of a
development organization in the throws of creating a new
product. Davis et al. [97] proposed an approach that was
designed to facilitate life-long learning and stimulate the
continuous improvement of RE practice. One of the most
significant planned contributions of their project was the
establishment of a test bed in order to integrate RE research
and education which has a direct impact on practice. Hai-
ney et al. [40] evaluated a game to teach requirements
collection and analysis in software engineering at tertiary
education level. The results indicate that the game did not
meet the expectations of Further Education (FE) learners
(college level) to the same degree as the Higher Education
(HE) learners (university level). The approach seems to be
more suited to HE learners in terms of knowledge acqui-
sition, aspects of the game, and perceptions as opposed to
FE learners.
4.5 RQ4. Are REE studies empirically validated?
The result for this question revealed that 34 % of the
studies are not empirical studies because they do not pro-
vide any type of validation and do not present any proof of
concept. About 52 % of the nonempirical studies identified
in this paper suggest solutions with which to enhance REE
without empirically validating the proposed approaches.
Another 44 % of the nonempirical studies report their
authors’ experiences in teaching RE. A review [112] was
also identified, which presents a comparison of practitio-
ners’ perspectives of the usefulness of the RE content
knowledge included in higher education programs. In
around 38 % of the selected papers, experiments were
conducted with students in order to evaluate the effec-
tiveness of the authors’ approach in REE. 6 % of the papers
reported case studies in industry, and 22 % reported the
results of surveys.
Requirements Eng

Table 2 Classification
References Classification Quality assessment

P. Channel P. Year Research type Empirical Approach RE activity Curricula (a) (b) (c) (d) Score
type

[7] Workshop 2004 Evaluation research Survey Model Process CCCS 1 0.5 1 1 3.5
[8] Conference 2008 Experience paper Experiment Method Engineering No 1 1 1 1.5 4.5
[16] Conference 2008 Solution proposal Survey Tool Process Other 1 1 1 1.5 4.5
[17] Conference 2003 Experience paper Experiment Tool Process SWEBOK 1 1 1 1.5 4.5
[31] Conference 2010 Evaluation research Survey Guideline Engineering No 1 0.5 1 0.5 3
[39] Conference 2006 Experience paper Experiment Method Process No 1 1 1 1.5 4.5
[40] Journal 2011 Evaluation research Experiment Method Analysis No 1 0.5 1 2 4.5
[41] Journal 2007 Evaluation research Experiment Tool Engineering No 1 0.5 1 2 4.5
[42] Symposium 2011 Experience paper Survey Model Process No 1 1 1 1.5 4.5
[43] Workshop 2004 Solution proposal Experiment Framework Engineering No 1 1 1 1 4
[44] Conference 2004 Experience paper Survey Method Process No 1 1 1 1 4
[45] Conference 2008 Evaluation research Survey Tool Engineering No 1 0.5 1 1.5 4
[46] Conference 2008 Evaluation research Survey Method Process Other 1 0.5 1 1.5 4
[47] Conference 2000 Experience paper Survey Method Analysis No 1 1 1 1 4
[48] Conference 2005 Experience paper Case study Method Process No 1 1 1 1 4
[49] Conference 2007 Evaluation research Experiment Method Process No 1 0.5 1 1.5 4
[50] Conference 2005 Solution proposal Survey Framework Process No 1 1 1 1 4
[51] Symposium 2008 Evaluation research Survey Model Engineering No 1 0.5 1 1.5 4
[52] Conference 1995 Solution proposal Experiment Tool Elicitation No 1 1 1 1 4
[53] Conference 1999 Solution proposal Experiment Model Analysis No 1 1 1 1 4
[54] Conference 2009 Solution proposal Experiment Method Process SWEBOK 1 1 1 0.5 3.5
[55] Conference 2006 Experience paper Experiment Method Process No 1 1 1 0.5 3.5
[56] Journal 2009 Experience paper No Method Process No 1 1 0 1.5 3.5
[57] Conference 2009 Solution proposal No Model Engineering No 1 1 0 1.5 3.5
[58] Workshop 2008 Experience paper Experiment Framework Engineering No 1 1 1 0 3
[59] Workshop 2008 Experience paper Experiment Guideline Process No 1 1 1 0 3
[60] Workshop 2011 Experience paper Experiment Method Engineering No 1 1 1 0 3
[61] Conference 2009 Experience paper Survey Method Process No 1 1 1 0 3
[62] Workshop 2005 Experience paper Experiment Method Process No 1 1 1 0 3
[63] Conference 2010 Experience paper Experiment Method Engineering No 1 1 1 0 3
[64] Workshop 2010 Solution proposal Survey Method Process No 1 1 1 0 3
[65] Technical report 1997 Experience paper Experiment Tool Engineering No 1 1 1 0 3
[66] Workshop 2005 Experience paper Experiment Guideline Process SEEK 1 1 1 0 3
[67] Workshop 2007 Solution proposal Experiment Method Process Other 1 1 1 0 3
[68] Workshop 2009 Solution proposal Experiment Method Process SWEBOK 1 1 1 0 3
[69] Conference 2003 Experience paper No Method Elicitation IEEE830 1 1 0 1 3
[70] Workshop 2007 Experience paper Experiment Method Process No 1 1 1 0 3
[71] Workshop 2005 Solution proposal Experiment Method Process CCSE 1 1 1 0 3
[72] Working paper 2002 Solution proposal Experiment Framework Process No 1 1 1 0 3
[73] Conference 2011 Solution proposal Experiment Tool Process No 1 1 1 0 3
[74] Conference 2012 Solution proposal Case study Method Engineering No 1 1 1 0 3
[75] Conference 2008 Solution proposal No Tool Elicitation No 1 1 0 1 3
[76] Conference 2000 Experience paper No Method Process No 1 1 0 1 3
[77] Journal 2004 Solution proposal Experiment Tool Specification IEEE1233 1 1 1 0 3
[78] Conference 2005 Evaluation research Survey Method Process CCSE 1 0.5 1 0.5 3
[79] Workshop 2005 Solution proposal Experiment Method Process SWEBOK 1 1 1 0 3
[80] Journal 2010 Solution proposal Survey Tool Elicitation No 1 1 1 0 3
[81] Conference 2011 Evaluation research Experiment Method Analysis No 1 0.5 1 0 2.5

123
 Requirements Eng

Table 2 continued
References Classification Quality assessment

P. Channel P. Year Research type Empirical Approach RE activity Curricula (a) (b) (c) (d) Score
type

[82] Workshop 2007 Experience paper Case study Method Process No 1 0.5 1 0 2.5
[83] Workshop 2005 Experience paper No Model Process No 1 0.5 1 0 2.5
[84] Workshop 2005 Evaluation research Experiment Method Engineering CCSE 1 0.5 1 0 2.5
[85] Workshop 2011 Evaluation research Experiment Method Engineering No 1 0.5 1 0 2.5
[86] Conference 2012 Evaluation research Survey Method Engineering No 1 0.5 1 0 2.5
[87] Workshop 2010 Evaluation research Experiment Method Process No 1 0.5 1 0 2.5
[88] Workshop 2005 Evaluation research Survey Model Process No 1 0.5 1 0 2.5
[89] Workshop 2007 Evaluation research Case study Model Analysis No 1 0.5 1 0 2.5
[90] Workshop 2007 Evaluation research Case study Method Process No 1 0.5 1 0 2.5
[91] Workshop 2008 Experience paper No Tool Elicitation No 1 1 0 0 2
[92] Workshop 2012 Solution proposal No Method Engineering No 1 1 0 0 2
[93] Workshop 2010 Solution proposal No Method Analysis No 1 1 0 0 2
[94] Workshop 2005 Solution proposal No Guideline Elicitation No 1 1 0 0 2
[95] Workshop 2008 Experience paper No Method Process No 1 1 0 0 2
[96] Workshop 2004 Experience paper No Method Engineering No 1 1 0 0 2
[97] Workshop 2005 Experience paper No Method Process Other 1 1 0 0 2
[98] Workshop 2005 Experience paper No Method Process CCSE 1 1 0 0 2
[99] Workshop 2008 Solution proposal No Method Engineering No 1 1 0 0 2
[100] Conference 2005 Experience paper No Method Elicitation No 1 1 0 0 2
[101] Workshop 2008 Solution proposal No Tool Engineering No 1 1 0 0 2
[102] Workshop 2008 Solution proposal No Model Process No 1 1 0 0 2
[103] Workshop 2010 Solution proposal Survey Method Process IEEE830 1 1 0 0 2
[104] Journal 2012 Solution proposal No Method Engineering No 1 1 0 0 2
[105] Journal 2011 Solution proposal No Tool Engineering No 1 1 0 0 2
[106] Symposium 2010 Solution proposal No Tool Process No 1 1 0 0 2
[107] Conference 2008 Solution proposal No Tool Elicitation No 1 1 0 0 2
[108] Conference 2009 Solution proposal No Tool Elicitation No 1 1 0 0 2
[109] Conference 2005 Experience paper No Guideline Process IEEE830 1 1 0 0 2
[110] Workshop 2011 Solution proposal No Method Analysis No 1 1 0 0 2
[111] Workshop 2009 Experience paper No Model Process SWEBOK 1 1 0 0 2
[112] Workshop 2005 Review No Method Process No 1 0.5 0 0 1.5

Note that if the paper selected indicates that it contains a
case study and the authors used students as subjects for
their case study, then according to our classification crite-
ria, it is classified as an experiment. All of these empirical
studies were carried out by using previously trained stu-
dents who had a high level of control as regards the study.
The students first received lectures about an RE topic (for
example, the tools and methodologies used to manage
requirements) and then applied the knowledge and skills
acquired to the design of a system or even the creation of a
physical product in a laboratory environment. The inves-
tigations identified could be repeated under identical con-
ditions. The studies therefore had execution control and
ease of replication, which are the research strategy factors
of an experiment, not a case study. According to Wohlin
et al. [113], ‘‘the difference between case studies and
experiments is that experiments sample over variables that
are being manipulated, while case studies sample from the
variables representing the typical situation.’’ In our opin-
ion, a case study that is carried out with students in the
environments described does not represent a typical situa-
tion that is suitable for the industrial evaluation of RE
methods and tools because the students are manipulated by
the researchers to act in a defined scenario.
Figure 4 shows that around the half of the solution
proposals and experience papers were empirically vali-
dated through experiments. It also shows that in order to
evaluate an existing approach, authors mainly use

123
Requirements Eng

Table 3 Publication source

Publication source Channel References No. %

International Workshop on requirements engineering Workshop [58–60, 62, 64, 66–68, 70, 71, 79, 82–85, 87–91, 30 37.97
education and training 93–95, 97, 98, 102, 103, 110–112]
IEEE international requirements engineering conference Conference [8, 16, 17, 45, 46, 57] 6 7.59
Conference on software engineering education & training Conference [31, 54, 55, 78] 4 5.06
ASEE/IEEE rrontiers in education conference Conference [52, 53, 69, 76] 4 5.06
IEEE international conference and workshop on the Conference [44, 48] 2 2.53
engineering of computer-based systems
Australian workshop on requirements engineering Workshop [7, 43] 2 2.53
International workshop on multimedia and enjoyable Workshop [99, 101] 2 2.53
requirements engineering
International workshop on advances and applications of Workshop [96] 1 1.27
problem frames
International workshop on software engineering education Workshop [92] 1 1.27
based on real-world experiences
International world scientific and engineering academy and Conference [86] 1 1.27
society conference
Malaysian conference in software engineering Conference [81] 1 1.27
Mexican international conference on computer science Conference [108] 1 1.27
Norsk informatikkonferanse conference Conference [63] 1 1.27
ACM special interest group on computer science education Conference [39] 1 1.27
conference
World academy of science, engineering and technology Conference [109] 1 1.27
International technology, education and development Conference [73] 1 1.27
conference
International business conference Conference [50] 1 1.27
Australian software engineering conference Conference [47] 1 1.27
IEEE international conference on advanced learning Conference [75] 1 1.27
technologies
IEEE international conference on computer science and Conference [74] 1 1.27
automation engineering
International conference on computing in civil engineering Conference [100] 1 1.27
International conference on information technology: new Conference [61] 1 1.27
generations
International conference on intelligent virtual agents Conference [107] 1 1.27
International conference on software engineering Conference [49] 1 1.27
Symposium of the special commission of games and digital Symposium [106] 1 1.27
entertainment
ACM-IEEE international symposium on empirical software Symposium [51] 1 1.27
engineering and measurement
Symposium on computer science education Symposium [42] 1 1.27
Computers & education journal Journal [40] 1 1.27
Empirical software engineering Journal [41] 1 1.27
International journal of computer applications Journal [104] 1 1.27
International Journal of Computer Science Issues Journal [105] 1 1.27
International journal of engineering education Journal [77] 1 1.27
Requirements engineering journal Journal [56] 1 1.27
Revista Facultad de Ingeniera Universidad de Antioquia Journal [80] 1 1.27
Technical report Other [65] 1 1.27
Working paper Other [72] 1 1.27

123

questionnaires or conduct experiments. Examples of REE
empirical research are mentioned in the following
paragraph.
Beatty and Alexander [45] conducted an experience to
evaluate the existing research on the use of games in
training and their application to RE training. The authors’
experiences, the existing research, and theories from the
field of learning led them to conclude that games appeared
to be transferable and applicable to training in RE. Mead
Fig. 3 Number of articles published per year
Fig. 4 Research types and empirical type
Fig. 5 Approaches
123
Requirements Eng
et al. [68] presented a case study to validate a compre-
hensive teaching model for security Requirements Engi-
neering that centered on the employment of the SQUARE
method [114]. The results showed that when students learn
about security Requirements Engineering using SQUARE,
they have a better understanding of what is needed to
produce more secure software. Svahnberg et al. [51]
investigated students’ abilities to understand and assess
multiple perspective (basic skills, state of practice, and
state of the art) involvement in the requirements selection
process. The authors performed a survey in relation to
requirements selection as part of an Advanced Course on
Requirements Engineering at Blekinge Institute of Tech-
nology. The results indicated that students have a good
understanding of the way industry acts in the context of
requirements selection and that students may work well as
subjects in empirical studies in this area. Memon et al. [31]
attempted to assess the status of REE courses offered in
major public universities in Malaysia. The aim of their
paper was to discover out the problems that students con-
front on an RE course and compare them with those pre-
sented in the REE literature. The survey results confirmed
many of the problems presented in the literature but also
explored insights that may encourage researchers to be
aware of the limitations of the way in which RE courses are
commonly conducted.
4.6 RQ5. What are the approaches that were reported
in REE research that address REE problems?
The majority of all research type approaches describe
methods. About 20 % of REE approaches describe tools
and 16 % describe models. Guidelines (6 %) and frame-
works (5 %) are also proposed as solutions to REE prob-
lems. Figure 5 shows that the majority of authors report
Requirements Eng
their experience in delivering REE course methods. With
regard to proposing solutions, the authors’ tend to propose
both methods and tools. Models were almost equally
reported in solution proposals, in experience papers and in
evaluation research. The framework approach (in this
context ’framework’ is considered to be a structure that
forms a frame for REE) is mainly reported as an REE
proposed solution, while the guideline approach is reported
as the authors’ approach experience. Some of the REE
approaches reported in the selected papers are listed below.
Callele and Makaroff [39] presented a second-year
computer science course designed to simulate appreciation
for the need for RE and to provide students with an
opportunity to engage in RE activity. Damien et al. [62]
described a course that was taught in collaboration between
three universities in different locations, time zones, and
culture: Canada, Australia, and Italy. The students from the
three locations played the roles of a client and developer
and experienced the iterative development of requirements
specification in global projects. Smith and Gotel [16]
designed the RE-O-Poly game to explain and explore good
RE practices. Since requirements are often conflicting,
players have to learn to resolve conflicts and determine
priorities in the game. Zowghi and Paryani [17] used role
playing as a pedagogical tool to provide their students with
a fuller appreciation of the range of both technical and
nontechnical issues involved in RE practices. Knauss et al.
[101] presented a Web-based simulation game, which was
easy to understand and fun to play within just a few min-
utes in order to take RE more seriously. The Software
Quantum game helps students to understand one of the
principal challenges of RE which is to build the right
system within the available time. Barnes, Gause, and Way
[59] described sample techniques for the teaching of the
unknown (e.g., recommendations about requirements for
disaster recovery) and unknowable (e.g., requirements to
predict future trends) in RE for systems design and pre-
sented their experiences in teaching as part of requirements
analysis in an engineering systems design class. Armarego
[43] presented an approach based on problem-based
learning (PBL) which attempts to provide students with a
solid foundation in the subject matter while simultaneously
exposing them to real-world characteristics. It provides
students with a process to deal with problems within a
metacognitive-rich framework that makes complexity
apparent and allows students deal with it in an adaptive
manner.
4.7 RQ6. What are the RE activities that were
addressed in the REE literature?
About 52 % of the selected papers present approaches for
requirements processes, and 27 % of the selected papers
did not specify any RE activity and were therefore clas-
sified under Requirements Engineering. Requirements
elicitation was reported in 11 % of the selected papers,
9 % discussed requirements analysis, and only one paper
discussed requirements specification, while no require-
ments validation were identified. In Fig. 6, the answers
for RQ3, RQ4, and RQ5 were combined in order to
establish a mapping with the aim of providing an over-
view of RE activities. This mapping has allowed us to
obtain more information on how the results from each RQ
are related to the others. The diagram presents the
research facet related to RE activity, distributed over
approaches and research types. This figure allows us to
conclude that only one tool was proposed as a solution for
requirements specification that authors mainly report their
experience in teaching requirements processes and that
the majority of the solution proposals concern require-
ments processes. In addition, most of the REE tools are
designed for requirements elicitation activities. Further
information regarding the relationship between the three
facets is shown in Fig. 6.
4.8 RQ7. Were REE approaches reported
in the literature based on RE curricula?
Only 24 % of the selected papers take into consideration
RE model curricula, BoKs, or standards. SWEBOK, CCSE,
and IEEE std 830 are the main RE sources for authors.
Other curricula were identified: Business Analyst BOdy of
Knowledge (BABOK), IEEE Std 9003, and Requirements
Engineering Good Practice Guide (REGPG).
4.9 Quality assessment
Table 5 presents the quality assessment score for each
paper. About 59 % of the selected papers had an above
average score, and 13 % had an average score, while 28 %
were below average. This quality assessment may help
REE scholars to choose the relevant papers according to
the criteria defined in Sect. 3.4.
5 Discussion and implication
This section summarizes and discusses the results related to
the systematic mapping study. Advice for instructors is also
proposed in this section.
5.1 Principal findings
The goal of this systematic mapping study was to examine
the current knowledge in REE by selecting 79 papers from
a total of 148. They were then classified according to five
123
 Requirements Eng

Fig. 6 RE activities

the yearly Chaos Report series conducted by the

Table 4 Curricula
Standish Group [3] and the studies [5, 115, 116] that
Curricula References Total demonstrate the criticality of RE with regard to the
SWEBOK [17, 54, 68, 79, 111] 5
success of SE projects, we believe that REE will
probably gain much more attention in the future.
SEEK [66] 1
• About 77 % of the selected papers reported solutions to
CCSE [71, 78, 84, 98] 4
REE. This result shows that the REE field has not yet
CCCS [7] 1
attained sufficient maturity for evaluation and that the
IEEE Std 830 [69, 103, 109] 3
main concern for REE researchers is to propose
IEEE Std 1233 [77] 1
approaches with which to enhance REE. This is also
Other [16, 46, 67, 97] 4
shown by the fact that experience papers reporting the
authors’ teaching experience and solution proposals
were the most common research types found in the
Table 5 Quality assessment literature. The objective of the solutions identified in
References Score Total this study is principally to address students’ lack of
awareness of RE principles and practices in the
[112] 1.5 1 development of software projects [39, 50] and their
[91–111] 2 21 lack of interest in RE courses [16, 101].
[81–90] 2.5 10 • The most frequently reported approaches in the selected
[31, 58–80] 3 24 papers were methods which were mainly reported as
[7, 54–57] 3.5 5 RE courses. Some authors shared their teaching expe-
[43–53] 4 11 rience [8, 44, 70, 76] or their course designs [54, 74,
[8, 16, 17, 39–42] 4.5 7 103] in order to present solutions to REE. Other authors
preferred to evaluate the efficiency of the existing
training and teaching programs [40, 46] and their
criteria: research type, empirical type, approach type, RE
impact on REE improvement. Another approach cate-
activity, and RE model curricula. The principal findings of
gory that has received attention was that of tools.
our study are the following:
Authors proposed games as teaching tools because
• The REE research area has gained increasing attention games are fun and engaging and provide friendly
since 2004, and 2005 marked the shift in the REE competition [16, 17, 80, 106]. Many of the approaches
publication trend as it was the year in which an identified were suggested to enhance REE and to
International Workshop on REET began. About 46 % replace traditional RE course teaching methods, such as
of the selected papers were published in workshops, the use of simulated project examples that do not
while only 9 % had reached the maturity of a journal provide the students with a realistic experience of
publication. However, upon taking into consideration, client–developer interaction [8, 42].

123
Requirements Eng
• In 38 % of the selected papers, the authors conducted
experiences, and in only 6 % of the selected articles did
they include industrial case studies in their research. In
RE, a case study is an observational study which
usually aims to track a specific attribute or identify
relationships between different attributes [113]. The
reduced number of studies identified in this category
can be justified by the amount of people involved and
the inherent difficulty of finding industrial projects.
What is more, in 22 % of the selected papers, the
authors used surveys to collect quantitative information
about REE approaches. About 73 % of the selected
papers were empirical studies and 58 % of these
empirical studies used students as subjects. A common
criticism of controlled experiments in which students
are used as subjects is their external validity, and it is
therefore difficult to generalize the results to industrial
settings [117]. However, the study by Svahnberg et al.
[51] shows that in certain circumstances, it may be
possible to use students as subjects for empirical studies
and to influence them to provide answers that are in line
with industrial practice.
• Around the half of the selected papers presented
approaches for requirements processes. About 11 %
discussed requirements elicitation, 9 % requirements
analysis, and only one paper discussed requirements
specification. No study for requirements validation was
reported. Researchers mainly propose approaches with
which to address the teaching of requirements process
that involves the general aspects of RE. More research
is needed to address the broad teaching of each RE
activity, and more attention should be paid to require-
ments validation.
• The main RE sources identified that inspired authors
were SWEBOK, CCSE, and IEEE std 830. However,
only a quarter of the selected studies took into
consideration the well-known RE model curricula,
BoKs, and standards in the design of their REE
approaches. This result shows that the remaining
approaches do not conform to curricula, BoKs, or
standards.
• About 24 % of the courses reported in the selected
papers were designed for undergraduate students, and
around 16 % were designed for professional engineers,
while only 9 % could be carried out with both
undergraduate and post graduate students. More
research is therefore needed to assess new teaching
methods and tools in postgraduate courses which
requires the acquisition of a more specialized knowl-
edge of the RE discipline.
• The main audience interested in this systematic map-
ping study could be instructors in the field of REE,
since the majority of the REE literature is written by
teachers and instructors. To support this statement, two
points were investigated: the affiliation of the authors
and the publication source. About 84 % of the papers
selected were written by instructors from universities,
while 10 % were written by professionals who are
mainly consultants, and 6 % were written jointly
between instructors and professionals. Only a few
practitioners therefore appear to be interested in REE.
Also, as it was mentioned in the RQ1 result, around
46 % of the selected papers appeared in REE-related
workshops and 39 % in conferences, which are mainly
held for instructors in order to give them the opportu-
nity to share their knowledge and experiences. Observe
that no paper was found in IEEE Transactions on
Education journal, which is one of the most important
journals related to education and computer science.
5.2 Implications and advice for instructors
The findings of our systematic mapping study have
implications for instructors who are working in REE, since
this study will allow them to discover the existing
approaches in the literature concerning REE and to exploit
combined approaches in REE. Moreover, the empirical
studies presented can provide an overview of the efficiency
of each approach. In order to improve the quality of RE
courses, instructors could take into consideration some
advice.
These pieces of advice are addressed toward instructors
since they are probably the main audience interested in this
systematic mapping study, as has been demonstrated in the
previous section. The studies cited in this section were
chosen by considering their content. First, those papers in
the mapping study that provided any kind of practical
guidance for REE were classified among those found in the
‘‘Study selection’’ phase, by reading their abstracts. Sec-
ond, one author carried out an in-depth review of previ-
ously selected papers in order to make recommendations
for the teaching of RE. No predefined templates or
guidelines were used in this screening process. During this
process, important problems in RE training and discussions
addressing key issues in RE were searched for. Recom-
mendations were extracted from the papers as provided by
the authors, and their original purpose and meaning were
maintained.
According to the literature reviewed [16, 31, 39–41, 45,
101], various issues have been identified to be considered
in this section. Other sources have also been identified
[118–128], which are not necessarily REE focused, but are
related to these issues; in that, they provide interesting
complementary information regarding the following
recommendations:
123

• Teach how to define scope of the problem and avoid
general and vague specifications [39]. In order to
address this issue, instructors can take into account the
different personalities of students when teaching an RE
course. Instructors can ask the students to answer a
questionnaire in order to discover their personality
traits. The results of this questionnaire will allow the
instructors to form teams which exhibit a better
performance. Previous studies [119] show a significant
positive correlation between the personality factor
extraversion and software product quality, including
satisfied requirements.
• Show how to select and use an RE tool. More than 100
RE tools can be found on the market [120]. Students
should be aware of the features that the current RE tools
provide and should be taught how to select the best tool
for a project, according to its needs: requirements
elicitation, requirements analysis, requirements speci-
fication, requirements verification and validation, and
requirements management.
• Promote activities in requirements analysis and model-
ing in addition to requirements management and
introduce the concept of prototyping in the course.
Prototyping is an important technique in the RE phase
[121]. Through prototyping, an executable model of the
software product can be achieved before the final
product has been created [122]. Moreover, prototypes
presented to stakeholders are very effective in ensuring
valid requirements [123]. A part of the course could be
devoted to prototyping the system described in the
Software Requirements Specification (SRS) and the
enterprise models. Students should rewrite the SRS and
submit an executive summary, including features such
as Return On Investment (ROI) in a nontechnical
language [41]. The difficulties involved in learning
these issues on RE courses have been identified in the
literature [31].
• Involve students in industrial projects in order to allow
them to acquire sufficient knowledge and practice,
especially on postgraduate courses. Instructors could
also invite industry practitioners to present real projects
and to describe their accumulated industrial experience.
The difficulties involved in how to apply RE knowl-
edge in the real world have been reported in the
literature [41, 118]. By providing the student with an
accurate view of reality and the adequate tools,
instructors can offer them the opportunity to attain a
critical mindset in order to deal with RE in practice.
• Have the ability, skills, and strategies needed to align
RE courses with contemporary global software devel-
opment (GSD) conditions. REE must adapt to meet the
changing demands in the global development environ-
ment. Instructors can refer to the experience by Damien
123
Requirements Eng
et al. [124] in designing and evaluating a framework to
teach GSD courses and to extract ideas that can be
implemented in RE courses.
• Familiarize students with approaches to problem solv-
ing, development methodologies, and development
tools [31]. Instructors can improve their courses by
using some of the solutions proposed in Sect. 4.6. RE
courses are often given in a traditional lecture/exercise
format [41]. It often occurs that students do not place
importance on RE activities and do not see their impact
on the success or failure of projects [101]. Instructors
are encouraged to include alternatives approaches in the
curricula. Some studies [16, 40, 45, 101, 125] have
shown that learning through play provides a successful
education experience. In general, game playing consists
of rules, goals, engagement, challenge, feedback, fun,
interactivity, outcome. and immediate reward.
• Use mobile devices as teaching tools. Mobile phones
are a particularly attractive avenue for delivering
courses and training in REE. M-learning (learning with
mobile devices) promises a continued extension toward
‘‘anywhere, anytime’’ learning [126]. Instructors can
deliver their courses in an interactive classroom: The
students can share a virtual whiteboard, electronic
textbook, and data over a networked environment to
actively participate in the course discussions [127].
These devices will encourage students to share their
knowledge, and students will be far more active than on
traditional courses. We propose that instructors refer to
the experience of Yau et al. [128] in order to adopt the
concept of a smart classroom which facilitates collab-
orative learning among students and can be adapted to
REE.
5.3 Threats to validity
Four kinds of threats to validity are discussed below.
• Construct validity: Construct threats to validity in a
mapping study are related to the identification of
primary studies [129, 130]. In order to ensure that as
many relevant primary studies as possible were being
included, two authors identified and proposed potential
search keywords in several iterations. Seven terms
related to RE and nine terms related to education were
used in the search string. However, the list might not
have been complete, and additional or alternative terms
might have altered the final list of papers found [131].
The search was performed by using IEEE Digital
Library, ACM Digital Library, Science Direct, and
Google Scholar. According to the statistics of literature
search engines [132], we believe that most of the
research on RE can be found in these electronic
Requirements Eng
libraries. To decrease the risk of missing related and
important publications, the authors also sought related
papers in major RE research venues (e.g., REJ, ER,
REFSQ, ICSE, TSE).
We believe that the inclusion of publication sources
that are not top journals and conferences in the review
might decrease the quality standards of the primary
studies, but it signifies that the representativeness of the
primary studies is increased. In particular, the REET
workshop is an essential venue when undertaking a
study on REE, in spite of its reducing the mean score
for primary studies in Table 2 (2.94 out of 5).
Moreover, certain papers may have been overlooked as
the result of the subscription limitations of our univer-
sity library, as was the case of two conference papers
found in the IEEE Digital Library. This problem was
overcome by asking the authors of the papers to provide
us with a copy of their published articles.
Another threat concerns the potential mishandling of
duplications, which might have slightly altered our
results. Two cases of possible duplication were
detected, which were examined exhaustively to dis-
cover whether or not they were the same study.
Although certain content elements were common to
different papers, these papers are based on innovative
ideas or new studies.
The final decision to select a study depended on the two
authors who conducted the search process. If a
disagreement arose between them, then a discussion
took place until an agreement was reached.
Finally, note that although the same papers were not
found in a replication of this secondary study (which is
only possible in a relatively narrow area with experts in
the area conducting the study), the same general
conclusions may be drawn [133].
• Internal validity: Internal validity deals with extraction
and data analysis [129, 130]. Two authors carried out
the data extraction and classification of the primary
studies, while the other two authors reviewed the final
results.
The decision as to which data to collect and how to
classify the papers therefore depended on the judge-
ment of the two authors conducting the systematic
mapping study. These authors who were from different
culture and research groups carried out two different
classifications for reliability purposes [134].
The Kappa coefficient was 0.95, reflecting a high level
of agreement between the authors, which indicates a
similar understanding of relevance, thus reducing this
threat significantly. The authors were in different time
zones and communicated with each other using video-
conferencing via Skype, and therefore needed to agree
on a timetable for their meetings to avoid them
becoming tired—a situation that is identified as a threat
in literature [135].
Data extraction from prose could also result in a
misclassification, but this problem was addressed by
developing a classification scheme on the basis of
widely accepted guidelines [12] and terminology pro-
posed for use in RE [136]. This would, therefore, only
have a minor influence on the general classification
derived in this study.
• Conclusion validity: The threat to conclusion validity is
concerned with the identification of incorrect relation-
ships which may lead to incorrect conclusions. In the
case of a mapping study, this threat refers to factors
such as missing studies and incorrect data extraction
[130].
The aim is to control these factors so that a systematic
mapping study can be performed by other researchers
[129, 131, 137] who will draw the same conclusions
[134].
Bias both as regards selecting and classifying primary
studies and analyzing data may therefore affect the
interpretation of the results. In order to mitigate this
threat, every step performed in the selection and data
extraction activity was clearly described as discussed in
the previous paragraphs.
The traceability between the data extracted and the
conclusions was strengthened through the direct gen-
eration of bubble plots and frequency plots from the
data by using a statistical package. In our opinion,
slight differences based on publication selection bias
and misclassification would not alter the main conclu-
sions drawn from the 79 articles identified in our
mapping study.
• External validity: External validity is concerned with
the generalization of this study [113, 137, 138]. The
systematic mapping results were considered with regard
to the RE domain, and the validity of the conclusions
drawn in this paper concerns only the REE context.
Since no time restriction was introduced in the search
for published studies, the representativeness of the
selected studies was not affected. This threat is not
therefore present in this context.
The search string and the classification scheme pre-
sented in this paper may serve as a starting point for RE
researchers, and practitioners can search for and cate-
gorize additional papers accordingly.
6 Conclusions and future work
This paper has presented a systematic mapping study that
summarizes the existing research in REE. Of 1359 studies,
148 papers were identified between 1995 and 2012, 79 of
123

which were selected and classified according to five crite-
ria: research type, empirical type, approach type, RE
activity, and RE model curricula. Publication source and
trend were also identified.
The results obtained showed that an increasing amount
of attention has been paid to REE since 2004. Around half
of the selected papers appeared in workshops, and only a
few papers had reached the maturity of a journal publica-
tion. The two main research types found were experience
papers and solution proposals. The majority of the selected
papers were empirical studies. Many papers proposed
solutions to address the education of RE, and the main
contribution types were methods. The RE process is the
most frequently addressed RE activity in the literature, and
few authors take into consideration model curricula, bodies
of knowledge or standards in the design of their REE
approaches.
This research could be a starting point to investigate
better ways in which to give RE courses in the future.
Furthermore, the REE approaches presented in this study
may help instructors to identify approaches that can be
adopted in order to improve the quality of their courses.
For future research on REE, greater presence in journals
should be considered and more attention should be paid to
the teaching of each RE activity, particularly requirements
specification and validation. More evaluation research
should be carried out in order to evaluate existing REE
approaches. Moreover, it is advised that future proposed
approaches should conform to the existing model curricula,
BoKs, or standards.
Ongoing research is based on performing a systematic
literature review to assess the research on REE by taking
into consideration the results found in this systematic
mapping study and performing a joint experiment
between the University of Murcia and the University of
Mohammed V Souissi in Rabat in order to study the RE
in GSD.
Acknowledgments This research is part of the project PEGASO-
PANGEA (TIN2009-13718-C02-02) financed by the Spanish Minis-
try of Science and Innovation (Spain), and also part of the GEODAS-
REQ project (TIN2012-37493-C03-02) financed by the Spanish
Ministry of Economy and Competitiveness. This research is also part
of the project Software Project Management using Data Mining
Techniques, (AP2010-2013), financed by Mohammed V Souissi
University (Morocco). The mobility grant of Sofia Ouhbi is financed
by the Mediterranean Office for Youth (MOY).
References
1. Zave P (1997) Classification of research efforts in requirements
engineering. ACM Comput Surv 29(4):315–321
2. Nuseibeh B, Easterbrook S (2000) Requirements engineering: a
roadmap. In: Proceedings of the conference on the future of
software engineering, ICSE ’00, ACM, New York, pp 35–46
123
Requirements Eng
3. Standish-Group, CHAOS summary (2009) [cited 2013]. http://
blog.standishgroup.com/pmresearch
4. Damian D, Chisan J (2006) An empirical study of the complex
relationships between requirements engineering processes and
other processes that lead to payoffs in productivity, quality, and
risk management. IEEE Trans Softw Eng 32(7):433–453
5. Smith MJ (2001) Troubled IT Projects: prevention and turn-
around, Institution of Electrical Engineers
6. Sommerville I, Ransom J (2005) An empirical study of indus-
trial requirements engineering process assessment and
improvement. ACM Trans Softw Eng Method 14(1):85–117
7. Minor O, Armarego J (2004) Requirements engineering: a close
look at industry needs and model curricula. In: Proceedings of
the 9th Australian workshop on requirements engineering,
AWRE’04, Australian Workshop on Requirements Engineering,
Adelaide, pp 9.1–9.10
8. Regev G, Gause DC, Wegmann A (2008) Requirements engi-
neering education in the 21st century, an experiential learning
approach. In: Proceedings of the 16th IEEE international
requirements engineering conference, RE ’08, IEEE Computer
Society, Washington, pp 85–94
9. The body of Software Engineering Education Knowledge
(SEEK) (2003) [cited 2013]. http://www.acm.org/education/
curricula.html
10. IEEE/ACM JTF-SEC, Computing Curricula—Software Engi-
neering (CCSE) (2004) [cited 2013]. http://sites.computer.org/
ccse/
11. IEEE/ACM (Ed.). The Joint Task Force on Computing Curricula
IEEE/ACM. Computing Curricula—Computer Science (CCCS)
[online] (2001) [cited 2013]
12. Abran A, Moore JW (2004) Guide to the software engineering
body of knowledge (SWEBOK), IEEE Computer Society
13. Kitchenham B, Budgen D, Brereton P, Woodall P (2005) An
investigation of software engineering curricula. J Syst Softw
74(3):325–335
14. Aoyama M, Nakatani T, Saito S, Suzuki M, Fujita K, Nakazaki
H, Suzuki R (2010) A model and architecture of REBOK
(Requirements Engineering Body of Knowledge) and its eval-
uation. In: Proceedings of the Asia Pacific Software Engineering
conference, APSEC ’10, IEEE Computer Society, Washington,
DC, USA, pp 50–59
15. Armarego J (2007) Educating requirements engineers in Aus-
tralia: effective learning for professional practice, PhD Infor-
mation Technology, University of South Australia
16. Smith R, Gotel O (2008) Gameplay to introduce and reinforce
requirements engineering practices. In: Proceedings of the 16th
IEEE International Requirements Engineering, IEEE Computer
Society, Los Alamitos, CA, USA, pp 95–104
17. Zowghi D, Paryani S (2003) Teaching requirements engineering
through role playing: Lessons learnt. In: Proceedings of the 11th
IEEE international requirements engineering conference, IEEE
Computer Society, Los Alamitos, CA, USA, p 233
18. Idri A, Ouhbi S, Fernández-Alemán JL, Toval A (2012) A
survey of requirements engineering education. In: Proceedings
of the IEEE global engineering education conference, EDU-
CON’12, Marrakech, Morocco pp. 826–830
19. Petersen K, Feldt R, Mujtaba S, Mattsson M (2008) Systematic
mapping studies in software engineering. In: Proceedings of the
12th international conference on evaluation and assessment in
software engineering, EASE’08, Blekinge Institute of Technol-
ogy, Bari, Italy, pp 71–80
20. ISO/IEC Guide 2:1996 Standardization and related activities—
General vocabulary (1996)
21. ACM/AIS Curriculum Guidelines for undergraduate degree
programs in information systems (IS 2010) [cited 2013]. www.
acm.org/education/curricula-recommendations
Requirements Eng
22. Graduate Software Engineering (GSwE2009)–Curriculum
Guidelines for Graduate Degree Programs in Software Engi-
neering [cited 2013]. http://www.gswe2009.org/
23. IEEE Std 1233-1998, IEEE Guide for Developing System
Requirements Specifications (1998)
24. IEEE Std 1465-1998//ISO/IEC 12119:1994, IEEE Standard
Adoption of International Standard ISO/IEC 12119:1994(E),
Information Technology-Software Packages-Quality Require-
ments and Testing (1998)
25. IEEE Std 830-1998, IEEE Recommended Practice for Software
Requirements Specifications (1998)
26. ISO/IEC 25030:2007 Software engineering—Software product
Quality Requirements and Evaluation (SQuaRE)—Quality
requirements (2007)
27. ISO/IEC 9126-1 Software engineering—Product quality—Part
1: Quality model (2001)
28. ISO/IEC 15288 Systems and software engineering—System life
cycle processes (2008)
29. ISO/IEC TR 24766:2009 Information technology—Systems and
software engineering—Guide for requirements engineering tool
capabilities (2009)
30. Brereton P, Kitchenham BA, Budgen D, Turner M, Khalil M
(2007) Lessons from applying the systematic literature review
process within the software engineering domain. J Syst Softw
80(4):571–583
31. Memon RN, Ahmad R, Salim SS 2010 Problems in require-
ments engineering education: a survey. In: Preceedings of the
8th international conference on frontiers of information tech-
nology, FIT ’10, ACM, New York, NY, USA, pp 5:1–5:6
32. Rosenstreich D, Wooliscroft B (2009) Measuring the impact of
accounting journals using Google Scholar and the g-index. Br
Account Rev 41(4):227–239
33. Landis J, Koch G (1977) The measurement of observer agree-
ment for categorical data. Biometrics 33:159–174
34. Fernandez A, Insfran E, Abrahão S (2011) Usability evaluation
methods for the web: A systematic mapping study. Inform Softw
Technol 53:789–817
35. Computer science conference rankings CORE (2010) [cited
2013]. http://lamp.infosys.deakin.edu.au/era/
36. Condori-Fernandez N, Daneva M, Sikkel K, Wieringa R, Dieste
O, Pastor O (2009) A systematic mapping study on empirical
evaluation of software requirements specifications techniques.
In: Proceedings of the 3rd international symposium on empirical
software engineering and measurement, ESEM ’09, IEEE
Computer Society, Washington, DC, USA, pp 502–505
37. Tonella P, Torchiano M, Du Bois B, Systä T (2007) Empirical
studies in reverse engineering: state of the art and future trends.
Emp Softw Eng 12(5):551–571
38. Barmi ZA, Ebrahimi AH, Feldt R (2011) Alignment of
requirements specification and testing: A systematic mapping
study. In: Proceedings of the IEEE Fourth International Con-
ference on Software Testing, Verification and Validation
Workshops, ICSTW ’11, IEEE Computer Society, Washington,
DC, USA, pp 476–485
39. Callele D, Makaroff D (2006) Teaching requirements engineer-
ing to an unsuspecting audience. ACM SIGCSE Bull 38:433–437
40. Hainey T, Connolly TM, Stansfield M, Boyle EA (2011) Eval-
uation of a game to teach requirements collection and analysis in
software engineering at tertiary education level. Comput Educ
56:21–35
41. Karlsson L, Thelin T, Regnell B, Berander P, Wohlin C (2007)
Pair-wise comparisons versus planning game partitioning–
experiments on requirements prioritisation techniques. Emp
Softw Eng 12:3–33
42. Mohan S, Chenoweth S (2011) Teaching requirements engi-
neering to undergraduate students. In: Proceedings of the 42nd
ACM technical symposium on Computer science education,
SIGCSE ’11, ACM, New York, NY, USA, pp. 141–146
43. Armarego J (2004) Learning requirements engineering within an
engineering ethos. In: Proceedings of the 9th Australian work-
shop on requirements engineering, AWRE’04, Adelaide, Aus-
tralia, pp. 6–7
44. Al-Ani B, Yusop N (2004) Role-playing, group work and other
ambitious teaching methods in a large requirements engineering
course. In: Proceedings of the 11th IEEE international confer-
ence and workshop on the engineering of computer-based sys-
tems, IEEE Computer Society, Los Alamitos, CA, USA, p 299
45. Beatty J, Alexander M (2008) Games-based requirements
engineering training: An initial experience report. In: Proceed-
ings of the 16th IEEE international requirements engineering
conference, RE ’08, IEEE Computer Society, pp 211–216
46. Berenbach B, Rayment T (2008) The evaluation of a require-
ments engineering training program at Siemens. In: Proceedings
of the 16th IEEE international requirements engineering con-
ference, RE ’08, IEEE Computer Society, pp 205–210
47. Gibson JP (2000) Formal requirements engineering: Learning
from the students. In: Proceedings of the Australian software
engineering conference, ASWEC ’00, IEEE Computer Society,
Washington, DC, USA, pp 171–180
48. Jiang L, Eberlein A, Far BH (2005) Combining requirements
engineering techniques—Theory and case study. In: Proceedings
of the 12th IEEE international conference and workshops on
engineering of computer-based systems, ECBS ’05, IEEE
Computer Society, Washington, DC, USA, pp 105–112
49. Ludi S (2007) Introducing accessibility requirements through
external stakeholder utilization in an undergraduate require-
ments engineering course. In: Proceedings of the 29th interna-
tional conference on software engineering, ICSE ’07, IEEE
Computer Society, Washington, DC, USA, pp. 736–743
50. Nguyen L, Armarego J, Swatman P (2005) Understanding
requirements engineering process: a challenge for practice and
education. In: Proceedings of the 5th international business
information management conference, International Business
Information Management Association, Cairo, Egypt, pp 886–894
51. Svahnberg M, Aurum A, Wohlin C (2008) Using students as
subjects—an empirical evaluation. In: Proceedings of the Sec-
ond ACM-IEEE international symposium on empirical software
engineering and measurement, ESEM ’08, ACM, New York,
NY, USA, pp 288–290
52. Swigger KM, Brazile R, Shin D (1995) Teaching cooperation
and requirements elicitation via a computer-supported cooper-
ative problem solving environment. In: Proceedings of the
frontiers in education conference, vol 2 of FIE ’95, IEEE
Computer Society, Washington, DC, USA, pp 3c2–7
53. Tuya J, Garcia-Fanjul J (1999) Teaching requirements analysis
by means of student collaboration. In: Proceedings of the 29th
annual frontiers in education conference, vol 1, pp 11B4/11–15
54. Fernandes JM, Machado RJ, Seidman SB (2009) A requirements
engineering and management training course for software
development professionals. In: Proceedings of the 22nd con-
ference on software engineering education and training, CSEET
’09, IEEE Computer Society, Washington, USA, pp 20–25
55. Mead NR, Hough ED (2006) Security requirements engineering
for software systems: case studies in support of software engi-
neering education. In: Proceedings of the 19th conference on
software engineering education & training, CSEET ’06, IEEE
Computer Society, Los Alamitos, CA, USA, pp 149–158
56. Regev G, Gause DC, Wegmann A (2009) Experiential learning
approach for requirements engineering education. Requir Eng
14(4):269–287
57. Zowghi D (2009) Requirements engineering education and
training: Key challenges and practical solutions. In: Proceedings
123

of the 17th IEEE International Requirements Engineering
Conference, RE ’09, IEEE Computer Society, Los Alamitos,
CA, USA, p 358
58. Auriol G, Baron C, Fourniols JY (2008) Teaching requirements
skills within the context of a physical engineering project. In:
Proceedings of the 3rd international workshop on requirements
engineering education and training, REET ’08, IEEE Computer
Society, pp 6–11
59. Barnes Raymond J, Gause Donald C, Way Eileen C (2008)
Teaching the unknown and the unknowable in requirements
engineering education. In: Proceedings of the 3rd international
workshop on requirements engineering education and training,
REET ’08, IEEE Computer Society, Washington, DC, USA,
pp 30–37
60. Beus-Dukic L (2011) Final year project: A test case for
requirements engineering skills. In: Proceedings of the 6th
international workshop on requirements engineering education
and training, REET’11, IEEE Computer Society, Washington
DC, USA, pp 5–8.
61. Connor AM, Buchan J, Petrova K (2009) Bridging the research-
practice gap in requirements engineering through effective
teaching and peer learning. In: Proceedings of the 6th interna-
tional conference on information technology: new generations,
ITNG ’09, IEEE Computer Society, Washington, DC, USA,
pp 678–683
62. Damian D, Ban A, Cubranic D, Robles L (2005) Teaching
requirements engineering in global software development: a
report on a three-university collaboration. In: 1st International
Workshop on requirements engineering education and training,
REET’05, IEEE Computer Society, Washington DC, USA,
pp 121–127.
63. Danielsen A (2010) Teaching requirements engineering an
experimental approach. In: Proceedings of the Norsk informa-
tikkonferanse conference, NIK, Oslo, pp 77–86
64. Gabrysiak G, Giese H, Seibel A, Neumann S (2010) Teaching
requirements engineering with virtual stakeholders without
software engineering knowledge. In: Proceedings of the 5th
International Workshop on requirements engineering education
and training, REET’10, IEEE Computer Society, Washington
DC, USA, pp 36–45
65. Jones S, Britton C (1997) Using multimedia case study material
for teaching requirements engineering, Tech. rep., University of
Hertfordshire
66. Beus-Dukic L, Myers C (2005) Use and abuse cases. In: Pro-
ceedings of the 1st international workshop on requirements
engineering education and training, REET’05, IEEE Computer
Society, Washington DC, USA, pp 133–141
67. Martin M (2007) Improvisational theatre: an approach to soft
skills for requirements engineers. In: Proceedings of the 2nd
international workshop on requirements engineering education
and training, REET’08, IEEE Computer Society, Washington
DC, USA, pp 56–60
68. Mead N, Shoemaker D, Ingalsbe J (2009) Teaching security
requirements engineering using SQUARE. In: Proceedings of
the 4th international workshop on requirements engineering
education and training, REET’09, IEEE Computer Society,
Washington DC, USA, pp 20–27
69. Garcı́a F, Moreno M (2003) C-requirements specification
teaching. In: Proceedings of the 33rd annual frontiers in edu-
cation, vol 3 of FIE’03, pp 1–6
70. Takako N (2007) Improving the engineering mind in eliciting
requirements. In: Proceedings of the 2nd international workshop
on requirements engineering education and training, REET’07,
IEEE Computer Society, Washington DC, USA, pp 37–41
71. Madhavji NH, Miller J (2005) Investigation-based requirements
engineering education. In: Proceedings of the 1st international
123
Requirements Eng
workshop on requirements engineering education and training,
REET’05, IEEE Computer Society, Washington DC, USA, pp 68–72
72. Nguyen L, Armarego J, Swatman P (2002) Understanding
requirements engineering: a challenge for practice and educa-
tion, Tech. rep., Deakin University, School of Information
Systems
73. Periyasamy K, Qin X, He D (2011) A requirements editor for
teaching requirements engineering. In: Proceedings of the 5th
international technology, education and development confer-
ence, INTED’11, IATED, Valencia, Spain, pp 4092–4100
74. Shubhamangala B, Rao L, Dakshinamurthy A, Singh C (2012)
Ability based domain specific training: a pragmatic solution to
poor requirement engineering in CMM level 5 companies. In:
Proceedings of the IEEE international conference on computer
science and automation engineering, vol 3 of CSAE’12,
Zhangjiajie, China, pp 459–464
75. Romero M, Vizcaı́no A, Piattini M (2008) A simulator for
education and training in global requirements engineering: A
work in progress. In: Proceedings of the 8th IEEE international
conference on advanced learning technologies, ICALT’08, IEEE
Computer Society, Washington DC, USA, pp 123–125
76. Rosca D (2000) An active/collaborative approach in teaching
requirements engineering. In: Proceedings of the 30th annual
frontiers in education, vol 1 of FIE ’00, IEEE Computer Society,
Washington, DC, USA, pp T2C/9–T2C12
77. Salzer HT, Levin I (2004) Spreadsheet-based logic controller for
teaching fundamentals of requirements engineering. Int J Eng
Educ 20:939–948
78. Sindre G (2005) Teaching oral communication techniques in RE
by student-student role play: Initial experiences. In: Proceedings
of the 18th conference on software engineering education and
training, CSEET ’05, IEEE Computer Society, Washington, DC,
USA, pp 85–92
79. Svahnberg M, Gorschek T (2005) Multi-perspective require-
ments engineering education with focus on industry relevance.
In: Proceedings of the 1st international workshop on require-
ments engineering education and training, REET’05, IEEE
Computer Society, Washington DC, USA, pp 88–97
80. Mario ZJC (2010) Communication and traceability game: a way
to improve requirements elicitation process teaching, Revista
Facultad de Ingenierı́a Universidad de Antioquia (56), 213–221
81. Albakry K, Kamalrudin M (2011) Pair analysis of requirements
in software engineering education. In: Proceedings of the 5th
Malaysian conference in software engineering, MySEC’11, Jo-
hor Bahru, Malaysia, pp 43–47
82. Beatty J, Agourida V (2007) Developing requirements engi-
neering skills: a case study in training practitioners. In: Pro-
ceedings of the 2nd international workshop on requirements
engineering education and training, REET’07, IEEE Computer
Society, Washington DC, USA, pp 111–120
83. Huijs C, Sikkel K, Wieringa R (2005) Mission 2 solution:
requirements engineering education as central theme in the BIT
programme. In: Proceedings of the 1st international workshop
on requirements engineering education and training, REET’05,
IEEE Computer Society, Washington DC, USA, pp 73–77
84. Ferrari R, Madhavji NH (2005) Requirements engineering
education for novice software architects. In: Proceedings of the
1st international workshop on requirements engineering educa-
tion and training, REET’05, IEEE Computer Society, Wash-
ington DC, USA, pp 106–110
85. Gabrysiak G (2011) Why should I help you to teach require-
ments engineering?. In: Proceedings of the 6th workshop on
requirements engineering education and training, REET’11,
IEEE Computer Society, Washington DC, USA, pp 9–13
86. Jamaludin NAA, Sahibuddin S (2011) Measurement of rasch
analysis towards requirement engineering education industry
Requirements Eng
perspective. In: Proceedings of the 6th international world sci-
entific and engineering academy and society conference, vol 9,
WSEAS’11, Stevens Point, Wisconsin, USA, pp 298–304
87. Penzenstadler B, Callele D (2010) Prototyping RE experiments
in the classroom—an experience report. In: Proceedings of the
5th international workshop on requirements engineering edu-
cation and training, REET’10, IEEE Computer Society, Wash-
ington DC, USA, pp 7–16
88. Simmons E (2007) Reflection on a successful corporate
requirements engineering training curriculum. In: Proceedings
of the 2nd international workshop on requirements engineering
education and training, REET’07, IEEE Computer Society,
Washington DC, USA, pp 7–16
89. Tsumaki T, Kaiya H, Tahara Y, Yoshioka N, Taguchi K,
Honiden S (2007) Errors and misconceptions in learning. In:
Proceedings of the 2nd international workshop on requirements
engineering education and training, REET’07, IEEE Computer
Society, Washington DC, USA, pp 28–36
90. Yusop N, Mehboob Z, Zowghi D (2007) The role of conducting
stakeholder meetings in requirements engineering training. In:
Proceedings of the 2nd international workshop on requirements
engineering education and training, REET’07, IEEE Computer
Society, Washington DC, USA, pp 48–55
91. Alexander M, Beatty J (2008) Effective design and use of
requirements engineering training games. In: Proceedings of the
3rd international workshop on requirements engineering edu-
cation and training, REET ’08, IEEE Computer Society,
Washington DC, USA, pp 18–21
92. Gabrysiak G, Guentert M, Hebig R, Giese H (2012) Teaching
requirements engineering with authentic stakeholders: Towards
a scalable course setting. In: Proceedings of the 1st International
Workshop on Software Engineering Education based on Real-
World Experiences, EduRex’12, IEEE, pp. 1–4
93. Berry Daniel M, Kaplan Craig S (2010) Planned programming
problem gotchas as lessons in requirements engineering. In:
Proceedings of the 5th international workshop on requirements
engineering education and Training, REET’10, IEEE Computer
Society, Washington DC, USA, pp 20–25
94. Berenbach B (2005) A hole in the curriculum. In: Proceedings of
the 1st international workshop on requirements engineering
education and training, REET’05, IEEE Computer Society,
Washington DC, USA, pp 62–67
95. Beus Dukic L, Alexander I (2008) Learning how to discover
requirements. In: Proceedings of the 3rd international workshop
on requirements engineering education and training, REET ’08,
IEEE Computer Society, Washington DC, USA, pp 12–14
96. Bray IK (2004) Experiences of teaching problem frame based
requirements engineering to undergraduates. In: Proceedings of
the 26th international conference on software engineering—
W4S Workshop ‘‘1st international workshop on advances and
applications of problem frames (IWAAPF 2004)’’, pp 17–20
97. Davis AM, Hickey AM, Chamillard A (2005) Moving beyond
the classroom: Integrating requirements engineering research &
education to improve practice. In: Proceedings of the 1st inter-
national workshop on requirements engineering education and
training, REET’05, IEEE Computer Society, Washington DC,
USA, pp 78–87
98. Lami G (2005) Teaching requirements engineering in the small:
an under-graduate course experience. In: Proceedings of the 1st
international workshop on requirements engineering education
and training, REET’05, IEEE Computer Society, Washington
DC, USA, pp 128–132
99. Hoffmann A (2008) Teaching soft facts in requirements engi-
neering using improvisation theatre techniques. In: Proceedings
of the 3rd international workshop on multimedia and enjoyable
requirements engineering—beyond mere descriptions and with
more fun and games, MERE ’08, IEEE Computer Society,
Washington, DC, USA, pp 1–3
100. Ozkaya I, Akin O, Tomayko JE (2005) Teaching to think in
software terms: An interdisciplinary graduate software require-
ment engineering course for AEC students. In: Proceedings of
the international conference on computing in civil engineering,
American Society of Civil Engineers
101. Knauss E, Schneider K, Stapel K (2008) A game for taking
requirements engineering more seriously. In: Proceedings of the
3rd international workshop on multimedia and enjoyable
requirements engineering, MERE ’08, IEEE Computer Society,
Washington DC, USA, pp 22–26
102. Liu L, Jin Z (2008) Balancing academic and industrial needs in
RE courses. In: Proceedings of the 3rd international workshop
on requirements engineering education and training, REET’08,
IEEE Computer Society, Washington DC, USA, pp 15–17
103. Nakatani T, Tsumaki T, Tamai T (2010) Requirements engi-
neering education for senior engineers: Course design and its
evaluation. In: Proceedings of the 5th international workshop on
requirements engineering education and training, REET’10,
IEEE Computer Society, Washington DC, USA, pp 26–35
104. Jamaludin NAA, Sahibuddin S (2012) Challenges of project-
based learning towards requirement engineering. Int J Comput
Appl 50(3):1–5
105. Jamaludin NAA, Sahibuddin S (2011) Development strategy
using cognitive domain in e-requirement engineering learning
system. Int J Comput Sci Issues 8:318–322
106. Thiry RQ, Marcello G (2010) Development of a game to support
the teaching of requirements engineering: the requirements
island, In: Proceedings of SBGames, SBC, Florianópolis, Brazil,
pp 358–361
107. Romero M, Vizcaı́no A, Piattini M (2008) Using virtual agents
for the teaching of requirements elicitation in GSD. In: Pro-
ceedings of the 8th international conference on intelligent virtual
Agents, IVA ’08, Springer, Berlin, pp 539–540
108. Romero M, Vizcaı́no A, Piattini M (2009) Teaching require-
ments elicitation within the context of global software devel-
opment. In: Proceedings of the Mexican international
conference on computer science, ENC ’09, IEEE Computer
Society, Washington, DC, USA, pp 232–239
109. Sallim J (2005) Requirement engineering for enterprise appli-
cation development: seven challenges in higher education
environment. In: Proceedings of the 2nd World Enformatika
conference, WEC’05, pp 101–104
110. Sikkel K, Daneva M (2011) Getting the client into the loop in
information systems modelling courses. In: Proceedings of the 6th
workshop on requirements engineering education and training,
REET’11, IEEE Computer Society, Washington DC, USA, pp 1–4
111. Zowghi D (2009) Teaching requirements engineering to the
Bahai; students in Iran who are denied of higher education. In:
Proceedings of the 4th international workshop on requirements
engineering education and training, REET ’09, IEEE Computer
Society, Washington DC, USA, pp 38–48
112. Armarego J, Minor O (2005) Studio learning of requirements:
towards aligning teaching to practitioner needs. In: Proceedings
of the 1st international workshop on requirements engineering
education and training, REET’05, IEEE Computer Society,
Washington DC, USA, pp 111–120
113. Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wes-
slén A (2000) Experimentation in software engineering: an
introduction. Kluwer, Norwell
114. Mead NR, Stehney T (2005) Security quality requirements
engineering (SQUARE) methodology. ACM SIGSOFT Softw
Eng Notes 30(4):1–7
115. Glass LR (1998) Software runaways: monumental disasters.
Prentice Hall, New Jersey
123

116. Glass LR (2002) Software engineering: facts and fallacies.
Addison-Wesley, Boston
117. Benestad HC, Arisholm E, Sjøberg DIK (2005) How to recruit
professionals as subjects in software engineering experiments.
In: Proceedings of the 28th information systems research con-
ference in Scandinavia, IRIS’05, Department of Information
Systems, Agder University College, Kristiansand, Norway
118. Mich L, Anesi C, Berry DM (2004) Requirements engineering
and creativity: An innovative approach based on a model of the
pragmatics of communication. In: Proceedings of requirements
engineering: foundation of software quality, REFSQ’04
119. Acuña ST, Gómez M, Juristo N (2009) How do personality,
team processes and task characteristics relate to job satisfaction
and software quality?. Inform Softw Technol 51(3):627–639
120. Carrillo de Gea JM, Nicolás J, Fernández Alemán JL, Toval A,
Ebert C, Vizcaı́no A (2011) Requirements engineering tools.
IEEE Softw 28(4):86–91
121. Ramesh B (1993) Process knowledge based rapid prototyping
for requirements engineering. In: Proceedings of IEEE interna-
tional symposium on requirements engineering, pp 248–255
122. Lichter H, Schneider-Hufschmidt M, Züllighoven H (1993)
Prototyping in industrial software projects—bridging the gap
between theory and practice. In: Proceedings of the 15th inter-
national conference on software engineering, ICSE ’93, IEEE
Computer Society Press, Los Alamitos, CA, USA, pp 221–229
123. Gabrysiak G, Giese H, Seibel A (2009) Interactive visualization
for elicitation and validationn of requirements with scenario-
based prototyping. In: Proceedings of the fourth international
workshop on requirements engineering visualization, REV ’09,
IEEE Computer Society, Washington, DC, USA, pp 41–45
124. Damian D, Hadwin A, Al-Ani B (2006) Instructional design and
assessment strategies for teaching global software development: a
framework. In: Proceedings of the 28th international conference
on software engineering, ICSE ’06, ACM, New York, USA,
pp 685–690
125. Myint Swe K (2011) Games in Education, Vol. 5, Contemporary
Approaches to Research in Learning Innovations. Sense Pub-
lishers, Rotterdam
126. Houser C, Thornton P, Kluge D (2002) Mobile learning: Cell
phones and PDAs for education. In: Proceedings of the Inter-
national Conference on Computers in Education, ICCE ’02,
IEEE Computer Society, Washington, DC, USA, p 1149
127. Abut H, Ozturk Y (1997) Interactive classroom for DSP/com-
munication courses. In: Proceedings of IEEE International
123
Requirements Eng
Conference on Acoustics, Speech, and Signal Processing, Vol. 1
of ICASSP ’97, IEEE Computer Society, Washington DC, USA,
pp 15–18
128. Yau Stephen S, Gupta Eep KS, Fariaz K, Sheikh I A, Yu W, Bin
W (2003) Smart classroom: Enhancing collaborative learning
using pervasive computing technology. In: Proceedings of the
6th WFEO world congress on engineering education & 2nd
American Society of Engineering Education, ASEE’03, Nash-
ville, Tennessee, USA
129. Elberzhager F, Münch J, Nha VTN A (2012) systematic map-
ping study on the combination of static and dynamic quality
assurance techniques. Inform Softw Technol 54(1):1–15
130. Ampatzoglou A, Charalampidou S, Stamelos I (2013) Research
state of the art on GoF design patterns: a mapping study. J Syst
Softw 86(7):1945–1964
131. Garousi V, Mesbah A, Betin-Can A, Mirshokraie S (2013) A
systematic mapping study of web application testing. Inform
Softw Technol 55(8):1374–1396
132. Zhang H, Babar MA, Tell P (2011) Identifying relevant studies
in software engineering. Inform Softw Technol 53(6):625–637
133. Wohlin C, Runeson P, da MotaSilveira Neto PA, Engström E,
doCarmo Machado I, de Almeida ES (2013) On the reliability of
mapping studies in software engineering. J Syst Softw
86(10):2594–2610
134. Portillo-Rodrı́guez J, Vizcaı́no A, Piattini M, Beecham S (2012)
Tools used in global software engineering: a systematic map-
ping review. Inform Softw Technol 54(7):663–685
135. Fernández-Sáez AM, Genero M, Chaudron M (2013) Empirical
studies concerning the maintenance of UML diagrams and their
use in the maintenance of code: a systematic mapping study.
Inform Softw Technol 55(7):1119–1142
136. Wieringa R, Maiden N, Mead N, Rolland C (2006) Require-
ments engineering paper classification and evaluation criteria: a
proposal and a discussion. Requir Eng 11(1):102–107
137. Easterbrook S, Singer J, Storey M-A, Damian D (2008)
Selecting empirical methods for software engineering research.
In: Guide to advanced empirical software engineering, Springer,
pp 285–311
138. Mateo PR, Usaola MP, Fernández Alemán JL (2013) Validating
2nd-order mutation at system level. IEEE Trans Softw Eng
39(4):570–587