See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/320027777

What Questions do Requirements Engineers Ask?

Conference Paper · September 2017

DOI: 10.1109/RE.2017.76

CITATIONS READS
12 4,974

4 authors, including:

Michael Vierhauser Jane Cleland-Huang

Johannes Kepler University Linz DePaul University
66 PUBLICATIONS   617 CITATIONS    247 PUBLICATIONS   6,015 CITATIONS   

SEE PROFILE SEE PROFILE

Smita Ghaisas
Tata Research Development and Design Centre (TCS Research)
71 PUBLICATIONS   439 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Data Driven Requirements Engineering View project

Architectually Significant Requirements View project

All content following this page was uploaded by Smita Ghaisas on 10 July 2018.

The user has requested enhancement of the downloaded file.

 What Questions do Requirements Engineers Ask?
Sugandha Malviya Michael Vierhauser and Jane Cleland-Huang
School of Computing University of Notre Dame
DePaul University, Chicago, USA. South Bend, IN, USA
slohar@depaul.edu mvierhau@nd.edu, JaneClelandHuang@nd.edu
Abstract—Requirements Engineering (RE) is comprised of var-
ious tasks related to discovering, documenting, and maintaining
different kinds of requirements. To accomplish these tasks, a
Requirements Engineer or Business Analyst needs to retrieve
and combine information from multiple sources such as use case
models, interview scripts or business rules. However, collecting
and analyzing all the required data can be tedious and error-
prone and often results in untraceable or missing requirements
or inadequate requirements verification. To understand, and
moreover support requirements engineering activities in practice,
we conducted an online survey with requirements professionals
in the IT industry. Our main goal was to discover the kinds
of queries and artifacts they are interested in. Our analysis
included 29 surveys responses and a total of 159 natural language
queries. Using open coding and grounded theory, we analyzed and
grouped these queries into 9 different query purposes and 53 sub-
purposes, and also identified frequently used artifacts. Analyzing
real-world queries can shed light on the questions requirements
professionals would like to ask and the artifacts needed to
support such questions. This is potentially useful for project-level
planners, and could help them to identify important questions,
proactively instrument their environments with supporting tools,
and strategically collect data that is needed to answer the queries
of interest to their project.
Index Terms—Requirements Engineering Tasks, Requirements
Engineering Questions, Requirements Traceability
I. Introduction
Requirements Engineering is a vital process for creating
high quality software systems [9, 8]. It involves diverse
tasks including requirements elicitation, analysis, validation,
verification, risk management, and change control [18]. A
plethora of different tools, methods, and techniques are needed
to successfully perform these tasks; however, even with proper
tool-support in place, such tasks can be time-consuming and
difficult to perform. Researchers have identified numerous
problems associated with performing tasks as diverse as
stakeholder identification, requirements elicitation and analy-
sis, requirements specification, change management. However,
providing support for such requirements engineering tech-
niques involves collecting and managing information from
many different data sources. For example, performing stake-
holder analysis requires retrieving, collating, and analyzing
information from interview notes, scenarios, goals and other
artifacts [30]. Typically, such artifacts are neither managed by
a single person, nor stored in a single location but distributed
across multiple repositories (e.g., document management sys-
tems, source-code repositories, issue trackers, or application
lifecycle management (ALM) services). Furthermore, artifact
Smita Ghaisas
Tata Consultancy Services
Pune, India
smita.ghaisas@tcs.com
data is sometimes incomplete, inconsistent and important trace
links can be missing [22]. Accessing these data sources and
combining them to produce meaningful and desired results
can therefore be considered a cumbersome and error-prone en-
deavor [19]. Such problems can best be alleviated by strategic
upfront project planning and with appropriate instrumenting
of the environment.
Several user studies [12, 6, 25] have investigated and
collected important questions asked by software practitioners
in various project roles, and then analyzed the questions to
discover the information snippets needed to support their
tasks. These studies have focused primarily on the developer
and activities related to programming and testing and have
not focused on activities such as requirements elicitation,
stakeholder analysis, and change impact analysis. To shed light
on the tasks performed by requirements engineers and business
analysts in practice – and to provide better support for strate-
gic project planning – we conducted an online survey with
requirements professionals in the IT industry. In contrast to
previous studies we focused on discovering questions pertinent
to requirements engineers and business analysts. As a result
of this survey we collected 159 questions from 29 participants
and extracted 9 main purposes and 83 artifact types using open
coding and grounded theory [11, 1].
The goal of the study is to discover different kinds of
questions that business analysts and requirements engineers
are interested in asking to support requirements-related tasks
such as quality assessments, business rule analysis, and change
impact analysis. More specifically, we investigate two research
questions:
• RQ1 : What kinds of queries are useful for requirements
engineers and business analysts?
• RQ2 : What artifact types are required to address their
questions?
We then show how this knowledge can be used to help
project stakeholders strategically instrument their environ-
ments with the project data and tools needed to service their
questions.
The remainder of this paper is structured as follows.
Section II discusses related work and empirical research in
software engineering. Section III describes the survey design
and the procedures for data collection and analysis. Sections
IV- VI report results, describe a usage scenario, and provide a
detailed discussion of our findings. Finally, Sections VII and
VIII discuss threats to validity, conclusions and future work.
 II. Related Work
Several studies in the past decade have explored the kinds of
questions software practitioners are interested in asking. Many
of these studies focused on questions asked by software devel-
opers. In addition, numerous techniques exist for supporting
users in navigating through complex artifacts and information.
Various studies have identified questions that support soft-
ware practitioners’ tasks. For instance, Fritz et al. [6] inter-
viewed professional developers and identified 78 questions that
were of interest to them. They found that answering these
questions required integration of information from multiple
sources including source code, change sets, team information,
and work items. To support co-located software development
teams, Ko et al. [12] collected 21 questions and grouped them
into seven different categories including submitting a change,
and reproducing a failure. Other related studies focused on
specific development tasks, questions for detecting security
vulnerabilities [27], maintaining source code [25] and analyz-
ing bug reports [5]. Begel et al. [3] created a catalog of 145
software analytical questions of interest to Data Scientists.
The questions were gathered through two different surveys
and included participants from three disciplines: development,
testing, and program management. Based on the participants’
roles and interests, the questions were grouped into 12 differ-
ent categories. For example, a category of Testing Practices
covered questions related to unit-testing, test coverage, and
test automation.
These studies all focused on software development activi-
ties, while our interest [15, 16] is in requirements engineering.
Our survey therefore deliberately targets requirements engi-
neers and business analysts and focuses on their associated
activities. We collect questions they perceive as important
and identify data artifacts which are needed to answer such
questions and to support requirements engineering activities.
In similar work, Bouillon et al. [4] conducted a survey
involving practitioners where they identified 29 traceability
usage scenarios such as refining and detailing requirements
and test coverage analysis of specification and code. They
grouped these scenarios into 6 different groups, each pertaining
to a phase of the software development process.
In addition to surveys that were designed to directly elicit
queries of interest from practitioners, other researchers have
sought to observe and understand requirements engineering
activities. For example, Zowghi and Coulin [30] studied tech-
niques, approaches, and tools used to perform requirements
elicitation. Ramesh and Jarke [21] conducted an extensive
empirical study of RE practices across different organizations.
They first identified specific information needs for software
engineering tasks, and then proposed traceability models that
would provide the necessary support for answering a wide
range of queries. They documented their findings as a set
of three reference level traceability meta-models describing
artifact types and traceability links.
All of these studies provide invaluable insights into the
information needs of project stakeholders according to their
specific roles and responsibilities. Such knowledge drives
strategic decisions about which data should be collected, when
project stakeholders should invest time and effort constructing
trace links, which scripts should be created to retrieve data
from individual tools, and which tools are needed to col-
late and present data to developers, requirements engineers,
project managers, and other stakeholders. Furthermore, such
studies provide guidance for tool developers and researchers
interested in addressing informational needs. To the best of
our knowledge however, there has been no systematic effort
to understand the kind of questions Requirement Engineers
or Business Analysts working in the Software Engineering
industry are interested in asking.
III. Research Context & Survey Design
We now outline the procedures and guidelines followed in
the design and execution of the survey.
A. Identifying Requirements Tasks
As a preparatory step for the survey, we constructed a list of
common requirements engineering tasks. 15 tasks were iden-
tified primarily through analyzing requirements engineering
text books [13, 23, 14, 29]. Following extensive discussion
we eliminated similar tasks and reduced the list to 11 distinct
tasks that would form the focus of our survey. An overview of
these tasks, with a brief description, can be found in Table I.
B. Subjects
We targeted professional Requirements Engineers and Busi-
ness Analysts working in the IT industry. To recruit partici-
pants we created several different mailing lists. The first was
comprised of authors who had published papers in indus-
try tracks of requirements engineering related conferences.
We included papers from the past five years (2012-2016)
of the Requirements Engineering conference (RE), and the
Requirements Engineering for Software Quality conference
(REFSQ). Only authors affiliated with a concrete IT company
were included. Authors from academia were not included.
As a result, we created a list of 101 potential candidates.
The second list, containing contact information for 23 people,
was comprised from our personal networks of professional
Business Analysts and Requirements Engineers.
Moreover, to reach additional participants, we
used the professional social network site LinkedIn
(http://www.linkedin.com) to advertise the study in virtual
practitioner groups. We posted our survey information
and link to three different groups: the IREB Certified
Professional for Requirements Engineering (2,523 members),
the Requirements Engineering Specialist Group (2,093
members), and the Requirements Engineering Forum of
Asia-Pacific (737 members). For this part we used a separate
survey link for collecting responses.
C. Survey Design & Process
Interviews and surveys are both popular techniques for
performing empirical research [26]. We decided to conduct an
TABLE I: RE Tasks included in our study. The number in
parenthesis reflects the number of queries collected for each
of the selected task.
Task Description
Elicitation & This task involves working with stake-
Specification (43) holders such as customers to understand
the business domain, discover, and specify
functional requirements and give adequate
inputs to architects so that they can design a
technical solution that meets requirements.
Business Rules This task involves capturing and document-
Analysis (15) ing business rules such as regulations and
company policies. It also involves under-
standing the implicatioons of the rules.
Quality Assessing the quality of individual require-
Assessment (38) ments and the overall specification.
Safety In a safety-critical system, failure or mal-
Assessment (5) function may result in death, serious injury,
severe damage, or environmental harm. In
the safety-assessment task, an analyst as-
sesses whether identified hazards have been
fully mitigated [17].
Threat Identifying security threats and specifying
Modeling (0) security requirements.
Performance Dealing with requirements related to scala-
Requirements (6) bility, availability, performance etc.
Change Impact Determining those parts of a software sys-
Analysis (12) tem that might affected when a certain
change is performed [2].
Coverage Evaluating the extent to which requirements
Analysis (2) are covered in the design.
Compliance Ensuring that external regulatory codes have
Analysis (4) been fully addressed by the requirements.
User Acceptance This task involves testing against speci-
Testing (16) fied requirements and proving that the cus-
tomers’ needs are met by the developed
software.
Project Managing requirements including priorities
Management (31) and schedules.
online survey as this minimizes the effort for collecting data,
increases the number of potential participants, and allows sub-
jects to answer the questions according to their own schedule
and availability [20]. As an incentive for participating in the
study, we offered to share study results.
As recommended by Yin [28] we performed a pilot
study with a couple of experienced Requirements Engi-
neers/Business Analysts. Based on the responses received,
we made small updates to our survey. For example, in our
initial survey we included several sample project queries to
familiarize participants with the kinds of queries a software
practitioner could ask about their task such as “Have any
components repeatedly failed acceptance tests?” However, we
found that users tended to closely mimic the examples, rather
than creatively thinking of their own queries. We therefore
replaced the examples with more descriptive scenarios (cf.
Table II) and associated queries and then asked the participants
to think of their own scenarios and associated queries.
We also reduced the number of tasks from three to one,
in order to increase the chance that participants would com-
plete the survey [26, 20]. We removed a question that asked
participants how they currently performed each task as this
was overly time-consuming to answer while providing limited
benefit to the study. Finally, we limited the maximum number
of queries per task for each user to five and reduced the widget
space for entering the query text to five lines so as to avoid
verbose queries.
The final questionnaire comprised the following parts:
• An introductory information sheet including an overview
of tasks to be performed and a request for consent to
participate.
• Questions regarding the participant’s experience in the IT
industry and their role as a Business Analysts or a Re-
quirements Engineer. Participants without this experience
were excluded and directed to the end of the survey.
• A list of commonly performed requirements tasks as
shown in Table I. Participants were asked to select a task
they were familiar with.
• A series of questions about artifacts and tools used to
perform the selected task, the difficulty level for accom-
plishing the task, and frequency at which the task was
performed.
• Two sample project scenarios with example questions.
These are shown in Table II.
• A prompt for up to five useful queries associated with a
scenario relevant to the participant’s work.
• An option for the participant to select another task and
to provide additional queries.
• A request for additional comments about the study and
to optionally provide contact information for follow-up
purposes.
The online questionnaire was created using Qualtrics Sur-
vey. The survey link1 was shared through a recruitment email.
D. Analysis
The goal of the survey was to collect requirements related
questions from practitioners, to categorize them based on the
purpose of the query, and to identify artifact types needed
to support the query. From the completed surveys, the first
researcher filtered out surveys containing no queries and those
received from participants with no industry and requirements
engineering experience. The remaining surveys were consid-
ered valid responses.
1 http://depaul.qualtrics.com/SE/?SID=SV_7aMFE3v3CC8gqk5
 Step 1: Step 2: Step 3: Step 4: Step 5:
Requirements Aggregated
Purposes and
Surveys Queries Purposes and
Tasks Artifacts
Artifacts

... refinement

Purposes 54 Purposes 9
11 29 159 Artifacts 83 Sub-Purposes 53
Artifacts 83

Senior Senior
Researcher 2 Researcher 3
cross-check
Researcher 1 and validation

Fig. 1: Research method followed: after creating a list of 11 requirements-related tasks, survey responses and queries were
collected. The anticipated purposes of the queries and the extracted artifacts were then used to iteratively derive codes and
theories.

The first researcher then extracted quantitative data for the the end of the survey.
queries provided by the participants into a spreadsheet (cf. We followed the same approach for queries that were
Fig. 1 - Step 3). We then used open coding [24, 11] to analyze deemed out of scope, not useful, or provided too little in-
the queries and extract relevant information. In particular, we formation on either the involved artifacts or the purpose.
were interested in two main aspects of the query: (1) the After thoroughly discussing those queries we either removed
purpose of the query, i.e. the intention the user had when them from the set of used queries – if all three researchers
issuing the query and (2) the artifacts referenced by the query. agreed – or discussed their purpose and artifact references
until agreement was reached.
TABLE II: Example Queries presented to Survey Participants In the next step (cf. Fig. 1 - Step 5) we followed the
grounded theory [1, 7] approach to further group and cate-
Scenario Example Queries
gorize the queries according to their purpose. In a step-wise
Project Management - Ray wants to -Is there any schedule refinement all three researchers discussed the coded purposes
know if there are any delayed slippage?
requirements before announcing the -How many requirements and attempted to group them into high-level purposes with
release date of their new phone. What have their status as subsequent sub-groups. The final set of purposes, sub-purposes
will be his query? pending? and artifacts provides guidance for instrumenting real-world
Threat Modeling - Susie and her team -List all the identified projects.
are specifying security requirements for threats for the
the drone remote control system. She authentication application.
thinks she may have missed some of the -Have all the threats been IV. Survey Results
threats related to operator authentication. identified and prioritized
She issues a query to confirm this. What for the drone remote Using the first mailing list comprised of authors of con-
is her query? control system? ference papers in industry tracks, we were able to collect 26
survey responses. From our network of contacts we collected
We selected these two elements as our main focus of in- 13 additional responses for an overall response rate of 33%.
terest with the expectation of identifying frequently addressed There was only a single response from the LinkedIn group. We
purposes and/or artifacts across the different tasks. As a pilot thus were able to collect 40 responses in total. Out of these
for the coding procedure, all three researchers involved in the 40 responses, 11 (6 due to the lack of industry experience
process coded 30 randomly selected queries and discussed the and 5 responses based on our exclusion criteria discussed in
outcome until mutual agreement for both purpose and artifacts Section III-D) were removed, resulting in a final set of 29
was reached. The first researcher then coded the remaining surveys.
queries and the other two researchers, on a frequent basis, Based on the quantitative data collected in the first part of
checked random samples. In cases where the first research our survey we can conclude that we were able to reach a broad
was unsure about either the purpose or the artifact references, spectrum of practitioners and experts with several years of
the query was tagged and discussed by all three researchers. industrial requirements engineering experience (cf. Table III).
In the case of eight queries, additional clarification was sought From our valid 29 surveys we calculated that more than 62%
directly from the participants. This was only possible for 11 of the survey participants can be considered very experienced
cases where participants had provided their email address in with more than 10 years of general IT experience and 31%
TABLE III: Survey participants’ background and experience.
From the total number of 40 participants 6 were removed in
the due to no practical experience and 5 due to missing or
incomplete data.
Experience IT Experience BA Experience
None 4 2
Less than one year 1 1
1-5 years 5 13
6-10 years 8 10
more than 10 years 22 10
Total 40 36
with more than 10 years of business analysis and requirements
engineering.
We were able to collect queries for 10 out of the 11 origi-
nally targeted RE tasks (cf. Table I). None of the participants
selected the Threat Modeling task. Figure 2(a) presents an
overview of the selection rate of the different tasks. Further-
more, participants were asked to rate how frequently they
performed the selected tasks on a three-step scale (i.e., Rarely,
Sometimes, and Regularly). Figure 2(b) provides an overview
of the responses and confirms that most selected tasks were
performed on a regular basis.
As a result of the survey we collected 172 queries, with
each participant providing an average of four queries with a
minimum of 2 and a maximum of 10. Following an initial
screening we found that several queries were either incomplete
or inconclusive. We thus removed 13 queries resulting in a
final set of 159 which we used in the coding step. Our initial
effort to code the intentions of the users issuing the different
queries resulted in 59 different purposes.
In a joint effort we then tried to find and group related
purposes which led to a final aggregate set of 9 high-level
purposes. We categorized queries solely on the query’s
primary aim, even if a participant had initially selected a
different ‘umbrella’ task in the survey. We define query
purposes as follows:
• Business Rule Analysis: Queries where the user was
primarily interested in business related aspects. This, for
example, includes business objectives, business rules,
and business processes. One example query assigned to
this group was “What are the business rule
requirements [for this project]?”.
• Change Impact Analysis: Queries that referred to any
kind of change such as adding new features, change
requests or issues arising due to a change. For example,
“How to minimize system impact with historical data on
such changes?”.
• Metrics: Queries that employed any sort of
measurement as their primary purpose such as counting,
calculating maxima or averages were assigned to this
purpose. For instance, “How many requirements are in
this specification?” would require to perform a count on
requirements.
• Process: Queries related to a process step. For example,
“Is there any defect to test case mapping?”.
• Project Management: Queries related to activities such
as monitoring the project schedule, estimating project
costs or planning releases were added to this group. For
example, “List all requirements that have their status as
pending.”
• Quality: Queries addressing an artifact’s (or its
attributes) quality such as queries looking for undefined
terms or queries addressing quality analysis (cf. [10]).
For example, “Find all ambiguous words in
requirements.”
• Risk Management: Queries related to compliance or
hazard analysis were assigned to this category. For
example, “In what frequency can a hazard happen?”.
• Search: Queries that require searching through a set of
artifacts in order to retrieve the desired information
were assigned to this purpose. The search could either
employ a specific matching rule or perform a text-based
search, e.g., “What previous projects had requirements
describing synchronization between multiple
instruments?”
• Stakeholder Analysis: Queries primarily targeting an
individual (or a group of) customer, supplier, user etc.
such as “Show me the stakeholder(s) with an interest in
a given business rule.” or “For a given requirement,
who are the stakeholders of interest?” were assigned to
this purpose.
To preserve the diverse intentions behind the high-leve
categorization of queries we introduced sub-purposes. These
are depicted in Table IV alongside several example queries
from our survey. A more comprehensive list of gathered
questions and their encodings can be found here: https://
tinyurl.com/ j6flyts
We therefore address our first research question RQ1 by
identifying and reporting questions for Requirements Engi-
neers and Business Analysts.
As a second contribution of our survey we extracted the re-
lated artifacts for each query. In total, we collected 83 different
artifact types. Furthermore, we identified related artifacts for
each purpose. Table V provides a comprehensive overview
of artifacts and associated purposes. We then calculated the
occurrence of each artifact type across all 9 query purposes.
Figure 3 presents the artifacts that were used to address more
than one query responses. The size of each artifact bubble
is proportional to the number of purposes it addresses. We
also evaluated the occurrence of each artifact type across the
complete query set. Not surprisingly, Requirements appeared
in 83 queries, Business Rules in 17 queries, Stakeholder in 17
queries, Requirements Type in 15 queries, Test Cases in 15
queries and Business Processes in 10 queries Other artifacts
such as Change Requests, Release, and Processes appeared in
at least 6 queries. However, 45 artifact types appeared in only
one query.This suggests that a core set of artifacts are common
across many queries; however, project planners must also
consider less frequently used artifacts, that support questions
important to their specific project goals and environments.
We thus, address RQ2 by reporting all the associated artifact
TABLE IV: The nine extracted purposes with their respective Sub-Purposes, and Example Queries taken from the survey
Purpose # Queries Sub-Purposes Examples
Business Planning and Strategy -At what step in a business process X does a business rule A
Business Rationale apply?
Business Rule
7 Search Business Rules -What type of constraints are embedded in this rule?(
Analysis
Search Business Rules Constraints geographic constraints?, deadlines?)
Reverse Engineering Business Rules -List all business objectives.
Business Impact Analysis -Do we know the impact is minor or major and will it affect
Managing Sub-Systems upstream downstream applications?
Managing Release -Does the new enhancement creating any issues in the testing of
Change Impact Product Evolution existing functionalities?
Analysis 12
Rationale -What are the changes? and what are the business modules that
Requirements History are going to be impacted for the change?
Requirements Management -How many systems are impacted by these requirements and
Testing for Finding Issues which business/system process are they associated to?
Document Analysis -How many non-functional requirements are there?
Issue Tracking -Do all requirements have a direct traceability to documented
Project Management business processes?
Metrics 16
Requirements Coverage -How many features exist?
Security Analysis -Are there test cases for all requirements?
User Acceptance Testing
-Is there any defect to test case mapping?
Process Status -How many stories are there not done in the Release?
Process 6
Metrics -How many bugs are there for the Epic called "Support"?

Cost Estimation -List all requirements that have their status as pending.
Critical Path Analysis -Is there any schedule slippage?
Managing tasks -What is the current utilization of resources based on the RAG
Project Planning status of the project?
Project
29 Progress -How many security requirements have their status as pending?
Management
Resource Estimation -Have we estimated the task?
Release Planning -Do we have task prioritized?Critical tasks needs to be
Scheduling performed in a timely manner.
Tracking Project Performance
-Find all ambiguous words in requirements.
Analysis
-Are there any duplicate requirements?
Backward Tracing (origins)
-Are all requirements traced to a stakeholder need?
Completeness
-How many processes are identified as a GAP for this release
Meeting Project Objectives
and do all of them have a technical response identified?
Quality 33 Metrics
-Are there any duplicate requirements?
Quality Control
-Do all of the process steps have requirements defined for them?
Quality Set
-Are functional requirements traced to the relevant function?
Reviewing Requirements
-Did I missed any requirements from stakeholders?
Uncovering Errors or Inconsistencies

-What are regulations to comply with?

-What are the main hazards that can happen to the system?
Risk Compliance Analysis -What can cause a hazard?
Management 8
Hazard Analysis -What are the consequences of a hazard?
In what frequency can a hazard happen?

-Which requirements are related to the requirement "xyz"?

Attributes -How many concurrent users would be there for the application?
Business Rule -Please list the business rules relevant to Travel Insurance
Data Provenance applicable for Multicountry travel.
Identifying Requirements -Does requirement match to a business objective?
Search 36
Finding Matching Requirements -Do previous projects have requirements related to measuring
Project Glossary Extraction total harmonic distortion?
Rationale -Need to know the regulatory compliance requirements pertinent
Uncovering Inconsistencies to Payer process in Healthcare

-For a given requirement,who are the stakeholders of interest?

Stakeholder Identification -What kind of users are going to use the system?
Stakeholder
14 Use Case Analysis -How many bugs have been found during acceptance testing?
Analysis
User Acceptance Testing -For a given business rule, identify the originator of the business
rule
 Project  Management    
Threat  Modeling     User  Acceptance  Tes.ng    
Compliance   0%  
Analysis     Project   Compliance  Analysis    
3%   Management    
12%   Coverage  Analysis   Rarely  
Elicita'on  and  
Specifica'on   Safety     Change  Impact  Analysis   Some.mes    
22%   Assessment   Performance  Requirements    
Business  Rules   2%   Regularly  
Analysis     Threat  Modeling    
15%   Quality   Safety  Assessment  
Assessment     Quality  Assessment    
Coverage  Analysis   17%  
5%   Business  Rules  Analysis    
User  Acceptance   Performance   Elicita.on  and  Specifica.on  
Change  Impact   Tes'ng    
Analysis   Requirements  
12%   3%   0   2   4   6   8   10   12   14   16  
9%  
(a) Distribution of selected tasks. (b) Frequency of performing the task as reported by the partipants.

Fig. 2: Analysis of covered RE Tasks regarding their relative occurrence (a) and frequency (b).

that address the collected questions and by presenting the
artifacts that are important in the context of different query
purposes.
V. A Usage Scenario
Collecting and analyzing queries and uncovering frequently
used artifacts can provide foundations for successfully im-
plementing query support in real-world projects. With the
following scenario we illustrate how this information could
be used. Imagine that a software firm called JMS Corp. is in
the process of developing a product.The company uses diverse
artifacts and respective tools for managing them. Their require-
ments are managed in the JIRA Issue Tracking system, impact
analysis reports of previous projects are stored as an Excel
spreadsheets and test cases stored in their own proprietary
tools. Let us further assume that JMS utilizes the artifacts
identified in Table V to strategically construct the Traceability
Information Model (TIM) shown in Fig. 4. This TIM in-
cludes the artifacts used by our survey respondents to support
“Change Impact Analysis” queries and now guides JMS’
decisions for creating infrastructure, for example, deploying
tools such as TaskTop or Mylyn to integrate heterogeneous
data sources, or to write customized data retrieval scripts to
make such data accessible for querying.
Now consider what happens if a customer submits an
enhancement change request and that the request has been
approved and is opened for implementation. Michael is work-
ing as a Business Analyst and requires information about the
change request.
The TIM displays the possible artifact types that can be used
as well as the associations established between artifacts. This
helps Michael determine what kind of queries are currently
supported in his project environment. For example, he could
consider searching for current features related to the en-
hancement request and retrieve their historical impact reports
to better understand how much effort would be involved in
implementing the change. He could also search for any related
requirements, and if necessary find stakeholders who might
be knowledgeable about the topic of the change request.
The fact that the TIM was established in advance, necessary
data collected and preserved, and the project environment
instrumented appropriately allows Michael to answer questions
that support his current review to enhance an existing feature.
VI. Discussion
A. Task Coverage
Our initial set of RE tasks contained 11 different activ-
ities of which ten were covered by survey responses. We
collected only 2 queries for the least frequently selected task
of Coverage Analysis ranging to 43 queries for the Elicitation
& Specification task. No queries were collected for Threat
Modeling. This is perhaps indicative of the lack of threat
modeling activities performed in general, or could also reflect
lack of diversity in our participant pool. The same holds true
for Coverage Analysis (2 queries) and Compliance Analysis
(4 queries). Although we did include Business Analysts in our
list of potential participants these tasks are only very sparsely
populated. We believe this may be because other types of
project stakeholders such as risk management officers or even
developers are more likely to perform these tasks.
B. Purpose Overlaps
While extracting the query purposes and during the aggre-
gation phase we noticed an overlap of query purpose and sub-
purpose. This means that a single query may serve multiple
purposes. For instance, we defined Metrics as a purpose but
also as a sub-purpose under Process. Consider the query,
How many bugs are there for the Epic called “Support”?
whose immediate objective is Process, but achieving this
objective requirements counting bugs. Therefore Metrics was
recognized as a sub-purpose. We deliberately refrained from
further aggregating or merging such cases since our intention
 Change Change
Impact Requests Metrics
TBD Items
Analysis

Process
Test Status

Business
Rules

Stakeholders Release
Analysis
Test Cases
System
Specification
System
Design
Business Rule
Analysis
Bugs

Requirements
Search Project
Business
Objective Management
Model/Goals

Reqs. Type

Constraints

Stakeholder Processes

Process Reqs.
Status
Steps
Reqs.
Quality
Attributes
Business
Processes

Quality Risk
Trace Data Management

Fig. 3: Overview of frequently used artifact types and their occurrence in the different purposes.

was to preserve the original motivation when issuing the
query. We furthermore noticed that some of the sub-purposes
were across multiple purposes. For instance, the sub-purpose
Rationale is defined under both Change Impact Analysis and
Search.
Likewise, some sub-purposes dominated their respective
categories. For example,13 of the 29 project management
queries were related to progress. Similarly Finding matching
requirements and Finding Business Rules covered 42% of
search queries.

C. Related Sub-Purposes E. Artifacts Type Coverage

Although we identified, a large number of artifact types
Some of the extracted sub-purposes are closely related.
from our query set, only a small subset of them are used across
For example, Finding Matching Requirements and Identifying
the majority of query purposes. Most artifacts addressed only
Requirements both belong to Search. However, the first type
one or two query purpose categories. This is partially due
of query finds matches based on artifact content, for example
to the fact that we did not aggregate similar artifact types.
for queries such as “Which requirements are related to the
However, such aggregation would be possible. For exam-
requirement xyz?”, while the second uses trace links or at-
ple, Hazard Cause, Hazard Consequence, Hazard Occurrence
tributes to identify artifacts based on constraints. An example
Rate, Hazard Type could be consolidated into a composite
is “Which requirements have pending status?” Each type of
artifact such as Hazard or Fault Tree. Here we decided to
query requires completely different project tooling.
report raw artifact references and leave artifact analysis for
D. Purpose and Sub-Purpose Coverage future work.

Another observation was that 61% of queries were rec-
ognized under the query purposes of Search, Quality and
Project Management, and additional 26% of queries were
tagged under the purposes of Change Impact Analysis, Metrics
and Stakeholder Analysis, while only 13% were associated
with the query purposes of Risk Management, Business Rule
Analysis and Process. This can partially be attributed to
overlap between categories (especially ones such as metrics
or search); while results suggest that some tasks are more
common than others, further studies are needed to determine
if this observation is generalizable.
VII. Threats To Validity
Construct validity: The queries collected during the survey
were produced outside the context of real software engineering
tasks and thus need to be evaluated in practice. However
all queries were provided by practitioners with several years
of experience and therefore serve as a good starting point
for further study. Another threat to validity stems from the
interpretation and analysis of the queries. To mitigate possible
misinterpretations three researchers were involved in the pro-
cess. The first researcher conducted the surveys and collected
survey and query data. The other researchers participated in the
TABLE V: Artifact Types Referenced by each Query Group
Business
  Business
  Business
 
Business Business Objectives Model/Business Goals, Business Rules
  Processes
 
Modules
 
Rule Processes, Business Rule Constraints, Business Rules, supported
 by
 
Analysis Reqs., Comply
  derived
 
to
  from
 
Change Business Modules, Business Processes, Business Rules,
Business
 
Impact Change Requests, Functionalities, Impact, Issues, Items, Historical
 Data
  Func,onali,es
  Requirements
 
Analysis Reqs., Stakeholders, Sub-System, System
scope
 
Specifications, Test Cases, implements
  change
 
Metrics Business Processes, Change Requests, Design, Features,
Feature Tree, Reqs., Reqs. Type, Test Cases, Test Goals
  Requirements
  Change
 
describes
 
Scenarios, Test Status, Trace Links, Requests
 
affects
 
Process Bugs, Defects, Epic, Release, Reqs., Reqs. History, submits
 
 
Solution Reqs.,Test Cases, Test Plan, User Stories, System
  Impact
 
Project Budget, Business Objectives Model/Business Goals, Specifica,ons
  Stakeholders
 
tests
  affects
 
Manage- Business Processes, Development Status, Documents,
ment Estimation Metrics, Process Status, Processes, Project
Test
 Cases
  Issues
 
Goals, Project Health, QA Team, Release, Release
Schedule, Reqs., Reqs. Priority, Reqs. Status, Reqs.
Type, Resource Usage, ROI, Schedule, Signature, Fig. 4: A Traceability Information Model constructed to
Signature Status, Task Priority, Task Type, Tasks, Test support “Change Impact Analysis” queries using artifacts
Cases, Version referenced by survey respondents.
Quality Bugs, Business Processes, Business Teams, Functional
Areas, Functions, GAP Analysis, Metrics(Defect
Density, High Impact, System Sensitiveness),
Performance Measures, Process Steps, Processes, Reqs.,
personal network of contacts.
Reqs. Model, Reqs. Quality Attributes, Reqs. Status,
Reqs. Type, Stakeholders, Stakeholder’s Reqs., Regarding external validity and the generalizability of the
Technical response/solution, Test Cases, Trace Data, results we reached out for practitioners in different domains
Transactions, User Profiles, User Stories, with different kinds of skills and experience. In total 29
Risk Hazard Cause, Hazard Consequence, Hazard practitioners participated in our survey working on different
Manage- Occurrence(frequency), Hazard Type, Hazards,
ment Products, Reqs. Type, Stakeholders, System
tasks such as requirements elicitation, business analysis and
Specifications, project management. We are thus confident that our survey
Search Business Objectives Model/Business Goals, Business covers a sufficiently broad area of practice. As part of our
Processes, Business Rules, Components, Constraints, future work we plan to extend the initial survey to reach
Design, Disaster Recovery, Functional Scope additional participants from different domains to cover those
Boundaries, Interface, Items, Metrics(output resistance,
tasks that are currently only sparsely populated with queries.
output impedance, or path resistance), Model, Previous
Projects, Process Steps, Processes, Reqs., Reqs. Quality
Attributes, Resolver Reqs., Reqs. Type, Schedule, VIII. Conclusion and Future Work
Stakeholders, System Specifications, Test Cases,
Stake- Activity Diagram, Bugs, Business Rules, Checklists, In this paper we presented an empirical study to elicit and
holder Data(Critical transactions pertaining to business analyze queries of interest to Requirements Engineers and
Analysis Processes), Reqs., Stakeholders, Systems Model, Business Analysts. The data was collected through an online
System Specifications, Test Cases, Test Status,
survey including 29 requirements professionals with the aim
of understanding the kinds of questions they would like to ask
to accomplish those tasks. In total we collected 172 questions
coding and cross-checked the results in all subsequent steps. and used 159 question for further analysis. From analyzing
In case an e-mail address was provided (on an optional basis) these queries we were able to extract 9 different purposes
we contacted the participant to clarify any uncertainties about and 83 different types of artifacts that our participants were
their queries. interested in. Furthermore, we identified the artifacts that were
Internal Validity: One of the challenges with surveys is used frequently across queries of different purposes; however,
recruiting a target group of respondents. To increase the prob- we did not aggregate the identified list of artifacts. We reserve
ability of receiving more responses, we conducted an online this as a part of the future work.
survey. However, as all the respondents were IT engineers, Furthermore, we aim to collect additional queries for tasks
taking an online survey was not an issue in our case [20]. that were not sufficiently covered and create a comprehensive
We distributed a separate anonymous survey link on LinkedIn set of query purposes with their associated artifacts. We are
Requirements engineering groups. This introduces a risk of confident that providing query support for requirements-related
inviting participants outside the focus group. We however re- artifacts can ease the task of combining and linking project-
ceived a single survey response. All the other survey responses specific information from different sources and in the long run
were received from the requirements engineers recognized simplify these tasks hence improving the quality of the whole
from the requirements related conferences and through our project.
 Acknowledgements
The work described in this paper was partially funded by US
National Science Foundation grants CCF-0959924 and CCF-
1265178.
References
[1] S. Adolph, W. Hall, and P. Kruchten. Using grounded
theory to study the experience of software development.
Empirical Software Engineering, 16(4):487–513, 2011.
[2] R. S. Arnold and S. A. Bohner. Impact analysis-towards
a framework for comparison. In Proc. of the 1993
Conference on Software Maintenance, pages 292–301,
1993.
[3] A. Begel and T. Zimmermann. Analyze this! 145
questions for data scientists in software engineering. In
Proc. of the 36th Int'l Conf. on Software Engineering,
pages 12–23. ACM, 2014.
[4] E. Bouillon, P. Mäder, and I. Philippow. A survey on
usage scenarios for requirements traceability in practice.
In Proc. of the Int'l Working Conf. on Requirements
Engineering: Foundation for Software Quality, pages
158–173. Springer, 2013.
[5] S. Breu, R. Premraj, J. Sillito, and T. Zimmermann.
Information needs in bug reports: improving cooperation
between developers and users. In Proc. of the 2010 ACM
Conf. on Computer supported cooperative work, pages
301–310. ACM, 2010.
[6] T. Fritz and G. C. Murphy. Using information fragments
to answer the questions developers ask. In Proc. of the
32nd ACM/IEEE Int'l Conf. on Software Engineering-
Volume 1, pages 175–184. ACM, 2010.
[7] B. G. Glaser and A. L. Strauss. The discovery of
grounded theory: Strategies for qualitative research.
Transaction Books, 2009.
[8] H. F. Hofmann and F. Lehner. Requirements engineering
as a success factor in software projects. IEEE Software,
18(4), 2001.
[9] E. Hull, K. Jackson, and J. Dick. Requirements engineer-
ing. Springer Science & Business Media, 2010.
[10] IEEE Computer Society. Software Engineering Standards
Committee and IEEE-SA Standards Board. IEEE Rec-
ommended Practice for Software Requirements Specifi-
cations. Institute of Electrical and Electronics Engineers,
1998.
[11] S. H. Khandkar. Open coding, 2009.
[12] A. J. Ko, R. DeLine, and G. Venolia. Information needs
in collocated software development teams. In Proc. of
the 29th Int'l Conf. on Software Engineering, pages 344–
353. IEEE Computer Society, 2007.
[13] G. Kotonya and I. Sommerville. Requirements engineer-
ing: processes and techniques. Wiley Publishing, 1998.
[14] P. A. Laplante. Requirements engineering for software
and systems. CRC Press, 2013.
[15] X. Lian, M. Rahimi, J. Cleland-Huang, L. Zhang, R. Fer-
rai, and M. Smith. Mining requirements knowledge from
View publication stats
collections of domain documents. In Proc. of the 24th
Int'l Requirements Engineering Conf., pages 156–165.
IEEE, 2016.
[16] S. Lohar, J. Cleland-Huang, and A. Rasin. Evaluating
the interpretation of natural language trace queries. In
Proc. of the Int'l Working Conf. on Requirements Engi-
neering: Foundation for Software Quality, pages 85–101.
Springer, 2016.
[17] NASA. NASA Software Safety Guidebook. 2004.
[18] B. Nuseibeh and S. Easterbrook. Requirements engineer-
ing: A roadmap. In Proc. of the Conf. on the Future of
Software Engineering, pages 35–46. ACM, 2000.
[19] P. Pruski, S. Lohar, R. Aquanette, G. Ott, S. Amorn-
borvornwong, A. Rasin, and J. Cleland-Huang. Tiqi:
Towards natural language trace queries. In Proc. of the
IEEE 22nd Int'l Requirements Engineering Conf., pages
123–132, 2014.
[20] T. Punter, M. Ciolkowski, B. Freimut, and I. John.
Conducting on-line surveys in software engineering. In
Proc. of the 2003 Int'l Symp. on Empirical Software
Engineering, pages 80–88. IEEE, 2003.
[21] B. Ramesh and M. Jarke. Toward reference models for
requirements traceability. IEEE Transactions on Software
Engineering, 27(1):58–93, 2001.
[22] P. Rempel, P. Mäder, T. Kuschke, and J. Cleland-Huang.
Mind the gap: Assessing the conformance of software
traceability to relevant guidelines. In Proc. of the 36th
Int'l Conf. on Software Engineering, pages 943–954,
2014.
[23] S. Robertson and J. Robertson. Mastering the require-
ments process: Getting requirements right. Addison-
wesley, 2012.
[24] C. Robson. Real World Research. John Wiley & Sons,
2011.
[25] J. Sillito, G. C. Murphy, and K. De Volder. Questions
programmers ask during software evolution tasks. In
Proc. of the 14th ACM SIGSOFT Int'l Symp. on Founda-
tions of Software Engineering, pages 23–34. ACM, 2006.
[26] J. Singer, S. E. Sim, and T. C. Lethbridge. Software
engineering data collection for field studies. In Proc. of
the Guide to Advanced Empirical Software Engineering,
pages 9–34. Springer, 2008.
[27] J. Smith, B. Johnson, E. Murphy-Hill, B. Chu, and H. R.
Lipford. Questions developers ask while diagnosing
potential security vulnerabilities with static analysis. In
Proc. of the 10th Joint Meeting on Foundations of
Software Engineering, pages 248–259. ACM, 2015.
[28] R. K. Yin. Case study research: Design and methods.
Sage, 2009.
[29] R. R. Young. Effective requirements practices. Addison-
Wesley Longman Publishing Co., Inc, 2001.
[30] D. Zowghi and C. Coulin. Requirements elicitation: A
survey of techniques, approaches, and tools. In Proc. of
the Engineering and Mmanaging Software Requirements,
pages 19–46. Springer, 2005.