See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/313805822

Requirements' elicitation needs for eLearning Systems

Conference Paper · December 2016

DOI: 10.1109/ICENCO.2016.7856459

CITATIONS READS

4 862

3 authors:

Nancy Rizk Eman Nasr

Arab Academy for Science, Technology & Maritime Transport, Cairo, Egypt 36 PUBLICATIONS   134 CITATIONS   
6 PUBLICATIONS   16 CITATIONS   
SEE PROFILE
SEE PROFILE

Mervat H. Gheith

58 PUBLICATIONS   304 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Requirements Engineering for Service and Cloud Computing View project

ArabAgent View project

All content following this page was uploaded by Nancy Rizk on 07 November 2017.

The user has requested enhancement of the downloaded file.

 Requirements’ Elicitation Needs for eLearning
Systems
Nancy M. Rizk, Mervat H. Gheith
Information Systems & Technology Department, Institute of
Statistical Studies and Research (ISSR), Cairo University,
Egypt
nancyrizk@gmail.com, mervat_gheith@yahoo.com
Abstract—Electronic Learning, or more popularly known as
eLearning, is generally defined to be the use of technology in the
delivery of education or training. eLearning Systems (eLS) are
now integral parts of educational organizations. eLS are diverse
in nature and size. They are nowadays integral parts also of some
commercial or governmental organizations as they are cost-
effective means of delivering training to employees. With the
diversity of people using eLS, there is a need for continuous
improvement, and software development teams need to better
understand the stakeholders’ requirements for faster delivery,
enhancement, or personalization of eLS. Requirements elicitation
is an activity within requirements engineering that is concerned
with discovering needs of stakeholders, either for software
development from scratch or evolution. In this paper we identify
the special properties of eLS that characterize them from other
software systems to help with better understanding of such
domain, discuss the special requirements elicitation challenges
that the special properties introduce, and introduce the main
current requirements elicitation approaches used for the domain.
Our research so far revealed that there are very limited
approaches that are especially tailored for such domain. Hence
we propose in this paper the use of crowdsourcing, which means
exploiting the power of the crowd in performing tasks, as a new
approach for eliciting requirements of eLS, the paper presents a
framework of the necessary elements needed under the umbrella
of this new approach to fill in the identified current research gap
in the domain.
Keywords—Requirements engineering; RE; requirements
elicitation; electronic learning; eLearning.
I. INTRODUCTION
Electronic Learning, or more popularly known as
eLearning, is generally defined to be the use of technology in
the delivery of education or training [1]. There is actually no
unified definition for eLearning. Another common definition is
the online access to learning resources, anywhere and anytime
[2], [3]. The use of technology in learning has led to mixing
and confusing terminologies for distance education such as,
distance learning, eLearning, online learning and blended
learning [4]. In this paper we will consider all of these terms to
be under eLearning.
eLearning Systems (eLS) are now integral parts of
educational organizations. They can be used as web-based
systems for online education, or as an additional tool for
physical on-campus study by providing a supportive eLearning
Eman S. Nasr
Independent Researcher,
Cairo, Egypt
nasr.eman.s@gmail.com
tool for students [5]. eLS are diverse in nature and size. They
are nowadays integral parts also of some commercial or
governmental organizations, as they are cost-effective means of
delivering training to employees [6]. eLS in this paper are not
necessarily meant to be Learning Management Systems (LMS)
which help in the administrative process of education and
training e.g. scheduling, and registration [7]. With the diversity
of people using eLS, there is always a need for continuous
improvement. Improvement in information systems means
better and enhanced software development process, which
started by requirements identification phase or the requirements
engineering (RE) phase. Software development teams need to
better understand the stakeholders’ requirements for faster
delivery, enhancement, or personalization of eLS.
Requirements elicitation is an activity within RE, that is
concerned with discovering needs of a system’s stakeholders in
order to reach the system’s goals [8], either for developing a
software from scratch or for software evolution.
RE is defined by the Department of Defense (DOD) to be
the software development phase that “involves all life-cycle
activities devoted to identification of user requirements,
analysis of the requirements to derive additional requirements,
documentation of the requirements as a specification, and
validation of the documented requirements against user needs,
as well as processes that support these activities” [9]. RE
activities according to Sommerville [10] are: requirements
elicitation, requirements analysis, requirements specification
and requirements validation. We are only concerned with the
requirements elicitation activity in this paper.
According to Zowghi and Coulin [11] there is no one
agreed on definition for requirements elicitation. Hickey and
Davis [12] define it as “learning, uncovering, extracting,
surfacing, or discovering the needs of customers, users, and
other potential stakeholders”. Zowghi and Coulin [11] define it
to be “concerned with learning and understanding the needs of
users and project sponsors with the ultimate aim of
communicating these needs to the system developers”. What is
agreed on is that requirements elicitation is an early software
development activity within the RE phase of any software
development. This activity is very important for the success of
the software development project, where detecting errors at the
initial stages of development can save time and money [13].
According to Standish group report [13], users involvement is
one of the top success factors in any software development
project and this begins to happen during the requirements
elicitation activity. It has a social nature because most of its
activities depend on people [14]. In many current software
systems requirements elicitation is also concerned with the
software system evolution, where user feedback is collected to
help in producing better future versions. Enhancing software is
imperative for all software producers, in order to achieve
higher usage and sales rates. Since software focuses on the
customers who are the final users and its success depends on
their acceptance of it, users’ feedback is usually the source of
evolution in software products from version to version [15],
[16].
This is an investigative paper of the current state of the art
of requirements elicitation for eLS, and present what we
envision to solve current problems. In this paper we identify
the special properties of eLS that characterize them from other
software systems to help with better understanding of such
domain, discuss the special requirements elicitation challenges
that the special properties introduce, and introduce the main
current requirements elicitation approaches used for the
domain. However when it comes to RE for eLS, there is no
special approach of RE for eLS, and because the current used
approaches for requirements elicitation, can’t cope with the
new technologies, and concepts used in eLS such as Web 2.0
technologies. These factors motivated the researchers to find a
new approach of requirements elicitation for eLS. This
approach can take the power of the social web, and the new
Web 2.0 technologies to better understand the stakeholders’
requirements. The paper proposes the use of crowdsourcing as
a new approach for eliciting requirements of eLS, and presents
a framework of the necessary elements needed for this new
approach.
The rest of this paper is organized as follows. Section II
gives an analysis of eLS and their types. Section III provides a
discussion of the special characteristics of eLS. Section IV
discusses eLS’ special needs in their software development.
Section V presents the current requirements elicitation
approaches available in the literature for eLS. Section VI
presents the use of crowdsourcing as a new approach for
eliciting requirements of eLS, by proposing a framework of the
necessary elements needed for this new approach, and finally
Section VII gives the conclusion and future work.
II. ELEARNING SYSTEMS’ TYPES
In this section we provide an overview of the different
types of eLS. Highlighting the types is expected to help in
investigating its special characteristics and its special
requirements in software development processes. These
characteristics will be discussed in the next section.
The literature reveals that there are different types of eLS.
They could be classified based on different criteria, e.g. the
instructor’s role, delivery method, or technology and tools
used. At the same time, this classification is overlapping with
eLearning methods and tools. In this section we discuss all of
them. It is a background part for the classifications of the
different types of eLS to show how broad eLS are, so each one
has different requirements and needs.
A. Instructor-led vs. Self-paced
Instructor-led learning means that there is an instructor who
controls the class with a definite start and end timing with an
exact learning material. Instructor led learning is like the
traditional way of learning; however it can be done using
virtual classrooms and hence considered eLearning. Self-paced
learning is also known as self-regulated learning; it is defined
according to Magill [17] as “an offering in which a learner
determines the pace and timing of content delivery”.
B. Technology Used
• Blended learning: It is a combination of offline (face-to-
face, traditional learning) and online learning in a way that
one compliments the other. It provides individuals with the
opportunity to enjoy the best of both worlds. For example, a
student might attend classes in a real-world classroom
setting, and then supplement the lesson plan by completing
online multimedia coursework. This can give flexibility to
physically attend class once a week and would be free to go
at their own pace (without worrying about scheduling
issues).
• Distributed/Distance learning: The term is often used to
describe the process of providing access to teaching
geographically distant learners [18]. With the use of
computers the term can be defined as “the delivery of
education, the instructional delivery including an instructor
who is physically located in a different place from the
learner, as well as possibly providing the instruction at
different times [18].
• Mobile learning: it is defined by “Any sort of learning that
happens when the learner is not at a fixed, predetermined
location” e.g. PDA, iPod, portable PlayStation, or Mobile
phones [19].
• Social learning: is a new term that emerged after the rise of
the social media and social networks. Corporate
organizations have realized the power of social learning and
encouraging their employees to interact more within this
domain. Employees collaborate and network to discuss
problems, queries, and experiences. Social collaboration
platforms are also built within the LMS so that the learners
do not have to discuss on public platforms.
• Game-based learning: It is a competitive activity in which
students are set educational goals intended to promote
knowledge acquisition. The games may either be designed
to promote learning or the development of cognitive skills,
or else take the form of simulations allowing learners to
practice their skills in a virtual environment [2].
• Personalized learning: the tailoring of pedagogy, curriculum
and learning environments to meet the needs and learning
styles of individual learners. Personalization is broader than
just individualization or differentiation in that it affords the
learner a degree of choice about what is learned, when it is
learned and how it is learned [2].
• eLearning 2.0: it is based on the use of Web 2.0 concept
which is read and write web with its applications e.g. social
networks, blogs, wiki [20]. The social media and digital
technologies are changing the way in which students learn
 and interact as they provide a platform for social learning to
take place [21].
C. Management Systems
• Learning Management Systems (LMS): help in the
administrative process of education and training. It is used
by managers, administrators, instructors, and learners. It is
used to schedule, register, bill, monitor and track learners
through courses and other learning events, manage training
programs and compile statistics and reports [7], e.g.
Moodle, BlackBoard.
• Learning Content Management Systems (LCMS) combine
the administrative and management dimensions of a
traditional LMS with the content creation and personalized
assembly dimensions. LCMS are used for authoring,
approving, publishing, and managing learning content,
which is referred to as Learning Objects (LO) [22].
III. THE SPECIAL CHARACTERISTICS OF ELEARNING SYSTEMS
eLS are special software systems. Studying their
characteristics can help us better understand what is special
about them, in order to best develop this type of systems. This
section presents the special properties of eLS, which are mainly
as follows:
• Stakeholders’ number. We discussed previously how
social learning and eLearning 2.0 increased the number of
eLearning users. We also discussed how the evolution of
the Web technologies led to a great increase in the number
of people using eLS. Users are learners, instructors, and
management.
• Collaboration issue. eLS require continuous collaboration
between students, learners and courses contents, because
learning is a social process [23]. Figure 1 illustrates some
forms of interactions between eLS’ participants. Improving
the social interaction in eLS lead to more satisfaction in the
learning process [24], which in turn leads to the success of
the eLS under use. Collaboration can take many forms like
open discussion, Storytelling, forums, project work, and
brainstorming [24].
• Diversity (background, culture, regulations and
geographical): we mentioned the large number of
stakeholders for the eLS and the different forms and types
of them, this leads to stakeholders’ diversity in different
aspects [5], [25], [26]. Geographical diversity as learners
and instructors may reside in different locations in the same
country or different countries. The diversity in location
leads to diversity in culture of people, their background, and
the regulations that control each country.
IV. THE SPECIAL REQUIREMENTS OF ELEARNING IN SOFTWARE
DEVELOPMENT
eLS have special nature, the paper discussed its special
characteristics in the last section. Therefore there must be
special requirements during RE phase in software development
process. These requirements are discussed next.
• Social aspect requirements: eLS as we mentioned before is
a social-based systems. It depends on the social interactions
Fig. 1. eLearning participants’ interactions.
and participation between the participants [27]. Improving
social interaction in eLS can improve user satisfaction [24].
Social interactions involve more collaborative activities.
Social aspects needs in eLS are translated in RE process as
social aspects requirements.
• Learner-centered design: since eLS’ main users are the
learner, so RE must focus on the learner requirements. As
we mentioned in the previous part, learning depends on the
social interactions. Accordingly, effective interaction is
needed between user and the application to reduce the time
needed. Thus, the eLearning developer must produce
accurate learner-centered design that suits learners since
learners have various learning styles, behaviors and
expectations towards the system [24], [28].
• Internationalization of requirements: is a process of
developing an adapted software application to various
languages and regions without engineering changes [29].
Currently, some educational institutions provide, or others
wish to provide, cross-countries learning material and
eLearning tools for multicultural students. To support their
learning activities, these institutions frequently use eLS
which should grant specific internationalization features
[29].
• Pedagogical requirements: according to Hammad et al. [30].
There are three elements for the success of eLS; Pedagogy,
technology, and learners capability. Pedagogical
requirements are important in eLS. It covers the
pedagogical aspects of eLearning such as learning theories
and models, assessment, learner behavior.
• Non-Functional Requirements (NFRs) (sustainability,
accessibility, flexibility, usability/UseCase Description
(UCD): Another important requirements in any systems and
so the eLS are NFRs [30].
V. REQUIREMENTS ELICITATION APPROACHES FOR ELEARNING
SYSTEMS
After analyzing the eLS nature, its special characteristics
and requirements, there is a need for a new approach to follow
when eliciting requirements for eLS. We searched for
publications talking about special requirements elicitation
approach for eLS, which can deal with its special
characteristics and requirements, but we couldn’t find any.
However, we found approaches taken from other disciplines,
researchers used for requirement elicitation for eLS. We
discuss some of these approaches in this section.
A. Activity Theory Hosseini et al. [38] provided initial work on investigation of
According to Fuentes-Fernández et al. [31] Activity Theory crowd and crowdsourcing features in requirements elicitation
(AT) is an approach for the analysis of social groups that and proved this investigation in their next paper using focus
focuses on their contextualized acts. People are involved in a groups and experts survey [39]. Table 1 gives a comparison
physical and socio-cultural context and their behavior cannot between crowdsourcing-based and the current requirements
be understood outside of it. At the same time, they actively elicitation approaches. The comparison criteria are based on
interact with the environment and change it. These crowd and crowdsourcing features found in [38], also
communication acts with the environment are called activities, supported by Groen et al. [40], in addition to the before
and their contexts activity systems. AT is used in requirements mentioned requirements elicitation approaches found in [11],
elicitation as it is a social activity, and it is also used in learning [31], [33], [32]. It is worth saying that this is a broad view
as well. AT satisfies the social and collaborative aspect of eLS. comparison not a specific one for each technique because it’s
not the target of the paper, but we need it to highlight the
B. Design Thinking importance of crowdsourcing. Below is an explanation of the
Design Thinking (DT), according to Soledad et al. [32], is a comparison found in Table 1.
group of user-centered techniques and tools that supports an • Number of users involved: most of traditional requirements
iterative process to produce solutions to face real challenges. elicitation techniques deal with small number of
An experience in refining requirements for an LMS is reported. stakeholders. However crowdsourcing approach is mainly
Users were involved in the DT process. As a result, DT was based on the use of large number of people so in systems
considered adequate to be applied in the refinement of such as eLS with large number of users, it’s better to use the
requirements in LMS. According to Souza and Silva [33]The crowdsourcing approach.
DT process is divided into three phases. The first one is called
immersion and it is responsible for gathering, analyzing and • Unknown stakeholders: it happens at the time of
summarizing data. The second is called ideation and is requirements elicitation in software development that a
responsible for defining the users’ profile that will cooperate in number of stakeholders maybe unknown for the developing
the creation of the solution based on innovative ideas. Finally, team. Late stakeholders’ discovery can lead to imprecise
the third phase is called prototyping, and it is the ability to requirements elicitation process. Crowdsourcing-based
represent the ideas concretely to promote the validation of the approach using supportive tools can help in this issue of
proposed solution. stakeholders’ discovery. Some new eLS can have unknown
users, thus requirement elicitation using crowdsourcing
VI. A FRAMEWORK OF THE PROPOSED REQUIREMENTS approach is a way that can help. Cost of activity: current
ELICITATION APPROACH FOR ELS requirements elicitation approaches are time-consuming and
The aforementioned approaches of requirements elicitation require professional staff, which lead to high cost
can satisfy the eLS needs during the software development requirements elicitation process. On the other hands the use
process, but they lack an important aspect which is dealing of crowdsourcing requires fewer numbers of professionals
with huge numbers of participants. Since the new concepts of so it’s less costly than the traditional approaches.
Web 2.0 and the social networking lead to a great increase in • Stakeholders’ diversity: in large systems there are diverse
the number of users of the systems, there is a need for a new stakeholders. Diversity of stakeholders can take many
approach to be used in requirements elicitation for eLS, which forms; geographical diversity, cultural diversity, or
can satisfy the special characteristics. background diversity, this may need more effort from the
Crowdsourcing is a new term that was started in 2006 by development team when using current requirements
Jeff Howe in his article “The Rise of Crowdsourcing” [34]. It is
defined in Merriam Webster as “the practice of obtaining TABLE I. CROWDSOURCING AND CURRENT REQUIREMENTS
ELICITATION APPROACHES COMPARISON
needed services, ideas, or content by soliciting contributions
from a large group of people, and especially from the online Comparison Current requirements Crowdsourcing- based
community rather than from traditional employees or criteria elicitation approaches approach
suppliers” [35]. Crowdsourcing is defined by Howe in a Number of users
another blog post [36] as “crowdsourcing represents the act of Small Large (Crowd)
involved
a company or institution taking a function once performed by Unknown Can't work with unknown There can be unknown
employees and outsourcing it to an undefined (and generally stakeholders stakeholder stakeholders
large) network of people in the form of an open call. This can High because it needs Low because uses
Cost of activity
take the form of peer-production (when the job is performed experts online social tools
collaboratively), but is also often undertaken by sole Stakeholders Great effort needed to deal Can deal with
individuals”. It has been advocated for the domain of diversity with stakeholders diversity stakeholders diversity
information systems’ development, through the involvement of Need of
Low need of motivation High need of motivation
users in documentation, analysis, design, coding, testing and motivation
evaluating the software [37]. These tasks were normally Feedback Partially use feedback Can rely on feedback
conducted by the software company or the contracted people, analysis analysis analysis
but now it can be outsourced to anyone in the general public Quality
High quality standards Low quality standards
considerations
[37].
 elicitation approaches. It requires the use of different
techniques, or better communication skills. Crowdsourcing-
based approach can solve this issue because every
stakeholder feels free to communicate with his own way,
using the available social tools.
• Need of motivation: the use of crowdsourcing-based
approach requires motivating the stakeholders because
stakeholders are volunteering the involvement in
requirements elicitation process. On the other hands current
approaches require less motivation because the developing
team interacts more lively with the stakeholders.
• Feedback analysis: new systems have users’ feedback
mechanism. The use of users’ feedback is very important
and considered as one of the powerful tools of
crowdsourcing-based approach. Conversely current
approaches don’t use feedback analysis in its techniques.
• Quality considerations: quality standards are well
established in current approaches, however in
crowdsourcing-based approach quality standards are not
followed because of the involvement of crowd in
requirements elicitation process.
After recognizing the features of current and
crowdsourcing-based approaches in requirements elicitation.
We can conclude from the previous comparison that
crowdsourcing based approach greatly matches eLS special
characteristics, so we believe that the use of a new approach
based on crowdsourcing can help in the requirements
elicitation process for eLS.
Figure 2 illustrates the crowdsourcing necessary elements,
which are needed in the requirements elicitation approach to be
used to satisfy what is special about eLS. The need for an
interactive way to use the power of social networking, applying
text mining tools and receiving users’ feedback are important
elements for the needed requirements elicitation approach.
Below is an explanation for each attribute in the framework.
• Interactivity: Interactivity between the eLearning
participants (Instructor, learner, course, and management) is
a must. Interactivity leads to better course results for
learners [41], and to know the participants’ opinions in the
different modules of the eLS, e.g. course material, quizzes,
assignments, scheduling, and eLearning process.
• Social collaboration: eLS have many supportive tools that
help in the continuity of learning process and satisfying its
social nature. Such eLS supportive tools are discussion
forums, social networking applications, or blogs and wikis.
Fig. 2. Framework of the necessary elements in proposed
requirements elicitation approach for eLS.
• Users’ feedback: There must be lots of feedback from
interactivity between eLearning participants in the different
forms of social collaborations that support eLS. Feedbacks
can be on the eLS itself, course and its material, or the
instructor and management. What important in this paper is
the feedback on eLS and its tools.
• Text mining tools: text mining is away for analyzing
unstructured text using data mining techniques. In this case
we use text mining in order to analyze eLearning
participants’ feedback, which exist in the different forms of
interactions happen. Interactions forms can be found in
discussion forums, social networking applications attached
to the eLS, comments spaces underneath blogs posts, or
other different posts. Using text mining techniques will
differentiate the feedback on the eLS and its tools; which is
the focus here; or any other feedback. Important feedback
will go through text mining process in order to extract
users’ requirements to help in requirements elicitation phase
for eLS development or revolution. Users’ feedback can
hide information about good, bad or recommended features,
which simply mean addition, omission or modifications of
requirements [42].
VII. CONCLUSION
eLS are special kinds of systems, which have special
characteristics that should be taken into consideration during
the software development process. The number of
stakeholders, the collaboration nature of eLS, and the diversity
of eLS’ users are among these characteristics. When it comes
to RE for eLS, there is no special approach of RE for eLS, and
because the current used approaches for requirements
elicitation can’t cope with the new technologies, and concepts
used in eLS such as Web 2.0 technologies. These factors
motivated us to find a new approach of requirements elicitation
for eLS. This approach can take the power of the social web,
and the new Web 2.0 technologies to better understand the
stakeholders’ requirements. The new approach will be based on
the crowdsourcing concept to suit what’s special about eLS.
The paper presented an overview and a comparison between
current requirements elicitation and crowdsourcing based
approaches as we envision that it can be useful and offers
solutions for current problems of the eLS domain. The paper
discussed a framework for the necessary elements in the
proposed requirements elicitation approach. As for our future
work, we intend to analyze in details, and evaluate by applying
different case studies, how crowdsourcing can help as a new
approach of requirements elicitation for eLS.
REFERENCES
[1] Australian Flexible Learning framework, "Practical Guide to E-learning
for Industry", http://industry.flexiblelearning.net.au/guide. Accessed
7/4/2016
[2] Epignosis LLC. (2014, Jan) E-learning: Concepts, Trends, and
Applications. http://www.talentlms.com/elearning/elearning-101-
jan2014-v1.1.pdf. Accessed 7/4/2016
[3] B. Holmes and J. Gardner, E-learning: Concepts and Practice.: Sage
Publications, 2006.
[4] Alexandru Pop. Distance learning Portal. [Online].
http://www.distancelearningportal.com/articles/269/blended-learning-e-
learning-and-online-learning-whats-important.html Accessed 7/4/2016
[5] A. D. Alharthi, M. Spichkova, and M. Hamilton, "Requirements [26] V. P. Docebo. [Online]. https://www.docebo.com/2015/04/23/8-types-
Engineering Aspects of ELearning Systems," in Proceedings of 24th elearning-students/. Accessed 10/5/2016
Australasian Software Engineering Conference, Adelaide, SA,
Australia, 2015. [27] E. L. Law, A. Chatterjee, D. Renzel, and R. Klamma, "The Social
Requirements Engineering (SRE) Approach to Developing a Large-
[6] M. Rose. Elearning Industry. [Online]. Scale Personal Learning Environment Infrastructure," in 21st Century
http://elearningindustry.com/blended-learning-ultimately-best-corporate- Learning for 21st Century Skills.: Springer Berlin Heidelberg, 2012, ch.
training Accessed 7/4/2016 Volume 7563 of the series Lecture Notes in Computer Science, pp. 194-
207.
[7] W. Horton and K. Horton, E-learning tools and techniques.: Wiley,
2003. [28] N. Abdul Rahman and S. Sahibuddin, "E-Learning Technology for
Problem-Based Learning Technique: An Introduction of Requirements
[8] C. Mellon. Software Engineering Institute. [Online]. Elicitation Model," , Kuching,Sarawak, 2010.
http://www.sei.cmu.edu/productlines/frame_report/req_eng.htm
Accessed 7/4/2016 [29] J. A. Cos and R. Toval, "Internationalization Requirements for E-
learning Audit Purpose," in Proceedings of Global Engineering
[9] E. Hull, K. Jackson, and J. Dick, Requirements Engineering, 3rd ed.: Education Conference, 2012 IEEE, Marrakech, 2012, pp. 1-6.
Springer-Verlag London, 2011.
[30] R. Hammad, M. Odeh, and Z. Khan, "Towards A Generic Requirements
[10] Ian Sommerville, Software Engineering, 9th ed. USA: Pearson, 2011. Model for Hybrid and Cloud-based e-Learning Systems," in
[11] D. Zowghi and C. Coulin, "Requirements Elicitation: A Survey of Proceedings of Cloud Computing Technology and Science, IEEE 5th
Techniques,Approaches, and Tools," in Proceedings of Engineering and International Conference On Volume 2, Bristol, 2013, pp. 106-111.
Managing Software Requirements. Berlin: Springer, 2005, pp. 19-46. [31] R. Fuentes-Fernández, J. J. Gómez-Sanz, and J. Pavón, "Understanding
[12] A. M. Hickey and A. M. Davis, "A Unified Model of Requirements The Human Context In Requirements Elicitation," Requirements
Elicitation," Journal of Management Information Systems, vol. 20, no. 4, Engineering, vol. 15, no. 3, pp. 267-283, August 2010.
pp. 65-84, 2004. [32] M. P. da P. Soledade et al., "Experimenting with Design Thinking in
[13] Standish Group International, "CHAOS manifesto ," 2013. Requirements Refinement for a Learning Management System," Anais
do Simpósio Brasileiro de Sistemas de Informação, 2013.
[14] J. Fernandes et al., "iThink: A Game-Based Approach Towards
Improving Collaboration and Participation in Requirement Elicitation," [33] C. L.de C. Souza and C. Silva, "An Experimental Study of the Use of
in Proceedings 4th International Conference on Games and Virtual Design Thinking as a Requirements Elicitation Approach for Mobile
Worlds for Serious Applications , 2012, pp. 66-77. Learning Environments," CLEI ELECTRONIC JOURNAL, vol. 18, no.
1, April 2015.
[15] L. V. Galvis Carreno and K. Winbladh, "Analysis of User Comments:
An Approach for Software Requirements Evolution," in Proceedings of [34] J. Howe. (2006, June) Wired. [Online].
the 2013 International Conference on Software Engineering, NJ, USA, http://archive.wired.com/wired/archive/14.06/crowds.html?pg=4&topic=
2013, pp. 582-591. crowds&topic_set= accessed 30/6/2016

[16] W. Jiang, H. Ruan, and L. Zhang, "Analysis of Economic Impact of [35] Merriam Webster. [Online]. http://www.merriam-
Online Reviews: An Approach for Market-Driven Requirements webster.com/dictionary/crowdsourcing accessed 30/6/2016
Evolution," Requirements Engineering Communications in Computer [36] J. Howe. (2006, June) Crowdsourcing: A Definition.
and Information Science, vol. 432, pp. 45-59, 2014. http://crowdsourcing.typepad.com/cs/2006/06/crowdsourcing_a.html.
[17] D. S. Magill, "What Part of Self Paced Don't You Understand," in accessed 15/10/2016
Proceedings of 24th Annual Conference of Distance Teaching and [37] M. Hosseini, K. Phalp, J. Taylor, and R. Ali, "The Four Pillars of
Learning, 2008. Crowdsourcing: a Reference Model," in Proceedings of IEEE Eighth
[18] J. L. Moore , C. Dickson-Deane, and K. Galyen, "E-Learning, Online International Conference on Research Challenges in Information
Learning, and Distance Learning Environments: Are They The Same," Science (RCIS), Marrakech, 2014, pp. 1-12.
Internet and Higher Education, vol. 14, pp. 129-135, 2011. [38] M., Phalp, K. T., Taylor, J. and Ali, R. Hosseini, "Towards
[19] N. Winters , "What is Mobile Learning?," in Big Issues in Mobile Crowdsourcing for Requirements Engineering," in Proceeding of The
Learning.: UNIVERSITY OF NOTTINGHAM, 2006, pp. 5-9. 20th International Working Conference on Requirements Engineering:
Foundation for Software Quality , Essen,Germany, 2014.
[20] T. O’Reilly. (2005) What is WEB 2.0 – design patterns and business
models for the next generation of software. [39] M. Hosseini, K. Phalp, J. Taylor, and R. Ali, "Toward Crowdsourcing
www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web- for Requirements Elicitation: Results from Expert Survey," in
20.html/. accessed 5/5/2016 Proceedings of The 20th International Working Conference on
Requirements Engineering: Foundation for Software Quality,
[21] X. Wang et al., "INTEGRATION OF E-LEARNING 2.0 WITH WEB Essen,Germany, April,2014.
2.0," Journal of Information Technology in Construction, vol. 17, pp.
387-396, September 2012. [40] E. C. Groen, J. Doerr, and S. Adam, "Towards Crowd-Based
Requirements Engineering A Research Preview," in Requirements
[22] M. Nichani. (2001, May) elearningpost. [Online]. Engineering: Foundation for Software Quality, Samuel A. Fricker and
http://www.elearningpost.com/articles/%20archives/lcms_lms_cms_rlos. Kurt Schneider, Eds.: Springer, 2015, pp. 247-253.
accessed 5/5/2016
[41] C. Palazuelos, D. García-Saiz, and M. Zorril, "Social Network Analysis
[23] Learning for sustainability. [Online]. and Data Mining: An Application to the E-Learning Context," in
http://learningforsustainability.net/social-learning/ accessed 7/4/2016 Computational Collective Intelligence. Technologies and Applications,
Costin Bǎdicǎ, Ngoc Thanh Nguyen, and Marius Brezovan, Eds. Craivo,
[24] N. Abdul Rahman and S. Sahibuddin, "Challenges in E-learning: an
Romania: Springer Berlin Heidelberg, 2013, pp. 651-660.
Requirements Engineering Prespective," in Proceedings of the 3rd
International Conference on Computing and Informatics, Bandung, [42] N. M. Rizk, A. Ebada, and E. S. Nasr, "Investigating Mobile
Indonesia, 2011. Applications' Requirements Evolution Through Sentiment Analysis of
[25] K. Goldsworthy and L. Rankine, "Identifying the characteristics of e- Users' Reviews," in Proceedings of 11th International Computer
Engineering Conference (ICENCO), Cairo, 2015, pp. 123-130.
learning environments used to support large units," in Proceedings of
Ascilite Auckland, Auckland, 2009.

View publication stats