See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/254019565

Analysis of students' behaviour in E-learning system

Conference Paper · June 2011

CITATIONS READS

17 5,738

3 authors, including:

Martin Cápay Zoltan Balogh

University of Constantinus the Philosopher in Nitra - Univerzita Konstant’na Filoz… University of Constantinus the Philosopher in Nitra - Univerzita Konstantína Filoz…
48 PUBLICATIONS   171 CITATIONS    71 PUBLICATIONS   345 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

IRNet Project www.irnet.us.edu.pl View project

IRNet – International Research Network View project

All content following this page was uploaded by Zoltan Balogh on 08 March 2017.

The user has requested enhancement of the downloaded file.

 Analysis of Students’ Behaviour in E-Learning
System

Martin Cápay, Miroslava Mesárošová, Zoltán Balogh

Department of Informatics, Faculty of Natural Sciences
Constantine the Philosopher University in Nitra
Tr. A. Hlinku 1, 949 74 Nitra, Slovakia
{mcapay, mmesarosova, zbalogh}@ukf.sk

Abstract—In certain stages of the educational process it is presenting, publishing and sharing learning content and
necessary to verify the quality of the study program that is information, as is the case in many other areas. Special
realized via e-learning. The aim of the research presented in the software called Learning Management System (Figure 1) is
paper was to get relevant feedback on the quality of one module generally used in most institutions providing web-based
of created study program for nurses, specifically the module learning [5].
“Role of a nurse in community care” which was one of the
outcomes of an international project E-learning in Community
Care supported by Leonardo da Vinci. We used two different
research methods, non-anonymous questionnaire and usage
analysis that is in fact a relatively hidden form of gaining the
data about the users in e-course. Our research question should be
formed as follows: Is there a significant difference between the
answers of questionnaire investigation and real behaviour of the
students in the e-course? This question was stated after the first
differences between how students responded in the questionnaire
and what the analysis of log file proved emerged. Comparing the
answers and outcomes of usage analysis we are able to find out
the real process of the participants’ study and eventually
interpret the differences.

Keywords- E-learning, Life Long Learning, E-course, Usage
Analysis, Questionnaire.
I. INTRODUCTION
The economic and social changes in the society in recent
years, mainly in highly developed countries, provoked several
changes also in the field of health care. It influenced its focus
as well as the goals; the governments enforce the community
health care development aimed to health support and
sustainability together with the prevention of diseases. The
modernization of education seems to be inevitable. New
knowledge based on recently discovered facts needs to be
reflected in the curricula – this requires regular innovation of
study programs [1]. However, not only the content needs to be
modernized but also the form has to be adapted to actual trends
and needs of the society. The above mentioned aims can be
accomplished via cooperation between the universities and
companies in the field of e-learning methodology [2][3][4] for
the educational system supported by e-learning offers modern
form of study which is highly flexible from the point of view of
time requirements and material resources. It represents very
good accessibility and easy and direct communication with the
tutor [2]. In recent years e-learning systems have become very
popular in all fields of higher education. The Internet and
related web technologies do offer great solutions for
Figure 1. Structure of an Learning Management System (LMS)
Now that the Internet is recognized as the main platform for
education, web-based applications are preferred when it comes
to educational activities, channels for communications and
systems to access knowledge. LMS are often viewed as the
starting point of any web-based learning program [6]. Many
pedagogues introducing their models of electronic education in
the early 20th century, but they had no sufficient tools to their
effective implementation at that time [7]. An LMS provides the
platform for the web-based learning environment.
Because of the huge number of e-learning systems, and the
availability of a large number of LMSs, one needs a systematic
way, or a tool to evaluate the quality, efficiency, and the
performance of LMSs and make a choice that will satisfy most
or all of the requirements [8]. LMS systems usually have large
number of features and it becomes a tedious task to make a
manual selection. One possibility is to automate this evaluation
process using computer aided techniques. [5].
Advanced Internet technology in health care learning –
including e-learning, web-based learning, online computer-
based educational training, Internet-based learning, and
distance learning – has been widely adopted in many
developed countries [9]. It can provide both – modern forms of
education reflecting the needs of present-day students as well
as the possibility to modify and update the educational content.

II. INTERNATIONAL COOPERATION – NEW POSSIBILITIES

International analysis of community care was carried out
already in 2005. It showed that the Czech Republic, Slovakia,
Poland and Hungary were, more or less, at the same level. At
that time, there were no educational programs which would
effectively prepare employees in this area for their future tasks.
On the other hand, the sphere of health care has already
been well known in the developed countries for many years
Figure 2. Structure of the educational program
and a serious attention has been paid to it thanks to its high
social importance [10]. Such status was also the aim in Czech
The technical part of the courses was realized by the
Republic, Slovakia and Poland so they joined to create the
Department of Informatics, Faculty of Natural Sciences,
study program for post-gradual specialization study and life-
Constantine the Philosopher University in Nitra. The main
long learning of nurses aimed at community care. This was
reason was the fact that the most of the content authors were
possible thanks to the international Leonardo da Vinci project
not digitally skilled enough to create their own e-learning
named E-learning in Community Care (CZ/07/LLP-
courses. The task of the Department of Informatics was not
LdV/TOI/134001) supported by European Commission in
only to create the resources and activities in the e-course and
Brussels.
fill them in with the materials from the authors but also to find
quick and easy way of collecting the materials and other
Project Leonardo da Vinci important information necessary for the course creation from
Due to the project, the supranational partnership of 5 the authors.
institutions (two from Czech Republic, two from Slovakia, one
from Poland) in the V4 countries and France was established. Mutual correspondence and communication were mostly
The coordinator of the project was National Centre of Nursing carried out via e-mail. For the materials in particular, the
and Other Health Professions in Brno. structured MS Word template was used – it was proved as an
applicable and also very convenient solution. Such document
The project reflected the absence of study programs in the contained all the information necessary for the course design in
field of providing the services oriented to health support and a logical and comprehensible structure.
sustainability of individual and society as well. The specific
aim of this two-year project was to create genuine continuity
and interconnectivity with the module-based specialization
study in community care and at the same time to offer suitable
alternative for those students who cannot attend conventional
form of education.
This should be achieved by creation of a new specialized
module-based educational program for distance form of study
and to prepare the modules for e-learning study so that the
nurses-to-be could learn effectively. The outcomes of the
project will be integrated into the continual education system in
particular countries.
E-learning form of study is very useful alternative since it is
acceptable in large groups of students. Web-based learning Figure 3. Sample of one module of e-course
provides them with a new environment that allows them to
develop professional skills and knowledge in self-initiated The structure of online course needs to be easy to follow,
learning [11]. Furthermore, thanks to its English version it is relevant, and learner-centered [12]. That is why we divided
also universally applicable. As the best software solution the each course into three main parts: introductory information,
LMS Moodle environment was chosen - among the main lessons and final part [13]. The elaborated e-courses followed
reasons was the positive experience with its usage for distance this unified structure which was approved in advance by the
as well as presence study at the Department of Informatics. representatives of each partner organization [14].

III. CREATION OF A NEW STUDY PROGRAM
The partners of the project agreed on uniform structure of
the educational program: the basic module, professional
module and a group of special modules - for general nurses and
for midwives separately (Figure 2).
IV. QUALITY VERIFICATION OF CHOSEN MODULE OF
STUDY PROGRAM
Project participants agreed in advance on quality
verification of two chosen modules of their national course
versions. In Slovakia, this was planned to be carried out via
pilot testing of 30 nurses and 30 midwives – the tested modules
were „The Role of a Nurse in the Community Care“ (Figure 3) e.g. range scaling, Lickert scales and open questions with free
and “Breastfeeding counselling”. All participants of the tested answers.
course had finished bachelor’s degree or bachelor’s degree
with specialization, their age was higher than 30 years old Since the questionnaire was realized non-anonymously, we
(Table I.) and most of them having more than 10 years of needed another, more objective sources that would enable us to
active practice (Table II.). evaluate real advantages and disadvantages of designed form of
study. It was necessary to get independent view of overall
TABLE I. AGE CATEGORIES OF PARTICIPANTS progress of the participants and the frequency of their accesses
to electronic resources.
< 30 year-old 31 - 45 year-old 46 – 60 year-old
26 % 62 % 12 %
B. Research methods - Usage analysis
TABLE II. NUMBER OF YEARS OF PRACTICE Very interesting and useful course usage information was
gained also from the log-on file analysis. That can help us
<5 6 – 10 11 - 20 > 20 better understand the behaviour of the student in the e-learning
15 % 6% 32 % 47 %
environment. During the data preparation we took into account
recommendations resulting from series of experiments
To get relevant data, the courses were tested on the students examining the impact of individual steps of data preprocessing
of distance form of study at the Department of Nursing, on quantity and quality of extracted rules [15][16][17].
Constantine the Philosopher University in Nitra as well as the
students of detached department in Prešov and all the The log-on file (Table III.) is created by LMS Moodle
participants were at the time of study actively working in the automatically and contains information of everything that
various health institutions as nurses and/or midwives. happens in the e-course. In order to analyse the data we have to
prepare them in advance – e.g. delete invalid data, convert
It is possible to get the information about the suitability of system time, create categories of actions, etc. After the
the structure of the content and the tests as well as the necessary data preparation we are able to find the model of
quantitative information about the frequency of usage of user’s behaviour in particular e-course - we can follow his
particular e-course modules in several ways. The research took navigation through the course, see what materials were visited
place in the final phase of the project as the analysis of the data and/or the time spent in the course. However, we cannot know
collected within pilot testing - the students were asked to fill in if the opened material was really read and understood.
the questionnaire at the end of the course (see the following
chapter).
TABLE III. ENTRIES OF ADJUSTED LOG-ON FILE
The aim of the research was to get relevant feedback on the
quality of particular module of created study program, Date 11.11.2009 11.11.2009 11.11.2009
Time 10:12:14 10:12:23 10:12:56
suitability of the division of study content and tests, as well as Unix_Time 1257934334 1257934343 1257934376
to get quantitative information on the frequency of usage of IP_Adress 62.168.65.122 62.168.65.122 62.168.65.122
particular e-course modules. There were several possibilities Access Out Out Out
how to verify the information: questionnaire, test (quiz), User_ID Uniq_User_ID Uniq_User_ID Uniq_User_ID
application task or dialogue – we employed them to various Action_Category main_page Reports main_page
extend. More detailed information on particular methods used Action course view user view course view
Action_Inf RNCC user view all RNCC
in our research is as follows.
Source: Moodle log-on file,
RNCC = The Role of a Nurse in the Community Care,
out = out of the university
A. Research methods - Questionnaire investigation
Questionnaire is one of the most frequently used research V. RESEARCH OUTCOMES
methods. It is very often employed in social sciences for quick
collecting of facts, opinions, preferences, values, motives, Comparing the data got by two research methods it is
needs, interests, etc. The experiences show that the possible to get more relevant information of the real process of
questionnaire is effective especially if it is anonymous. participants’ study, eventually also interpret the differences
Answers to open questions are usually more detailed and between the responses, system statistics and real behaviour.
expressed opinions are obviously more realistic than in non- In the following text we are going to deal into more details
anonymous questionnaire. Furthermore, loss of the with the research of “The Role of a Nurse in the Community
anonymousness often causes that the students answer only Care” course.
obligatory closed questions. However, not every time it is
possible and useful to use non-anonymous questionnaire – as it A. Outcomes of questionnaire investigation
was also in our case.
One part of the questionnaire was focused on the forms of
Participants in our research filled in the questionnaire in the distance learning. E-learning as the form of education
beginning of their study (information about the statistical interested quite a lot of participants in spite of the fact that only
sample) and in the final phase (overall evaluation of the quality 13 per cent of them had previous experience with an e-learning
of content, management, etc.). Both questionnaires were non- method of instruction. Most of them preferred e-learning to
anonymous, realized electronically as the activities in the e- traditional or combined education (Table IV.).
course environment. They contained several types of questions,
 TABLE IV. THE PREFERRED TYPE OF DISTANCE LEARNING time they studied. The responses say that most of the
E-learning Traditional, printed Blended learning participants studied in the evening (Figure 5).
via internet material, CD
63 % 23 % 14 %

According to the final questionnaire, almost 37% of the

students printed the electronic materials, 7% printed only some
topics, and the rest studied the materials directly from the
computer screen. Each chapter contained also the list of
additional resources. The outcomes of the questionnaires
showed that 73% of the students used them during their study.
Source: own research
TABLE V. THE PREFERRED TYPE OF STUDY MATERIAL

Type of study material Absolute Absolute Figure 5. Time of studying via e-learning
frequencies frequencies (%)
Text 7 23% As many as 90% of the students stated that they welcomed
Pictures 15 50% the opportunity to study at home. On the other hand, only one
Animation 5 17% third of the students expressed they were active in the forums
Podcast 4 13%
Movies 12 40% (Table VI.), the rest only occasionally.
The pictures and videos as the form of study materials were
preferred in comparison to text, animation and audio (Table TABLE VI. ACTIVITY IN THE DISCUSSIONS IN THE FORUMS
V.).
very often often rarely never
In the questionnaire, there were two questions asked using 3% 30% 57% 10%
Lickert scale for the answers:
Q1: What was your study supported by e-learning like?
B. Outcomes of usage analysis
Q2: How much have you learnt during the study?
Analysis of log-on file made it possible to view the
The chart (Figure 4) shows the number of responses to the students’ activities from more or less independent point of
questions. view. Gained data were visualized using statistical software in
two basic graphs, the interaction plot and web graph.
The interaction plot [10] visualizes interaction frequencies -
Category x Term. The graph shows that in the first phase the
students most often accessed the quizzes. The second most
common activity was submitting the assignments. In the
second phase, the number of accesses to both type of activities
decreased rapidly. However, they still belong among the most
visited course modules (either the resources or activities).
The tests used in the course served to get the feedback of
the quality of students’ theoretical knowledge before their state
exams. The outcome of the course was not a mark but credits
Source: own research that counted into participants’ professional growth. The
Q1 Answer: Very easy (1) … Very difficult (5)
Q2 Answer: Very much (1) … Very little (5)
condition to gain the credits was submitting the obligatory
assignments. Integrated tests were motivating for their learning
Figure 4. Responses on question Q1 and Q2
[2]. Also the fact that the self-test questions were issued from
the oral state exam syllabus was great inhibitor of the students’
Since the participants were at the same time students of
activities. The results might be significantly different if the
Constantine the Philosopher University in Nitra, we were
online program is perceived as relevant and applicable to the
interested in how they perceive the opportunity to study
employee’s professional role in the organization [12].
externally and learn via electronic means. The questionnaire
outcomes say that they recommend this method mainly (60%) In the interaction plot we did not differentiate between the
in life-long learning [18]. roles of participants in the course. More detailed analysis based
on longer time period considering also the users’ roles is
E-learning form of study is available and accessible 24 presented in Figure 6. It visualizes frequencies of activities -
hours a day. Each student has his/her own routines and is used Frequencies of access x Month, during 13 months. There are
to study in different time periods according to individual needs periods of study (Month) on the x axis and observed
[18]. One of the questions in the questionnaire was aimed at the frequencies on the y axis, while one polygon is drawn for each
role (student, teacher and creator of the course).
 Source: own research
Figure 6. Frequencies of Access for student, teacher and creator role
In the following part we are going to describe the results of
association rules analysis, which represents a non-sequential
attitude to the data being analysed. We shall not analyse
sequences, but transactions, i.e. we shall not include the time
variable into the analysis. In our case transaction represents a
variety of visited categories of the course by one user.
Regarding our data, we shall consider one transaction to be the
categories of parts of the course visited by one user for an
observed period of time. The size of the node in the web graph
(Figure 7) represents the support of an element (frequency of
category accesses), the line-width - the support of the rule
(frequency of occurrence of the pair of two consequently
accessed categories) and the brightness of the line - the lift of
the rule (specifies how many times more frequently the visited
categories occurred jointly than in case if they were statistically
independent) [10].
Source: own research
Figure 7. Web graph – visualization of found rules
In the graph (Figure 6), we can see the great difference in
the amount of access logs of the students in comparison with
those of the tutors and the course creators (in the course there
were several tutors as well as creators). In the first months after
enrolling into the course there was almost no or only very low
activity of the tutors. On the other hand, the course creators did
the last modifications and settings in the courses (their activity
was much higher in the preceding period). The greatest activity
of the students can be seen short before the deadlines of
particular activities and the tutors were very active at that time,
too [18]. The log file showed that the most of the course access
logs were from the computers outside the university.
Figure 8. Log on entries categorized by the time
VI. DISCUSSION
The obtained data, both from the questionnaire and mainly
from the adjusted log files of e-courses, were used for finding
out certain rules of behaviour of course participants, using
usage analysis. In some cases, the differences were found in
how students responded in the questionnaire and what the
analysis of log file proved. The possible reason for these
differences may be that they knew the questionnaire was not
anonymous while in case of the study itself they felt more
independently.
a) Difference 1 – On-line study material: According to
the final questionnaire, almost 37% of the students printed the
electronic materials, 7% printed only some topics, and the rest
studied the materials directly from the computer screen.
However, the log file analysis proved that study materials were
one of three least visited types of resources and/or activities.
Study materials were used much less than we expected.
However, that does not mean that the students would not read
them at all. It is possible that they printed or saved them to
their computers at their first display of the material.
b) Difference 2 – Additional materials: The outcomes
of the questionnaire showed that 73% of the students used them
during their study. The web graph (Figure 7), on the other
hand, shows that literature was the least visited category. We
can suppose that additional resources were seen by the great
amount of the students but most of them visited them only
once.
c) Difference 3 – Additional materials: We may judge
according to the responses in the questionnaire that only a few
students participated in the discussions – the results show that
only one third of the students were active in the forums (Table
VI.), the rest only occasionally. But this opposes the
association rules in the web graph (Figure 7) that proves that
the course was mainly used for the communication among the
participants as well as the tool for the assignment distribution
and collecting.
d) Difference 4 – Time of study: In the questionnaires,
half of the students wrote they studied mainly in the evenings.
However, the log file analysis showed that the most access
entries were between 4 pm and 5 pm, i.e. in the afternoon
(Figure 8).
 Another interesting aspect is testing in the e-course. The
research showed that students used tests quite a lot as they
seem to be a good preparation for the state exams (77 % of
them stated that self-tests were most helpful activity). The quiz
reports show that a lot of students repeated the quiz attempts
several times till they gained 100%. The outcomes of self-tests
were displayed for some time for all the students so they could
compare the results which increased the competition among
them [18].
VII. CONCLUSION
At the beginning we stated a question if there is a
difference between the responses of the participants in
questionnaire investigation and real data in the e-course. This
was proved in several cases. We can assume that using only the
questionnaire investigation is not relevant research method for
gaining the information about the process of study in the e-
course. It is likely that students’ individual habits as well as
their attitude to the tutor may considerably influence the
outcomes of the investigation.
Analysis of users’ behaviour via the method of usage
analysis seems to be one of more relevant possibilities how to
describe and visualize real behaviour of the user in the e-
course. On one hand, this method is quite complicated for
interpretation – it requires a professional in the field of
statistics and datamining. On the other hand, data collecting is
automated and is not influenced by any personal factors.
In general, the project is considered to be successfully
finished. Only 3 % of participants presented that such form of
study did not suit them, on the contrary, as many as 80 %
stated that they would like to participate in a course like this
again. Also the project partners expressed high satisfaction
with all created materials. Moreover, gained data will be used
also in realization of another project in the future. At this time,
the courses are available at the educational websites of project
partners and are used in distance education more or less
periodically. Since the project is over, the Department of
Informatics does not manage the courses anymore.
In spite of the fact that the project has already finished, the
cooperation among the Slovak partners continues in another
project, this time at the level of faculties. The project Virtual
Faculty – Distance Learning at the Faculty of Social Sciences
and Health Care of Constantine the Philosopher University in
Nitra started in 2010. Its aim is to create virtual faculty that
would be based on e-learning courses available for the students
24 hours a day, 7 days a week.
ACKNOWLEDGMENTS
This publication is published thanks to the financial support
of the project KEGA 080-019SPU-4/2010 named:
Development of creative potential in e-learning.
REFERENCES
[1] J. Hvorecký, M. Drlík. Enhancing Quality of E-Learning.In:
International Conference ICL 2008 : The Future of Learning -
View publication stats
Globalizing in Education. 2008: Kassel University Press, 2008. pp. 54-
65.
[2] Reimea, M. H., Harrisa, A., Aksnesb J., Mikkelsena J. The most
successful method in teaching nursing students infection control – E-
learning or lecture? Nurse Education Today Volume 28, Issue 7,
October 2008, pp. 798-806
[3] P.I. Pfefferlea, E. Van den Stockb, A. Nauerthc. The LEONARDO-DA-
VINCI pilot project “e-learning -assistant” – Situation-based learning in
nursing educationNurse Education Today Volume 30, Issue 5, July
2010, pp. 411-419.
[4] W. Changa., S. H. Sheen., P. Changa, P. Leea. ,Developing an e-learning
education programme for staff nurses: Processes and outcomes. Nurse
Education Today Volume 28, Issue 7, October 2008, pp. 822-828
[5] N. Cavus. Education technologies of the information age: course
management systems. Extend 2008;2:28.
[6] N. Cavus, H. Uzunboylu,D. Ibrahim. Assessing the success of students
using a learning management system and together with a collaborative
tool in webbased teaching of programming languages. J Educ Comput
Res 2007;36(3).pp. 301–21.
[7] J. Skalka, J., M. Drlík.. Avoiding Plagiarism in Computer Science E-
learning Courses. In Information & Communicaton Technology in
Natural Science Education., 2009, Vol. 16 (2009).
[8] N. Cavus, A. M. Momani. Computer aided evaluation of learning
management systems. Procedia-Soc Behav Sci 2009;1:426–30.
doi:10.1016/j.sbspro.2009. 01.076
[9] E. T.Anderson, Z.B. Mercer. Impact of community health content on
nurse practitioner practice: a comparison of classroom and web-based
teaching. Nursing Education Perspectives 25 (4), pp. 171–175.
[10] Z. Balogh, M. Munk., M. Turčáni, M. Cápay. Usage Analysis in e-
Learning System for Healthcare. 2010. In: The 4th International
Conference on Application of Information and Communication
Technologies AICT2010. Tashkent : IEEE, 2010. pp. 131-136.
[11] J. C. Liang., S. H. Wu. Nurses' motivations for web-based learning and
the role of Internet self-efficacy. Innovations in Education and Teaching
International 47 (1), pp. 25–37.
[12] M. Gabriel, M., S. Longman. Staff Perceptions of E-Learning in a
Community Health Care Organization [Online] In: Online Journal of
Distance Learning Administration, Volume VII, Number III. Available:
http://www.westga.edu/~distance/ojdla/fall73/gabriel73.html, accessed
22. February 2011.
[13] M. Cápay, J. Tomanová. Enhancing the Quality of Administration,
Teaching and Testing of Computer Science Using Learning
Management System. In: WSEAS Transactions on Information Science
& Applications. Vol. 7, no. 9, 2010.
[14] Z. Balogh., M. Turčáni., M. Burianová. . Modelling web-based
educational activities within the combined forms of education with the
support of applied informatics with an e-learning support. In: Proceeding
of the 7th International Conference Efficiency and Responsibility in
Education 2010. Praha: Czech University of Life Sciences, 2010. pp. 14-
24.
[15] M. Munk, J. Kapusta. P. Švec. Data preprocessing dependency for web
usage mining based on sequence rule analysis. IADIS European
Conference on Data Mining 2009, ECDM'09, Algarve, pp. 179-181.
[16] M. Munk, J. Kapusta. P. Švec, M.Turčáni. Data Advance Preparation
Factors Affecting Results of Sequence Rule Analysis in Web Log
Mining. E+M Economics and Management, Vol. 13, No. 4, pp. 143-160.
[17] M. Munk, J. Kapusta. P. Švec. Data Preprocessing Evaluation for Web
Log Mining: Reconstruction of Activities of a Web Visitor. In: Procedia
Computer Science. Vol 1, no. 1 (2010), p. 2267-2274.
[18] M. Cápay, Z. Balogh, M. Boledovičová, M. Mesárošová.. Interpretation
of Questionnaire Survey Results in Comparison with Usage Analysis in
E-Learning System for Healthcare, 2011. In: Communications in
Computer and Information Science : No. 166. - DICTAP 2011, Part I.
pp. 504–516.