Accepted Manuscript

Fuzzy Cognitive Mapping of LMS Users’ Quality of Interaction within Higher

Education Blended-Learning Environment

Sofia B. Dias, Sofia J. Hadjileontiadou, Leontios J. Hadjileontiadis, José A.

Diniz

PII: S0957-4174(15)00384-X
DOI: http://dx.doi.org/10.1016/j.eswa.2015.05.048
Reference: ESWA 10067

To appear in: Expert Systems with Applications

Please cite this article as: Dias, S.B., Hadjileontiadou, S.J., Hadjileontiadis, L.J., Diniz, J.A., Fuzzy Cognitive
Mapping of LMS Users’ Quality of Interaction within Higher Education Blended-Learning Environment, Expert
Systems with Applications (2015), doi: http://dx.doi.org/10.1016/j.eswa.2015.05.048

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers
we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and
review of the resulting proof before it is published in its final form. Please note that during the production process
errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
 Fuzzy Cognitive Mapping of LMS Users’ Quality of Interaction within
Higher Education Blended-Learning Environment
Sofia B. Diasa*, Sofia J. Hadjileontiadoub, Leontios J. Hadjileontiadisc and
José A. Diniza
a
Faculty of Human Kinetics, University of Lisbon, 1499-002 Cruz Quebrada, Lisbon, Portugal,
<sbalula@fmh.ulisboa.pt>; <jadiniz@fmh.ulisboa.pt>
b
Hellenic Open University, Praxitelous 23, 10562 Athens, Greece, <shadjileontiadou@gmail.com>
c
Department of Electrical and Computer Engineering, Aristotle University of Thessaloniki, GR-54124
Thessaloniki, Greece, <leontios@auth.gr>

Abstract Learning Management Systems (LMSs) under blended (b-) learning modality can
efficiently support online learning environments (OLEs) at Higher Education Institutions
(HEIs). Mining of LMS users’ data, involving artificial intelligence and incertitude modeling,
e.g., via fuzzy logic, is a fundamental challenge. This study addresses the hypothesis that the
structural characteristics of a Fuzzy Cognitive Map (FCM) can efficiently model the way
LMS users interact with it, by estimating their Quality of Interaction (QoI) within a b-
learning context. This study introduces the FCM-QoI model (combined with a model
visualizer) consisting of 14 input-one output concepts, dependences and trends, considering
one academic year of the LMS use from 75 professors and 1037 students at a HEI. The
experimental findings have shown that the proposed FCM-QoI model can provide concepts
interconnection and causal dependencies representation of Moodle LMS users’ QoI,
revealing perspectives of its evolution both at a micro and macro analysis level. Moreover,
the FCM-QoI model significantly adds to the evaluation and analysis of the QoI influential
concepts’ contribution to self-sustained cycles (static analysis) and their alterations, when the
time period of the LMS use is considered (dynamic analysis), showing potential to increase
the flexibility and adaptivity of the QoI modeling and feedback approaches. Clearly, based on
the FCM-QoI model, pedagogical instructors and decision-makers of HEIs could be assisted
to holistically visualize, understand and assess OLEs stakeholders’ needs within the teaching
and learning practices.

Keywords Blended learning; Moodle learning management system; Fuzzy cognitive maps
(FCMs); Quality of interaction (QoI); FCM-Viewer application.

*Corresponding author. Tel.: +351-214-149-151; FAX: +351-214-153-095

E-mail address: sbalula@fmh.ulisboa.pt; Postal address: Faculty of Human Kinetics, Estrada da Costa 1499-002
Cruz Quebrada-Dafundo, Lisbon, Portugal.

1
1 Introduction
Education can be seen as a set of processes designed to transmit knowledge, skills and

values to develop individual and collective abilities. At the same time, Learning Management

Systems (LMSs), such as Modular Object-Oriented Dynamic Learning Environment

(Moodle), within an Online Learning Environment (OLE), can provide educators an

environment to place their online course materials and for students to receive that education

while interacting with other students/teachers; however, students’ interactions, attention and

communications are seen as relatively low in the LMSs (Musbahtiti & Muhammad, 2013).

Nevertheless, it seems fair to say that higher education institutions (HEIs) are facing the need

of constant monitoring of users’ interaction with LMS, in order to identify key areas for

potential improvement. This could be expressed in terms of quality of interaction (QoI)

through the LMS use within a blended (b-)learning environment (Woltering, Herrler, Spitzer,

& Spreckelsen, 2009; Miyazoe & Anderson, 2010; López-Pérez, Pérez-López, & Rodríguez-

Ariza, 2011; Oliver & Stallings, 2014).

In fact, the lack of interaction is considered an important obstacle to online learning

(Parker, 1999); however, several improvements in Information and Communication

Technologies (ICTs) over the last years have enhanced the QoI in online environments. In

this vein, there are different types of interactions that educators can consider into the

curriculum, more specifically: learner to instructor, learner to learner, learner to content,

learner to tools, and learner to environment (Bastedo & Vargas, 2014).

Considering the aforementioned perspectives, QoI can be extended from individuals to

groups/networks, from closed to open learning environments, and from small groups to

massive opportunities. The QoI is usually estimated by employing statistical analysis of LMS

data, combined with transcripts of the discussions and exchanges of teacher and learners

within the online forums, specifically investigating the dimension, depth and category of

2
exchanges occurred (Ping, Cheng, & Manoharan, 2010). An alternative approach has been

proposed by Dias and Diniz (2013), who introduced for the first time a fuzzy logic (FL)-

based model, namely FuzzyQoI, to estimate the QoI, taking into account the users’

(professors’ and students’) interactions, as expressed through the LMS usage within a b-

learning environment. Note that for a state-of-the-art review upon the use of FL in education

the reader could refer the work of Dias and Diniz (2013) and Dias, Diniz, and Hadjileontiadis

(2014a). In the FuzzyQoI model, the knowledge of the experts in the field was translated into

fuzzy constructs, and, through five nested fuzzy inference systems (FISs), an estimation, in a

quantitative way, of a normalized index of the users’ QoI was achieved (Dias & Diniz, 2013).

In the FuzzyQoI model, the input (users’ LMS metrics)–output (normalized index of users’

QoI) mapping was integrated into a system as a quantitative map, yet, internally it can be

considered as a set of qualitative linguistic rules that could be used to model the type of

vague or ill-defined systems that are difficult to handle using conventional binary valued

(crisp) logic (Tsoukalas & Uhrig, 1996).

Stemming from the FL concept proposed in FuzzyQoI model (Dias & Diniz, 2013), an

alternative approach, yet within the field of FL, is proposed here, based on the concept of

Fuzzy Cognitive Map (FCM) (Kosko, 1986), namely FCM-QoI model. In general, FCMs can

model causal relationships in complex systems that evolve with time. From this line, several

studies have shown how FCMs can provide an understanding of problematic domains or

systems and/or knowledge for strategic purposes in terms of maximization of benefits,

minimization of risks and management issues (e.g., Sharif & Irani, 2006; Rodriguez-Repiso,

Setchi, & Salmeron, 2007; Glykas, 2013; Jetter & Sperry, 2013). The adoption of the FCM

approach within the FCM-QoI model is motivated by the fact that they are: i) easy to use,

create, visualize and parameterize, ii) flexible in representation/visualization, iii)

understandable for nontechnical experts and for different stakeholders groups, iv) able to

3
handle with complex and dynamic issues, and v) capable of providing an efficient feedback

mechanism that combines information drawn from distinct resources to create an enrich body

of knowledge.

In line to the above, the use of FCMs in modeling the QoI of the LMS users within b-

learning environment aims at providing a “strategic mapping” to better understand the

particular domain of HE in an interconnected way of thinking. In fact, the theory of FCM is

mainly concerned with modeling factors and their interrelationships within complex

domains/systems. Knowing that the adoption of LMSs in HEIs represents a complex domain

with multiple influencing aspects, FCMs seem quite suitable to model this particular domain;

simultaneously, considering that FCMs have the ability to model dynamic systems (Kosko,

1992), they are promising tools to capture the (internal/external) dynamic aspect of the

LMSs. Furthermore, the FCM-based model could present a predictive power for estimating

the QoI, involving concepts, dependences and trends, considering the time period of the LMS

use. From this perspective, experimental validation using real LMS Moodle data drawn from

a large database from a HEI, could reveal the efficiency of the proposed FCM-QoI model

under the LMS-based b-learning approach.

To accomplish the aforementioned goals, this study addresses the hypothesis that the

structural characteristics of a FCM can efficiently model the way LMS users interact with it

and, further, evaluate it by estimating their QoI within a b-learning context.

2 Literature Review
The literature review is divided into two major sections. The first section reviews

research done on the use of FCM in the educational context, while the second one focuses on

literature review dealing with improvements in the FCM functionality.

4
2.1 FCM within the Educational Context

Originated from the theories of FL, neural networks, soft computing and computational

intelligence techniques, FCMs can be understood as a modeling methodology based on

exploiting knowledge and experience (Kosko, 1986, 1992; Papageorgiou & Stylios, 2008).

Nowadays, FCMs have played an important role in different scientific fields, such as

behavioural sciences (e.g., Carvalho, 2013), medicine (e.g., ourgani, St lios, Manis,

eorgopoulos, ), engineering (e.g., asc k & Reyes, 2014), management and business

(e.g., Campaña, Medina, & Vila, 2014; Glykas, 2013), environment (e.g., Wildenberg,

Bachhofer, Isak, & Skov, 2014), and information systems (e.g., Irani, Sharif, Kamal, & Love,

2014). Regarding to the FCMs use for educational scenarios/issues, examples of research

studies (spanning from 2008 up to 2015) are tabulated in Table 1, where their objective(s)

and related problem solving are listed, with the latter being the basis of their order of

appearance.

----------------- (Table 1 about here) -----------------

Table 1 reveals that most of the listed works use the FCM as a basis for modeling,

decision-making, factors analysis, learning and prediction, while some of them focus on

knowledge representation, derivation of progress indicators, framework setting, control and

guiding, along with analysis and evaluation. The variety of these problem solving categories

reveals the ample space for applicability of the FCM in the area of education. In this vein, the

related findings show that: i) insight into the context of educational software adoption in

schools could be achieved, which can guide both educational decision-makers and software

developers in terms of more appropriate software development efforts (Hossain & Brooks,

2008), ii) support to the online learning community could be provided, by allowing prediction

comparisons to be made between numerous tools measured by multiple factors and its

relations, so decision makers can be helped to efficiently/effectively select e-learning

5
technologies (Salmeron, 2009), iii) causalities of the education management could be easily

understood by linked graph representation (Nownaisin, Chomsuwan, & Hongkrailert, 2012),

iv) the success factors of educational organizations could better be understood (Yesil, Ozturk,

Dodurka, & Sahin, 2013), v) the assessment of learning on interactive environments could be

facilitated (Barón, Crespo, Espada, & Martínez, 2014), vi) learning style could be recognized,

by handling the uncertainty and fuzziness of a learning style diagnosis in an efficient way

(Georgiou & Botsios, 2008), vii) game-based learning could be promoted (Luo, Wei, &

Zhang, 2009), viii) highly participatory scenario frameworks, which involve a blend of

qualitative, semi-quantitative, and quantitative methods, could be established, linking

stakeholders and modelers in scenario studies (van Vliet, Kok, and Veldkamp, 2010), ix) the

domain knowledge could be represented in a more realistic way, allowing the adaptive and/or

personalized tutoring system to dynamically deliver the learning material to each individual

learner, taking into account his/her learning needs and his/her different learning pace

(Chrysafiadi & Virvou, 2013), x) decision-making services could be provided by an

intelligent and adaptive web-based educational s stem, provoking learners’ traits to adapt

lectures to enhance their apprenticeship (Peña-Ayala & Sossa-Azuela, 2014), xi) critical

decision-making in medical education could be supported, b exploring extensive “what-if”

scenarios in case studies and preparing for dealing with critical adverse events

( eorgopoulos, Chouliara, St lios, ), xii) student’s grade evaluation and prediction

the forthcoming semesters could achieved (Takács, Rudas, & Lantos, 2014), xiii) stress

factors during learning could be identified (Anusha & Ramana, 2015), xiv) making decisions

concerning networked learning could be supported from both a static and dynamic

perspective (Tsadiras Stamatis, 8), xv) a better understanding on students’ progress

could be offered both to teachers and students via appropriate indicators (Yang, Li, & Lau,

2011), and xvi) modeling and solving decision problems with multiple, conflicting, and

6
interdependent attributes under complex, poorly defined and uncertain environments, such

those identified in some educational contexts, is achievable ( a kasoğlu ölcük, 5).

2.2 Improving FCM Performance

In general, FCM has the ability to analyze and depict human perception of a given

system. This is due to the inherent production of a conceptual model, which is not limited by

exact values and measurements, and thus, is well suited to represent relatively unstructured

knowledge and causalities expressed in imprecise forms (Glykas, 2010; Papageorgiou, 2014).

Usually, there is a manual construction of FCMs, and, thus, they cannot be applied when

dealing with large number of variables. In such cases, their development could be

significantly affected by the limited knowledge and skills of the knowledge holder; thus, it is

vital to use learning algorithms to accomplish this task (Papageorgiou, 2014).

In order to improve the performance of FCMs, several methodologies have been explored;

for instance, Peña, Sossa, and Gutierrez (2007) proposed a model for the design and

development of ontology agents to manage rule-based FCMs (i.e., RB-FCM), taking into

account the ontology agents and the FCMs represented by fuzzy rule bases. Based on a

theoretic analysis, Pedrycz (2010) presented the synergy of granular computing and

evolutionary optimization to design efficiently FCMs. On the other hand, Song, Miao, Roel,

Shen, and Catthoor (2010) proposed a fuzzy neural network to improve the learning ability of

FCMs, incorporating the inference mechanism of conventional FCMs with the determination

of membership functions and the quantification of causalities. Cai, Miao, Tan, Shen, and Li

(2010) developed the Evolutionary Fuzzy Cognitive Map (E-FCM), as an extension of the

FCMs. Basically, the E-FCM approach was used to model dynamic concepts and their causal

relationships in the virtual world, including modeling characters and contexts. In this way, the

virtual world can become more dynamic, with plausible characters/contexts, enhancing

pla er’s experience in the interactions with the environment. Compared with the FCM, the E-

7
FCM presented three main improvements, in particular: i) it allowed a different update time

schedule for each variable; ii) it enabled self-mutation of the context variables, showing the

dynamics of the world variables as evolving behavior; and iii) it involved the probabilistic

causality among the variables, reflecting realistic relationships among the concepts and

adding more dynamics to the character. Papageorgiou (2011) proposed a methodology to

design augmented FCMs combining knowledge from experts and from different data sources

in the form of fuzzy rules generated from rule-based knowledge discovery methods. In

another study (Acampora, Loia, & Vitiello, 2011), a FCM-based emotional agent in an

Ambient Intelligence (AmI) environment was developed, in order to provide advanced

services and to enhance the quality of life. In general, this approach tried to maximize the

s stem’s usabilit in terms of efficienc , accurac and emotional response. Chunying, Lu,

Dong, and Ruitao (2011) proposed the Rough Cognitive Maps (RCM) as an FCM extension

to model the whole set of relations between two concepts in real world, in order to improve

the simulation capabilities of the FCM. In general, they concluded that RCM is more flexible

and robust than FCM, simulating the real world in a better way. Finally, more recentl , Štula,

Stipaničev, and Šerić ( ) investigated the possibilit of multi-agent systems (MAS)

application in distributed computation by using FCM technique. For a thorough description

of the literature review upon the FCM and its use the reader could refer to (Papageorgiou,

2012, 2014; Papageorgiou & ) and to the online FCM Community

(http://joomla.epapageorgiou.com/index.php/about-fuzzy-cognitive-maps). The way FCM is

used here it is explained in the methodology section.

2.3 Contribution of the Proposed Approach

Considering the research presented in the previous subsections, this paper contributes to

the literature in the following ways:

1. Bridges the field of FCM with the QoI of LMS users within the b-learning context;

8
 2. Introduces a new model that extends the FL-based modeling of the QoI of the LMS

users further, by providing FCM-based structural maps that reveal the causal

interconnections of the important factors within the complex domain of LMS-based

online learning;

3. Contributes to the QoI estimation, due to the predictive power of the proposed FCM-

QoI model, taking into account concepts, dependences and trends;

4. Provides FCM structural approaches based on the time (in)dependencies, that could

reveal bilateral perspectives in the evolution of the LMS users’ QoI, both at a micro

and macro analysis level;

5. Evaluates the role of the influential factors of the QoI and analyzes their contribution

to self-sustained cycles (static analysis) and their alterations, when the time period of

the LMS use is considered (dynamic analysis);

6. Assists the decision-making of pedagogical planners to (re)adjust online tools, in order

to maximize the use of the LMS Moodle within the teaching and learning practices;

7. Provides specific structural numerical data (e.g., estimated weight matrices), validated

across a large amount of LMS Moodle logs under different scenarios, taking into

account the user’s t pe (i.e., professor/student), ready to be used by other researchers

in the field and/or LMS Moodle developers, exhibiting potential to be integrated

within LMS users’ QoI modeling and feedback schemes, towards increased flexibility

and adaptivity; and

8. Introduces a new FCM visualization tool.

The aforementioned contributions are justified by the analysis presented in the subsequent

sections.

9
3 Methodology

3.1 FCM Functional Structure

A FCM is depicted as a fuzzy causal graph, where nodes represent concepts, whereas

directed edges between the concepts denote causal relationships present between them

(Kosko, 1986). Under the FCM approach, a given system is defined as a collection of

concepts (e.g., events, actions, values, goals, etc.) that possess influential interconnections,

expressing cause-effect relationships, which are quantified through a weighting

correspondence, usually normalized to the [-1, 1] interval. In the latter, positive values of

weights describe promoting effect, whereas negative ones describe inhibiting effect; other

values correspond to different intermediate levels of the causal effect. A generic

representation of the FCM model is presented in Figure 1, consisting of four concepts, i.e.,

, interconnected with the following 4x4 weight matrix :

. (1)

----------------- (Fig. 1 about here) -----------------

In general, a FCM is defined using a 4-tuple , where

denotes the set of concepts of the graph, is a function that associates a

causal value to each pair of nodes of the connection matrix, denoting the weight

of the directed edge from to , representing the causality degree between the

interconnected concepts; hence, the weight matrix gathers the system causality, usually

determined by knowledge experts. Every label (in the example of (1) is

mapped to its activation value , i.e., , with 0 and 1 corresponding to

no and full activation, respectively. The activation value of concepts is also fuzzy in nature

10
and regularly takes values in the range [0,1], although the scale [-1,1] is also allowed

(Nápoles, Bello & Vanhoof, 2013). In the proposed FCM-QoI model, all activation values

refer to parameters that lie within the [0,1], as they express FIS inputs/outputs from the

FuzzyQoI model (Dias & Diniz, 2013). The higher the activation value, the stronger the

influence of the concept over the system, which helps to understand the modeling. In fact,

is a function that associates the activation degree to each concept at the iteration

. Finally, a transformation (or threshold) function is employed

to keep the activation value of concept within the interval . The most frequently used

threshold functions include the bivalent function, the trivalent function and the sigmoid

variants (Bueno & Salmeron, 2009). The labels from can be interpreted as linguistic terms

that point to fuzzy sets (Kosko, 1986; Glykas, 2010), meaning that each is interpreted as

the value of fuzzy membership function that measures the degree in which an observed value

belongs to the fuzzy set pointed by the related term. Moreover, the labels (in the example

of (1) ) denote the real valued variables, with their domains assumed to be

normalized into the interval (Papageorgiou, 2014).

For the manual construction of the FCM, the number and kind of concepts (e.g., input or

output) are determined by a group of knowledge experts and each interconnection is

described by either with an IF-THEN rule that infers a fuzzy linguistic variable from a

determined set or with a direct fuzzy linguistic weight, which associates the relationship

between the two concepts and determines the grade of causality between the two concepts.

Then, all the linguistic variables suggested by the knowledge experts are aggregated using the

SUM method (Kosko, 1986) and an overall linguistic weight is produced, which is then

transformed into a numerical weight , belonging to the interval , by using a

defuzzification method (e.g., the center of gravity (Kosko, 1986)), and, finally, a numerical

weight for is calculated. In this way, all the weights of the FCM model are inferred.

11
 As it was mentioned above, an automated estimation of the can be achieved,

overcoming the dubious effort and potential subjectivity of the knowledge expert in his/her

involvement in its definition process, by adopting learning algorithms. Towards such

endeavor, adaptive Hebbian-based learning algorithms, the evolutionary-based (e.g., genetic

algorithms) and the hybrid approaches (composed of Hebbian type and genetic algorithm)

(Papageorgiou, Stylios, & Groumpos, 2003; Stach, Kurgan, Pedrycz, & Reformat, 2005;

Glykas, 2010; Papageorgiou, 2012, 2014), have been proposed and widely used as the most

efficient methods for training FCMs.

Once the FCM is constructed, it can receive data from its input concepts, perform

reasoning and infer decisions as values of its output concepts. During reasoning, the FCM

iteratively updates its state by multiplying the causal weight matrix by the current state vector

until convergence is reached or a stopping condition is met (e.g., the number of iterations

has been reached). Note that, in the case of the involvement of a learning algorithm within the

construction of the FCM, apart from the , the matrix is also updated at each iteration.

3.2 The Proposed FCM-QoI Modeling Approach

A schematic representation of the proposed FCM-QoI model is depicted in Figure 2. As it

is clear from the Figure 2(a), the user interacts with the LMS Moodle and 110 metrics are

acquired, exactly the same as the ones used in the FuzzyQoI model (Dias & Diniz, 2013, see

also Appendix A-Table A.1). These metrics are then categorized into 14 categories, denoted

as , corresponding to the 14 categories used in the FuzzyQoI model (Dias &

Diniz, 2013), i.e., : {Journal/Wiki/Blog/Form (J/W/B/F)}, : {Forum/Discussion/Chat

(F/D/C)}, : {Submission/Report/Quiz/Feedback (S/R/Q/F)}, : {Course Page (CP)}, :

{Module (M)}, : {Post/Activity (P/A)}, : {Resource/Assignment (R/A)}, : {Label

(L)}, : {Upload (UP)}, : {Update (U)}, : {Assign (A)}, : {Edit/Delete (E/D)},

12
 : {Time Period (TP)}, and : {Engagement Time (ET)}. These 14 concepts are

considered the inputs of the FCM within the FCM-QoI model and the additional FCM-QoI

concept is considered its output (see Figure 2(b)). The FuzzyQoI model (Figure 2(a)) outputs

the , which is used in the training phase of the FCM within the FCM-QoI model

(Figure 2(b)). In the proposed FCM-QoI model, the Nonlinear Hebbian Rule (NHR)-based

learning algorithm (Papageorgiou et al., 2003) is adopted. For the sake of completeness, a

mathematical description of the adopted FCM updating process (both of and ) is

presented in Appendix B. During the iterative learning process ( maximum

iterations), the weights , along with the corresponding labels , are updated, using

Eqs. (A.3) and (A.5), respectively, towards the satisfaction of Eq. (A.6) (see Appendix B).

When the latter is met, the FCM is considered trained and the updated weights ( ) are used

in the testing phase of the FCM-QoI model using Eq. (A.8), considering the satisfaction of

Eq. (A.7) (see Appendix B). The fulfilment of the latter denotes the final estimation of the

output of the FCM-QoI model.

----------------- (Fig. 2 about here) -----------------

As it is clear from the aforementioned description of the FCM-QoI model, the involved

FCM plays the role of a system representation, dismantling the 14 inputs-1 output relations,

as represented by the estimated matrix. This actuall reflects the expert’s knowledge

representation hidden in the IF/THEN fuzzy rules of the FuzzyQoI model, yet in a more

quantitative way, i.e., in the form of the interconnection weight values.

The efficiency of the proposed FCM-QoI model has been evaluated through its application

to the same data used in the FuzzyQoI model (Dias & Diniz, 2013), drawn from a real-life

LMS Moodle use case from higher education (HE), involving both professors and students,

as thoroughly described in the succeeding section.

13
4 Experimental Validation of the FCM-QoI Model

4.1 Data Characteristics

The LMS Moodle data for the validation of the proposed FCM-QoI model were drawn

from a b-learning environment related to five undergraduate courses (Sport Sciences,

Ergonomics, Dance, Sport Management and Psychomotor Rehabilitation) offered by the first

Author’s affiliated HEI. As it was mentioned before, the data were identical to the ones used

for the validation of the FuzzyQoI model (Dias & Diniz, 2013) for comparison reasons. LMS

Moodle data from both professors [75 in total, 49% male, aged from 24 to 54 yrs (mean±std

= 47.19±8.8 yrs)] and students [1037 in total, 45% male, aged from 18 to 48 yrs (mean±std =

25.05±5.9 yrs)] were used. All of them started to use LMS Moodle in the 2009/2010

academic year. The time-period involved spanned 51 weeks (August 26, 2009-August 18,

2010), including 610775 LMS users’ interactions in total (94288 from professors and 516487

from students). The number of the daily data loggings belonging to the same category per

user was summed across the duration of one week (the analysis time-unit), and all derived

input variable values per week were normalized to the corresponding maximum value across

the analyzed total time-period, i.e., 51 weeks, for each user. Furthermore, some distinct dates

across the whole examined time-period were taken under consideration to segment the latter;

this segmentation is depicted in Figure 3, involving Lectures, Interruptions and Exams time-

period sections for the first (soft-stems) and the second (heavy-stems) semesters.

----------------- (Fig. 3 about here) -----------------

4.2 Training and Testing Dataset Configuration

The whole dataset of the 14 input-1 output data from the FuzzyQoI model was randomly

split into 75% as a training dataset and 25% as a testing dataset for the FCM-QoI model. The

latter was kept separate from the former one, in order to test the generalization power of the

14
FCM model. Table summarizes the anal sis scenarios/phases per users’ categor adopted.

In particular, a bilateral approach was realized via the time-dependent and time-independent

scenarios, both including a training and a testing phase, accordingly. In the time-dependent

scenario, the time unit of analysis of one week is taken into account, in order to obtain the

best estimation per week, whereas in the time-independent scenario, the time unit of

analysis of the whole academic year was considered, so to obtain the best estimation

per academic year. In this way, a fine-grained (per week) and a more abstract (per academic

year) perspective are examined, showing the potentialities of the FCM-QoI model at both

micro- and macro-levels.

----------------- (Table 2 about here) -----------------

In particular, in the time-dependent training scenario, 1000 randomized selections of the

75% of the training set were considered and, for each random selection, the mean value

across the users per input/output per week was estimated, setting, in this way, the initial

values of the , per week in the updating equation of Eq. (A.5) (see

Appendix B). The initial values of the matrix ( in (Eq. A.3)) per week were

randomly selected from the range of (apparently . Consequently,

the training process in the time-dependent scenario outputted a per week .

In the time-independent training scenario, the same process as in the case of the time-

dependent scenario was followed, yet here, for each random selection of the 1000

randomized selections of the 75% of the training set, the mean value across the users per

input/output and across the weeks was estimated. In this way, the training process in the time-

independent scenario outputted a per academic year .

In the time-dependent and time-independent testing scenarios, the estimated and

were used in the Eq. (A.8), respectively, conducted upon the initially selected testing

15
set (25% of the initial data) to infer the per week and per academic year,

respectively.

In all cases, the evaluation of the performance of the FCM-QoI model was realized via the

estimation of the Root Mean Squared Error (RMSE) and the correlation coefficient between

the estimated and the derived from the FuzzyQoI model.

4.3 Implementation and Visualization Issues

The implementation of the whole analysis of the FCM-QoI model was carried out in

Matlab 2014a (The Mathworks, Inc., Natick, USA), using custom-made programming code.

The archived data in the LMS Moodle repository were exported from .xml to .xlsm

(Microsoft Excel format) and imported to the Matlab environment and archived as .mat files.

The values used for the updating process of the FCM were selected as the optimum ones that

minimize the number of iterations for meeting the termination criteria of Eqs. (A.6) and (A.7)

(see Appendix A). In particular, (see Eq. (A.3)), ,

(see Eq. (Α.4)), (see Eq.

(A.5)), (see Eqs. (A.6) and (A.7)).

The visualization of the estimated FCM graphs was carried out via an especially

programmed Matlab application, namely FCM-Viewer (see Appendix B). It should be noted

that FCM-Viewer is freely available from the authors upon request.

The results from the analysis of the aforementioned LMS Moodle data with the proposed

FCM-QoI model scheme are described and discussed in the succeeding subsections.

5 Results and Discussion

5.1 Professors’ QoI

5.1.1 Time-dependent scenario: Training phase

16
 The mean RMSE between the estimated and the across 1000

iterations during the time-dependent training phase of the FCM-QoI model for the professors’

case (denoted by the prefix of ) was found equal to 0.0201, spanning between 0.015 and

0.024, and showing a similar behavior across the 1000 iterations. This denotes a consistency

in the behavior of the FCM-QoI model during the time-dependent training phase, sustaining

the RMSE in low values. The latter is reflected in Figure 4, where the estimated

(dense black line), which corresponds to the iteration (291) that exhibits the

minimum RMSE, is illustrated in Figure 4(a), whereas the estimated (dense

black line), which corresponds to the iteration (415) that exhibits the maximum RMSE, is

shown in Figure 4(b). In both subplots, the corresponding (light grey line)

derived from the FuzzyQoI model (Dias & Diniz, 2013) and used for the time-dependent

training phase of the FCM-QoI model is superimposed to facilitate the comparison.

Moreover, the vertical lines illustrate the time-period segmentation according to Figure 3. As

it can be seen from Figure 4, the estimated and exhibit a

very good phase synchronization with the , denoting a successful training

outcome (even in the case of the maximum RMSE, Figure 4(b)). This is justified by the

corresponding correlation coefficients, i.e.,

and .

----------------- (Fig. 4 about here) -----------------

The higher correlation seen in the case of maximum RMSE, compared to the case of

minimum RMSE, is justified by Figure 4, where the (Figure 4(b)) follows

more uniformly, compared to the (Figure 4(a)), the phase alterations of

, yet exhibiting an underestimation of the true values of ; this is

reduced in the case of , justifying the corresponding minimum RMSE value.

17
As it can be seen from Figure 4(a), the shows its main phase mismatch with

the at weeks 17 and 20 of the first semester, whereas in the second semester it

clearly follows the changes of .

The corresponding matrix exhibits a degree of sparseness with sufficient

balance between the negative and positive weight values, showing an increase in the latter

ones around week 32 (just after the Carnival interruption at week 31). Obviously, the

complexity of the 4D-matrix of does not allow for easy visualization of the

corresponding FCM per week across the academic year. Nevertheless, specific weeks in

distinct time-periods are isolated and the corresponding FCMs are visualized in the

forthcoming Time-Period Dependences subsection.

5.1.2 Time-dependent scenario: Testing phase

Figure 5 illustrates the results from the testing phase of the time-dependent scenario of the

FCM-QoI model. As it was noted before, the data used in the testing phase were totally

independent from the ones involved in training phase of the time-dependent scenario.

Apparently, the was used in the construction of the FCMs per week for the

estimation of the (thick grey line), as it corresponds to the minimum

RSME form the training phase of the time-dependent scenario. For comparison reasons, the

(light black line), which corresponds to the estimated from the FuzzyQoI

model across the same professors that are incorporated in the construction of the testing data

of FCM under the time-dependent scenario, is superimposed. The RMSE and the correlation

coefficient between the and are equal to 0.0368 and

0.4045 ( ), respectively. Clearly, the derived RMSE is quite low, showing an

efficient generalization of the FCM-QoI model in predicting the after a training

procedure based on historical data.

18
 ----------------- (Fig. 5 about here) -----------------

The estimated correlation coefficient shows a good phase synchronization of the

with , yet not in all weeks across the academic year. From

Figure 5 it can be seen that the estimated matches better the phase

alterations of after week 26, i.e., in the second semester, in comparison to the

ones in the first semester, with the exception of the underestimation of the

peak at week 25. In general, however, the is in good agreement with

the , showing a satisfactory performance of the FCM-QoI model to the

estimation for the case of professors.

5.1.3 Time-independent scenario: Training phase

The mean RMSE between the estimated and the across 1000

iterations during the time-independent training phase of the FCM-QoI model for the

professors’ case was found equal to 0.024, spanning between 10-5 and 0.0316, and showing a

similar behavior across the 1000 iterations mostly between 0.005 and 0.0316 RMSE values.

Unlike the case of the time-dependent scenario, where the week is the time-analysis unit, in

the case of time-independent scenario the academic year serves as the analysis unit; hence,

the results of the training phase under this scenario are depicted across the 1000 iterations.

This is illustrated in Figure 6, where the estimated (diamonds) along with the

(dots), corresponding to the double averaged (across the professors of the training

phase and across the 51 weeks of the academic year) values estimated from the

FuzzyQoI model (Dias & Diniz, 2013), are depicted for the 1000 iterations of the training

phase of the time-independent scenario.

----------------- (Fig. 6 about here) -----------------

19
 Moreover, the and are also

shown as horizontal solid lines, respectively. As it can be seen from Figure 6 and deduced

from the corresponding mean values, the FCM-QoI model exhibits a kind of underestimation

of the values per academic year derived from the FuzzyQoI model (training set).

Table 3 presents the estimated interconnection weight matrix between the 15

concepts (14 input concepts and as the output concept) of the time-independent

training phase of the FCM-QoI model, which corresponds to the iteration (597) that exhibits

the minimum RMSE (10-5). As it is shown in Table 3, the estimated interconnection weights

lie within the range of and exhibits about 73% of sparseness. This is

reflected into the FCM that corresponds to depicted in Figure 7. In the latter, the

way the 14 input concepts and the output concept (within the red border) of the

FCM-QoI model are interconnected is visualized via the FCM-Viewer (see Appendix B).

Note that , enclosed by thick border, are those concepts that are

directly connected to the output concept.

----------------- (Table 3 about here) -----------------

----------------- (Fig. 7 about here) -----------------

5.1.4 Time-independent scenario: Testing phase

Since this scenario involves the double averaged values of the I, i.e., ,

of the professors participating in the testing data, the output of the FCM-QoI model, i.e.,

, is a single value. In this vein, for the specific testing data it was found that

, close enough to the . Moreover, although the

time-independent scenario does not involve week-based analysis in the training phase, an

effort to test the generalization power across the academic year (i.e., per week analysis) was

attempted, by predicting the per week, yet based on the matrix,

20
rather than on a one. The results from this approach are shown in Figure 8, which

illustrates the estimated professors’ (thick grey line) during the

testing phase of the time-independent scenario, along with the corresponding

(light black line) derived from the FuzzyQoI model (Dias & Diniz, 2013) and used for the

time-independent testing phase of the FCM-QoI model; the vertical lines illustrate the time-

period segmentation of Figure 3.

----------------- (Fig. 8 about here) -----------------

The matrix of Table 3 was used here, derived from the training phase of the

time-independent scenario. The RMSE and the correlation coefficient between the

and are equal to 0.0346 and 0.84 ( ),

respectively. The derived RMSE supports the generalization efficiency of the FCM-QoI

model in predicting the after a training procedure based on historical data. Moreover, the

estimated correlation coefficient shows a good phase synchronization of the

with , yet without providing a good estimation of the

amplitude of the in all weeks across the academic year. Consequently, from

Figure 8 it can be seen that the estimated identifies better the

morphology of the per week and provides, mainly, the trend of the latter,

rather than an absolute matching of the values.

5.2 Students' QoI

5.2.1 Time-dependent scenario: Training phase

The same analysis trajectory as in the case of professors was followed for the case of

students. The mean RMSE between the estimated and the across

1000 iterations during the time-dependent training phase of the FCM-QoI model for the

21
students’ case (denoted b the prefix of ) was found equal to 0.013. Similarly to the case of

professors, the estimated RMSE exhibits a similar behavior across the 1000 iterations,

covering a range between 0.0085 and 0.0175, showing a good consistency, as justified via the

estimated depicted in Figure 9(a) as dense black line (iteration 87) that

exhibits the minimum RMSE, along with the estimated shown in Figure

9(b) as dense black line (iteration 554) that exhibits the maximum RMSE. In both subplots,

the corresponding (light grey line) is superimposed to facilitate the comparison.

As usually, the vertical lines illustrate the time-period segmentation according to Figure 3. As

Figure 9 shows, the estimated and exhibit an efficient

phase synchronization with the , even in the case of the maximum RMSE (Figure

9(b)), justified by the corresponding correlation coefficients, i.e.,

and . As it can be seen

from Figure 9(a), the shows its main phase mismatch with the

between weeks 2 and 16 of the first semester, whereas in the second semester it clearly

follows the changes of .

----------------- (Fig. 9 about here) -----------------

The corresponding matrix is characterized by some sparseness with sufficient

balance between the negative and positive weight values, with the latter ones increasing

around weeks 18-22 and 38-44, coinciding within the exam periods of the first and second

semester, respectively. The alteration of the corresponding FCMs across specific time periods

is further examined in the forthcoming Time-Period Dependences subsection.

5.2.2 Time-dependent scenario: Testing phase

Figure 10 depicts the results from the testing phase of the time-dependent scenario of the

FCM-QoI model for the students’ case. The that corresponds to the minimum

22
RSME form the training phase of the time-dependent scenario was used in the construction of

the FCMs per week for the estimation of the (thick black line),

which compared with the superimposed (light grey line). The RMSE and the

correlation coefficient between the and are equal to

0.0264 and 0.5363 ( ), respectively. Clearly, the derived RMSE is quite low (even

lower compared to the corresponding one in the professors’ case), justif ing the

generalization power of the FCM-QoI model, whereas the estimated correlation coefficient

(higher compared to the corresponding one in the professors’ case), shows a good phase

synchronization of the with , yet with a phase

mismatch occurring mainly at the beginning (weeks 1-4) and the end (weeks 44-51) of the

academic year. Nevertheless, the FCM-QoI model exhibits overall a satisfactory

estimation performance, even higher than that seen in the case of professors.

----------------- (Fig. 10 about here) -----------------

5.2.3 Time-independent scenario: Training phase

During the time-independent training phase of the FCM-QoI model for the students’ case,

the mean RMSE between the estimated and the across 1000

iterations was found equal to 0.017 (range to 0.0316; similar behaviour across the

1000 iterations mostly between 0.0022 and 0.0316). Similarly to the corresponding results in

the professors’ case, the estimated (diamonds) along with the (dots),

the and (horizontal solid lines)

are all illustrated in Figure 11. The latter shows that the FCM-QoI model underestimates the

values per academic year derived from the FuzzyQoI model (training set), yet less when

compared to the corresponding case of the professors (Figure 6). This is also reflected in the

23
estimated and values, since their distance is less

than that between the and values.

----------------- (Fig. 11 about here) -----------------

The of the time-independent training phase of the FCM-QoI model, which

corresponds to the iteration (954) that exhibits the minimum RMSE ( ), is tabulated in

Table 4; exhibits about 71.5% of sparseness and its weights lie within the range of

----------------- (Table 4 about here) -----------------

The corresponding FCM (visualized via the FCM-Viewer, see Appendix C) that

incorporates the is depicted in Figure 12. From the latter, it can be seen that from

the 14 input concepts only the , enclosed by thick border, are directly

connected to the output concept. Compared to the case of professors, the

concepts are common to the both user’s t pe, with regard to their direct connection

with the corresponding output concept (i.e., ).

----------------- (Fig. 12 about here) -----------------

5.2.4 Time-independent scenario: Testing phase

As in the case of professors, for the specific testing data a single valued output was

estimated, i.e., , comparable to the . Moreover,

the generalization power of the FCM-QoI model across the academic year (i.e., per week

analysis) for the case of students was tested by predicting the per week, yet

based on the matrix derived during the training phase of the time-independent

scenario (Table 4), rather than on a one. The related findings are shown in Figure

3, which illustrates the estimated students’ (dense black line) and

24
the corresponding (light grey line); as before, the vertical lines illustrate the

time-period segmentation of Figure 3.

----------------- (Fig. 13 about here) -----------------

The RMSE and the correlation coefficient between the and

are equal to 0.0274 and 0.8047 ( ), respectively. As in the case of

professors, the estimated RMSE supports the generalization efficiency of the trained FCM-

QoI model in predicting the , exhibiting a good phase synchronization between the

and , et, similarl to the professors’ case, without

providing an efficient amplitude estimation of the in all weeks across the

academic year. Consequently, from Figures 11 and 13 it can be deduced that the estimated

better identify the morphology of the corresponding

per week, rather than their actual amplitude, mainly providing an estimation of

their trends across the whole academic year.

5.3 Static Analysis of the Derived Main FCMs

In an effort to further deep in the derived results, a static analysis of the derived main

FCMs under the time-independent scenario for both users’ categories (Figures 7 and 12,

respectively) was carried out. As a first level of this analysis, the causal relationships between

the concepts that are directly connected with the , illustrated as bold circles in

Figures 7 and 12, respectivel , are interpreted. In particular, focusing at the professors’ case

(Figure 7, Table 3), the main concepts are and , which are

directly connected with the . Based on the corresponding weights (Table 3), :

{Edit/Delete (E/D)} is the concept that negatively affects most the , whereas :

{Journal/Wiki/Blog/Form (J/W/B/F)} is the one that positively affects most the ,

closely followed by : {Course Page (CP)}. From an overall perspective across the whole

25
academic year and despite the moderate values of weights, these findings allow for some

guidance to the Professors, i.e., the alterations (E/D) they perform during their interaction

with the LMS should be taken with care, while the use of J/W/B/F and CP should further be

encouraged. With regard to the Students’ case (Figure 12, Table 4), the main concepts are

and , which are directly connected with the . According to the

corresponding weights of Table 4, : {Submission/Report/Quiz/Feedback (S/R/Q/F)} is the

concept that negatively affects most the , whereas : {Resource/Assignment

(R/A)} is the one that positively affects most the . Similarly to the case of

professors, these findings allow for some guidance to the students, indicating the need for

attention in the way they perform the S/R/Q/F, while their engagement with R/A should

further be encouraged.

However, as there are a number of interrelationships between the concepts within the

FCMs of Figures 7 and 12, there is a potential for feedback cycles to exist within these

FCMs, whereby a change in one concept can have a cumulative and/or self-sustaining effect.

In this way, similarly to the work of Hossain and Brooks (2008), the second level of the

static analysis adopted was based on studying the characteristics of the weighted directed

graph that represent each derived FCM, using graph theory techniques (Tsadiras et al., 2001).

In particular, the main feedback cycles were identified within the estimated FCMs of Figures

7 and 12, structured by the concepts that are directly connected with the ,

illustrated as bold circles in Figures 7 and 12. The identified main feedback cycles are

tabulated in Table 5 for both users’ categories (Professors/Students). Each identified feedback

cycle was allocated a positive or negative sign (last column of Table 5), which was

determined by multiplying the signs of the arcs present in each feedback cycle (Tsadiras et

al., 2001). In fact, positive (negative) feedback cycle behavior is that of amplifying

(counteracting) any initial change, leading to a constant increase (decrease) in case an

26
increase is introduced to the system (Tsadiras et al., 2001). For example, the P5 cycle (Table

5, Professors’ case) involves the : {Post/Activity (P/A)}, , : {Module

(M)}, : {J/W/B/F}, and : {CP} concepts, showing that P/A affects , which

affects M, which affects J/W/B/F, which affects CP, which then affects back P/A. This cycle

could interpreted as follows: the involvement of professors in P/A could affect the

and then lead to the engagement of M, which could evoke J/W/B/F activity in the

related area; hence, involvement of CP, leading back to more P/A. This cycle exhibits a

positive sign (Table 5, last column) and shows how the feedback of could

contribute in fueling a self-sustaining action ( ), which has a cumulative effect on other

actions, i.e., and within the derived FCM. As a second example, the S5 cycle

(Table 5, Students’ case) involves the : {R/A}, , and : {E/D}, implying

that R/A affects the , which then influences the content alterations (E/D), which

could affect back the way Students could handle the R/A. The negative sign of S5 (Table 5,

last column) indicates that in this cycle, the increase in R/A will not necessary lead to a self-

sustained action, but as an overall, it would negatively be affected by the changes in

and ; hence, careful design in the students’ R/A activities should be considered,

without boosting the need for any unnecessary alterations (E/D).

From an overall perspective of the cycles in Table 5, it seems that there is a balance

between the number of c cles with positive and negative sign, in both users’ categories

(Professors/Students). This means that the involvement of within these cycles

could affect them in a bilateral way, either by boosting or by decreasing them. From a cross-

users’ perspective, it is interesting to notice that affect in the same way the

cycles initiated by the concepts that are common between the two users’ cases

(Professors/Students), i.e., P4/S4 ( , P6-P7/S5 ( , and P9/S6 ( ), with the

exception of S7 ( . This finding, however, indicates that the interconnection of these

27
common concepts with other ones and the corresponding output of the FCM-QoI model

follows a similar pattern, independentl of the user’s categor ; hence, the should be

considered as factors that affect in a similar wa both main stakeholders’ (i.e.,

Professors/Students) QoI.

----------------- (Table 5 about here) -----------------

5.4 Time-period Dependences

As it was shown in the previous two subsections, the structure of the FCM-QoI model

supports the monitoring of the dynamics in the change of the per week across the

academic year, both for professors and students. This provides the opportunity to focus upon

some specific time-periods (see Figure 3) and compare the changes in the FCMs of the FCM-

QoI model when shifting from one specific time-period to another. Apparently, all

discontinuities in Figure 3 could be considered as points of interest for time-period

dependences; yet, due to space limitations, the most profound ones that involve the main

distinct periods of Figure 3 are analyzed here. Obviously, the same approach could be easily

expanded to the rest of the transitional time-periods. In particular, the week-pairs of

PR1={15,16}, PR2={17,18}, PR3={22,23} and PR4={37,38} were selected, as they

represent the transition from Lecture (1st semester) to Interruption (Christmas), Interruption

(Christmas) to Exam (1st semester), Exam (1st semester) to Lecture (2nd semester) and Lecture

(2nd semester) to Exam (2nd semester), respectively. Figure 20 summarizes the corresponding

FCMs (derived from the training phase of the time-dependent scenario) to the aforementioned

week-pairs for the case of professors (Figure 14(a), P-PR1, P-PR2, P-PR3, P-PR4) and

students (Figure 14(b), S-PR1, S-PR2, S-PR3, S-PR4), using the FCM-Viewer (see Appendix

B). In all FCMs of Figure 14, the concept interconnection weights have been omitted for

better visualization of the changes in the FCMs structure across the week-pairs (note that all

numerical material corresponding to the FCM-QoI model parameters are freely available

28
upon request from the authors). Moreover, the concepts of each FCM that are directly

connected to the corresponding output (i.e., ) are visualised as circles with thicker

border than the rest ones.

As it can be seen from Figure 14(a) (professors’ case), the number of concepts that are

directly connected to is decreased from six to three within the P-PR1 and increased

from three to eight within the P-PR2; moreover, it is increased from four to six within P-PR3

and from two to 10 within P-PR4. In this context, the concepts ({Module (M)}),

({Assign (A)}) and ({Edit/Delete (E/D)}) appear in five out of eight weeks of the P-

PR1:PR4 week-pairs, leaving behind the rest ones that appear in one up to three weeks of the

week-pairs. In an analogy, from Figure 14(b) (students’ case), it is deduced that the number

of concepts that are directly connected to is decreased from eight to three within the

S-PR1 and increased from seven to 10 within the S-PR2; furthermore, it is increased from six

to eight within S-PR3 and decreased from nine to six within S-PR4. Moreover, the concept

({Submission/Report/Quiz/Feedback (S/R/Q/F)}) appears in seven out of eight weeks of

the S-PR1:PR4 week-pairs, concepts ({Journal/Wiki/Blog/Form (J/W/B/F)}) and

({Module (M)}) appear in six out of eight weeks, concepts {Label (L)} and

({Engagement Time (ET)}) appear in five out of eight weeks, while the rest ones appear in

one up to four weeks of the week-pairs. These characteristics, reflected in Figure 14, show

that time-period dependency alters the number and the type of concepts of FCM-QoI model

that directly affect the . More specifically, in the case of professors, it seems that

entrance to Lecture or Exam time-periods increases the number of concepts that directly

influence . This is also observed, in general, in the case of students, with the

exception of S-PR4, where the transition from Lecture (2nd semester) to Exam time-period

decreases the number of concepts that directly influence . This time-period

dependency seen in the structural characteristics of the FCM-QoI model actually translates

29
the changes in the users’ interaction with the LMS due to the temporal changes of the

educational context (e.g., transition from the Lecture to Exam time-period) into role changes

of the concepts involved, by enabling them to be more or less direct effective on the FCM-

QoI output. Consequently, the latter is involved in more complex or simpler feedback cycles,

as those examined in §5.3, yet in a dynamic way. This information could be useful in a

dynamic design of the online part of the b-learning, by considering which concepts across

time become more important and can contribute to self-sustained cycles that positively affect

the users’ QoI; hence, encouragement of users’ interaction with these concepts could be

promoted by the LMS and the curriculum of the b-learning, in general.

----------------- (Fig. 14 about here) -----------------

Moreover, it seems that the nature of the aforementioned concepts with high appearance

frequency, that have a direct impact on the estimation of within the examined

week-pairs, comply with the corresponding role of the user’s type; that is, concepts ,

and refer to main activities of professors in the area of information addition and

alteration (Dias & Diniz, 2013), reflecting the teaching role, whereas concepts , , ,

and (sorted from high to low appearance frequency) express students’ LMS

interactions in the area of information view, addition and engagement (Dias & Diniz, 2013),

potentially reflecting main activities of the learning process.

Apparently, the structure of the proposed FCM-QoI model allows for zooming into

specific time-dependent characteristics of the users’ interaction with the LMS Moodle,

capturing the internal changes that appear in the dynamics of the interconnected concepts that

directly influence the final across the academic year.

30
5.5 Overall Perspective

5.5.1 Hypothesis justification

The proposed FCM-QoI model, when placed within the panorama of the works that

combine FCMs with the educational context (see Literature Review section), fills a gap that

relates to the way the users interact with LMS within a b-learning context. In this perspective,

it justifies the hypothesis adopted (see Introduction section), by providing an estimation of

the LMS users’ QoI, compl ing with the findings of Hossain and Brooks (2008) and further

contributing to shed light into the way LMS is used in OLEs; hence, providing valuable

information to both educational decision-makers and software developers towards more

appropriate software development efforts. Moreover, the FCM-QoI model counterparts the

CSFs of Salmeron (2009) and Yesil et al. (2013), providing quantitative measure (i.e., the

QoI) for the LMS users’ interaction attitude, assisting to more effective categorization of the

main activities that are essential towards optimization of the OLEs functionality. Similarly to

the work of Nownaisin et al. (2012), the FCM-QoI model identifies the causalities within the

different t pes of users’ interactions with the LMS that influence their QoI, presented in an

easy to use and easy to understand interlinked graph representation. In accordance to the aims

of Chrysafiadi and Virvou (2013), the FCM-QoI model allows dynamic monitoring of LMS

users’ QoI across the academic ear; hence, contributing to the perspective of OLEs from a

dynamic rather static view, taking into account the alterations in the QoI of the LMS users

with time. Furthermore, since the FCM-QoI model identifies the different LMS interaction

attitude between professors and students, in terms of their QoI, could be combined with the

findings of Peña-Ayala and Sossa-Azuela (2014), for more holistically approaching their

modeling of the teaching-learning processes within OLEs.

5.5.2 Comparison with the FuzzyQoI Model

31
 Apparently, the FCM-QoI model stems from the FuzzyQoI one (Dias & Diniz, 2013),

since its training phase is based on the data provided by the FuzzyQoI model.

Nevertheless, when looking from an overall perspective at these two models, the following

could be identified:

 The knowledge representation in FuzzyQoI model was realized through 600

IF/THEN rules provided by an expert in the field (Dias & Diniz, 2013). In the

case of FCM-QoI model, however, this knowledge is reflected in the

or matrices, automatically derived after the training process.

 In the FuzzyQoI model, the way the estimated QoI output is interconnected with

all 14 categorized LMS input metrics is not explicitly tangible, as it is implied

through the 600 expert’s defined IF/THEN fuzzy rules (Dias & Diniz, 2013). On

the contrary, in the FCM-QoI model, there is a direct and explicit representation

of the way each one of the 14 categorized LMS input metrics is interconnected

with the others and the estimated QoI output, visualized with an easy and

comprehensible way via the FCM.

 Unlike the FuzzyQoI model, the FCM-QoI one incorporates a training process,

since it draws its inference upon QoI after learning from historical data.

 Dissimilarly to the FuzzyQoI model that remains the same for both professors and

students, the FCM-QoI one exhibits a different structure, in terms of the concept

interconnection weight values, for the professors and the students. This provides a

higher degree of adaptation to the FCM-QoI model compared to the FuzzyQoI

one, according to the characteristics of each user’s t pe.

 Both FCM-QoI and FuzzyQoI models share the same 14 inputs - 1 output scheme,

handling the LMS users’ interaction data via the same metrics categorization. This

32
 makes both models comparable in terms of data handling, offering alternative

ways of approaching and analyzing the same LMS data.

5.5.3 FCM-QoI model limitations

Considering the structure and the functionality of the proposed FCM-QoI model, some

limitations could be identified. In particular, the need for historical data to achieve its training

phase reduces its flexibility to adapt to other educational scenarios where there is a lack of

available data. In these cases, however, the interconnection weight matrix could be manually

formed, based on the fusion of the experts’ opinions (e.g., by forming a FL-based fusion

schema). Moreover, according to the presented results, the use of the provides more

efficient estimation of the QoI compared to use of the . This, however, increases the

complexity of the FCM-QoI model, as the former case incorporates a matrix

rather than the latter case, which incorporates just a matrix. Finally, the FCM-QoI

model was trained using the mean values of across users provided by the FuzzyQoI

model; this, however, merges the specific characteristics of each user to an average behavior.

Apparently, following the course of each user across the academic years (or semesters), a

“personalized” set of FCMs per user could be formed, using as training data the ones

from the first academic year (or semester) and estimating the at the next academic

years (or semesters).

5.5.4 FCM-QoI model implications

As mentioned before, recent research has demonstrated that the users’ interactions are

relatively low in the LMSs. Considering the impact and benefits/opportunities of LMSs that

have been created in the last decade, it is essential to ensure their successful

implementation/adoption and development. From this perspective, the proposed FCM-QoI

model arising from this study, based on QoI of the LMS users, clearly constitutes a practical

33
implication to the domain of FCM research in the HE context. In the latter, managerial

implications could be traced; actually, based on the concepts and the causal dependencies, the

proposed FCM-QoI model can help pedagogical planners/instructors to holistically visualize,

capture, understand, and assess stakeholders’ needs and their interdependencies. In other

words, the FCM-QoI approach, here, can be used as ideal mechanism/feedback to support

different stakeholder groups (including department heads, teachers, administrators, technical

support staff, and learners) in the domain of HE, combining knowledge by mathematically

aggregating individual/collective FCM models. Moreover, the causal relationships within the

model are weighted in the sense that the strength of the influence of a causal factor on an

effect factor can be assigned. Objectively, the assigned strength values to causal relationships

in the proposed FCM-QoI model can reveal which factors are believed to have a greater

effect on the adoption of online tools in the LMS Moodle, in a kind of simplified and global

visualization for all combinations of inputs and output. Generally speaking, each concept can

be examined, not only on its own values but also on the associated ones interconnected to it

through the causal fuzzy weights. In this way, the analysis of a particular model can allow the

expert to view the holistic picture, as the identified factors evolve, allowing the incorporation

of an eclectic (strategic) perspective. Under this line, the fuzzy weights express the

magnitude of change that a variable may undertake due its causal relationship with other

variables.

Seen as a holistic and dynamic model, the FCM-QoI approach has the potential to explore

possibilities and scenarios from different perspectives; for instance, pedagogical planners and

decision makers can (re)adjust online tools, towards maximum use of the LMS Moodle

within the teaching and learning practices. To exemplify such capabilities, the five

undergraduate courses that data come from could be considered as an application space.

Although each course was not considered the unit of analysis here, as in the case of Dias et al.

34
(2014b, 2014c), some useful general directions, instructions and directives could be

identified. From one hand, as it was shown in §5.3 and §5.4, some concepts contribute more

than others to self-sustained feedback cycles within the FCM-QoI model, whereas the

complexity of these cycles, in terms of the number of concepts involved that directly affect

the FCM-QoI output, varies according to the time-period. From the other hand, the learning

goals of the b-learning courses are pursued within the same time-period; hence, an optimum

direction to be followed would be the adaptation of the online part of b-learning towards the

boosting of these important concepts identified by the FMC-QoI model, resulting in the

users’ increased QoI with the LMS, thus, contributing in their effort to achieve such learning

goals.

Taking for example the course of dance, from a pedagogical planning the aims are set in

the construction of a place of experience and experimentation with different materials,

choreographic and contextual, along with specific techniques for anal sis, leading to “know-

how” and the enlargement and consolidation of formal and expressive repertoire of the

students. Moreover, it is intended that students will be able to play with fluency and accuracy

through demonstration sequences danced in technical context; cooperate with colleagues in

group tasks; interact with faculty and/or colleagues by actively participating in the tasks;

assess their technical performance and that of others and their participation in groups (Dias et

al., 2014c). Considering the concepts involved in the main self-sustained feedback cycles

(e.g., S3 in Table 5), it could be instructed that a shift towards more interactive and appealing

online resources (e.g., quizzes, discussion forums, blogs, videos, e-portfolios), triggered at

specific time-periods, would increase the interest to LMS interaction, seeing the latter as a

more enhanced source of information, which could accompany the face-to-face interaction

and complement the effort towards multifaceted way of learning.

35
 The aforementioned exemplifies how the FCM-QoI model could attach a dynamic

character to the role of LMS towards an ‘intelligent’ LMS, assisting both teachers and

students to enhance the quality of the instructional environment. The main challenge in this

endeavor, however, is the integration of different learning settings, different learner groups

and different sources, along with the challenge to provide a dynamic and flexible interface to

set up and maintain the list of external resources. In this vein, the FCM-QoI model could

extend the view of the LMS within the knowledge society, which demands competencies and

skills that require innovative, personalized educational practices, based on open sharing and

the evaluation of ideas, fostering creativity and teamwork among the learners. Towards such

endeavor, new approaches to content and network analysis, technical integration of different

LMS, equipped with flexibility and adaptivity, should be considered.

In addition, a large real-life database was used to illustrate the power and applicability of

the method adopted; in fact, very few organizations/HEIs are far along the maturity curve in

dealing with the process of discovering useful patterns and trends in large data sets (i.e., big

data); however the ability to address big data is going to be the most important infrastructure

change for information technology in the next decade, with major implications for knowledge

management (Larose & Larose, 2014). Furthermore, a relatively good visualization of the

LMS use in the form of FCM can be obtained in a short time, giving a clear “mental

landscape” of the s stem and revealing some of the assumption of the “mental modeling”

behind the stakeholders’ perspectives. This could assist the pedagogical instructors of HEIs to

manage the constant change and development that we are likely to see over the next few

years, not just in the application of new technologies to learning, but in the overall approach

to teaching. In this vein, HEIs will need to have in place clear strategic directions. If

innovation is to be a key strategy, then HEIs will need to have in place strong governance

mechanisms for decision-making around the choice and use of technology, to ensure

36
consistency and quality, while allowing as much freedom as possible for instructors to choose

the technologies and learning environments that best fit the needs of their students. Given the

inexorable development of new technologies, it is not really possible to isolate the HEIs from

the changes going on all around it. ICT and teaching will continue to bounce off one another

over time. Technology feeds and amplifies this trend towards change. These changes are not

necessarily good (or bad), but HEIs, instructors and students do need strategies to prepare

themselves for a continuously and rapidly changing environment for teaching and learning.

The scalability of the FCM-QoI model to capture both fine-grained and more abstract

characteristics of the users’ QoI with LMS, along with its adaptivity to time-dependencies,

provide pedagogical instructors of HEIs with different levels of ‘anal sis resolution’ of the

concepts involved and their functional interconnections (causalit ) that promote the users’

QoI. This allows for better understanding, quantitative assessment of transitional and

dynamic changes that could not be accessed before (e.g., access to and evaluation of the role

of specific concepts within feedback cycles at specific time-periods), allowing for flexible

regulations, targeted enhancements and directed scenarios that accommodate the needs of the

stakeholders involved within the b-learning context.

6 Conclusions and Future Work

An original approach to the estimation of the QoI of both professors and students when

interacting with the LMS Moodle within a b-learning context has been proposed in the

present work, introducing the FCM-QoI modeling approach. The FCM-based structure of the

FCM-QoI model provided an easy way to represent LMS Moodle academic community

understanding, in a form of scaled up mental modeling, as a kind of internal representation of

external reality. The efficient performance of the FCM-QoI model was validated on real data,

incorporating adequate number of users and LMS data logs from a HEI. Combined with the

FuzzyQoI model (Dias & Diniz, 2013), the FCM-QoI one extends further the potentiality of

37
the FL to represent the LMS users’ attitude in terms of their QoI. As an expert and intelligent

system, the presented FCM-QoI model resembles the form of adaptive inferencing about QoI,

since it represents a recursive ‘learning’ procedure for gradually modifying connection

strengths driven by the available data, up to a satisfactory RMSE, shaping the inferential

mechanisms housed by the finally estimated connection weights matrix. The latter translates

the expert’s direct or indirect knowledge acquisition from observation, from sensing and,

perhaps, from measuring the flux of experience. This, however, is a significantly complex

task for humans, especially when multiple analysis dimensions are involved, like the ones

examined here. Most decision tree induction methods used for extracting knowledge in

classification problems do not deal with cognitive uncertainties, such as vagueness and

ambiguity associated with human thinking and perception (Glykas, 2010). The FCM

equipped with the NHR-based learning algorithm embedded within the FCM-QoI model, in

turn, allows for adaptiveness and intelligence, representing the classification knowledge more

naturally to the way of human thinking; hence, the FCM-QoI model constructs a manageable

representation of the system that governs the LMS-based QoI and unlocks the ‘black-box’, by

offering access and control to its elements (concepts), unveiling the way they influence the

final output (i.e., QoI).

Probing to the future, some additional aspects related to the FCM-QoI model could be

considered. In particular, here, the five courses involved in the training and testing data were

unified; in this way, a course effect analysis could be performed and related with the

estimated FCM-QoI outputs, so to examine how the course content affects the users’ QoI.

Since the structure and functionality of the proposed FCM-QoI model are not related to a

specific educational context, it could easily be expanded to the analysis of LMS data from

various fields, e.g., Social Sciences and/or Engineering Education (Lawton et al., 2012),

exploring possible dis/similarities and correlations in the LMS users’ QoI, from an

38
institutional perspective. Moreover, the data used here refer to the LMS Moodle usage from

one year only; the FCM-QoI model, however, could also be applied to similar data from

consequent academic years (e.g., 2010/2011-2013/2014), revealing possible causal

dependencies, converged and/or dispersed interaction trends, all reflected at the FCM-QoI

model response, in order to promote a more objective interpretation of the way LMS Moodle-

based b-learning environments could be realized within the online learning context.

Finally, a hybrid model (e.g., FISFCM-QoI) could also be examined, by considering the

intermediate variables outputted by the FIS1-FIS3 of the FuzzyQoI model, i.e., View,

Addition, Alteration, respectively, along with the two inputs to the FIS5, i.e., Time Period and

Engagement Time (Dias & Diniz, 2013), as five-input concepts to the FISFCM-QoI, keeping

the QoI as its desired output. In this way, the number of concepts will be reduced from 15 of

the FCM-QoI model down to six of the hybrid FISFCM-QoI one, reducing, simultaneously,

the computational complexity.

The exploratory study presented here aims at providing a framework for possible

rethinking of the LMS-based b-learning modeling, extending its views as a more fruitful

source of information about the attitude of its users’ interaction.

Acknowledgments

The first author has been supported by the Foundation for Science and Technology (FCT,

Portugal) (Postdoctoral Grant SFRH/BPD/496004/2013). Moreover, this work was realized

within the framework of the EU FP7-ICT-2011-9-ICT-2011.8.2, under the grant agreement

n° 600676: "i-Treasures" Project (http://i-treasures.eu). Finally, the authors would like to

thank Dr. C. Ferreira of the Faculty of Human Kinetics, University of Lisbon, Portugal, for

his assistance in the retrieval of the LMS Moodle data.

39
Appendix A: LMS Moodle Metrics Categorization

A typical LMS Moodle includes metrics that relate with the user’s activit in the

s stem (both as ‘Action’ and combination of ‘Module’ and ‘Action’) and theoreticall could

all be used as useful input variables to the modeling approach, since they convey information

for all the facets of the user’s activit , expressed via his/her actions in the LMS Moodle

environment. Categorization of such LMS Moodle metrics into 12 basic categories, in which

the summation of the metrics belonging to the same category defines the value of the

corresponding variable, could significantly reduce the number of the input parameters

involved in the modeling design. This approach allows all metrics to be employed in the

inference process and it was initially proposed by Dias and Diniz (2013) in their FuzzyQoI

model. Table A.1 tabulates these 110 metrics used as inputs to the FuzzyQoI model (Dias &

Diniz, 2013) and the corresponding categories ( , also adopted by the

proposed FCM-QoI model. Note that, like in (Dias & Diniz, 2013), two additional categories,

i.e., : {Time Period (TP)}, and : {Engagement Time (ET)}, were also used.

----------------- (Table A.1 about here) -----------------

40
Appendix B: FCM Parameter Updating Process

Adopting the general FCM 4-tuple defined in Methodology section, the FCM

parameter updating process involved in the proposed FCM-QoI model is realized through the

Nonlinear Hebbian Learning (NHL) (Papageorgiou et al., 2003) as follows.

Under the view of NHL, the value of each concept at the iteration is calculated

taking into account the influence of the other concepts with which it is interconnected via the

updated interconnection weights, i.e.:

, (A.1)

where is the number of concepts involved in the FCM, is the maximum number of

iterations, denotes the value of the concept at the iteration , is the value

of the concept at the previous iteration , corresponds to the weight of the

interconnection between and concepts at the previous iteration , whereas is the

sigmoid threshold function, i.e.,

. (A.2)

The updating equation of the in Eq. (A.1) is given by (Papageorgiou et al., 2003):

, (A.3)

where the coefficient is a very small positive scalar factor, namely learning parameter. As

Tsadiras (2008) demonstrated, the inference capability of FCM may be strongly influenced

by the selection of the concept transformation function. To this end, Eq. (A.2) can take the

more general form of:

, (A.4)

41
where is a constant. Moreover, as Papageorgiou, Oikonomou and Kannappan, (2012)

suggest, if convenient, the Eq. (A.1) could be used in a rescaled form, like:

, (A.5)

where are constants. The Eq. (A.5) is employed in order to remove the spurious influence

of inactive concepts (with ) on other concepts, and avoids the conflicts emerge in

cases where the initial values of concepts are 0 or 0.5 (Papageorgiou et al., 2012).

In the FCM-QoI model, the iterative process is terminated when a criterion is satisfied.

The latter depends upon the phase of the FCM, and has the form of:

Training phase criterion: , (A.6)

Testing phase criterion: , (A.7)

where is a tolerance level keeping the variation of the values in Eqs. (A.6) and

(A.7) as low as possible. Apparently, if Eq. (A.6) or Eq. (A.7) is not met during the iterative

procedure, the is adopted as the output of the FCM-QoI model. In the training

phase, when the termination conditions are met, the final weight matrix

is derived and used as the one that expresses the concepts

interconnection knowledge within the FCM of the FCM-QoI model. Consequently, the Eq.

(A.5) during the testing phase of the FCM-QoI model takes the form of

. (A.8)

42
Appendix C: The FCM-Viewer Application

Figure A.1(a) illustrates the interface of the developed FCM-Viewer application. As it can

be seen from the latter, a series of user-selection categories is available. In particular, the

requirement for the FCM-Viewer to function is the loading of the concept interconnection

weights matrix. This could be realized in two ways, i.e., either from a Matlab format (.mat)

file or as a variable from the Matlab workspace. As soon as the weights matrix is loaded, a

series of concept layout parameters becomes available. With the navigation buttons

(PREVIOUS-NEXT-GO TO), the user could navigate to each concept and alter/save its

characteristics or use the defaults button that directly corresponds the default values to all

variables. Moreover, the user could set the weights layout, select the FCM layout type,

enable/disable nodes auto size, set the scaling factor (values greater (smaller) than 1 shrink

(expand) the FCM representation), select the arrows characteristics and the weights layout.

Via the HELP button, a help video is automatically triggered that visually explains all the

functionalities of the FCM-Viewer.

When the aforementioned parameters are set, the FCM could be created and visualized in

a new window (Figure A.1(b)) with available menu functionalities of saving, exporting,

moving, zoom in/out, and node right-click functionalities, such as highlighting of the

ancestors/descendants or both, along with the node properties. The visualized FCM could

also be exported as a visual object handle and loaded again as a separate object (see Figure

A.1(a)-bottom), to facilitate the easy access to existing FCMs.

----------------- (Fig. A.1 about here) -----------------

It should be noted that the FCM-Viewer requires prior installation of the Matlab

Bioinformatics Toolbox; hence, with the initiation of the FCM-Viewer application, the user is

automatically informed about the existence or not of the specific toolbox in the currently

43
running Matalb version (>R2011 version is preferable). Finally, a mouse-over help is

available for each button of the FCM-Viewer, whereas a series of warnings/information

messages or returns to previous states is evoked when a wrong selection is performed by the

user.

44
References

Acampora, G., Loia, V., & Vitiello, A. (2011). Distributing emotional services in Ambient

Intelligence through cognitive agents. Service Oriented Computing and Applications,

5(1), 17-35.

Anusha, G., & Ramana, P. V. (2015). A study on symptoms of stress on college students

using combined disjoint block fuzzy cognitive maps (CDBFCM). International

Journal of Applied Mathematics and Mechanics, 2(3), 177-182.

Barón, H. B., Crespo, R. G., Espada, J. P., & Martínez, O. S. (2014). Assessment of learning

in environments interactive through fuzzy cognitive maps. Soft Computing. DOI:

10.1007/s00500-014-1313-x.

Bastedo, K., & Vargas, J. (2014). Assistive technology research, practice, and theory. In B.

DaCosta, & S. Seok (Eds.), Assistive technology and distance learning: making

content accessible (pp. 233-251). Hershey, USA: IGI Global.

a kasoğlu, A., ölcük, İ. ( 5). Development of a novel multiple-attribute decision

making model via fuzzy cognitive maps and hierarchical fuzzy TOPSIS. Information

Sciences, 301, 75-98.

Bourgani, E., Stylios, C. D., Manis, G., & Georgopoulos, V. C. (2014). Time Dependent

Fuzzy Cognitive Maps for Medical Diagnosis. In A. Likas, K. Blekas, & D. Kalles

(Eds.), Artificial Intelligence: Methods and Applications, Lecture Notes in Computer

Science (Vol. 8445, pp. 544-554). Switzerland: Springer International Publishing.

Bueno, S., & Salmeron, J. L. (2009). Benchmarking main activation functions in fuzzy

cognitive maps. Expert Systems with Applications, 36, 5221-5229.

Cai, Y., Miao, C., Tan, A. H., Shen, Z., & Li, B. (2010). Creating an immersive game world

with evolutionary fuzzy cognitive maps. IEEE computer graphics and applications,

30(2), 58-70.

45
Campaña, J. R., Medina, J. M., & Vila, M. A. (2014). Semantic data management using fuzzy

relational databases. In O. Pivert, S. Zadrożn (Eds.), Flexible approaches in data,

information and knowledge management, studies in computational intelligence (Vol.

497, pp. 115-140). Switzerland: Springer International Publishing.

Carvalho, J. P. (2013). On the semantics and the use of fuzzy cognitive maps and dynamic

cognitive maps in social sciences. Fuzzy Sets and Systems, 214, 6-19.

Chrysafiadi, K., & Virvou, M. (2013). A knowledge representation approach using fuzzy

cognitive maps for better navigation support in an adaptive learning system.

SpringerPlus, 2(1), 1-13.

Chunying, Z., Lu, L., Dong, O., & Ruitao, L. (2011). Research of rough cognitive map

model. In G. Shen, & X. Huang (Eds.), Advanced research on electronic commerce,

web application, and communication, communications in computer and information

science (Vol. 144, pp. 224-229). Berlin Heidelberg: Springer.

Dias, S. B., & Diniz, J. A. (2013). FuzzyQoI model: A fuzzy logic-based modelling of users’

quality of interaction with a learning management system under blended learning.

Computers & Education, 69, 38-59.

Dias, S. B., Diniz, J. A., & Hadjileontiadis, L. J. (2014a). Towards an intelligent learning

management system under blended learning: trends, profiles and modelling

perspectives. In J. Kacprzyk, & L. C. Jain (Eds.), Intelligent Systems Reference

Library, Volume 59, Berlin/Heidelberg: Springer-Verlag, ISBN: 978-3-319-02077-8.

Dias, S. B., Diniz, J. A., & Hadjileontiadis, L. J. (2014b). Exploring B-Learning Scenarios

Using Fuzzy Logic- ased Modeling of Users’ LMS Qualit of Interaction in

Ergonomics and Psychomotor Rehabilitation Academic Courses. Lecture Notes in

Computer Science, 8512, 233-243.

46
Dias, S. B., Hadjileontiadis, L. J., & Diniz, J. A. (2014c). On Enhancing Blended-Learning

Scenarios Through Fuzzy Logic- ased Modeling of Users’ LMS Qualit of

Interaction: The Rare & Contemporary Dance Paradigms. In Proceedings of the 9th

International Joint Conference on Computer Vision, Imaging and Computer Graphic

(VISIGRAPP/IAMICH 2014) (vol. II, pp. 765-772).

Georgiou, D. A., & Botsios, S. D. (2008). Learning style recognition: A three layers fuzzy

cognitive map schema. Proceedings of the IEEE International Conference on Fuzzy

Systems (pp. 2202-2207). Hong Kong, China.

Georgopoulos, V. C., Chouliara, S., & Stylios, C. D. (2014). Fuzzy Cognitive Map scenario-

based medical decision support systems for education. Proceedings of the 36th

Annual International Conference of the Engineering in Medicine and Biology Society

(EMBC) (pp. 1813-1816). Chicago, IL: IEEE.

Glykas, G. (2010). Fuzzy cognitive maps: Advances in theory, methodologies, tools and

applications (1st ed.). Heidelberg: Springer-Verlag.

Glykas, M. (2013). Fuzzy cognitive strategic maps in business process performance

measurement. Expert Systems with Applications, 40(1), 1-14.

Hebb, D.O. (1949). The Organization of Behavior. New York: Wiley & Sons.

Hossain, S., & Brooks, L. (2008). Fuzzy cognitive map modelling educational software

adoption. Computers & Education, 51(4), 1569-1588.

Irani, Z., Sharif, A., Kamal, M. M., & Love, P. E. (2014). Visualising a knowledge mapping

of information systems investment evaluation. Expert Systems with Applications,

41(1), 105-125.

Jetter, A. J., & Sperry, R. C. (2013). Fuzzy cognitive maps for product planning: using

stakeholder knowledge to achieve corporate responsibility. Proceedings of the 46th

47
 Hawaii International Conference on System Sciences (HICSS), (pp. 925-934), Maui,

Hawaii.

Kosko, B. (1986). Fuzzy cognitive maps. International Journal of man-machine studies,

24(1), 65-75.

Kosko, B. (1992). Fuzzy associative memory systems. In A. Kandel (Ed.), Fuzzy Expert

Systems (135-162). Boca Raton, Florida: CRC Press.

Larose, D. T., and Larose, C. D. (2014). Discovering knowledge in data: an introduction to

data mining (2nd ed.). USA, New Jersey: John Wiley & Sons.

Lawton, D., Vye, N., Bransford, J., Sanders, E., Richey, M., French, D., & Stephens, R.

(2012). Online learning based on essential concepts and formative assessment.

Journal of Engineering Education, 101(2), 244-287.

López-Pérez, M., Pérez-López, M. C., & Rodríguez-Ariza, L. (2011). Blended learning in

higher education: Students’ perceptions and their relation to outcomes. Computers &

Education, 56(3), 818-826.

Luo, X., Wei, X., & Zhang, J. (2009). Game-based learning model using fuzzy cognitive

map. Proceedings of the first ACM international workshop on Multimedia

technologies for distance learning (pp. 67-76). Beijing, China.

Mi azoe, T., Anderson, T. ( ). Learning outcomes and students’ perceptions of online

writing: Simultaneous implementation of a forum, blog, and wiki in an EFL blended

learning setting. System, 38(2), 185-199.

Musbahtiti, K., & Muhammad, A. (2013). Improvement quality of LMS through application

of social networking sites. International Journal of Emerging Technologies in

Learning, 8(3), 48-51.

Nápoles, G., Bello, R., & Vanhoof, K. (2013). Learning stability features on sigmoid Fuzzy

Cognitive Maps through a Swarm Intelligence approach. In Progress in Pattern

48
 Recognition, Image Analysis, Computer Vision, and Applications (pp. 270-277).

Springer Berlin Heidelberg.

Nownaisin, P., Chomsuwan, K., & Hongkrailert, N. (2012). Utilization of fuzzy cognitive

map in modeling of Thailand science-based technology school. Proceedings of the

International Conference on Teaching, Assessment and Learning for Engineering

(TALE), (pp. T2D5- T2D7), Hong Kong, China.

Oliver, K., & Stallings, D. (2014). Blended learning: core concepts and evaluation strategies.

Journal of Technology and Teacher Education, 22(1), 57-81.

Papageorgiou, E. I. (2011). A new methodology for decisions in medical informatics using

fuzzy cognitive maps based on fuzzy rule-extraction techniques. Applied Soft

Computing, 11(1), 500-513.

Papageorgiou, E. I. (2012). Learning algorithms for fuzzy cognitive maps-A review study.

IEEE Trans. on Systems, Man, and Cybernetics-Part C: Applications and Reviews,

42(2), 150-163.

Papageorgiou, E. I. (Ed.) (2014). Fuzzy cognitive maps for applied sciences and engineering,

Intelligent Systems Reference Library 54, DOI: 10.1007/978-3-642-39739-4. Berlin

Heidelberg: Springer-Verlag.

Papageorgiou, E. I., & Salmeron, J. L. (2013). A review of fuzzy cognitive maps research

during the last decade. IEEE Transactions on Fuzzy Systems, 21(1), 66-79.

Papageorgiou, E. I., & Stylios, C. D. (2008). Fuzzy cognitive maps. In W. Pedrycz, A.

Skowron, & V. Kreinovich (Eds.), Handbook of Granular Computing (pp. 755-774).

Chichester, England: John Wiley & Son Ltd, Publication Atrium.

Papageorgiou, E. I., Oikonomou, P., & Kannappan, A. (2012). Bagged nonlinear hebbian

learning algorithm for fuzzy cognitive maps working on classification tasks. In I.

Maglogiannis, V. Plagianakos, I. Vlahavas (Eds.), Artificial Intelligence: Theories

49
 and Applications, Lecture Notes in Computer Science (Vol. 7297, pp 157-164). Berlin

Heidelberg: Springer.

Papageorgiou, E. I., Stylios, C., & Groumpos, P. (2003). Fuzzy cognitive map learning based

on nonlinear Hebbian rule. In: T. D., Gedeon, & L.C., Fung (Eds.), AI 2003:

Advances in artificial intelligence (Vol. 2903, pp. 256-268). Heidelberg: Springer.

Parker, A, (1999). Interaction in distance education: The critical conversation. AACE Journal,

1(12), 13-17.

Pedrycz, W. (2010). The design of cognitive maps: A study in synergy of granular computing

and evolutionary optimization. Expert Systems with Applications, 37(10), 7288-7294.

Peña, A., Sossa, H., & Gutierrez, F. (2007). Ontology agent based rule base fuzzy cognitive

maps. In N. T. Nguyen, A. G.Adam, R. J. Howlett, & L.C. Jain (Eds.), Agent and

Multi-Agent Systems: Technologies and Applications, Lecture Notes in Computer

Science (Vol. 4496, pp. 328-337). Berlin Heidelberg: Springer.

Peña-Ayala, A., & Sossa-Azuela, J. H. (2014). Decision making by rule-based fuzzy

cognitive maps: an approach to implement student-centered education. In E. I.

Papageorgiou, (Ed.) Fuzzy cognitive maps for applied sciences and engineering,

Intelligent Systems Reference Library (Vol., 54, pp. 107-120). Berlin Heidelberg:

Springer.

Ping, T. A., Cheng., A. Y., Manoharan, K. ( ). Students’ interaction in the online

learning management systems: A comparative study of undergraduate and

postgraduate courses. Proceedings of the AAOU-2010 Annual Conference (1-14),

Hanoi, Vietnam.

Rodriguez-Repiso, L., Setchi, R., & Salmeron, J. L. (2007). Modelling IT projects success

with fuzzy cognitive maps. Expert Systems with Applications, 32(2), 543-559.

50
Salmeron, J. L. (2009). Augmented fuzzy cognitive maps for modelling LMS critical success

factors. Knowledge-based systems, 22(4), 275-278.

Sharif, A. M., & Irani, Z. (2006). Exploring fuzzy cognitive mapping for IS evaluation.

European Journal of Operational Research, 173(3), 1175-1187.

Song, H., Miao, C., Roel, W., Shen, Z., & Catthoor, F. (2010). Implementation of fuzzy

cognitive maps based on fuzzy neural network and application in prediction of time

series. IEEE Transactions on Fuzzy Systems, 18(2), 233-250.

Stach, W., Kurgan, L., Pedrycz, W., & Reformat, M. (2005). Genetic learning of fuzzy

cognitive maps. Fuzzy Sets and Systems, 153(3), 371–401.

Štula, M., Stipaničev, D., Šerić, L. ( ). Multi-Agent systems in distributed

computation. In G. Jezic, M. Kusek, N.-T. Nguyen, R. J. Howlett, & Lakhmi C. Jain

(Eds.) Agent and Multi-Agent Systems. Technologies and Applications, Lecture Notes

in Computer Science (Vol. 7327, pp. 629-637). Berlin Heidelberg: Springer.

Takács, M., Rudas, I. J., & Lantos, Z. (2014). Fuzzy Cognitive Map for student evaluation

model. Proceedings of the International Conference on System Science and

Engineering (ICSSE) (pp. 284-288). Shanghai: IEEE.

Tsadiras, A. & Stamatis, D. (2008). Decisions on Networked Learning based on Fuzzy

Cognitive Maps. Proceedings of the 6th International Conference on Networked

Learning (NLC2008) (pp. 376-383). Halkidiki, Greece.

Tsadiras, A. K. (2008). Comparing the inference capabilities of binary, trivalent and sigmoid

fuzzy cognitive maps. Information Science, 178, 3880-3894.

Tsadiras, A. K., Kouskouvelis, I., & Margaritis, K. G. (2001). Making political decisions

using fuzzy cognitive maps: The FYROM crisis. In Proceedings of the 8th

Panhellenic conference on informatics (Vol. 2, pp. 501–510).

51
van Vliet, M., Kok, K., & Veldkamp, T. (2010). Linking stakeholders and modellers in

scenario studies: The use of Fuzzy Cognitive Maps as a communication and learning

tool. Futures, 42(1), 1-14.

ašč k, J., Re es, N. H. ( ). Use and perspectives of fuzz cognitive maps in robotics.

In E. I. Papageorgiou (Ed.), Fuzzy Cognitive Maps for Applied Sciences and

Engineering, Intelligent Systems Reference Library (Vol. 54, pp. 253-266). Berlin

Heidelberg: Springer.

Wildenberg, M., Bachhofer, M., Isak, K. G., & Skov, F. (2014). Use and evaluation of FCM

as a tool for long term socio ecological research. In E. I. Papageorgiou (Ed.), Fuzzy

cognitive maps for applied sciences and engineering, Intelligent Systems Reference

Library (Vol. 54, pp. 221-236). Berlin Heidelberg: Springer.

Woltering, V., Herrler, A., Spitzer, K., & Spreckelsen, C. (2009). Blended learning positively

affects students’ satisfaction and the role of the tutor in the problem-based learning

process: results of a mixed-method evaluation. Advances in Health Sciences

Education, 14(5), 725-738.

Yang, F., Li, F. W., & Lau, R. W. (2011). Fuzzy cognitive map based student progress

indicators. In H. Leung, E. Popescu, Y. Cao, R.W.H. Lau, & W. Nejdl (Eds.)

Advances in Web-Based Learning (ICWL 2011), Lecture Notes in Computer Science

(Vol. 7048, pp. 174-187). Berlin Heidelberg: Springer.

Yesil, E., Ozturk, C., Dodurka, M. F., & Sahin, A. (2013). Control engineering education

critical success factors modeling via Fuzzy Cognitive Maps. Proceedings of the 12th

International Conference on Information Technology based Higher Education and

Training (ITHET) (pp. 1-8). Antalya, Turkey.

52
53
Figure captions

Fig. 1. A schematic representation of a FCM model with four concepts (i.e., ), corresponding to the
interconnection weight matrix of (1).

Fig. 2. A schematic representation of the proposed FCM-QoI model, with the LMS Moodle user’s interaction
metrics ( categorized into 14 input parameters ( fed to (a) the FuzzyQoI model (Dias &
Diniz, 2013) outputting the estimated and (b) the FCM-QoI model as input concepts interconnected with
the FCM-QoI as an output concept to estimate the . Note that the estimated is fed to the FCM-
QoI model during the training phase only, to correctly adjust the interconnection weights of the latter towards
the minimization of the error in the knowledge representation process.

Fig. 3. A schematic representation of the time-period segmentation of the examined academic year 2009/2010,
spanning 51 weeks, involving three distinct periods, i.e., Lecture, Interruption (from left to right: Christmas,
Carnival, Easter, Summer) and Exam, corresponded to specific weeks, for the first (soft-stems) and the second
(heavy-stems) semester.

Fig. 4. (a) The estimated (dense black line), which corresponds to the iteration 291 that
exhibits the minimum RMSE, and (b) the estimated (dense black line), which corresponds to
the iteration (415) that exhibits the maximum RMSE. In both subplots, the corresponding (light gray
line) derived from the FuzzyQoI model (Dias & Diniz, 2013) and used for the time-dependent training phase of
the FCM-QoI model is superimposed for comparison reasons. The vertical lines illustrate the time-period
segmentation of Fig. 3.

Fig. 5. The estimated professors’ (thick grey line) during the testing phase of the time-
dependent scenario, along with the corresponding (light black line) derived from the FuzzyQoI
model (Dias & Diniz, 2013) and used for the time-dependent testing phase of the FCM-QoI model. The vertical
lines illustrate the time-period segmentation of Fig. 3.

Fig. 6. The estimated (diamonds) along with the (dots), corresponding to the double
averaged (across the professors and across the weeks) values estimated from the FuzzyQoI model
(Dias & Diniz, 2013), across the 1000 iterations of the training phase of the time-independent scenario. The
solid lines correspond to the and , respectively.

Fig. 7. The FCM corresponding to the interconnection weights matrix between the 15 concepts [14
input concepts and as the output concept of the FCM-QoI model], shown in Table 1. The
, enclosed by thick circle border, correspond to the concepts that are directly connected to the
output concept.

Fig. 8. The estimated professors’ (dense black line) during the testing phase of the
time-independent scenario, along with the corresponding (light grey line) derived from the
FuzzyQoI model (Dias & Diniz, 2013) and used for the time-independent testing phase of the FCM-QoI model.
The vertical lines illustrate the time-period segmentation of Fig. 3.

Fig. 9. (a) The estimated (grey line), which corresponds to the iteration 87 of Fig. 12 that
exhibits the minimum RMSE, and (b) the estimated (grey line), which corresponds to the
iteration (554) that exhibits the maximum RMSE. In both subplots, the corresponding (black line)
derived from the FuzzyQoI model (Dias & Diniz, 2013) and used for the time-dependent training phase of the
FCM-QoI model is superimposed for comparison reasons. The vertical lines illustrate the time-period
segmentation of Fig. 3.

Fig. 10. The estimated students’ (grey line) during the testing phase of the time-
dependent scenario, along with the corresponding (black line) derived from the FuzzyQoI model
(Dias & Diniz, 2013) and used for the time-dependent testing phase of the FCM-QoI model. The vertical lines
illustrate the time-period segmentation of Fig. 3.

54
Fig. 11. The estimated (diamonds) along with the (dots), corresponding to the double
averaged (across the students and across the weeks) values estimated from the FuzzyQoI model
(Dias & Diniz, 2013), across the 1000 iterations of the training phase of the time-independent scenario. The
solid lines correspond to the and , respectively.

Fig. 12. The FCM corresponding to the interconnection weights matrix between the 15 concepts [14
input concepts and as the output concept of the FCM-QoI model], shown in Table 2. The
, enclosed by thick circle border, correspond to the concepts that are directly connected to the output
concept.

Fig. 13. The estimated students’ (grey line) during the testing phase of the time-
independent scenario, along with the corresponding (black line) derived from the FuzzyQoI
model (Dias & Diniz, 2013) and used for the time-independent testing phase of the FCM-QoI model. The
vertical lines illustrate the time-period segmentation of Fig. 3.

Fig. 14. The estimated FCMs from the training phase of the time-dependent scenario that correspond to the
week-pairs of (a) professors (P-PR1, P-PR2, P-PR3, P-PR4) and (b) students (S-PR1, S-PR2, S-PR3, S-PR4),
visualised using the FCM-Viewer (see Appendix B). For both cases, PR1={15,16}, PR2={17,18}, PR3={22,23}
and PR4={37,38}, representing the transition from Lecture (1st semester) to Interruption (Christmas),
Interruption (Christmas) to Exam (1st semester), Exam (1st semester) to Lecture (2nd semester) and Lecture (2nd
semester) to Exam (2nd semester), respectively. In (a) and (b), the concepts with thick border represent those
that are directly connected to the corresponding output concept, i.e., , where # denotes the
current week. Note that in all FCMs, the interconnection weights have been omitted to facilitate the visualisation
of the FCMs.

Fig. A.1. (a) The interface of the FCM-Viewer developed application with the main user-selection categories,
including: loading of the weights matrix, selection of the layout type, nodes auto size and scaling, concept
navigation, concept layout characteristics selection, concept settings saving, arrows characteristics selection,
weights layout, default use, help recall, exporting/loading visual object handles and, finally, creation and
visualization of the FCM in a new window, as shown in (b), with additional menu and node right-click
functionalities of the FCM.

55
Fig1
 Fig2

𝑀1

𝑀110
…

LMS MOODLE
FCM-QoI model
METRICS
CATEGORIZATION
𝐶1

𝑄𝑜𝐼 𝐹𝐶𝑀
𝐶14

FuzzyQoI model

Phase
Training
(a)

𝑄𝑜𝐼 𝐹𝐼𝑆

(b)
Fig3
Fig4

(a)

(b)
Fig5
Fig6
Fig7
Fig8
Fig9

(a)

(b)
Fig10
Fig11
Fig12

Fig. 12. The FCM corresponding to the interconnection weights matrix between the 15
concepts [14 input concepts and as the output concept of the FCM-QoI model], shown in
Table 2. The , enclosed by thick circle border, correspond to the concepts that are
directly connected to the output concept.
Fig13
Fig14

P-PR1

P-PR2

P-PR3

P-PR4

(a)
S-PR1

S-PR2

S-PR3

S-PR4

(b)
FigA1

(a)

(b)
Table 1
Research studies regarding to the FCMs use for educational scenarios along with corresponding
objective(s) and problem solving areas.

Problem Solving

Control and guiding

Analysis-Evaluation
Progress indicators
Framework setting
Decision-making

Factors analysis

representation
Research Study Objective(s)

Knowledge
Prediction
Modeling

Learning
Planning
Tsadiras and To examine the use of    
Stamatis (2008) FCMs for planning
Network Learning and for
enhancing the success of
learning programs.
Salmeron (2009) To model critical success   
factors in LMS.
Hossain and Brooks To model educational  
(2008) software adoption at UK
secondary schools based on
stakeholders’ perceptions.
Yang, Li and Lau To propose a set of  
(2011) student’s progress
indicators based on the
FCM and describe his/her
progress on various aspects
and their causal
relationships.
Nownaisin, To identify the causalities  
Chomsuwan and of the education
Hongkrailert (2012) management of the
Thailand science-based
technology school.
Yesil, Ozturk, To model the control  
Dodurka and Sahin engineering educational
(2013) critical success factors.
Barón, Crespo, To present a learning  
Espada and Martínez assessment system that uses
(2014) multivariate analysis based
on structural equation
modeling, facilitating the
assessment of learning on
interactive environments.
Peña-Ayala and To model the teaching-    
Sossa-Azuela (2014) learning environment and
simulate the bias exerted by
authored lectures on the

56
 Problem Solving

Control and guiding

Analysis-Evaluation
Progress indicators
Framework setting
Decision-making

Factors analysis

representation
Research Study Objective(s)

Knowledge
Prediction
Modeling

Learning
Planning
student’s learning.
a kasoğlu and To propose a new model,  
Gölcük (2015) that integrates the
Technique for Order
Preference by Similarity to
Ideal Solution and FCMs
for modeling and solving
decision problems related to
industrial engineering.
Georgiou and To identify learning styles  
Botsios (2008) in Adaptive Educational
Hypermedia Systems.
Georgopoulos, To expand the utilization of  
Chouliara and FCM based on Medical
Stylios (2014) Decision Support Systems
for learning and educational
purposes using a scenario-
based learning approach.
Takács, Rudas and To introduce a novel FCM   
Lantos (2014) based algorithm for the
calculation of the values of
the interrelation levels
between the factors in a
system for the student’s
grade evaluation.
Chrysafiadi and To contribute to the  
Virvou (2013) improvement of the
navigation support in an e-
learning adaptive system,
for teaching computer
programming.
van Vliet, Kok and To describe an updated 
Veldkamp (2010) framework for participatory
scenario, bridging the gap
between storylines and
models.
Luo, Wei and Zhang To design a game-based 
(2009) learning system.
Anusha and Ramana To analyze the symptoms of 
(2015) stress on college students
using combined disjoint
block fuzzy cognitive maps
model.

57
Table 2
The scenarios and their phases per user (Professors, Students) involved in the FCM-QoI model.

SCENARIOS
Users’ Categor
Time Dependent Time Independent
Professor Training Phase Testing Phase Training Phase Testing Phase
Student Training Phase Testing Phase Training Phase Testing Phase

58
Table 3
The concept interconnection weights of the FCM-QoI model corresponding to the
embedded within the FCM of Figure 7.

0 0 0 0.10 0 0 0 -0.13 0 0.13 0 0 0 0 0.24

0.13 0 -0.13 0.11 0.1 0 -0.12 0 0 0 0.17 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0.24 0.11 0 0 -0.11 0 0 0 0.15 0 0 0 0 0.23
-0.13 0 0 0 0 0 -0.11 0.23 0 0 0 0 0 0 -0.17
-0.19 0 0.1 0 -0.11 0 0 0 0 0 0.14 0 0 0 0.13
0 0 -0.17 0 0 0 0 0 -0.11 0 0 0 0 -0.12 0.12
0 0 0 -0.11 0 -0.16 0 0 0 0 0 0 0 0.11 0
0 -0.21 -0.22 0.13 0 0 -0.21 -0.21 0 0 0 0 0.13 0 0
0 0 0 0 0 0 -0.25 0 0 0 0 0 0 0 0
0 0 0 0.22 0 0.24 0 -0.11 -0.13 0.11 0 0 0 0 0
0.13 -0.13 0 0.14 0 0 0 0 0 0 0 0 0.16 0 -0.29
0 0 0 0.24 0 0.11 0.17 0 0.1 -0.14 0 0 0 0.1 0
0 0 0 -0.23 0 0 0 0 0 0 0 -0.11 0 0 0.14
0 -0.14 0 0 0.11 0 0 0 0.12 0 0 0 0 0 0

59
Table 4
The concept interconnection weights of the FCM-QoI model corresponding to the
embedded within the FCM of Figure 12.

0 0 0 0 0 0.17 0 0 -0.12 -0.11 0 0 0 0 0

0 0 0.13 0 0 0 0 0 0 0.13 -0.12 0 0 0 0.11
0 0 0 -0.11 0 0 0 0 0.1 0 0 -0.18 -0.25 0 -0.18
-0.17 0 0 0 0 0.2 -0.11 0.1 -0.15 -0.13 -0.2 0 0 0 0
0 0 0 0.19 0 0 0 0 0 0 0 0 -0.14 0 0.13
0 0 0 0 -0.11 0 0 0 -0.15 -0.15 0 0 0 0 0
0 0 0 0 0 -0.16 0 0.15 0 0.19 0.18 0 0.15 -0.17 0.2
0.13 0 0.1 0 0 -0.12 -0.18 0 0 -0.12 0 0 0 0 0
0 -0.25 0 0 0 0 0 0 0 0 0 0 0.18 -0.15 0
0 0.12 0 -0.12 0 0 0 0.1 0 0 0 -0.15 0 0 0
0 0 0 -0.23 0 0 -0.15 0 0 0 0 0 0 0 0
-0.13 0 0 0 -0.13 0 -0.12 0.13 -0.13 0 -0.17 0 0 0 0
-0.17 0 0 -0.12 0.2 0 0 0 0 -0.14 0 0.1 0 0 0
0 -0.13 0 0 0 0 0 0 0 0.26 0 0 0 0 -0.14
0 0 -0.11 0 -0.14 0 0 0 0.13 0 0 0.26 0 0 0

60
Table 5
The main feedback cycles of the estimated FCMs for the cases of Professors and Students under the
time-independent scenarios (see Figures 7 and 12, respectively). Note that these feedback cycles are
structured by the concepts that are directly connected with the , illustrated as bold
circles in Figures 7 and , respectivel ; in the case of Students, however, some ‘auxiliar ’ concepts
(denoted within brackets) are needed to close such feedback cycles.

Users No. Main Feedback Cycles Cycle Sign

P1 negative
P2 negative
Professors (Figure 7)

P3 negative
P4 negative
P5 positive
P6 negative
P7 negative
P8 positive
P9 positive
S1 positive
Students (Figure 12)

S2 negative
S3 positive
S4 negative
S5 negative
S6 positive
S7 negative

61
Table A.1
Categorization of all LMS Moodle metrics (in their original appearance) used as inputs to the FuzzyQoI model (Dias & Diniz, 2013) to the corresponding
categories ( , also adopted by the proposed FCM-QoI model. Note that, like in (Dias & Diniz, 2013), two additional categories, i.e., :
{Time Period (TP)}, and : {Engagement Time (ET)}, were also used (not included in the list below).

LMS Moodle metrics

Combination of 'Module' and 'Action' Category Name
'Action'
'Module' 'Action'
View all 4 Course Page (CP)
View discussion 2 Forum/Discussion/Chat (F/D/C)
View form 1 Journal/Wiki/Blog/Form (J/W/B/F)
View forum(s) 2 Forum/Discussion/Chat (F/D/C)
View grade 3 Submission/Report/Quiz/Feedback (S/R/Q/F)
View graph 3 Submission/Report/Quiz/Feedback (S/R/Q/F)
View report 3 Submission/Report/Quiz/Feedback (S/R/Q/F)
View responses 2 Forum/Discussion/Chat (F/D/C)
View submission 3 Submission/Report/Quiz/Feedback (S/R/Q/F)
View subscribers 1 Journal/Wiki/Blog/Form (J/W/B/F)
Preview 4 Course Page (CP)
Info 1 Journal/Wiki/Blog/Form (J/W/B/F)
Links 1 Journal/Wiki/Blog/Form (J/W/B/F)
Search 4 Course Page (CP)
Mail error 2 Forum/Discussion/Chat (F/D/C)
Mark read 2 Forum/Discussion/Chat (F/D/C)
Templates view 4 Course Page (CP)
Course View 4 Course Page (CP)
Glossary View 1 Journal/Wiki/Blog/Form (J/W/B/F)
Resource View 4 Course Page (CP)
Assignment View 3 Submission/Report/Quiz/Feedback (S/R/Q/F)
Feedback View 3 Submission/Report/Quiz/Feedback (S/R/Q/F)
User View 4 Course Page (CP)
Choice View 3 Submission/Report/Quiz/Feedback (S/R/Q/F)

62
 LMS Moodle metrics
Combination of 'Module' and 'Action' Category Name
'Action'
'Module' 'Action'
Wiki View 1 Journal/Wiki/Blog/Form (J/W/B/F)
Lesson View 1 Journal/Wiki/Blog/Form (J/W/B/F)
Blog View 1 Journal/Wiki/Blog/Form (J/W/B/F)
Notes View 1 Journal/Wiki/Blog/Form (J/W/B/F)
Quiz View 3 Submission/Report/Quiz/Feedback (S/R/Q/F)
Scorm View 3 Submission/Report/Quiz/Feedback (S/R/Q/F)
Data View 4 Course Page (CP)
Flashchat View 2 Forum/Discussion/Chat (F/D/C)
Hotpot View 3 Submission/Report/Quiz/Feedback (S/R/Q/F)
Chat View 2 Forum/Discussion/Chat (F/D/C)
Journal View 1 Journal/Wiki/Blog/Form (J/W/B/F)
Add mod 5 Module (M)
Add post 6 Post/Activity (P/A)
Add entry 7 Resource/Assignment (R/A)
Add discussion 6 Post/Activity (P/A)
Add comment 6 Post/Activity (P/A)
Choose 6 Post/Activity (P/A)
Choose again 6 Post/Activity (P/A)
Start 5 Module (M)
Start complete 5 Module (M)
End 5 Module (M)
Fields add 5 Module (M)
Report 6 Post/Activity (P/A)
Report live 6 Post/Activity (P/A)
Report log 6 Post/Activity (P/A)
Report outline 6 Post/Activity (P/A)
Report participation 6 Post/Activity (P/A)
Report stats 6 Post/Activity (P/A)
Submit 6 Post/Activity (P/A)
Subscribe 6 Post/Activity (P/A)
Subscribe all 6 Post/Activity (P/A)

63
 LMS Moodle metrics
Combination of 'Module' and 'Action' Category Name
'Action'
'Module' 'Action'
Talk 6 Post/Activity (P/A)
Attempt 6 Post/Activity (P/A)
Continue attempt 6 Post/Activity (P/A)
User report 6 Post/Activity (P/A)
Bogus 6 Post/Activity (P/A)
Set page flags 6 Post/Activity (P/A)
Resource Add 7 Resource/Assignment (R/A)
Chat Add 6 Post/Activity (P/A)
Forum Add 6 Post/Activity (P/A)
Label Add 8 Label (L)
Assignment Add 6 Post/Activity (P/A)
Feedback Add 6 Post/Activity (P/A)
Choice Add 6 Post/Activity (P/A)
Calendar Add 6 Post/Activity (P/A)
Wiki Add 6 Post/Activity (P/A)
Glossary Add 6 Post/Activity (P/A)
Data Add 5 Module (M)
Lesson Add 5 Module (M)
Flashchat Add 6 Post/Activity (P/A)
Hotpot Add 6 Post/Activity (P/A)
Journal Add 6 Post/Activity (P/A)
Quiz Add 6 Post/Activity (P/A)
Scorm Add 5 Module (M)
Survey Add 5 Module (M)
Assign 11 Assign (A)
Unassign 12 Edit/Delete (E/D)
Enrol 11 Assign (A)
Unenrol 12 Edit/Delete (E/D)
Unsubscribe 12 Edit/Delete (E/D)
Upload 9 Upload (UP)
Attachment 9 Upload (UP)

64
 LMS Moodle metrics
Combination of 'Module' and 'Action' Category Name
'Action'
'Module' 'Action'
Update 10 Update (U)
Update comment 10 Update (U)
Update entry 10 Update (U)
Update feedback 10 Update (U)
Update grades 10 Update (U)
Update mod 10 Update (U)
Update post 10 Update (U)
Fields update 10 Update (U)
Recent 10 Update (U)
Move discussions 10 Update (U)
Templates saved 9 Upload (UP)
Edit 12 Edit/Delete (E/D)
Edit section 12 Edit/Delete (E/D)
Edit questions 12 Edit/Delete (E/D)
Delete 12 Edit/Delete (E/D)
Delete all 12 Edit/Delete (E/D)
Delete attempt 12 Edit/Delete (E/D)
Delete comment 12 Edit/Delete (E/D)
Delete discussion 12 Edit/Delete (E/D)
Delete entry 12 Edit/Delete (E/D)
Delete mod 12 Edit/Delete (E/D)
Delete post 12 Edit/Delete (E/D)
Change password 12 Edit/Delete (E/D)
Review 12 Edit/Delete (E/D)

65
Highlights
 Fuzzy modelling of LMS Moodle users’ quality of interaction within b-learning

 FCM-QoI model efficiently identifies LMS interaction trends

 QoI dependencies exist with the time-period of the LMS use

 FCM-QoI supports better understanding of the LMS users’ interaction behaviour

66