Computers & Education 68 (2013) 105–116

Contents lists available at SciVerse ScienceDirect

Computers & Education

journal homepage: www.elsevier.com/locate/compedu

Using a concept map knowledge management system to enhance the

learning of biology
Shih-Hwa Liu*, Gwo-Guang Lee
Department of Information and Management, National Taiwan University of Science and Technology, No. 43, Sec. 4, Keelung Rd., Da’an Dist., Taipei 106,
Taiwan, ROC

a r t i c l e i n f o a b s t r a c t

Article history: This study was undertaken with the goal of developing a Concept Map Knowledge Management System
Received 8 June 2012 (CMKMS) for use as a tool in observing change in a student’s understanding of biology concepts over
Received in revised form time. The CMKMS should be useful in assessing the extent of a student’s knowledge and in revealing
2 May 2013
their unique thought processes. The study was concerned not only with a student’s self-evaluation of
Accepted 5 May 2013
learning but also with their level of satisfaction after using the CMKMS. The CMKMS combines the
diagnosis of concept mapping with the style of thought processing, and promotes teaching activities
Keywords:
step-by-step, in order to promote effective student learning. In analyzing the factors that influence
Architectures for educational technology
system the effect of teaching, the teacher can determine a student’s knowledge structure and highlight mis-
Cooperative/collaborative learning conceptions by inspecting the concept maps and logs. The results of this study show that knowledge
Teaching/learning strategies management involved in computer-aided instruction in the teaching of biology had a positive influence
on learning effectiveness. The CMKMS also was useful in promoting the student’s thought processing,
creativity, and ability to judge.
Ó 2013 Elsevier Ltd. All rights reserved.

1. Introduction

Although modern biology problems are complex, teachers want students to learn to explain patterns and processes in the natural world
and to be able to make predictions about system behaviors. Concept mapping as a means for students to learn new information has been
widely recommended and accepted in science, mathematics, and educational psychology (Brown, Nagel, Zlatanova & Kolbe, 2013; Chiou, Lee
& Liu, 2012; Empson, Greenstein, Maldonado & Roschelle, 2013; Wijeysundera et al., 2013). Concept mapping allows students to apply
nodes, labels, and links in order to graphically represent the relationships among multiple concepts (Tseng, Chang, Lou, Tan & Chiu, 2012).
Students can draw concept maps to explore the incompleteness of a concept or a defect in understanding. Therefore, concept mapping can
help students to improve their conceptual understanding and improve their attitude toward a new course (Novak, 1990).
A concept map is also an effective and efficient tool that can be used to evaluate student learning, because concept maps reflect a
student’s internal semantic networks with regard to the knowledge they perceive, accumulate, and comprehend (Hung, Hwang, Su & I-Hua,
2012; Wu, Hwang, Milrad, Ke & Huang, 2012). There, the teacher can estimate a student’s learning by evaluating the content and structure of
a concept map they have constructed. The traditional method of concept map scoring was based on the components and structure of the
concept map, whereby points are assigned for valid lists of concepts, cross-links, propositions, and examples derived from expert-prepared
maps: 1 point for each proposition, 1 point for each branch, 1 point for each example, 5 points for each level of hierarchy, and 10 points for
each valid cross-link (Novak & Gowin, 1984). Some researchers have sought to confirm and extend a computer-based technique for auto-
matically scoring concept maps (Cline, Brewster & Fell, 2010; Lee, Lee & Leu, 2009). Ruiz-Primo and Shavelson (1996) have described
methods that can be used to compare a student’s map to that of an expert. Expert maps may be constructed by a teacher, a domain expert or
a group of teachers or domain experts. Some researchers have experimented with a combination of methods based on traditional
component-based scoring and methods based on comparisons to expert maps in order to weigh the propositions put forth in student maps
(Jeong & Lee, 2012; Soleimani & Nabizadeh, 2012).

* Corresponding author. Tel.: þ886 2 27376764; fax: þ886 2 27376777.

E-mail address: tbrainliu@msn.com (S.-H. Liu).

0360-1315/$ – see front matter Ó 2013 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.compedu.2013.05.007
106 S.-H. Liu, G.-G. Lee / Computers & Education 68 (2013) 105–116

However, as far as could be ascertained, the previous citations in the literature have surprisingly neglected the process that students use
to develop their concept maps, since these methods focused on examining the finished concept map. The lack of information about how
students categorize, reorganize, and link their perceived concepts, would result in teachers not understanding the learning patterns of
students with different personalities, cultures, and backgrounds. This would lead to an inability by teachers to objectively evaluate learning
performance and to provide constructive suggestions. However, with regard to cooperative learning, the final concept map does not
demonstrate the learning performance and cooperative behavior of each team member, which leads teacher difficulty in observing the
knowledge accumulation and organization processes for each of the individuals within a team. Therefore, the teacher cannot deliberately
guide the discussion of a team during the development of the concept map, and neither can he/she assist the members who do not follow up
on the concepts constructed by the team. Therefore, the need of a new framework to both observe the developing procedure and to evaluate
the results of the concept map is inescapable.
Biology courses are often taught in the traditional teacher-centered style. Classes are usually dominated by direct instruction. Students
are generally expected to take responsibility for their own learning without questioning the teacher. The teacher-centered approach as-
sumes that all students have similar levels of background knowledge in the subject being taught and that they absorb new material at a
similar pace (Lord, 1999). In contrast, the constructivist learning approach allows students to work together in small groups and answer a
posed question (Yager, 1991). These students can be at different levels of knowledge and abilities to learn since students assist each other in
the process. For the constructivist, knowledge is created rather than discovered. The use of concept mapping is often linked to the
‘constructivist’ view of learning, because a concept map makes a good starting point for constructivist teaching.
When teachers step into the classroom, they must find effective teaching strategies to assist the students. Although there are many
different ways to teach students, direct instruction and constructivist theories of learning are two of the most common methods. Each of
these methods has advantages and disadvantages for the teacher and the students. No matter what method the teacher selects, it is
important that the teacher take any means necessary to improve student motivation and engagement in the learning process. Biology
teachers must clearly modify the way they teach in order to develop students who are enthused about the subject and who really un-
derstand the material.
The aim of the present study was to compare two sections of a biology course - one that was dominated by lecture and one that used the
constructivist model. After surveying the literature on problem-solving in e-learning and biology, proposal was made as to how biology
education researchers could apply research-supported pedagogical techniques from concept mapping to enhance biology problem-solving
by students. The need for the development of a knowledge management system using concept maps for biology was apparent. The first step
was to characterize the problems that biology students are typically asked to solve. In the present study, a framework was developed for the
analysis of individual learning performance and cooperative behavior on knowledge accumulation and organization using concept maps.
The Concept Map Knowledge Management System (CMKMS) was then developed to empirically evaluate and confirm the effectiveness of
the proposed framework. The CMKMS maintains current and historical versions of concept maps, and it also records the discussion/
cooperation history of the concept map developing process. Finally, in the present study, proposals were made as to how biology teachers
can apply the best practices in biology education research. Recording the evolution process of a concept map allows a teacher to observe the
concept map as it is being created/modified, by whom, at what time, and with which modifications. This enables a teacher (1) to guide
students in the appropriate developing direction of the concept map in a timely manner, and (2) to help students improve their learning
performance and to adjust to the cooperative learning process. Moreover, from the perspective of students, the CMKMS can be applied in
practice to measure their learning quality and to enable them to evaluate alternative concept linking and the construction of viewpoints
through the maps drawn either by teachers or by other students. This could provide another valuable format for the evaluation of different
teaching styles for similar biology curricula.

2. Literature review

2.1. E-learning in biology teaching

Technology advances have allowed educators to create lessons that will interest and engage students during the learning process. E-
learning is defined as the use of any of the new technologies or applications in the service of learning or learner support (Sangrà,
Vlachopoulos & Cabrera, 2011). When a new technology is introduced, earlier studies focus on demonstrations of efficacy, followed by
studies comparing the new technology against the old. These comparisons are important because e-learning can make a significant dif-
ference: to how students learn; how quickly students master a skill; how easy it is to study; and, equally important, how much the students
enjoy learning.
Recent research provides reassurance that e-learning is better than nothing and similar to traditional instruction. E-learning has been
used very effectively in biology classes to enhance the traditional forms of teaching and administration (Efferth, 2013; Rajkumar & Imad,
2012; Wang, 2010). Owusu, Monney, Appiah and Wilmot (2010) investigated the comparative efficiency of computer-assisted instruction
(CAI) and conventional teaching methods in biology for senior high school students. The results indicated that even though the CAI group did
not perform better than the conventional approach group, the students in the CAI group perceived CAI to be interesting when they were
interviewed. Quinnell, May and Peat (2012) used hierarchical cluster analysis to survey freshman students’ conceptions of biology. By
comparing cluster membership at the start and end of the semester, they were able to assess whether students (1) maintained their incoming
approach to study and conception of the discipline of biology, and (2) whether certain Learner Profiles were more persistent than others.
Teaching through traditional passive lectures makes learning difficult, conceptual materials more difficult, and should be replaced with
more interactive lectures coupled with more practical inquiry-based and small group-learning sessions to increase student engagement and
interest in the subject. Flowers, White and Raynor (2012) have suggested that a reduction in the transactional distances for online learning
may positively influence retention rates and student learning outcomes of biology courses. Interactive computers could be used to give
students an alternative to writing as a form of active participation in knowledge-building. E-learning enables students to communicate
through networks of communities practicing a cybernetic approach that embraces change and innovation as an inherent property of the
system. At the same time, we need a way to create a common infrastructure of agreed standards of interoperability that will enable, rather
 S.-H. Liu, G.-G. Lee / Computers & Education 68 (2013) 105–116 107

than frustrate, innovation. In general, the use of collaborative inquiry learning can promote dynamic scaffolding in order to support
argumentative discussions between groups of students during the co-creation of knowledge claims (Laru, Järvelä & Clariana, 2012).
Research shows that active learning strategies can increase students’ learning, yet few biology instructors use all identified active
learning strategies (Gardner & Belland, 2012). These changes do not happen without planning and coordination. The change towards e-
learning creates the peculiar challenge that it needs both a network-style approach to innovation, and a command and control approach to
shared infrastructure and standardization. This is possible if some influence is exerted over the way in which e-learning is used in the
teaching of biology, and if power is overtly directed towards the needs of students. Pontelli et al. (2009) met the challenges of the afore-
mentioned concepts with visualization and immediate feedback, and established a web-based interactive learning environment for
biology. Students gain immediate feedback from the exercises, and teachers receive real-time feedback on student performance.
E-learning makes many incremental changes to local ways of working, but it also requires the pooling of resources to create shared
networks, and agreed technical standards to enable those networks to interoperate. E-learning featured by active participation, interaction
and collaboration of learners and educators is becoming more and more important in education both for learners and educators (Du, Fu,
Zhao, Liu & Liu, 2013). Another source for this kind of analysis is the literature on knowledge management, which draws our attention
to the importance of continual innovation for, organizations to remain competitive. Senge’s analysis derives from a systems approach, and
concludes that the organization must be continually expanding its capacity to create its future, and adaptive learning must be joined by
generative learning that enhances the capacity to create (Senge, 1993). Nonaka made the link between knowledge creation and competition
in his seminal paper on organizational knowledge, and his model draws attention to the relationship between individual learning and
organizational learning (Nonaka & Takeuchi, 1995). Organizational knowledge creation is seen as a continual dynamic process of conversion
between tacit and explicit knowledge, iterating between the different levels of the individual, the group and the organization. Adaptive e-
learning materials can help teachers educate heterogeneous group of student, by allowing the students to pursue their own learning paths
and strategies (van Seters, Ossevoort, Tramper & Goedhart, 2012). The generation and acquisition of new knowledge is widespread and rapid
in a maturing knowledge organization. Students being educated to cope with it such change must not be sheltered from the processes of
knowledge development.
As we create and generate knowledge and information we naturally use different media, depending on the nature of the content and the
objectives we want to achieve. The scope of e-learning can be broadened to engage students in the social construction of knowledge. In
addition, the experiences of e-learning use could be improved if factors adversely affecting engagement were addressed (Moule, Ward &
Lockyer, 2010). Change in the teaching of biology is an aspect of an organization, and again, the opportunities of new learning technolo-
gies, including all available capabilities for information processing, communications, mass participation, design, and creativity, support the
management structure that would enable change to be organic and progressive – adaptive rather than mechanistic. It is a given that solving
authentic, complex biology problems requires the use of many skills in representing problems, seeking relationships, making predictions,
and verifying or checking solutions. So if biology teachers are rethinking their methods of teaching, they need a management structure that
is capable of continuously reconfiguring itself in order to create new sources of public value.

2.2. Concept mapping

A concept map can be used at the beginning of a classroom to assess what students already know. Over the past three decades, concept
mapping has been a widely popular approach for knowledge modeling in classroom teaching. Novak (1990) described a concept map as a
graphical tool that can be used to represent an organizational approach to a topic in a pictorial form. Concept maps consist of concepts and
propositions, which are usually enclosed in circles or rectangles. Unidirectional arrows link the concepts, thereby demonstrating the re-
lationships between them (Novak, 2010). Concept maps are an alternative medium for summarizing information with diagrams showing
networks of concept nodes connected by relationship links. Novak (2010) defines a concept as a perceived regularity in events or objects that
is represented by words or symbols. A proposition is defined as a meaningful statement about some objects or events, either naturally
occurring or constructed.
Concept mapping is a multi-step process that employs quantitative and qualitative techniques to establish a conceptual map of related
concepts. Concept maps are directed graphs in which labeled nodes represent concepts and labeled arcs represent relationships between
concepts (Cline et al., 2010). Novak and Gowin (1984) recommend six steps in creating a concept map: Select a topic, write general concepts,
identify more specific concepts with precise words, tie general and specific concepts together with propositional words that identify how
the concepts are linked, make cross-linkages to show connections, and, finally, reflect and revise the concept map.
In Biology science education, Assaraf, Dodick and Tripto (2013) used concept maps to characterize the system thinking skills among
students in the 10th grade within the context of the human body. Hilbert and Renkl (2008) accomplished think-aloud research to analyze
the relationships between cognitive processes as well as the characteristics of the concept maps the learners produced. Person, Olney,
D’Mello and Lehman (2012) developed an intelligent tutoring system, called Guru, for high school biology. Guru is a dialogue-based sys-
tem that intersperses concept map tasks within a tutorial dialogue. It was concluded that concept mapping could serve as an appropriate
alternative for studying biology since what is learned through it can be retained for a long period of time (Patrick, 2011). In addition to being
a learning tool, concept mapping is also one of the most powerful assessment evaluation tools, encouraging students to use meaningful-
mode learning patterns (Gouli, Gogoulou, Papanikolaou & Grigoriadou, 2004; Kane & Trochim, 2007; Mintzes, Wandersee, & Novak,
2000). Specifications for concept mapping assessment usually include the following: (1) defining the task demands in terms of a general
topic of concepts and (2) scoring the concept maps using the Herl metric for scoring concept structure, semantic content, number of links
used in the structure, and comparison with expert maps that are used as criteria (Herl, Baker, & Niemi, 1996; O’Neil & Schacter, 1997). Tripto,
Assaraf and Amit (2013) examined the effectiveness of the concept map as an assessment tool for the thinking of students within a system.
An extensive amount of qualitative data suggests that the strength of the concept map is in its ability to describe the first two levels of
system thinking (analysis and synthesis). However, the concept map proved less successful in eliciting evidence of the third and highest
level, particularly of students’ understanding of patterns, of homeostasis, and of their capacity for temporal thinking.
There is considerable potential educational benefit in collaborative concept mapping (Martínez Maldonado, Kay & Yacef, 2010). Being a
cooperative learning tool, in order to learn successfully by concept mapping, learners should specialize in planning processes, should aim at
108 S.-H. Liu, G.-G. Lee / Computers & Education 68 (2013) 105–116

constructing a coherent concept map and should master the progress of their map (Van Boxtel, van der Linden & Kanselaar, 2000). Using
collaborative concept mapping to learn science could increase the opportunities for discussion between peers, thus fostering better or-
ganization and understanding of the content (Jang, 2010). Concept mapping could be effectively used in collaborative learning and problem
solving activities with a particular focus on online concept mapping environments. Students can achieve not only the development of their
creativity and imagination but also their cognitive and social awareness (Gkotzos & Potamias, 2012).
The application of concept mapping makes it possible to relate the structure of the curriculum to the structure of the discipline, in order
to support the development of robust student knowledge structures in ways that reflect the professional practices of subject experts
(Kinchin, 2011). Concept maps provide a visual and holistic way for sharing ideas in an accessible and concise form, thereby turning
knowledge-sharing into a promising model (Wang, Cheung, Lee & Kwok, 2008). Students can achieve meaningful learning more effectively
when they construct their own knowledge using computer technology as opposed to learning in a conventional classroom structure
(Jonassen, Howland, Marra, & Crismond, 2008). Concept mapping as a knowledge-consolidation phase positively affected an increase in
knowledge retention rates (Gerstner & Bogner, 2010). However, few have investigated how students assess their own ability to successfully
construct concept maps using computer programs that can represent their level of knowledge.

3. Methodology

3.1. Teaching procedure

This study adopted Miller’s (2009) concept mapping instruction procedures for biology curriculum. There are five key points in the
teaching procedure:

(1) Decide the general topic of the map.

(2) Consider the ideas related to the general topic.
(3) Select words that are appropriate to the general topic and that will support the main ideas of biology.
(4) Draw and connect these words to the main topic with a line referring to the relationship.
(5) Repeat the above process for each subtopic until students have enough ideas and information to write about.

3.2. Learning procedure

The students’ learning procedures could be formed into five basic tips:

(1) Preparation

The preparation step involves defining the focus statement and determining the general topic of the biology curriculum. A focus
statement is an open-ended prompt that students are asked to complete (Trochim, 1989).

(2) Brainstorming

The brainstorming sessions are designed to generate a large set of statements related to the general topic from step one. Students are
asked to generate statements or ideas in response to the identified focus statement with a couple of words (Yampolskaya, Nesman,
Hernandez & Koch, 2004).

(3) Ranking

Select the words that are appropriate to the general topic from each brainstorming session, which will support the main ideas of biology
(Smith & Humphreys, 2006).

(4) Structuring

To allow for comparisons of group similarities and differences, a separate map is created for each of the groups. The final concept map
indicates the average importance of the items according to the students’ rankings (Roth & Roychoudhury, 1993).

(5) Interpretation

Presentation of the data to highlight the themes and barriers identified (Trochim, 1989).

3.3. System design

For the educational innovator, who seriously wishes to improve the quality of education and the learning experience, it is imperative to create
a CMKMS (Concept Map Knowledge Management System) that clarifies its values and sets its aims and ambitions high. The CMKMS provides a
means for representing information and ideas not simply as words and pictures, but as structured systems. The CMKMS is an information
system, which embodies a working model with which the user can interact – not just analyzing and reworking, but testing and challenging.
The basic requirements for this system are concept constructing, creating ease of operation, to establishing a unique repository of data,
and having the ability to easily browse and search a student’s learning log. A CMKMS should simplify the design for the map classification
 S.-H. Liu, G.-G. Lee / Computers & Education 68 (2013) 105–116 109

with functionality. The comment functionality is added and it is the necessary procedure before posting the concept map in the CMKMS (See
Fig. 1).
Students can hold a forum to demonstrate their knowledge or engage in discussion. Teachers can attend the discussion if necessary (See
Fig. 2).
The term Knowledge Management refers to the capacity of the system to demonstrate the same knowledge in multiple versions.
Teachers can browse through the process of each concept map while students can retrieve information and realize each classmate’s thought
processes with the help of the log base. Since this application focuses on the sharing of knowledge, a knowledge base was added to give the
students not only the concept of map presentation but also an understanding of the meaning of other maps. When more than one concept
map is available, the most popular concept map is selected and uploaded to the knowledge base by the teacher (See Fig. 3).

3.4. Participants

The present study was conducted using 229 high school students. The participants were all K-11 students enrolled at two Taipei high
schools. The students on both high schools had no significant differences in biology achievement from K-10, as they had all been taught by
the same respective biology teachers. The two biology teachers were experienced and familiar with the skills needed to operate a CMKMS.
The biology classes were taught for 2 h every week.

3.5. Experiment design

The CMKMS provides the opportunity to learn and practice creating a concept map. The experiments divided the students into several
groups that included the following: “Without a CMKMS,” “Individual learning with a CMKMS,” “Cooperative learning with a CMKMS.”

(1). Without a CMKMS

This training session taught biology using traditional teaching methods.

(2). Individual learning with CMKMS

This training session taught biology with the assistance of a CMKMS. Students were personally in charge of retrieving and leveling
concepts. The teacher presented the biology information to a class and asked each student to summarize it in a concept map. The CMKMS
allowed students to search for important concepts, relationships, and structures in order to organize their individual understanding of
biology.

(3). Cooperative learning with a CMKMS

The students were divided into subgroups of 6 or 7 persons. When receiving cooperative-mapping treatment, the students cooperated in
finishing a map. The teacher provided biology information and asked the students to summarize it in a single concept map through a
communicative process. During the cooperative-mapping process, the students got chances to communicate, interact, and exchange their

Fig. 1. Student uploads concept map and comment.

110 S.-H. Liu, G.-G. Lee / Computers & Education 68 (2013) 105–116

Fig. 2. Student holds forum to provide their knowledge or discussion.

opinions with other members. Because concept mapping is a new learning strategy for students, the students were able to use the CMKMS
and take part in a comprehensive training program.

3.6. Quality Data collection

Quality Data were gathered from participant-observation notes, teacher interviews, and CMKMS artifacts. These data were analyzed
using a category construction method (Merriam, 2009) that involved open coding and axial coding.

(1) Observation

The first data collection technique employed was classroom observation. Classroom observation data affords the researcher an oppor-
tunity to gather live data based on what people do and say in real-life situations (Cohen, Manion & Morrison, 2000). The purpose of the

Fig. 3. The structure of CMKMS.

 S.-H. Liu, G.-G. Lee / Computers & Education 68 (2013) 105–116 111

Table 1
Results.

School 1 School 2

Without CMKMS Individual CMKMS Cooperative CMKMS Without CMKMS Individual CMKMS Cooperative CMKMS
P
X 3002 3036 3238 3020 3182 3276
P 2
X 238,500 250,032 276,948 240,704 260,324 275,592
N 38 37 38 38 39 39
Average score 79.00 82.05 85.21 79.24 81.59 83.84

classroom observations was to shed more light on the actual instructional practices in the classrooms and to identify the nature of the
teacher–student interactions in terms of the biology concepts being facilitated. It was important to create an opportunity to identify the
cognitive levels of activities and language occurring relative to the students’ thinking about biology. Furthermore, the observations were
used as an opportunity to gain insight as to whether the prior understanding of the students was considered and to what extent cognitive
dialogue was entertained.

(2) Interview

After the observation and transcription of field notes, the biology teachers were interviewed about their observations of the biology
tutoring activities (Maree & Pietersen, 2007). The real benefit of this step was to allow the teacher to give greater clarity to interactions that
were perhaps not clear in the recording or observation, so that any misunderstandings could be cleared up immediately (Whites, 2000).
Furthermore, the interview conferred the opportunity to contribute additional information that may have been used for other perspectives.
The interview emphasized the centrality of human interaction for knowledge production and provided a social context for the research data.

3.7. Statistics method

(1) ANOVA

During the semester prior to student usage, teachers presented a half-day course to the students about the application of the CMKMS for
biology education. A biology examination was held over six weeks. For the comparisons between different teaching and practice activities, a
two-way ANOVA with a ¼ 0.05 was conducted.

(2) Tukey’s HSD Test

Tukey’s HSD (Honest Significant Difference) test is a post-hoc test, meaning that it is performed after an analysis of variance (ANOVA)
test. In statistics, post-hoc tests are used only for further data analysis; these types of tests are not pre-planned. The purpose of the Tukey’s
HSD test was to determine which groups differed in the sample.

(3) McNemar test

We wanted to know whether or not the students liked the teaching model after the experiment. We performed a McNemar test ( Kotz &
Johnson, 1983; McNemar, 1947; Sheskin, 2000) to test the marginal homogeneity in 2  2 tables, which included all 4 possible combinations
that differed significantly from the expected count. These binary outcomes were the same outcome variable on matching pairs (like or not)
from the treatment group. This non-parametric test had the null hypothesis that differences in both samples would be equal in both di-
rections. The test used matched-pairs of labels (A, B) with a ¼ 0.05, and a Chi–Square test of significance to determine if the proportion of A
and B labels were equal for both members. Proportion A was conducted before the experiment to examine student likes and dislikes.
Proportion B was conducted after the experiment.

3.8. Data mining

We used data mining techniques to extract useful information from the log base.

(1) CHAMELEON Algorithm

Table 2
School 1 different CMKMS treatment ANOVA summary table.

Source of variation SS Degree Mean square F

SSb 732.91 2 366.45 12.24
SSw 3294.21 110 29.95
Total 4027.12 112

F0.95(2,110) ¼ 8.23.
SS: Sum of square of deviations from the mean.
SSb: Sum of square of deviations from the mean between groups.
SSw: Sum of square of deviations from the mean within groups.
112 S.-H. Liu, G.-G. Lee / Computers & Education 68 (2013) 105–116

Table 3
School 2 different CMKMS treatment ANOVA summary table.

Source of variation SS degree Mean square F

SSb 395.12 2 197.56 12.36
SSw 1806.91 113 15.99
Total 2202.03 115

F0.95(2,113) ¼ 8.22.

In the present study, we applied a CHAMELEON clustering algorithm to group data into a series of clusters that would maximize the
similarities within clusters and minimize the similarities between two different clusters. A CHAMELEON clustering algorithm involves two
phases: identifying small tight clusters between each pair of clusters, Ci and Cj, based on a k-nearest neighbor graph, then merging similar
clusters together based on both interconnectivity RI(Ci, Cj) and closeness RC(Ci, Cj). CHAMELEON clustering algorithm facilitates the dis-
covery of natural and homogeneous clusters.
The internal interconnectivity of a cluster, Ci, can be easily captured by the size of its min-cut bisector, ECCi. Hence, in a CHAMELEON
clustering algorithm, the relative interconnectivity between a pair of clusters, Ci and Cj, is given by the following equation.
  
  EC Ci ; Cj 
RI Ci ; Cj ¼   
jECðCi Þj þ EC Cj 
2
This equation normalizes the absolute inter-connectivity with the average internal inter-connectivity of the two clusters.
The relative closeness between a pair of clusters, Ci and Cj, is computed as follow:
 
  SEC Ci ; Cj
RC Ci ; Cj ¼  
jCi j C j   
 SECðCi Þ þ  
C i þ C j  Ci þ Cj SEC Cj

where SECCi and SECCj are the average weights of the edges that belong in the min-cut bisector of clusters Ci and Cj, respectively, and SEC{Ci,
Cj} is the average weight of the edges that connect the vertices in Ci to the vertices in Cj.

(2) Apriori Algorithm

Apriori is a frequent-pattern mining algorithm for discovering the association rules between different sets of data. The results of as-
sociation rule mining shows rules that have a group of related patterns called an “item set.” An item set is considered in each transaction. The
association rule has two parts that describe its significance: Its confidence is defined as a percentage of data sets associated with each
discovered pattern, and the threshold of minimum confidence values is set to filter the item sets specified by requirement and usage; and,
the support of an associated pattern is a percentage of the data sets for which the pattern is true, and the minimum support value threshold
is set to limit the item sets that are related to the specified criteria (Agrawal & Srikant, 1994).
The simple formulas are as follows:

ConfidenceðA/BÞ ¼ #tuples containing both A B=#tuples containing A ¼ PðBjAÞ ¼ PðAWBÞ=PðAÞ

SupportðA/BÞ ¼ #tuples containing both AB=total number of tuples ¼ PðAWBÞ

Apriori uses a level-wise search and a hash tree structure to efficiently count candidate item sets. It generates candidate item sets of
length k from the k1 item sets and avoids expanding all the item set’s graph. The algorithm finds all frequent item sets until there are no
further successful extensions, and generates strong association rules from the frequency of the item sets. Rules that have both high con-
fidence and support are called strong rules. Based on the Apriori technique, the algorithm can decrease time processing in generating fewer
groups of item sets and avoid the expansion of infrequent candidate item sets.

4. Results and analysis

The experiment results are consisted of quantitative part and quality part.

4.1. Quantitative results

1. The first research question assessed the interaction effect that the CMKMS treatment had on students and how this transferred to the
effectiveness of learning biology. The scores from a biology examination are listed in Table 1: For school 1, the ‘Cooperative CMKMS’

Table 4
School 1 Tukey’s HSD test.

KSM treatment Test I u1 ¼ 85.21 Test II u2 ¼ 82.05 Test III u3 ¼ 79.00

Test I u1 ¼ 85.21 – 3.16* 6.21*
Test II u2 ¼ 82.05 – 3.05*
Test III u3 ¼ 79.00 –

Level of significance *J(HSD) ¼ q0.95(3,110) ¼ 2.99.

 S.-H. Liu, G.-G. Lee / Computers & Education 68 (2013) 105–116 113

Table 5
School 2 Tukey’s HSD test.

KSM treatment Test I u1 ¼ 83.84 Test II u2 ¼ 81.59 Test III u3 ¼ 79.24

Test I u1 ¼ 83.84 – 2.25* 4.60*
Test II u2 ¼ 81.59 – 2.35*
Test III u3 ¼ 79.24 –

Level of significance *J(HSD) ¼ q0.95(3,113) ¼ 2.15.

group had a mean of 85.21; the ‘Individual CMKMS’ group had a mean of 82.05; and, the ‘Without a CMKMS’ group had a mean of 79.00.
As shown in Table 2, with a 0.05 level of significance, a 95% confidence level, and a critical region F (2, 110) ¼ 8.23, the ANOVA results
indicated a significant main effect for the CMKMS factor, F ¼ 12.24. The hypothesis (H0: u1 ¼ u2 ¼ u3) should be rejected. Results were
similar for School 2: the ‘Cooperative CMKMS’ group had a mean of 83.84, the ‘Individual CMKMS’ group had a mean of 81.59, and the
‘Without a CMKMS’ group had a mean of 79.24. As shown in Table 3, with a 0.05 level of significance, a 95% confidence level, and a
critical region F (2, 113) ¼ 8.26, the ANOVA results indicated a significant main effect for the CMKMS factor, F ¼ 12.36. The hypothesis
(H0: u1 ¼ u2 ¼ u3) should be rejected.

There is at least a pairwise comparison that reached the 0.05 level of significance. Results for both the posterior and Tukey’s HSD tests are
listed in Table 4. The differences found in the posterior test (u2, u1), (u3, u1), (u2, u3) were larger than those found with the HSD test (2.99).
These results indicated that the students who received the Cooperative CMKMS treatment had the highest scores. Also, the students who
used the CMKMS performed better on an individual basis than those who did not use it. Based on the results of the Tukey’s HSD test, school 2
showed similar results. The results are listed in Table 5.

2. The McNemar test is on a 2  2 classification table and tests the differences between paired proportions in studies with a ‘before and
after design.’ We recorded the students’ preferences before and after the experiment. With the significance level set at a ¼ 0.05, the
critical region Z (0.025) ¼ - 1.96, and Z (0.975) ¼ 1.96, the conclusion was that the students had changed their opinions. As shown in
Table 6, for the three groups, the Z-Statistic values were estimated to be 1.46, 3.16 and 4.72, respectively. For both the ‘Individual
CMKMS’ and the ‘Cooperative CMKMS’ groups, the possibility of changing an opinion from dislike (before the experiment) to like (after
the experiment) was greater than from like to dislike. Most students changed their opinions to a negative one after the exam. In contrast,
the opinions of the group ‘Without a CMKMS’ did not change over time.
3. Analysis of 498 pages of the forum log showed that the deepest level reached 7. Using this similarity measure, all the log data were set in
the correct clusters on the second pass of the merging phase based on a threshold value of 20 (See Fig. 4). The log data with a low density
were eliminated as noise. The merging results of dynamic modeling used in the CHAMELEON clustering algorithm resulted in the
construction of 6 categorizations (See Fig. 5).
4. We used an Apriori algorithm to identify 42 correspondence rules. We set the confidence threshold at 50%, the support threshold at 50%,
and established 8 high-correlation dependency network links (See Table 7). On the whole, the conclusions agreed with the anticipated
requests, and were used to detect the details of a website’s frequency of use in order to demonstrate a student’s learning path, and could
also be used for decision-making with regards to teaching strategy.

4.2. Quality results

After collecting the data from interviews with biology teachers and from classroom observation, the results were summarized as follows.

1. The CMKMS made a positive contribution to the teaching of biology.

2. Concept mapping is an effective learning strategy for students.
3. Computerized concept maps facilitate the modification of nodes and links and make it easier to draw and revise the structure.
4. Some students were unaware of how to initiate and organize their thoughts and ideas during the process of constructing a concept.
5. Interaction with students through discussion boards is an important motivating factor for the completion of biology courses.
6. The CMKMS helped students to develop conceptual mapping abilities.
7. The CMKMS enabled students to intuitively construct and organize their tacit knowledge.
8. A student’s graphic representation of biology concepts can be tracked over time.
9. Students reduced their anxiety, increased their motivation, and promoted greater creation and achievement, particularly within the
context of a group.

Table 6
Results of McNemar test.

After experiment

Without CMKMS Individual CMKMS Cooperative CMKMS

Dislike Like Z Dislike Like Z Dislike Like Z

Before Experiment Like 11 20 1.46 10 17 3.16 7 18 4.72
Dislike 26 19 19 30 13 39
114 S.-H. Liu, G.-G. Lee / Computers & Education 68 (2013) 105–116

Fig. 4. Stability plots for CHAMELEON dataset.

5. Discussion

1. The present study used the teaching of biology to prove that a CMKMS can be a positive learning strategy. The utilization of a computer
allowed the students to present concept maps more easily. The students’ success on the examination was related to the complexity of
their concept maps, and those students who chose to draw concept maps or write prose summaries were more likely to succeed on the
test than those who wrote nothing on the scratch paper. Also, the students who used a CMKMS cooperatively tended to perform best.
The CMKMS promoted greater collaboration among students and encouraged the sharing of knowledge. The results of the present study
support the previous results found by other scholars - biology students who create concept maps collaboratively learn more than those
who construct concept maps individually (e.g, Hung, Chou, Chen, & Own, 2010; Roper, 2007; Sansone, Fraughton, Zachary, Butner, &
Heiner, 2011).
2. According to the results of the McNemar test, the CMKMS was effective in changing students’ attitudes. Students who were taught
without a CMKMS did not change their opinions. The students valued the CMKMS process and felt that the concept mapping approach
may be a valuable tool for the sharing of ideas. When the students grasped the use of the mapping strategy for learning, their attitude
about the learning process became more positive. Mutual discussion improved the students’ interest and presentation abilities. Both of
the biology teachers agreed that utilization of the CMKMS had changed students’ learning intentions, motivations, and even self-
regulation.
3. The CMKMS allows the learning process to be analyzed by teachers. The CHAMELEON clustering algorithm can reveal the source of a
student’s feedback the knowledge gained from each session. Low density in the log data of a student can reveal a lack of familiarity
with the concept of mapping; moreover, some students do not understand how bring the potential values of concept mapping into
full play. The concept maps created by students comprise a visual learning record, and the quality of change in understanding can be
measured empirically. Some students put more effort into the practice of concept mapping than into the analytical training. This
result agrees with the conclusion of Vassilis, Marida, and Vassiliki (2007). Asking students to construct a concept map on
the CMKMS platform as a learning strategy ignores the role of problem-solving. The students may structure their maps to meet the
local area and, thus, not expand their thinking to the greatest extent possible. More importantly, concept mapping is a meta-
cognitive strategy that helps students to learn how to learn. The students can examine what ideas are missing or irrelevant in
their writing.
4. The CMKMS provides the opportunity for students to divide blocks of new knowledge into smaller parts, arrange and order concepts
to make sense, and make connections between concepts. According to the analysis results, a teacher can propose proper improve-
ments. In the present study, an Apriori algorithm helped to extract 42 correspondence rules into 8 high correlation dependency
network links. Meaningful learning was purported to produce a series of changes within the entire cognitive structure with the
modification of existing concepts and the formation of new links between those concepts. The duration of concept mapping in-
terventions in biology education showed that meaningful learning is lasting, powerful, and, hence, superior to the rote learning that is
easily forgotten.

Fig. 5. Result for CHAMELEON dataset.

 S.-H. Liu, G.-G. Lee / Computers & Education 68 (2013) 105–116 115

Table 7
Result of Apriori frequent item sets.

Item Strong rule Confidence Support

1 K1 / K14 78% 61%
2 K2 / K13 77% 60%
3 K3 / K14 70% 56%
4 K6 / K12 68% 55%
5 K3 / K5 67% 55%
6 K3 / K32 67% 53%
7 K9 / K40 65% 52%
8 K7 / K23 63% 51%

6. Conclusion

In this research, we took students’ learning proficiency and different mapping procedures into account in order to investigate the effects
of computerized concept maps on different learners. Concept mapping has been identified as a technique for developing and evaluating
critical thinking abilities. The concept mapping that was involved in the knowledge management system had a more positive effect than
traditional biology teaching methods. The utilization of the CMKMS can improve a student’s analytical performance. Using the CMKMS
helped these students enhance their biology learning attitudes and motivation. Rather than the traditional learning method of memorizing
facts in a biology context, concept mapping helped these students recognize how multiple parts result in a cohesive whole.
The CMKMS is a perfected process that fosters the creation and development of knowledge. As a means by which to transform and
externalize tacit knowledge, the value of the CMKMS is readily apparent. The CMKMS can function as a graphic organizer by helping
students to visualize knowledge on a specific topic and to connect the relationships among the facts. Students can apply the CMKMS as a
planning tool with their peers to maximize varieties of ideas held within a group where members may have differing knowledge structures.
The CMKMS definitely helps students avoid awkward situations while sharing and challenging each other’s ideas. These characteristics
enable individuals to utilize the CMKMS in order to problem-solve and communicate with one another. The cooperative mapping group
works most effectively when group members bring different perspectives to the process of learning.
The CMKMS captures the twin tasks of both generating new knowledge and monitoring existing activities, to ensure adaptive change in
response to the external environment. This research offers a platform on which group brainstorming sessions can be structured, knowledge
and values can be integrated, and a shared conceptual framework can be created to address evaluation problems. With the assistance of the
CMKMS, a student’s graphic representation of biology concepts can be tracked over time. The CMKMS allows the process of understanding
to be analyzed by teachers. Teachers can utilize the CMKMS to monitor a student’s conceptual development and examine their under-
standing of the concepts of biology. The CMKMS platform connects a student’s course network with his/her social and knowledge networks.
As a result, the CMKMS helps students to build personalized social and knowledge networks during the process of learning.

References

Agrawal, R., & Ramakrishnan, S. (1994). Fast algorithms for mining association rules. In The 20th international conference on Very Large Databases (VLDB), Santiago, Chile (pp.
487–499).
Assaraf, B.-Z., Dodick, & Tripto. (2013). High school students’ understanding of the human body system. Research Science Education, 43, 33–56.
Brown, G., Nagel, C., Zlatanova, S., & Kolbe, T. H. (2013). Modelling 3D topographic space against indoor navigation requirements. Progress and New Trends in 3D Geo-
information Sciences, 1–22.
Chiou, C.-C., Lee, L.-T., & Liu, Y.-Q. (2012). Effect of Novak colorful concept map with digital teaching materials on student academic achievement. Procedia – Social and
Behavioral Sciences, 64(9), 192–201.
Cline, B. E., Brewster, C. C., & Fell, R. D. (2010). A rule-based system for automatically evaluating student concept maps. Expert Systems with Applications, 37(3), 2282–2291.
Cohen, L., Manion, L., & Morrison, K. (2000). Research in education (5th ed.). London: Routledge Falmer.
Du, Z., Fu, X., Zhao, C., Liu, Q., & Liu, T. (2013). Interactive and collaborative E-learning platform with integrated social software and learning management system. In
Proceedings of the 2012 international conference on information technology and software engineering (pp. 11–18). Berlin Heidelberg: Springer.
Efferth, T. (2013). Ten years’ experience with an e-learning lecture series on cancer biology and pharmacology. Education Sciences, 3(1), 1–16.
Empson, S. B., Greenstein, S., Maldonado, L., & Roschelle, J. (2013). Scaling up innovative mathematics in the middle grades: Case studies of “good enough” enactments. In The
SimCalc Vision and contributions: Democratizing access to important mathematics. 251.
Flowers, L. O., White, E. N., & Raynor, J. E., Jr. (2012). Examining the transactional distance theory in a web-enhanced biology course. Journal of Studies in Education, 2(3), 144–154.
Gardner, J., & Belland, B. R. (2012). A conceptual framework for organizing active learning experiences in biology instruction. Journal of Science Education and Technology, 1–11.
Gerstner, S., & Bogner, F. X. (2010). Cognitive achievement and motivation in hands-on and teacher-centred science classes: does an additional hands-on consolidation phase
(concept mapping) optimise cognitive learning at work stations? International Journal of Science Education, 32(7), 849–870.
Gkotzos, D., & Potamias, G. (2012). Collaborative concept mapping via Multimodal ICT tools. In INTED2012 proceedings (pp. 4594–4600).
Gouli, E., Gogoulou, A., Papanikolaou, K., & Grigoriadou, M. (2004). Designing an adaptive feedback scheme to support reflection in concept mapping. In Proceedings of the
adaptive Hypermedia Conference 2004: Workshop on individual differences in adaptive Hypermedia, (August 2004, Eindhoven, Netherlands) (pp. 126–135).
Herl, H. E., Baker, E. L., & Niemi, D. (1996). Construct validation of an approach to modeling cognitive science structure of U.S. history knowledge. Journal of Educational
Research, 89, 206–218.
Hilbert, T. S., & Renkl, A. (2008). Concept mapping as a follow-up strategy to learning from texts: what characterizes good and poor mappers? Instructional Science, 36, 53–73.
Hung, M.-L., Chou, C., Chen, C.-H., & Own, Z.-Y. (2010). Learner readiness for online-learning: scale development and student perceptions. Computers & Education, 55(3),
1080–1090.
Hung, P. H., Hwang, G. J., Su, I. H., & I-Hua, L. (2012). A concept-map integrated dynamic assessment system for improving ecology observation competences in mobile
learning activities. The Turkish Online Journal of Educational Technology (TOJET), 11(1), 10–19.
Jang, S. J. (2010). The impact on incorporating collaborative concept mapping with coteaching techniques in elementary science classes. School Science and Mathematics,
110(2), 86–97.
Jeong, A., & Lee, W. J. (2012). Developing causal understanding with causal maps: the impact of total links, temporal flow, and lateral position of outcome nodes. Educational
Technology Research and Development, 1–16.
Jonassen, D., Howland, J., Marra, R. M., & Crismond, D. (2008). Meaningful learning with technology. NJ: Pearson Education Ltd.
Kane, M., & Trochim, W. M. K. (2007). Concept mapping for planning and evaluation. Thousand Oaks, CA: Sage.
Kinchin, I. M. (2011). Visualising knowledge structures in biology: discipline, curriculum and student understanding. Journal of Biological Education, 45(4), 183–189.
Kotz, S., & Johnson, N. (1983)Somes G. McNemar test, encyclopedia of statistical sciences, Vol. 5, (pp. 361–363)New York: Wiley.
116 S.-H. Liu, G.-G. Lee / Computers & Education 68 (2013) 105–116

Laru, J., Järvelä, S., & Clariana, R. B. (2012). Supporting collaborative inquiry during a biology field trip with mobile peer-to-peer tools for learning: a case study with K-12
learners. Interactive Learning Environments, 20(2), 103–117.
Lee, C. H., Lee, G. G., & Leu, Y. (2009). Application of automatically constructed concept map of learning to conceptual diagnosis of e-learning. Expert Systems with Applications,
36(2), 1675–1684.
Lord, T. R. (1999). A comparison between traditional and constructivist teaching in environmental science. Journal of Environmental Education, 30(3), 22–28.
McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12, 153–157.
Maree, K., & Pietersen, J. (2007). Sampling. In K. Maree (Ed.), First steps in research). Pretoria: Van Schaik Publishers.
Martínez Maldonado, R., Kay, J., & Yacef, K. (2010). Collaborative concept mapping at the tabletop. In ACM international conference on interactive tabletops and surfaces (pp.
207–210). ACM.
Merriam, S. B. (2009). Qualitative research: A guide to design and implementation. Sacramento, CA: Jossey-Bass.
Miller, K. J., Koury, K., Fitzgerald, G., Hollingsead, C., Mitchem, K., Tsai, H., et al. (2009). Concept mapping as a research tool to evaluate conceptual change related to
instructional method. Teacher Education and Special Education, 32(4), 365–378.
Mintzes, J., Wandersee, J., & Novak, J. D. (2000). Assessing science understanding. San Diego, CA: Academic Press.
Moule, P., Ward, R., & Lockyer, L. (2010). Nursing and healthcare students’ experiences and use of e-learning in higher education. Journal of Advanced Nursing, 66(12), 2785.
Nonaka, I., & Takeuchi, H. (1995). The knowledge-creating company. New York: Oxford Press.
Novak, J. D. (1990). Concept mapping: a useful tool for science education. Journal of Research in Science Teaching, 27(10), 937–949.
Novak, J. D. (2010). Learning, creating, and using knowledge: Concept maps as facilitative tools in schools and corporations. Taylor & Francis.
Novak, J. D., & Gowin, D. B. (1984). Learning how to learn. Cambridge, UK: Cambridge University Press.
O’Neil, H. F., & Schacter, J. (1997). Test Specifications for problem solving assessment. CSE Technical Report 463. Los Angeles, CA: Center for the Study of Evaluation.
Owusu, K. A., Monney, K. A., Appiah, J. Y., & Wilmot, E. M. (2010). Effects of computer-assisted instruction on performance of senior high school biology students in Ghana.
Computers & Education, 55(2), 904–910.
Patrick, A. O. (2011). Concept mapping as a study skill: effects on students achievement in biology. International Journal of Educational Sciences, 3(1), 49–57.
Person, N., Olney, A., D’Mello, S., & Lehman, B. (2012). Interactive concept maps and learning outcomes in Guru. In Proceedings of 25th Florida Artificial Intelligence Research
Society Conference, Menlo Park, CA (pp. 456–461).
Pontelli, E., Pinto, J., Qin, X., He, J., Bevan, D., MacCuish, N., et al. (2009). BIOPS Interactive: an e-Learning Platform Focused on Protein Structure and DNA. Bioscene: Journal of
College Biology Teaching, 35(2), 6–15.
Quinnell, R., May, E., & Peat, M. (2012). Conceptions of biology and approaches to learning of first year biology students: introducing a technique for tracking changes in
learner profiles over time. International Journal of Science Education, 34(7), 1053–1074.
Rajkumar, R., & Imad, M. K. (2012). Blot e-learning system: next generation learning system. In Computer communication and informatics (ICCCI), 2012 international conference
on (pp. 1–6). IEEE.
Roper, A. R. (2007). How students develop online learning skills. EDUCAUSE Quarterly, 1, 62–65.
Roth, W. M., & Roychoudhury, A. (1993). The concept map as a tool for the collaborative construction of knowledge: a microanalysis of high school physics students. Journal of
Research in Science Teaching, 30(5), 503–534.
Ruiz-Primo, M. A., & Shavelson, R. J. (1996). Problems and issues in the use of concept maps in science assessment. Journal of Research in Science Teaching, 33, 569–600.
Sangrà, A., Vlachopoulos, D., & Cabrera, N. (2011). Defining e-learning inclusively. Barcelona: Open University of Catalonia.
Sansone, C., Fraughton, T., Zachary, J. L., Butner, J., & Heiner, C. (2011). Self-regulation of motivation when learning online: the importance of who, why and how. Educational
Technology Research and Development, 59(2), 199–212.
Senge, P. M. (1993). The fifth discipline: The art & practice of the learning organization. London: Century Nusiness.
van Seters, J. R., Ossevoort, M. A., Tramper, J., & Goedhart, M. J. (2012). The influence of student characteristics on the use of adaptive e-learning material. Computers &
Education, 58(3), 942–952.
Sheskin, D. J. (2000). Handbook of parametric and nonparametric statistical procedures (2nd ed.). Boca Raton: Chapman & Hall.
Smith, A. E., & Humphreys, M. S. (2006). Evaluation of unsupervised semantic mapping of natural language with Leximancer concept mapping. Behavior Research Methods,
38(2), 262–279.
Soleimani, H., & Nabizadeh, F. (2012). The effect of learner constructed, fill in the map concept map technique, and summarizing strategy on Iranian pre-university students’
reading comprehension. English Language Teaching, 5(9), 78.
Tripto, J., Assaraf, O. B. Z., & Amit, M. (2013). Mapping what they know: concept maps as an effective tool for assessing students’ systems thinking. American Journal of
Operations Research, 3, 245–258.
Trochim, W. M. (1989). An introduction to concept mapping for planning and evaluation. Evaluation and Program Planning, 12(1), 1–16.
Tseng, K.-H., Chang, C.-C., Lou, S.-J., Tan, Y., & Chiu, C.-J. (2012). How concept-mapping perception navigates student knowledge transfer performance. Educational Technology
& Society, 15(1), 102–115.
Van Boxtel, C., van der Linden, J. L., & Kanselaar, G. (2000). Collaborative learning tasks and the elaboration of conceptual knowledge. Learning and Instruction, 10(4), 311–330.
Vassilis, K., Marida, E., & Vassiliki, Z. (2007). Comparing computer-supported dynamic modeling and ‘paper & pencil’ concept mapping technique in students’ collaborative
activity. Computers & Education, 49(4), 991–1017.
Wang, T. H. (2010). Web-based dynamic assessment: taking assessment as teaching and learning strategy for improving students’ e-Learning effectiveness. Computers &
Education, 54(4), 1157–1166.
Wang, W. M., Cheung, C. F., Lee, W. B., & Kwok, S. K. (2008). Mining knowledge from natural language texts using fuzzy associated concept mapping. Information Processing
and Management, 44(5), 1707–1719.
White, M. D. (2000). Questioning behavior on a consumer health electronic list. Library Quarterly, 70(3), 302–334.
Wijeysundera, H. C., Trubiani, G., Wang, X., Mitsakakis, N., Austin, P. C., Ko, D. T., et al. (2013). A population-based study to evaluate the effectiveness of multidisciplinary heart
failure clinics and identify important service components clinical perspective. Circulation: Heart Failure, 6(1), 68–75.
Wu, P. H., Hwang, G. J., Milrad, M., Ke, H. R., & Huang, Y. M. (2012). An innovative concept map approach for improving students’ learning performance with an instant
feedback mechanism. British Journal of Educational Technology, 43(2), 217–232.
Yager, R. E. (1991). The constructivist learning model. The Science Teacher, 58(6), 53–57.
Yampolskaya, S., Nesman, T. M., Hernandez, M., & Koch, D. (2004). Using concept mapping to develop a logic model and articulate a program theory: a case example. American
Journal of Evaluation, 25(2), 191–207.