Analyzing Collaborative Learning

Activities in Wikis using Social Network

Analysis


Abstract
This paper investigates the potential of existing
methods for analyzing collaboration in wiki
environments. After a short description of the
motivation for this research paper a presentation of
analysis methods in CSCL will be given, including a
special focus on Social Network Analysis. The next
section points out the main characteristics of wikis and
their differences compared to other CSCL tools. In the
following step, the methods for collaboration analysis
are transferred to wiki contexts. The paper concludes
with open issues and an outlook on future research on
this topic area.
Keywords
Wikis, CSCL, Collaboration Analysis, Social Network
Analysis
ACM Classification Keywords
H.4.m. Information Systems Applications:
Miscellaneous. H.1.2. User/Machine Systems.
Motivation
Since the emergence of the Web 2.0, wikis have been
applied in various areas. Because of their openness to
Copyright is held by the author/owner(s).
CHI 2009, April 49, 2009, Boston, Massachusetts, USA.
ACM 978-1-60558-247-4/09/04.
Saskia-Janina Kepp
Information Science
Hildesheim University
Marienburger Platz 22
31141 Hildesheim, Germany
keppsa@uni-hildesheim.de

Heidemarie Schorr
English Language and Literature
Hildesheim University
Marienburger Platz 22
31141 Hildesheim, Germany
schorr@uni-hildesheim.de



CHI 2009 ~ Spotlight on Works in Progress ~ Session 2 April 4-9, 2009 ~ Boston, MA, USA
4201

every user and every topic without restriction of access
they have been applied in higher education to
encourage self-determined, self-organized and self-
regulated learning, e.g. [3,4,7]. But the existence of
wikis alone, like any other software, is not enough to
guarantee successful learning scenarios. Support and
moderation are important in every learning platform,
especially those allowing constructivist learning, in
order to guide and scaffold the learning process, e.g.
[6]. With every user becoming a potential author and
the amount of content rising exponentially, the lost in
hyperspace syndrome and the cognitive or information
overload problems that are well-known in the context
of hypermedia have an even stronger impact [14].
Thus far, there have been attempts to motivate and
encourage participation as well as collaboration in
collaborative settings by giving incentives. One
example for an incentive is rewarding contributions and
giving ranks to the participants according to the
amount of their contributions [5].
However, our assumption is that motivation is not the
main problem. From our evaluations of wiki-based
learning scenarios [8], we concluded that the students
are very motivated, but feel overwhelmed by the
number of articles and authors in the Wikis in use.
Moreover, they feel they destroy their peers work if
they edit and change the contents others have created.
Therefore, in our wiki-based learning scenarios, editing
and adding to other peoples work has been made an
explicit task and the common goal of collaboratively
creating a knowledge base is emphasized over the
individual goals of externalizing ones knowledge. In
addition to that, different technical support channels
are made available to the students. While these basic
conditions are creating a framework for using wikis for
collaborative learning, they need extensive moderation
accompanying and scaffolding the learning process.
Since this is too much work for one lecturer alone,
supportive software could be employed. According to
[13], there are different possibilities for realizing
automatic moderation to some degree: the software
can either visualize the current state of collaboration
(mirroring) or visualize the current state as well as an
ideal state of collaboration so that the users can adapt
their behavior in order to get closer to the ideal state
(metacognitive tools). A third option is to let the
system interpret the current state of collaboration and
infer feedback for the users from it (guiding systems).
Irrespective of the approach, the state of collaboration
has to be measured first. This work-in-progress paper
describes existing methods for the analysis of
collaboration in CSCL systems focussing on Social
Network Analysis. These methods will then be
transferred to wiki environments. Open issues and an
outlook on future research will conclude the paper.
Collaboration Analysis in CSCL
Before taking a look at the existing methods for
analyzing collaboration, a definition for the term
collaboration is given. According to [12], collaboration
and cooperation can be distinguished by the way tasks
are completed. Cooperative tasks, on the one hand, are
divided into subtasks that are being accomplished
separately by single group members. The result of
cooperation is therefore the sum of the results of these
subtasks. Collaboration, on the other hand, means that
the group members directly work together towards a
common goal. The result of collaboration is more than
the sum of the parts, because every person adds their
CHI 2009 ~ Spotlight on Works in Progress ~ Session 2 April 4-9, 2009 ~ Boston, MA, USA
4202

own knowledge and extracts new knowledge by
working on the task. Cooperation, therefore, mainly
involves communication, while collaboration involves
communication as well as co-construction [12]. When
analyzing collaboration it is important to differentiate
between these two possibilities.
If the participants communication is to be analyzed,
conversation analysis can be applied. The co-
construction of knowledge can be analyzed using
activity-based methods, for example. With Social
Network Analysis it is possible to consider both
communication and co-construction. Using this method,
each user is represented by a node in the network and
the relationship between them is represented by edges
between these nodes. A relationship is consequently
formed by working on a wiki entry together, i.e.
someone edits an entry somebody else has worked on.
Another way of establishing a relation between two
network nodes is to comment on an entry created
and/or edited by one or more peers. This network can
be analyzed on the individual, the group or the
community level.
By pointing out the main differences between Wikis and
other CSCL tools, the next section will show the
effectiveness of this method for analyzing wiki
collaboration.
Differences between Wikis and other tools
Wikis are websites consisting of several interconnected
pages that can be created, edited and commented on
by every visitor to the site [9]. Each wiki page can be
viewed in read mode, like regular websites, or in edit
mode, where the content of the page can be changed
or added to by using so-called wiki syntax [9]. Wiki
syntax is simplified HTML code that allows users to
format and layout the page without having to be
familiar with HTML. Moreover each page usually has a
discussion or commenting section where the
communication about the contents takes place.
Nevertheless, co-construction is the main focus of
wikis. For learning scenarios they therefore offer the
opportunity for collaborative learning and let students
take an active role in their learning process. As a
consequence, conversation-oriented methods are not
sufficient for analyzing wiki contents.
Another difference between wikis and traditional CSCL
software is the absence of fixed groups and roles.
Instead, wikis are characterized by being open to every
user, by not restricting access rights and by allowing
free collaboration within the community. That means
that one user can either work alone on an article or in a
group, together with others. The composition of the
group, however, can change constantly, because a user
can switch groups at any given time and can also be in
several groups at the same time. As a result,
techniques that analyze collaboration on the group level
are not sufficient. Instead, techniques analyzing wiki
collaboration have to focus on the individual and the
community level rather than on groups.
A wikis openness also extends to the topics covered.
For instance, with many conversation-oriented methods
there comes a restriction to specific sentence openers
in order to map predesigned conversation paths to the
analysis results to discover conflicts. Such restrictions
of usage should be avoided in order to maintain the
wikis openness.
CHI 2009 ~ Spotlight on Works in Progress ~ Session 2 April 4-9, 2009 ~ Boston, MA, USA
4203

One last aspect is the fact that using wikis is an
asynchronous process. Therefore the analysis methods
have to take into account that the collaboration has to
be tracked over an undetermined period of time.
Analysis of Wiki Collaboration
There have been attempts to analyze wiki collaboration
by using different methods. [1] for example, have
considered an entrys longevity to infer the quality of
contributions to a wiki article. [15] have developed a
way of measuring the degree of co-authorship of wiki
articles.
However, when wikis are deployed in learning settings
additional measures have to be considered. For
example it is important to measure the amount and
types of activity of all learners to identify those who are
active participants and those who are passive. While
passive browsing, i.e. reading the contents, can still be
considered learning, it mainly enables the readers to
memorize and reproduce knowledge [2]. For higher
order goals, according to [2], the active processing and
construction of knowledge is important. Since our goal
is to enable competence development, we need to
identify passive learners in order to be able to guide
them towards being more active. In order to determine
the degree of collaboration, the number of contributions
can be counted. Contributions can be, for example, the
creation of new wiki entries, editing or commenting on
existing ones.
Social Network Analysis can be used to describe this
collaboration. It has been applied in CSCL contexts
before, but has so far mainly focused on the aspect of
communication. [11], for example, have used cluster
and clique analyses to monitor e-mail communication
and forum discourse. Another example is SAMSA, a tool
that analyzes the actors degree centrality, the network
density and the network degree centralization in order
to study participatory aspects of learning [10]. This
tool has been applied to various settings in which
synchronous as well as asynchronous cooperation and
collaboration was supported. Thus, it was possible to
analyze communication as well as co-construction. The
results show that Social Network Analysis is especially
useful in asynchronous settings and therefore could
possibly be transferred to analyzing collaboration in
wiki contexts.
Regarding the analysis object, SAMSA mainly applies
Social Network Analysis to the group level. As has been
said before, the focus concerning the analysis of wiki
collaboration should be on the individual and the
community levels, which can also be analyzed with this
method.
To give a working example: we measured the actors
degree centrality in one of our wikis which is a glossary
that is filled collaboratively. During the last semester
students worked on creating wiki entries and thereby
produced a set of log data consisting of 1773 records.
After cleaning the dataset, 167 records could be
identified as collaborative activities. The activity types
considered as collaborative are as follows: edit,
comment, upload and tag. According to this data, 11
out of the 21 participating students were actually
collaborating whereas the other 10 participants were
mainly working by themselves. The resulting degree
centralities for the nodes therefore lie between 0 and
0.3.
CHI 2009 ~ Spotlight on Works in Progress ~ Session 2 April 4-9, 2009 ~ Boston, MA, USA
4204

Social Network Analysis should be supplemented by
qualitative measures that look at the quality of the
contents created. The approach developed by [1], i.e.
considering the longevity, is not necessarily a useful
indicator for the quality of an article in respect of
learning settings. Instead, an entrys longevity could
result from a lack of motivation or courage of peers to
edit it, a lack of knowledge on how editing is done or a
lack of factual knowledge on the topic of the entry.
Therefore other means of quality measurement need to
be found that are not biased by these factors.
Important measurements that can be used to infer the
quality of a wiki page are formal aspects like
appropriate formatting, and especially the mentioning
of references. Another indicator that especially applies
in the wiki context is the interconnectedness of a wiki
article, i.e. the amount of hyperlinks to related
concepts within the text of the wiki page. A rather
simple measuring device for determining the quality of
a wiki entry in a learning setting can be the length of
an entry. It often takes more than a few lines of text to
describe a concept in a scholarly way. Therefore a very
short entry is, in many cases, unlikely to be a complete
one.
Open issues and future work
This paper investigates the applicability of Social
Network Analysis to wiki-based learning scenarios
based on a literature review. This theory-driven study
and a working example show the potential the method
could have in wiki contexts. The next step is an
extensive study to verify the assumptions. In this study
further measuring methods will be applied to the wiki
log files and the results will be discussed with
pedagogues in order to evaluate their validity.
By describing how Social Network Analysis can be
applied to model collaboration in wikis we have not yet
dealt with the fact that wiki entries, and thereby their
authors, can be connected by setting hyperlinks inside
the wiki article. That relationship is not as strong as the
direct collaboration by working on the same entry and
therefore would need to be defined as another type of
relationship. As a consequence we are comparing two
different kinds of relationships: direct collaboration
(working on the same wiki entry) and indirect
collaboration (working on related wiki entries that are
connected to each other through hyperlinks).
Commenting should then be defined as the third type of
relationship. Future work will address these issues. The
results of this analysis can be used to support
collaborative processes either by visualization of the
results or by interpreting them in order to give
automatic guidance to the learners and users of the
wiki. This guidance can, for example, lessen the lost-in-
hyperspace effect mentioned in the beginning and help
learners organize their self-regulated work.
CHI 2009 ~ Spotlight on Works in Progress ~ Session 2 April 4-9, 2009 ~ Boston, MA, USA
4205

Acknowledgements
I thank the German county of Lower Saxony for funding
the CELEB project that is realized within the ELAN
context and in which the use of wiki and other tools fr
self-organized learning is studied.
Citations
[1] Adler, B.T., de Alfaro, L., Pye, I., and Raman, V.
Measuring Author Contributions to the Wikipedia. In
Proceedings WikiSym 2008.
http://www.wikisym.org/ws2008/proceedings/research
papers/18500027.pdf
[2] Bloom, B. S. Taxonomy of Educational Objektives.
Handbook I: Cognitive Domain. New York, 1956.
[3] Duffy, P., Bruns, A. The Use of Blogs, Wikis and
RSS in Education: A Conversation of Possibilities. In
Proceedings of the Online Learning and Teaching
Conference, Brisbane, 2006, 31-38.
[4] Ebner, M., Kickmeier-Rust, M., Holzinger, A.
Utilizing Wiki-Systems in Higher Education Classes: a
Chance for Universal Access? In Universal Access in the
Information Society, volume 7, issue 4, 2008, 199-207.
[5] Farzan, R., DiMicco, J., Millen, D. R., Dugan, C.,
Geyer, W., and Brownholtz, E. Results from deploying a
participation incentive mechanism within the
enterprise. In Proceeding of the Twenty-Sixth Annual
SIGCHI Conference on Human Factors in Computing
Systems (CHI '08). ACM Press (2008), 563-572.
[6] Hmelo-Silver, C. E., Duncan, R. G., Chinn, C. A.
Scaffolding and achievement in problem-based and
inquiry learning: A response to Kirschner, Sweller, and
Clark (2006). Educational Psychologist, volume 42,
2007, 99-107.
[7] Jadin, T. Social Software fr kollaboratives Lernen.
In Batinic, B., Koller, A., Sikora, H. (Eds.). E-Learning
in Obersterreich. Digitale Medien und lebenslanges
Lernen. Trauner, Linz (2008), 2335.
[8] Jaksch, B., Kepp, S.-J., Womser-Hacker, C.
Integration of a Wiki for collaborative knowledge
development in an E-Learning context for university
teaching. In A. Holzinger (Ed.). USAB 2008, LNCS
5298, Springer, Berlin, 2008, 7796.
[9] Lamb, B. Wide Open Spaces: Wikis, Ready or Not.
In Educause Review, volume 39, issue 5, 2004, 36-48.
[10] Martnez, A., Dimitriadis, Y., Gmez-Snchez, E.,
Rubia-Avi, B., Jorrn-Abelln, I., Marcos, J. A. Studying
participation networks in collaboration using mixed
methods. In International Journal of Computer-
Supported Collaborative Learning, volume 1, issue 3,
2006, 383-408.
[11] Reffay, C., and Chanier, T. How Social Network
Analysis can help to Measure Cohesion in Collaborative
Distance-Learning. In Proceedings of Computer
Supported Collaborative Learning Conference 2003
(CSCL'2003), Kluwer Academic Publishers, Dordrecht,
NL, 2003, 343-352.
[12] Roschelle, J., Teasley, S. D. The construction of
shared knowledge in collaborative problem solving. In
OMalley, C. (Ed.) Computer Supported Collaborative
Learning. Berlin, Springer, 1995, 69-97.
[13] Soller, A., Martinez, A., Jermann, P., and
Mhlenbrock, M. From Mirroring to Guiding: A Review
of State of the Art Technology for Supporting
Collaborative Learning. In International Journal of
Artificial Intelligence in Education, volume 15, 2005,
261-290.
[14] Stickel, C., Ebner, M., Holzinger, A. Useful Oblivion
Versus Information Overload in e-Learning Examples in
the Context of Wiki Systems. In Journal of Computing
and Information Technology, volume 16, issue 4, 2008,
271-277.
[15] Tang, L.V.-S., Biuk-Aghai, R.P., and Fong, S. A
Method for Measuring Co-authorship Relationships in
MediaWiki. In Proceedings WikiSym 2008.
http://www.wikisym.org/ws2008/proceedings/research
papers/18500125.pdf
CHI 2009 ~ Spotlight on Works in Progress ~ Session 2 April 4-9, 2009 ~ Boston, MA, USA
4206