TOOLS AND EVALUATION TECHNIQUES FOR COLLABORATIVE E-LEARNING COMMUNITIES

Niki Lambropoulos (Lampropoulou)

Submitted for Examination of Doctor of Philosophy

Center for Interactive Systems Engineering London South Bank University London United Kingdom

Tools and Evaluation Techniques for Collaborative E-Learning Communities

To my friends, family and the Greek teachers

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

ii

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Is the sand other than the rocks? That is, is the sand perhaps nothing but a greater number of very tiny stones? Is the moon a greater rock? If we understood rocks, would we also understand the sand and the moon? Is the wind a sloshing of the air analogous to the sloshing motion of the water in the sea? What common features do different movements have? What is common to different kinds of sound? How many different colours are there? And so on. In this way we try gradually to analyse all things, to put together things which at first sight look different, with the hope that we may be able to reduce the number of different things and thereby understand them better. Richard P. Feynman, Six easy Pieces. 1995, p. 23-24.

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

iii

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Acknowledgments
First, I want to express my thanks to Xristine Faulkner and Fintan Culwin, my Supervisors, for all they have done to support me in my studies. Both have given generously of their time and talents, and it is my (possibly biased) view that they provide a model of PhD supervising at its best. I cannot thank them enough. In addition, Louise Campbell and Chung Lam from the Research Office for helping all research students at LSBU on an individual basis. Many other people have also provided valued input to my research through discussions, participating in empirical work, or commenting on written work (or more than one of these). I particularly wish to thank Sophi Danis, Sara BenIsaac, my Yoga mates, especially Ilana Isserow, Jennifer Pearl and Lesley Todd, who have been through the course with me; John Henderson for all the fun and support; also Catherine Spiro, Betty Shane, Ben Daniel, Martha Christopoulou and last but not least, Mariza Smirli for their help and support. Special thanks to my Dad, Konstantinos, my Mum, Aphrodite and my Sister, Georgia for being such a tolerant and supportive family. Most of my work has been funded by the Greek Ministry of Education and Religious Affairs given as three years of educational paid leave of absence. For this, I need to thank Theodoros Birbas, Socratis Papathanasiou, Epaminondas Georgopoulos, and Panayiotis Zevlas. In addition, many thanks to the Greek School Network and in particular, Michael Paraskevas and Vangelis Grigoropoulos for their help and providing access to research space. I need to thank my mates in the Dream-e-Team, Marianna Vivitsou, Dimitris Konetas, Alexander Gkikas, Sofia Papadimitriou, Panayiotis Kampylis, Elias Economakos, and especially Nikos Minaoglou, for the unimaginable creative collaboration all these years. Great thanks to Alexander Muir, Jenny Preece and Ben Shneiderman for the continuous inspiration as well as their insights and energy that enabled me to overcome obstacles. Finally, special thanks to my examiners, Dr Judy Ramsay and Professor Stephen Lerman for the detailed comments and insights that improved this thesis.

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

iv

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Declaration
I grant powers of discretion to the University Librarian to allow this thesis to be copied in whole or in part without further reference to me. This permission covers only single copies made for study purposes, subject to normal conditions of acknowledgment.

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

v

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Abstract

This study provides new multidisciplinary approaches for tools and evaluation techniques to ensure quality in collaborative e-learning communities. The research problem was the Greek teachers’ absence of participation in e-learning discussions for 3 years. Three conceptual frameworks were used to understand and evaluate the situation: passive and active participation, the collaborative e-learning episode, and the sense of e-learning community index. Two interventions were made, collaborative e-learning and the introduction of associated software-based tools: participation graphs and avatars, MessageTag, a tool to depict the levels of critical thinking in collaborative e-learning, and social network analysis tools, the visualisation interactions nodes and centrality. Ethnotechnology was the research design triangulating quantitative and qualitative data as well as social network analysis. The originality of this study lies in the investigation of the processes on the social and learning aspects of collaborative e-learning and associated tools. The results indicated that the suggested frameworks and tools can be useful in supporting collaborative e-learning communities. The wider implications from the findings emphasise the need for: the organisations’ e-learning readiness; the e-learners’ prior knowledge and ability to interact; the facilitation of e-learners’ social awareness; the change of teaching and learning approaches for different levels and types of interactions, participation, and critical thinking; the development of collaborative e-learning communities; the use of tools anchored in learner-centred design and solid pedagogical frameworks. If all components are present in an online course then e-learning quality can be ensured.

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

vi

Τρία εννοιολογικά πλαίσια χρησιμοποιήθηκαν για να κατανοηθεί και να αξιολογηθεί η περίπτωση: παθητική και ενεργητική συμμετοχή. Εάν όλα τα στοιχεία είναι υπαρκτά σε ένα διαδικτυακό μάθημα τότε η ποιότητα της διαδικτυακής εκπαίδευσης μπορεί να εγγυηθεί. το επεισόδιο συνεργατικής μάθησης.D. την ανάπτυξη διαδικτυακών μαθησιακών κοινοτήτων. Τα αποτελέσματα έδειξαν ότι τα προτεινόμενα πλαίσια και εργαλεία μπορεί να είναι χρήσιμα για την υποστήριξη των διαδικτυακών μαθησιακών κοινοτήτων. και τέλος τη χρήση εργαλείων βασισμένων στο χρηστο-κεντρικό σχεδιασμό και ισχυρές παιδαγωγικές προσεγγίσεις. Έγιναν δυο παρεμβάσεις. Οι ευρύτερες επιπτώσεις από τα αποτελέσματα δείχνουν την ανάγκη για: την ετοιμότητα των οργανισμών όσον αφορά τη διαδικτυακή εκπαίδευση. και ο δείκτης αισθήματος ‘ανήκειν’ στη διαδικτυακή μαθησιακή κοινότητα.Tools and Evaluation Techniques for Collaborative E-Learning Communities Περίληψη Αυτή η έρευνα έχει βασιστεί σε διεπιστημονικές προσεγγίσεις για εργαλεία και τεχνικές αξιολόγησης με σκοπό να εξασφαλίσει την ποιότητα στις συνεργατικές διαδικτυακές μαθησιακές κοινότητες. Ph. τη διευκόλυνσή τους για κοινωνική συνειδητοποίηση. Thesis Centre for Interactive Systems Engineering LSBU vii . συμμετοχής και κριτικής σκέψης. και τα εργαλεία ανάλυσης κοινωνικών δικτύων για την απεικόνηση των διαδράσεων με βάση τους διαδραστικούς κόμβους και την κεντρικότητα. Το ερευνητικό πρόβλημα στηρίχτηκε στην απουσία συμετοχής των Ελλήνων εκπαιδευτικών σε συζητήσεις διαδικτυακών μαθημάτων για 3 χρόνια. Η ερευνητική μεθοδολογία ήταν η Εθνοτεχνολογία και χρησιμοποιήθηκε για το συνδυασμό ποσοτικής και ποιοτικής ανάλυσης και ανάλυσης κοινωνικών δικτύων με σκοπό την εγκυρότητα των αποτελεσμάτων. την προηγούμενη γνώση και ικανότητα των εκπαιδευομένων για διάδραση. η συνεργατική μάθηση και η εισαγωγή σχετικών εφαρμογών: αυτά είναι γραφήματα και άβαταρ συμμετοχής. το εργαλείο MessageTag για την αναπαράσταση των επιπέδων κριτικής σκέψης στη συνεργατική διαδικτυακή μάθηση. Η πρωτοτυπία αυτής της έρευνας έγκειται στην ερευνητική διαδικασία για την κοινωνική και μαθησιακή διάσταση της συνεργατικής μάθησης και των σχετικών εργαλείων. την αλλαγή που έχει υπάρξει στις μεθόδους διδασκαλίας και μάθησης για διαφορετικά επίπεδα και τύπους διάδρασης.

1..1 Socio-Technical & User-Centred Design ………………….. Chapter 1: Introduction ……………………………………………………..1.. 2.1.. 2.. Thesis Centre for Interactive Systems Engineering LSBU viii . iv v vi vii viii xi xiii xiv xv 1 2 3 5 7 9 11 12 Chapter 2: The Literature Review ……………………………………… 2.. QUALITY IN ONLINE EDUCATION …………………………………… 1..1.5. 2.6..2..1 From the Individual to the Community: a Brief History … 2.3.2 Instructional Design & Engineering ………………………. Design Principles for e-Learning Communities …. Collaborative e-Learning Episodes …………………… 2.4. REFERENCES ………………………………………………………………….2. 2.1 Greek teachers’ online training ……………………………… 1..2.5. COLLABORATIVE LEARNING …………………………………….1.3.. 2.4.. 2. 2. Index of Graphs ……………………………………………………………. Tools to Support Collaborative e-Learning Communities ……………………………………………...6 PROPOSITIONS FOR DESIGN ……………………………………….. 2.1.1 Towards a Learner-Centred Design ……………..2...5..6.. 2. EDUCATORS’ ONLINE TRAINING ………………………………….1 INTRODUCTION ………………………………………………………….4... 2..D.1 INTRODUCTION ………………………………………………………. 2. 2. The Sense of e-Learning Community Index ………….2. Tools to Structure Dialogical Sequences …….. AIMS AND RESEARCH QUESTIONS ………………………………… 1.3. E-Learning Tools & Interface Design …………...Tools and Evaluation Techniques for Collaborative E-Learning Communities Table of Contents Acknowledgments …………………………………………………………… Declaration …………………………………………………………………….. Index of Appendices ………………………………………………………..4 DESIGN FOR LEARNERS AS USERS & USERS AS LEARNERS . When the Online Community Met E-Learning ………. Levels of Participation …………………………………..1.. Περίληψη (Abstract in Greek) ……………………………………………… Table of Contents …………………………………………………………… Index of Tables ……………………………………………………………… Index of Figures …………………………………………………………….1.2.. … 2.. TOOLS FOR COLLABORATIVE E-LEARNING COMMUNITIES .. REFERENCES ………………………………………………………………… 14 15 15 18 24 25 27 31 31 33 34 38 41 42 44 45 46 51 51 53 55 56 Ph...5.2 Passive Participation in e-Learning Communities ….. 2..2.5.2.6.. Passive Participation in Online Communities …. 2. 1.5. Abstract ………………………………………………………………………. OVERVIEW OF THE CHAPTERS …………………………………….4. 2.5.. 1.3. 1....3 ONLINE & E-LEARNING COMMUNITIES …………………………… 2.

2. REFERENCES …………………………………………………………………..2.1..1.1 Users and activity at Moodle@GSN …………….1.. 3.2 IDENTIFYING INTENTIONS …………………………………………… 4.1. Greek teachers Moodle developers ………………………. 3..2.2.... 5.2 INTRODUCTION ………………………………………………………. 67 68 68 70 70 71 72 73 76 76 79 81 83 86 87 Chapter 4: The Research Context ………………………………………. Participation Awareness: Evaluation Participation Tools …………………… 5.1.2 Ethnotechnology: the virtual ethnography ….1.2.2.1.2. 5..1. 3. E-Research Coordination: Time-short series design ………… 3. Questionnaire Design ………………………………………….3 PLANNING ……………………………………………………………….. 3.2.1.3. 3.3.2. 3... Moodle@GSN: Moodle at the Greek School Network .1. REFERENCES ………………………………………………………………… 4.2.1. The E-mmersion Block ……………………………………… 5. REFERENCES ………………………………………………………………… 108 109 110 110 111 112 113 115 118 125 127 130 133 136 Ph. 3.1 INTRODUCTION ……………………………………………………….2..1 92 93 93 96 99 99 100 107 Chapter 5: Tools & Evaluation Techniques ……………………….2.2. Implications for Research design ………………… 5.2. INTRODUCTION ………………………………………………………… 4.4.2.3...1 5.1.. 5.. The Native’s Point of View: background and Characteristics …………………………………………….2.D.5.1.1.2..2.2 RESEARCH DESIGN ……………………………………………………. Structuring Collaborative E-Learning: MessageTag ……………………………………… 5. 4. Application of guidelines and heuristics from feedback in design ……………………………. 3. E-Learning Research ………………………………………… 3. 4. 3.3.……….2. COLLABORATIVE E-LEARNING TOOLS …………………………… 5.3.1.2.3...2.2..2. Presence & Co-Presence Awareness: Visualisations Interactions Tools ………………… 5.2.2.2 Interactions: Social network analysis ………………… 3.3 Ethnotechnological methods ………………………….1..3.. Research Constraints ………………………………………….. Tools Evaluation Pool ………………………………. ….1 Ethnography …………………………………. 4.2. 5.1 Posts: Thematic analysis ……………………………… 3.2. Application of guidelines and heuristics from feedback in design ……………………………. Design for initial design: Create prototype for testing by the e-learners …………………………………… 5.2. Thesis Centre for Interactive Systems Engineering LSBU ix .1. Examining the Research Context ……………………………..2.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 3: The Research Design ………………………………………. In the Search for Quality: Human-Human Interaction Analysis …………………………………………………….

Conditions of working and learning online ………………… 6.... 6.………………………….. 6.2 7.3.6 PEDAGOGICAL USABILITY .1. 6.5..2. 218 219 219 222 222 223 225 225 232 234 285 Appendices ……………………………………………………………………… Glossary …………………………………………………………………………… Ph.2.1.2..5 THE SENSE OF THE E-LEARNING COMMUNITY INDEX ………. 6.3. Key contributions ……………………………………………...3..2. Trust …………………………………………………………… 6.2.. 7. 6..1..2. Quantitative variables ……………………………………... Global centrality ……………………………………… 6....6....3.1. 6.1.1 INTRODUCTION ……………………………………………………….6. 6. Qualitative variables …………………………………………. Passive participation levels ………………………….5..3. Active participation levels …………………………… 6.. Thesis Centre for Interactive Systems Engineering LSBU x .2.2.5. Active and passive participants ……………………....4..... 7.. Local nodes and centrality in real time ……………..2 DESCRIPTION OF THE RESEARCH CONTEXT …………………. 7.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study …………………………………………………....…………………… 6. Secondary contributions …………………………………….7.1..4..7..5. … 6.3..1. 6.. 7.3.7... …..3..5.6...4 THESIS LIMITATIONS ………………………………………………… 7. On Participation ………………………………………………. Global cohesion ……………………………………… 6.1...5.…………………………… CONTRIBUTIONS ……………………………………………………….1.2..7. 6. Community evolution ………………………………………… 6...1 7.4 THE COLLABORATIVE E-LEARNING EPISODES ………………… 6.3.4. Frequency of visits …………………………………………… 6.. 6.. Previous Knowledge of collaborative e-learning techniques & participation …………………………………………….D..5.2....2.3 TRACING PARTICIPATION …………………………………………… 6.………………………………………………………………….. Correlations & crosstabulations ……………………………… 6.. 6..2...5.2. … 6.7 INTERVENTION ANALYSIS ………………………………………. 6.. Pedagogical Usability . Global Social Network Analysis ……………………………. 6..3 INTRODUCTION ………………………………………………………… SCOPE OF FINDINGS ……………………. Sense of belonging ………………………………………. Initial activities ……………………………………………….5 CONCLUSIONS ………………………………………………………… REFERENCE . REFERENCES ………………………………………………………………… 137 138 139 141 141 142 144 147 147 151 151 152 153 158 158 160 166 167 172 174 177 178 182 184 186 192 197 201 201 204 209 215 Chapter 7: Conclusions………………………………………………. E-learning quality …………………………………………….. 6. 6.5.2..1.5... Intensity ……………………………………………………….5..3.3. 6. Empathy ………………………………………………………. Who are the Greek teachers? ……………………………… 6.

.. 103 Table 4..5-1...3-1.....3.5..1-3.4.2-1...2. 85 Chapter 4 Table 4.. Demographics for the three Greek Teachers / Moodle Developers 130 Table 5..4-1.2. The Socially-Shared Cognition Approach …………………………..2. MessageForum Collaborative Learning Attributes ………………… 48 Table 2.1-1. e-Learners’ replies and dates ………………………………………. Courses categories and number of e-learners ………………… 100 Table 4....1-2. Temporal overview of posted messages (add post/forum) …….2..3.3.3..1-1...1-1.3. Research Design ………………………………………………………….2-1.. The questionnaires’ selection process ……………………………… 139 Table 6. Thesis Centre for Interactive Systems Engineering LSBU xi .. Number of active and passive participants …………………….2-2... Questionnaire open questions ……………………………………..3. 36 Table2. e-Learners’ posts and views in the 6 active courses …………….-1..2..3..3. 152 Table 6... Forums and users view log files in the research pool …………… 148 Table 6.. 123 Table 5.. Greek teachers’ knowledge and attitudes on collaboration and participation ……………………………………………………… 145 Table 6.1. 42 Table 2.3-1...1-3... 154 Ph. Design principles for online and e-learning communities ...2.3-1..2.. 151 Table 6. 132 Chapter 6 Table 6. Active participation levels (Second proposition) ………………..2.. Limitations and strengths in Time-Short Series ……………………..1..4-2.1-2...2-1....1.. 46 Table 2..2-2. E-Learning Engineering for Moodle@GSN ………………………. Tools Pedagogical Usability Scores …………………………………... Moodle@GSN forums and users view log files ………………….1.....3-1...1-1.. Observations..2..2....2. interventions and evaluation ………………………… 84 Table 3.5...D.2.2.1-1. Learning and Instructional Activities ………………………………....1...3.2..2..1-4.…....3.5. Research methodology …………………………….6..2-2.. 153 Table 6. 104 Table 4.1-1.2. Collaborative e-Learning Episodes Coding Matrix ………………… 78 Table 3... Use of MessageTag …………………………………………………... 54 Chapter 3 Table 3..1-4.. Instructional Design problems …………………………………………..2. 150 Table 6. Iterative Design Blocks ………………………………………………… 110 Table 5. 84 Table 3.3-2.6. Reasons for lurking in e-learning communities .4-1.2.1....... 70 Table 3.... 147 Table 6. Similarities and differences between MessageForum & InterLock 50 Table 2.... Chapters Overview ……………………………………………………… 11 Chapter 2 Table 2.3.2.. 53 Table 2..1-2.1-5.1-1.2-1. Strategies for enhancing activity in e-learning . Levels of activity measurement ……………………………………….1-1.3-3.2-1.... 29 Table 2...2....3..3.2. Levels of participation measurement ………………………………….. Temporal overview of all activities ………………………………… 149 Table 6. Research Design …………………………………………………… 129 Table 5.3. Passive Participation Levels ……………………………………..Tools and Evaluation Techniques for Collaborative E-Learning Communities Index of Tables Chapter 1 Table 1. 131 Table 5. Active participation levels (Initial proposition) …………………..5..2. 106 Chapter 5 Table 5.1-1....3. Total number of posts ……………………………………………… 148 Table 6..1-3. 29 Table 2. 21 Table 2.

Knowing the community …………………………………………… Table 6. Training on LMS * Frequency of Use ……………………………… Table 6.4-1.2-4.5.3-1. Collaborative e-Learning Episodes selected for analysis ……… Table 6.5.Tools and Evaluation Techniques for Collaborative E-Learning Communities Table 6.7-2. Table 6.2-2.5-3. The Sense of e-Learning Community Index Checklist ……………. Table 6. Time using LMS * Frequency of Use ……………………………… Table 6.7-3. Cliques ……………………………………………………………. Table 6.5.D. Table 6.7.2-1: Collaborative E-Learning Episodes Overview ………………….2-2..6. Table 6.5. Table 6. Posts from passive participants ………………………………… Table 6. Empathy factors ……………………………………………………..1-1. Table 6.5.. Top 10 Scorers in Out-Degree Centrality ………………………. Trust levels …………………………………………………………… Table 6.5. 155 160 161 162 163 169 174 176 177 177 179 181 186 188 192 193 194 195 201 202 206 207 209 210 212 Ph.. Persistence in Moodle@GSN ……………………………………. Table 6. Table 6.7.2-1.2-3.4. Table 6.6.3-2. Group Network Cohesion: Density & Reciprocity …………….1-3.4-2.. Table 6.7..1-1.5.5.. Table 6.7.. Cliques in Structural Equivalence ……………………………….2-1.6..5.3-2.7-1.1-2. Use of MessageTag in the Research pool ……………………….2-2. Persistence in the research pool ………………………………….6. CeLEs: e-learners and e-tutors’ contributions ………………….7.1-2. Thesis Centre for Interactive Systems Engineering LSBU xii .7.2-3.4.5.3. Intervention Analysis ………………………………………………….5-2. Group Centrality …………………………………………………… Table 6. Correlations for Empathy factors …………………………………. Table 6.2. Table 6.4..1-1. Overall scores for the collaborative tools’ usability and utility ….4.5. Trust development towards individuals …………………………. Table 6. Top 10 Scorers in In-Degree Centrality ………………………… Table 6. E-learning engineering ……………………………………………….5. Collaborative e-Learning Episodes temporal overview ………… Table 6.5.

1-1. Discussion on tools’ Greek names ……………………………….ti) ……………… 118 Figure 5. Participants’ location in Greece …………………………………… 142 Figure 6..7-1. Thesis Centre for Interactive Systems Engineering LSBU xiii ..2....1..... Pedagogical Usability Attributes .. InterLock Interface ………………………………………………. 140 Figure 6..... The Collaborative E-learning Episode (CeLE) ………………………...5.2-1..1.1...1-2.. 135 Chapter 6 Figure 6.2-1..2...1.5...2.6.2-1: Categorisation of collaborative learning tools .2...1-1. 113 Figure 5...5.2..2....2.1-1...2-1....2..1-2.6.. Redesign of participation graphs ………………………………….....….. Visualisation Interactions Tools (VIT) production line …………. 112 Figure 5.1-2.1.1..2. 39 Figure 2... GSN adjacency matrix in UCINET ………………………………. PLATO III ………………………………………………………………… 40 Figure 2. MessageTag ………………………………………………………….1-1.2.2-2..3. VIT Centrality ……………………………………………………….5-1. 185 Figure 6..0 ………………………………… 204 Ph.3.2.1-5.1-1.2... 45 Figure 2... 114 Figure 5.2...2.. 124 Figure 5.... Course and individual participation levels graph ……………….2.....2-2...5-2... Questionnaire Design Methodology ………………………………… 82 Chapter 4 Figure 4. Visualisation Interaction Tools (VIT) Nodes & Centrality …. 111 Figure 5....6.. Organisation of the Education System in Greece 2003/04 ………….52 Figure 2. 47 Figure 2.. VIT Nodes …………………………………………………………… 119 Figure 5.2.....2.. 117 Figure 5... 122 Figure 5.... Participants and number of messages (ATLAS.2.1-1.2.........2. Normality overview for tools pedagogical usability and utility in HCE ………………………………………………………………..1-1. Initial Design for Message Tagging ………………………………...2. MessageForum Attributes in a Discussion Topic ………………… 49 Figure 2... 134 Figure 5.Tools and Evaluation Techniques for Collaborative E-Learning Communities Index of Figures Chapter 2 Figure 2.2. 111 Figure 5.2. 138 Figure 6.1. 126 Figure 5...1-4..3-1 Ethnotechnology and methods …………………………………….2-3 CeLE MessageTag tool ……………………………………………… 127 Figure 5.3.2.5.2-1.1. The e-tutors in the online course ……………………………………..2... 119 Figure 5.2. Location of VIT on the discussion forum …………………………...2-1... 119 Figure 5.2.. Lurkers overall view in VIT Centrality (right) …………………….3-1. 96 Figure 4.D.. 99 Chapter 5 Figure 5...1-1 The Eyeball of Participation ………………………………………… 22 Figure 2..2. 75 Figure 3.2-1 Total Codes Network in the Web Design pool .2.. Participation evaluation graphs production line …………………..2... Correlations analysis in HCE 3.....1-2. Google trend history for online community and e-learning ………….. Moodle@GSN research context …………………………………….1-2..2-1..1-3.1-1. Participation Levels in Collaborative e-Learning Communities …… 54 Chapter 3 Figure 3..

. The e-learning facilitators ………………………………………….4.5...5-1..2. Participating countries ……………………………………………….. Online course categories in Moodle@GSN ……………………… 101 Graph 4.2...... Graph 6.2. and Learning Management Systems (LMS) …………………….Tools and Evaluation Techniques for Collaborative E-Learning Communities Index of Graphs Chapter 4 Graph 4.D..6-3.5...... Graph 6. Graph 6.. Scatter plot for empathy …………………………………………… Graph 6. Correlations between time in education. Active & Passive Participation locus from the same participants . Comparison between sent messages and messages for analysis . 102 Graph 4....2-1. Graph 6.2-1... New members’ contributions ………………………………………… Graph 6.. Graph 6. Participants’ opinions on e-learning community elements ……..5. Reciprocal ties in GSN (a) & the research pool (b) ………….. 104 Chapter 5 Graph 5.. Structural equivalence dendrogrammes in GSN (all) ………. Graph 6. Graph 6..7.3... Roles in the e-learning community ……………………………….... Graph 6.5.....2. CeLEs factors’ comparison graph …………………………………... Comparison for number of words posted by e-learners and e-tutors ………………………………………………………. Elements that show community evolution: e-learning ………….. richness of text.2-1...1-2.. And discussion depth …………………………………………….....1-1.2.2.1-2....2-2..4.7... Comparison between number of messages and replies ...7.5. use of computers.. Community evolution elements …………………………………….... Graph 6....5. Correlations between codes on collaborative e-learning quality Graph 6..1-3.1-2.1.......3-1: VIT Nodes in CeLE IX …………………………………………...5....7.1-1.1-1. 31/03/2006 – 5/28 e-learners ……………………………………… 121 Graph 5.1..3-1..1-5.. 08/04/2006 – 10/28 e-learners …………………………………….1-3..7...1-3.6-2.2-2..5..2.1-2..7... Thesis Centre for Interactive Systems Engineering LSBU xiv . Graph 6. Graph 6.3-2...1-1.1..2. 04/04/2006 – 10/28 e-learners …………………………………….1-1. Comparison between messages for analysis..5..1-1.3-1. Activity in the online course …….. Structural equivalence dendrogramme in GSN (e-learners) . Graph 6.3. 121 Chapter 6 Graph 6..6-1. Structural equivalence dendrogrammes in the research pool (all) …………………………………………… Graph 6..3.. Logs of overall activity VS posting ………………………………… Graph 6.3..2. Graph 6.....4... Structural equivalence dendrogramme in the research pool (e-learners) ………………………………… Graph 6..1-2..2..2-1....5.. 121 Graph 5. Graph 6... Communication with the educational authorities …………………. 143 146 149 150 155 159 162 164 167 170 172 173 174 175 178 182 183 184 187 170 170 191 191 198 199 Ph.5.3.3...1-4.....5.. 115 Graph 5.. Passive & Active Participation Process …………………………. Graph 6...5... Comparison of themes and community elements ………………. VIT Centrality in CeLE IX ………………………………………...7... Graph 6.. Graph 6.....3-1..5.1-3........ Graph 6..5.5..………………………………….5..

................ A_X_3........................ A_VII_10. Crosstabulation: Moodle Use * Time using LMS ........................................................................................ Participation in e-learning Communities ........ Moodle Usability .................................. Post-retreat opinions on new members’ contribution . Invitation to the study ..................... Collaborative e-Learning Episode IX ............ Suggestions for Writing Online Messages ................. Appendix IV: Questionnaire Main study I (sample) …………………………….................................... Appendix VI: Participants’ Documents ……………………………………………............................... Appendix II: Risk Management …………………………………………………… Appendix III: Initial Questionnaire (sample) …………………………………….................... A_IX_3. A_VI_3....... CeLE-III Code Network . A_VII_13....... Post-retreat opinions on participation ... A_VII_4.................................................... Post-retreat opinions on communities ..... A_X_5......... A_VII_3.. Appendix X: Collaborative e-Learning Episodes (Examples from the Main Study ……….................. A_X_2......... A_VI_2...Tools and Evaluation Techniques for Collaborative E-Learning Communities Index of Appendixes Appendix I: Online Community Management …………………………………… A_I_1.......... Crosstabulation: Internet Use in class * Use for educational purposes ................................ CeLE-III Analysis .. A_VI_1. Post-retreat opinions on e-learners’ participation in the project .................. Demographic data .................... A_III_1..........................D...... Demographics ............................. 234 234 235 236 239 239 240 240 242 250 250 250 251 252 252 253 253 253 253 254 254 254 255 255 256 257 258 262 263 264 264 264 265 266 266 267 267 268 268 269 270 270 271 271 Ph.. A_VII_12................... Use of e-learning tools ... A_IX_6............................................. Post-retreat opinions on learning ..... A_VII_6.... ……………………………………………….......... Messages Quantitative Analysis ....................... A_VII_7...............................Interactivity Management ... Post-retreat opinions on new members’ contribution ....... A_VII_5................................ A_X_4. A_IX_4........................ A_VII_14. A_IX_2....................... Reasons for participating in e-learning communities ....... Messages Analysis: Collaborative E-Learning Episode I ............... Training in Educational Technologies ..... Collaborative e-Learning Episode III ......................... A_IX_7.... A_IV_1................ Post-retreat opinions on other communities .......................... A_I_2.... Forums and users view log files .......................................................................... A_VII_9...................... Thesis Centre for Interactive Systems Engineering LSBU xv ........................................... CeLE-III Locus (CeLE-CIII: Stanzas 4-23) ............. Appendix IX: Thematic Analysis in the Main Study ……………………………… A_IX_1..... Internalisation and externalisation thought process .................................................. Participant SP1: Thought Processes ............................... CeLE 1 (AIa-1:stanzas1-25) .................... Moderator’s Responsibilities . Appendix V: Questionnaire Main study II ……………………………………….... Appendix VII: The E-mmersion Data Analysis ……………………………………...................... A_VII_1....... Appendix VIII: Conditions of working and learning online …………………….................. A_X_6.......................... A_X_1.................. The usability section in the Initial Questionnaire ...................... A_VII_11................. A_VII_8....................... Netiquette .............. Instructions of use Moodle and the Research Pool ............. Online course experience ....................................... A_VII_2............................ A_IX_5.............

....... Pedagogical Usability – Utility results ………………………… A_XII_2. Appendix XI: Messages Quantitative Analysis ……………………………………..D....... 276 276 277 278 280 280 281 282 282 285 286 288 289 Ph............................ Thesis Centre for Interactive Systems Engineering LSBU xvi .. Messages Quantitative Analysis in Moodle@GSN ............. CeLE-IX Locus (CeLE-GIX: Stanzas 2-84) .. Internalisation and externalisation thought process ................. Most important correlations in the Hierarchical Clustering Explorer …........ Messages Quantitative Analysis in the Research Pool .…....... A_XI_2. A_XI_1........................................... A_XII_1... A_X_9........... The best thing in the project ……………………………………… Appendix XIII: Recommendations …………………………………………………. Correlations in SPSS …………… ……………………………… A_XII_3..................... A_XII_4..... CeLE-IX Analysis .......... Appendix XII: Main Study Data & Reports ………………………………………….... A_X_8... A_X_10.....................................Tools and Evaluation Techniques for Collaborative E-Learning Communities A_X_7.................... CeLE-IX Code Network ...................

Consequently. Because socio-cultural learning theories and e-learning design techniques evolved separately. An example is presented from the Greek teachers’ e-learning community project aimed at their professional training and development. this study targets the development of evaluation techniques and associated tools to enhance participation in collaborative e-learning communities. they lack convergence.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 1: Introduction 1 • • • • • Introduction: The problem of e-learning quality related to socio-cultural learning and participation in e-learning communities Key Topics Covered in This Chapter: What the study is about Why educators’ online training is important for their professional development The problem of quality in online education as a distinction between provision of information and acquisition of knowledge Aims and research questions Overall structure of the thesis Chapter 1 introduces the research context. Ph. An initial study showed that the mere provision of information did not facilitate new-knowledge construction. the research problem and the aims and objectives of this study.D. This is one of the causes of quality problems in e-learning. Thesis Centre for Interactive Systems Engineering LSBU Page 1 .

Socio-technical design and user-centred design were planning approaches aiming to acknowledge that the development of interactive technologies increasingly relies on an appreciation of the social circumstances in which systems are used. However. if educational design could understand the technology of collaborative practice. It appears that learning within e-learning communities is not always successful. Educational or instructional design is the systematic processing of activities to solve an instructional problem with the aid of technologies. For this reason they fail to support e-learners’ transition between internalisation to externalisation and becoming active participants. the socio-cultural focus and the use of technology. Thesis Centre for Interactive Systems Engineering LSBU Page 2 . mere provision of information points to poor e-learning quality. started in 2003 as part of a project for online teachers’ training and aimed at enabling teachers to acquire new competencies. e-learning quality could be improved. she acts as both a user and a learner. So. the e-learning systems were found to be information-based mainly supporting monologue instead of being communication-based towards dialogue. In addition. Nonetheless. However.D. educational design and in particular e-learning design neglected the dual and situated persona of the learner. these two trends have evolved almost separately. Ph. Thus. these aims were not met because of passive participation and this implies that information acquisition may not be automatically related to collaborative learning. This chapter introduces the concept of quality in e-learning and examines its relationship to socio-cultural collaborative learning and associated design. The research context is the Greek teachers’ e-learning community.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 1: Introduction 1.1 INTRODUCTION For the past 50 years two main trends have been observed in education.

ICT is the backbone of the knowledge economy and has been recognised as an effective tool for promoting economic growth and development (World Bank Report. Scientific and Cultural Organization (UNESCO) report (2002). but rather. To solve this problem. the current shift occurring in the Web from a static content environment where end users are the recipients of information—defined as Web 1. 2000:8). 2007). The integration of Information and Communication Technology (ICT) in education as well as the social and collaborative nature of the Internet provided another medium for communication and training.D.0 is a platform where “knowledge-working is no longer thought of as the gathering and accumulation of facts. 2006). 2005). In his foreword for the United Nations Educational. To Berners-Lee (2007). and countries even within the same continent. Thesis Centre for Interactive Systems Engineering LSBU Page 3 . 1997. Thompson & Schmidt.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 1: Introduction 1. 2005). indicating a digital divide (Reddy & Manjulika. Web 2. general access to ICT varies across continents. despite the advantages. Daniels said that within a short time ICT has become one of the basic building blocks of the modern society. however. the Web is not only a technological tool but also a social phenomenon that enables collaboration and creativity.0—can be described as a transition to a more distributed. E-Learning is a component of TEL and describes learning via the Internet. the riding of waves in a dynamic environment” (Downes.2 EDUCATORS’ ONLINE TRAINING Education is generally acknowledged as one of the crucial components of personal and professional development. teachers’ education has been severely criticized on the grounds of both quantity and quality (e. 2002). UNESCO suggests that countries need to keep pace with technological development and the changing competencies. reflected in the Ph.g. participatory. the problem of e-learning quality is evident worldwide. intranet. The freedom that e-learning offers and the increasing number of online courses provided by educational organisations offer new opportunities for personal and professional development in a life-long learning course. Nevertheless. and extranet (WR Hambrecht and Co. or in what ESRC now calls Technology Enhanced Learning (TEL) (ESRC. Chen & Kee. Perraton & Potashnik. DarlingHammond & Bransford. 2005. Furthermore. Despite the expansion of ICT. and collaborative environment (Delich. educational institutions and business have been investing in the use of ICT in Education. 2006). Organisations.0—to one where they are active content creators—defined as Web 2.

2 There was no additional information for this project other than the website: http://fecone. the Australian National Quality Schooling Framework (NQSF) (Hartnell-Young.. an example that failed to engage teachers in an e-learning community is the European Minerva Project ‘Star Science’ aimed at collaboration between science teachers from Ireland. However. Oliver. the international project ‘Tapped In’ on a voluntary basis (Schlager & Fusco. their productivity and their wages should increase. UK. and Blackboard. or Global University Alliance. 2004. not all projects were successful.org/sites/.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 1: Introduction curriculum and teacher training. According to Harvey. Commercial LMS like Centra.K. for example.eu/. for example. 2004). 2003). 2002:633-640). Research studies have demonstrated that e-learning has both positive and negative impacts in terms of effectiveness and achievement of outcomes (Franklin et al.. 1. companies and institutions often use commercial Learning Management Systems (LMS) for online teachers’ training. 2001.passionforlearning. and 97 in Greece.forth. One of the reasons is because e-learning has created confusion between the mere supply of information and knowledge-building (Barbera. However. and Open Source software as Bodington. Dokeos or Moodle are nowadays widely used. 2003). Thesis Centre for Interactive Systems Engineering LSBU Page 4 . (http://moodle. but they can also expect the nature of their jobs to change with a great deal of specialization (UNESCO. This in turn would cause educators to become more important. or the European projects ‘Implementing Standards for European e-Tutor Training’ (on going. ISEeTT aims to define the core curriculum and quality standards for European e-Tutor training in relation to different national contexts. The Greek partner is the Educational Research and Evaluation Group. To support this life-long learning context. the UK failed to Leonardo DaVinci project ‘Implementing Standards for European e-Tutor Training’ (ISEeTT. have not succeeded in meeting a number of promises (Garrett. and capital.etutorportal. Sims et al. educational systems will become a national resource as important as the traditional factors of production-land. et al. A number of projects have addressed teachers’ online training using LMS successfully. one of them is in the service provided by the Greek School Network (GSN). Foundation for Research Technology .gr/). 1 Ph. These competencies are not only related to using the devices but also working on procedures that give access to information and skilfully transforming information into knowledge. and Bulgaria (Harvey.Hellas (http://www.net/). such as UK eUniversity. labour.124 registered sites from 182 countries. last access 29/05/2007). 2006-2007 1 ) and ‘E-Learning Fundamentals’ (started in May 2007 2 ).D. 2006). 2005). http://www. Economic advantage will accrue to a population that acquires competencies in processing information into knowledge and applying it in work and everyday life. 2004. Whatley. Universitas 21.855 in the U. As this is the task of the educator. there are more than 26. 2004). e-learning international and national projects.

Collaboration in design and planning by all stakeholders to bring pedagogy and technology together is lacking as the most valuable idea/suggestion: learner / user involvement in the design”. there is no evidence that the Greek teachers use the e-learning service effectively and skilfully to transform information into knowledge.2.1 Greek Teachers’ Online Training In Greece. for example at the University of the Aegean (Hlapanis & Dimitrakopoulou. money for) planning and organisation are lacking. In short. (There is no agreement between researchers on a single definition of community. Thesis Centre for Interactive Systems Engineering LSBU Page 5 . In this study. 2005) or projects based on collaborative activities between the Greek Pedagogical Institute. 1. There were efforts to facilitate online teacher training either from universities or governmental organisations in collaboration with Higher Education Institutes. An online community is a community where social interactions are facilitated by information and communication technologies. and other Greek universities. One of these initiatives is the use of Moodle as part of the Greek School Network (GSN) services. teachers’ training is mainly onsite and organised by the Greek Pedagogical Institute in collaboration with the Greek Ministry of Education and Religious Affairs. there was no significant evidence of socio-cultural learning in the European project ‘E-learning Fundamentals’ in which I took part in June of 2007. for more than three years (1077 days on the 13/10/2006 according to the log files). the Research Academic Computer Technology Institute.) However.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 1: Introduction participate and Bulgaria’s participation was minimal. Ph. a community is “a group of people who consciously share a sense of belonging anchored in common interests and enhanced by social interactions”. e-learning outcomes seem to be unpredictable. This means that isolated courses were developed as part of PhD studies. time. that is absence of posting. Although the Greek teachers do visit and download material as apparent from the log files. Following one of the participants in one of the studies “absence of participation in GSN is probably because (desire. Moodle@GSN was built to aid Greek teachers’ online training by developing an online community.D. Similarly. It also appears that broad collaboration is now crucial for contemporary organisations. Moodle@GSN appears not to have worked in that there has been a high level of passive participation.

e-learning is based on activities and experience (active and experiential learning) within groups and communities (collaborative learning). there are more opportunities to access learning resources. Nevertheless. instructional. time and cost are less because of the use of the electronic form of resources. and communication is nonlinear. some advantages have been reported in using e-learning for teachers’ training (Golian. 2000): e-learning is individualised and self-paced. 2000). Thesis Centre for Interactive Systems Engineering LSBU Page 6 . technical. there are several obstacles: institutional. Ph. and personal (Piotrowski & Vodanovich.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 1: Introduction Even though e-learning has delivered mixed results.D.

However. showed that 61% of the 433 respondents rated the overall e-learning quality somewhat negatively. the European Agency for Vocational Training. why it is important. 2002:3). as ‘fair’ or ‘poor (Massy. Nevertheless. Collaboration is here seen as the act of shared creation and/or Ph. quality refers to fitness for purpose. 5. and a further 7% finished the two basic sections on quality in e-learning. and the high levels of interest among educators. Overall. et al. the collapse of several initiatives may not indicate the failure of the e-learning concept per se but rather a lack of quality. despite the huge investments in technology and e-infrastructure. In general. 2005). and whether quality assurance is feasible. Ehlers. thinking together over demands and tackling complexities.D. administrators and policy makers worldwide. Socio-cultural learning is related to e-learning systems and tools used to facilitate and support socio-cultural learning principles and in particular collaborative learning.3 QUALITY IN ONLINE EDUCATION Online Education can support employees’ new competencies and training. The questions that arise are related to what constitutes e-learning quality.. 2005). The researchers stressed that ‘learners must play a key part in determining the quality of e-learning services’ and insisted on the involvement of all e-learning participants in quality design and development by 2010 (p.023 people called up the questionnaire. the PANORAMA report revealed the importance of e-learning quality: the need for critical awareness. In this survey. this is not only a Greek but also a global phenomenon.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 1: Introduction 1. UNESCO’s definition suggests that collaborative learning takes place: when learners work in groups on the same task simultaneously. of whom 28 % actually completed it. Thesis Centre for Interactive Systems Engineering LSBU Page 7 . and the need for quality requirements in e-learning design. According to the results. design for socio-cultural learning appears to be connected to e-learning quality. and in this context as applied to learning (Stephenson. e-learning remains an unproven experiment.11). quality relates to obtaining the best learning achievements (50%) and ‘something that is excellent in performance’ (19%). The European Foundation for Quality in eLearning (EFQUEL) conducted a European survey between 15 August 2004 and 15 November 2004 (Panorama Report. A survey on e-learning quality for CEDEFOP. In brief. the need for specific analytical frameworks as the respondents although believed that they knew about quality they showed a general lack of information.

) This means that collaborative learning is related to co-creativity and has the potential to occur online. Therefore. Technology & Learning definitions. Within the context of electronic communication.D. Thesis Centre for Interactive Systems Engineering LSBU Page 8 .Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 1: Introduction discovery. UNESCO (n. collaborative learning can take place without members being physically in the same location. collaborative learning and associated tools may influence e-learning quality.d. Ph. interventions to support social interactions. It also suggests that if one of the elements is missing then e-learning quality may be impaired.

Thus. The first set of research questions will be investigated in the literature review. Is there an educational design approach that can ensure quality in specific e-learning contexts such as the Greek teachers’ community? Q2.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 1: Introduction 1. Are there any collaborative e-learning frameworks to support CeLC? Q2. the goal of this thesis is to shed light on passive participants in e-learning communities and propose conceptual frameworks to ensure their participation. Are there any design principles for specific educational contexts? Ph.A. Thesis Centre for Interactive Systems Engineering LSBU Page 9 .B. These are: Q1.4 AIMS AND RESEARCH QUESTIONS The problem this research addresses is the lack of e-learning quality as evidenced by the Greek teachers’ passive participation in e-learning communities. Since this appears to be a common problem it is worth investigating.B. What is design? Q3.A.D. More specifically. In what ways can studies of Collaborative e-Learning Communities (CeLC) be exploited in a concrete way by educational designers? Q1. In what ways has collaborative learning research evolved up to the present? Q1. this study aims to carry out the following in regard to Collaborative e-Learning Communities: The study translates into two sets of questions in order to study the conditions to tackle e-learning quality.

Are tools for evaluating participation helpful to enable participation? Q3. Are there any tools and techniques that can be used to facilitate the formation of CeLC? If so.Ex1.A. Tools to Support CeLC Q3.B. Are tools and techniques for evaluating participation helpful to enable participation? Q3. Are there any evaluation techniques that can be used to facilitate the formation and maintenance of CeLC? 2. what are the effective characteristics and usage of these tools? Q3.Ex1.Ex1. Is there a collaborative learning scheme to identify.Ex3. Are tools for observing and analysing interactions helpful to enable participation in the Greek e-learning community? Q3.Ex2. Educational Design Q2.Ex4.Ex2.Ex2. In what ways can quality by design be achieved for the Greek elearning communities? 3. In what ways can core design principles be integrated in the process of educational design? Q2. Are tools for structuring collaborative learning helpful to enable participation? Q3.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 1: Introduction Q3. Collaborative e-Learning Communities (CeLC) Q1. analyse and evaluate CeLC? Q1. Are tools for observing reporting information on interactions helpful in enabling participation in the Greek e-learning community? Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 10 . Are tools for observing and analysing interactions helpful to enable participation? The second set of questions has an exploratory nature and will be based on the previous results: 1.

Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 1: Introduction 1.D. research problem Aims and objectives Research questions Chapters overview Collaborative e-learning E-Learning communities and participation Tools and evaluation techniques Design Conceptual framework Research methodologies Understanding the research context Ethnotechnological inputs Preliminary studies Planning and design Implementation of previous conceptual framework Findings Discussion Summary of the Thesis Conclusions Recommendations Future trends 2 Literature review 3 4 5 6 Research design Research context Tools and evaluation techniques Main study 7 Conclusions Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 11 . Chapters Overview TOOLS AND EVALUATION TECHNIQUES FOR COLLABORATIVE E-LEARNING COMMUNITIES Chapters 1 Introduction Description Motivation.5 OVERVIEW OF THE CHAPTERS The chapters in this thesis are developed as follows: Table 1.5-1.

& Sims. (2006). 25-30 June 2001. U. Ehlers. Published doctoral dissertation. from http://proquest. (2005). Ph. Bodington (n.) Retrieved 15/02/2006. CEDEFOP (n.computing. Lewis. R. M.S.uk/. ‘Utilizing Internet Resources by Education Professionals in the New Millennium’. A Model on Knowledge and Endogenous Growth World Bank Policy Research Working Paper 3539. Retrieved 21/08/07. & McGuinness. the 13th Annual World Conference on Educational Multimedia.0. Tampere.). (2002) “Foreword” in Information and Communication Technology in Education – A Curriculum for Schools and Programme for Teacher Development. (2004). British Journal of Educational Technology.). Finland. Berners-Lee.uk/ESRCInfoCentre/opportunities/current_fun ding_opportunities/tel_call. Hartnell-Young.d. L. http://www.dokeos.org.C.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 1: Introduction REFERENCES Barbera.org/etools/library/latestversion. Inc. S. Paper presented at ED-MEDIA. USA: Idea Group. from http://elearnmag. (2004).blackboard. Retrieved 20/08/2007. Golian.d.co. Retrieved 17/06/2007. 3-6. H. Evaluation of an Online Community: Australia's National Quality Schooling Framework.D. Preparing Teachers for a Changing World: What Teachers Should Learn and be Able to Do.sch.eu/.eun. Downes.com/pqdlink?Ver=1&Exp=09-092012&FMT=7&DID=1144195631&RQT=309&attempt=1. The Greek School Network.org/.d. CA: Jossey-Bass.europa. P.). A.. Quality in virtual education environments. from http://www. Retrieved 12/11/2006.. & Kee. from http://www.aspx?ComponentId=14262&SourcePageId=5964.org/subpage. User-Centred Design of E-Learning Communities. J. Retrieved 12/04/2006. R. L. E. Dokeos (n. Chen. from http://www. Association for Computing Machinery.d. European Foundation for Quality in eLearning. 35(1). Delich. Peat. Paris: UNESCO Darling-Hammond. from http://www.qualityfoundation. B. D. S.centra. CA.uk/computing/analysis/2186086/berners-lee-warnschanges-ahead.. (2007). 19(3): 136-143.asp?135703. Educause Quarterly.com/. Cedefop: The European Centre for the Development of Vocational Training. Retrieved 13/05/2006. (2005).d. Blackboard (n. Retrieved 22/10/2006. 27(4). L.L. Hershey. ESRC (2006).umi. The real story behind the failure of the UK eUniversity. Computing Magazine. from http://www. In Niki Lambropoulos and Panayiotis Zaphiris (Eds) (2006).cedefop. Garrett. CENTRA Education & Training (n. from http://www. K.Quality in European elearning – PANORAMA report. E-learning 2.).cfm?section=articles&article=29-1 EFQUEL (n. from http://info. from http://cms. Technology at the cutting edge: A large scale evaluation of the effectiveness of educational resources.H.org/.com/. M. (2005). Franklin. from http://www.worldbank.esrcsocietytoday. Pepperdine University..d. Daniels J. Berners-Lee warns of changes ahead. GSN (n. Pedagogical and interface modifications: What instructors change after teaching online. Hildebrandt. 13-20. from http://www. Retrieved 15/02/2006. & Bransford..bodington. (2000). Görtz. Information Technology and Libraries. Economic and Social Research council. Hypermedia & Telecommunications.pdf.org/shared/data/pdf/quality_in_elearning_panorama_en. (2005).). Thesis Centre for Interactive Systems Engineering LSBU Page 12 . Malibu. San Francisco.). A study by the European Quality Observatory. eLearn Magazine. T.d. Retrieved 10/09/2007. (2001). Retrieved 15/02/2006. & Pawlowski J. (2006).ac. E. Retrieved 15/02/2006.gr/en/.

Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 1: Introduction Harvey. 3. Australia.sch. 13(3). T.astd.org/shared/data/pdf/qual_onlinehe. M. WR Hambrecht & Co (2000). Schlager. Paper presented at a workshop for the European Schoolnet in Saragossa.htm. Melbourne.org/tappedin/web/papers/2003/TPDBarab.org/NR/rdonlyres/E2CF5659B67B-4D96-9D85-BFAC308D0E28/0/hambrecht. & Hand.V..org/documents/Vol3/v3p053-063-033.. Designing for Virtual Communities in the Service of Learning. A. & Manjulika. V. J.) Towards VirtualizationOpen and Distance Learning.). A. UNESCO (2002). Information and Communication Technology in Education – A Curriculum for Schools and Programme for Teacher Development.). Oliver. Vol. Meeting at the Crossroads.pdf. (2004). & Potashnik. Peratton. Thesis Centre for Interactive Systems Engineering LSBU Page 13 . Stephenson. Winter 2006-2007. Greece. 53-63. Paper presented at the 18th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE). S.d. Spain. 48-53. Hlapanis. from http://www. in V. (2003).org/education/educprog/lwf/doc/portfolio/definitions. December 9-12. (2002).. (2005). from http://e-learning.unesco. ‘Are the Reported Barriers to Internet-Based Instruction Warranteed?. Retrieved 12/07/2004. pp. H. Paper presented at the Conference Communities of Practice. J. from http://tappedin. from http://www.uk/docs/qualitysummary. technology. Athens.. 29 September – 3 October 2004. from http://unesdoc. Massy. J. 2001. Paper presented at the 4th Conference in ICT in Education. & Fusco. Supporting Communities in Practice: an exploration of two case studies.d. Retrieved 22/11/2006.d. J. Journal of Computing in Teacher Education (JCTE). and communities of practice: Are we putting the cart before the horse? In S. Research in Learning Technology. Whatley. Retrieved 14/03/2006. Quality and eLearning: BizMedia.D. (2001). Retrieved 24/10/2005. Teacher professional development. R. Moodle (n.ascilite. The University of Sheffield & Sheffield Hallem University (UK). Cambridge. Retrieved 22/10/2007.pdf. Greek Teachers' Training via the Internet: Presentation of the Greek Online Community KME from the University of the Aegean.V.pdf. Sims. G. 8-10 September 2003. Proactive evaluation: New perspectives for ensuring quality in online learning applications. Gray (Eds. Moodle at the Greek School Network. & Schmidt. An Agent System to Support Student Teams Working Online. Moodle@GSN (n.pdf..org. ALTJ. 509 – 517. & J. J. and Vodanovich (2000). 173-187. (1997) Teacher Education at a Distance. from http://cms. (2002) The Changing Context of Higher Education in the 21st Century. and S. E. Retrieved 20/08/2007. Retrieved 22/05/2006 . 2. from http://www.pdf. Retrieved 12/11/2006. UNESCO (n. New Delhi: Kogan Page. Corporate E-learning: exploring a new frontier. Paris: UNESCO. Reddy and S Manjulika (eds. Ph. 120153. Dobbs.org/images/0012/001295/129538e.co. Piotrowski. No 2. pp.unesco. (2007). (2004). Journal of Information Technology Education. UK: Cambridge University Press. (2004). from http://jite. Thompson. Barab. from http://moodle.gr/. M. Pp. J.pdf.org.eun. Quality assurance and elearning: Blue skies and pragmatism. D. from http://www. (2005).) Technology & Learning definitions: Collaborative Learning.elearningage. A. C. 10th Association for Learning Technology Conference (ALT-C). & Dimitrakopoulou.au/conferences/melbourne01/pdf/papers/simsr. Retrieved 21/08/2007.) Modular Object-Oriented Dynamic Learning Environment. A Synthesis of Recent Research Education 121(1). Reddy.pdf. Education and Technology Technical Notes. R. Kling. 1-47. Definitions of indicators of quality on the application of ICT to University Teaching. Retrieved 21/06/2003. R. 23(2).

However. Thesis Centre for Interactive Systems Engineering LSBU Page 14 .D. Ph.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review 2 • • • • • • Literature Review: Design for Collaborative e-Learning Communities Key Topics Covered in This Chapter: Why collaborative learning is important for today’s e-learners How and why educational research moved from the study of the individual to the study of the community Participation in online and e-learning communities How design supports the user as a learner within a community The need for further investigation for tools and evaluation techniques to support collaborative e-learning communities New propositions Chapter 2 discusses the importance of collaborative learning to effective elearning. Collaborative learning research as well as current design for e-learning suggest that the research focus has evolved from the study of the individual to the study of the community. there are still questions about instructional design as well as quality in e-learners’ interactions. Investigation of the literature pointed to new propositions for tools and evaluation techniques to enhance participation in collaborative e-learning communities.

D.g. Only recently. there should be specific targets relative to their backgrounds. in his theory of transformative learning. 2. critical.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review 2. Activity in the form of discussion of shared experience has been considered an effective means for adult learning (e. Brown & Duguid. their observation and study is difficult. Consequently. and passive. Other than the importance attached to socio-cultural learning. it is possible to study human activity and inference from this cognitive change. Thesis Centre for Interactive Systems Engineering LSBU Page 15 . Brookfield. Nonetheless. instruction should be task oriented. to explore its social nature and the role of tools in collaborative elearning communities. where the individual participate by posting. 2001). Because the intermediate variables are invisible.1 INTRODUCTION Education as a discipline was initially anchored in Cognitive Psychology and Pedagogy. 2004).g. adults not only accept but also pursue passive learning (Rogers. cognition is a complex social phenomenon that occurs within the individual’s head. collaborative learning as well as passive and active participation in dialogue are concepts also related to recent approaches to adult learning. external factors such as organisational and financial problems could be major obstacles (e.2 COLLABORATIVE LEARNING According to Lave (1988:1). It refers to intermediate variables that describe social interactions and their relationships with the conditions that facilitate learning. without active participation passive participation is not possible. proposed that adults were reflective. 1990. has learning with the use of tools introduced design issues. thus. additionally. where the individual does not. participation in discussions can be active. However. and open to others’ opinions. Brookfield. Knowles (1984) proposed the theory of andragogy (adult learning) to complement pedagogy (child learning): adults were responsible for their learning. This chapter attempts to shed light on the multidisciplinary nature of e-learning. Mezirow (2000). 2000). In reality. 2002. active participation in groups was essential. adults were less flexible towards Ph. Jarvis. However. and discovery should be guided and facilitated thus the relationship between the instructor and the learner needs to be redefined.

However. Thesis Centre for Interactive Systems Engineering LSBU Page 16 . Mezirow proposed that dialogue. in the pursuit of shared goals. the enjoinment of the joint activity. UNESCO’s definition embraces most of the aforementioned concepts.) An attempt to propose a distinction between cooperation and collaboration was made by Teasley and Roschelle (1993): Collaboration is a coordinated.D. such division is not deliberately required from the participants. Two terms have been used interchangeably in collaborative learning history: cooperation and collaboration. the collaborative learning pillar. 2002). It empowered and enabled learners to solve problems and understand subjects more easily since discussing ideas and constructing arguments through dialogue could shape in-depth learning. was the solution. 1994 provide an overview on cooperative learning. synchronous activity that is the result of a continued attempt to construct and maintain a shared conception of a problem… Cooperative work is accomplished by the division of labour among participants. collaborative learning occurs when learners work in groups on the same task simultaneously. Teasley & Roschelle. thinking together over demands and tackling complexities. 1993:235 Teasley and Roschelle provided a clear distinction anchored in the idea that tasks are divided between participants: “each person is responsible for a portion of the problem solving”. although roles exist naturally as a spontaneous division of labour. or simply furthering the relationship” (Argyle. as an activity where each person is responsible for a portion of the problem solving. in a coordinated way at work. Nonetheless. Cooperation was the basis of sociability “acting together. Borgers and Baranauskas (2003) advocated collaborative learning as it had more advantages than other types of group learning. Collaboration is an interactive process that engages two or more participants working together to achieve outcomes they could not accomplish independently (Salmons & Wilson. Ph. they did not acknowledge their passive participation (Illeris. In their review. or in social relationships. 2008).Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review change and for this reason. 1991:15). (Johnson & Johnson.

According to the Free Online Dictionary for Computing (http://foldoc. et al. as. while the other becomes an observer. it is posting occasionally or not at all. Joint Information Systems Committee online conference discussions on ‘Innovating E-Learning’. http://www. If passive and active modes are acceptable.D. According to Miyake (1986:174): "The person who has more to say about the current topic takes the task-doer's role. but reading the group's postings regularly. This term was not pejorative. Thesis Centre for Interactive Systems Engineering LSBU Page 17 . UNESCO (n..jisc. they had to: talk to create the context.d. there should be some processes and pedagogical methodologies to create and maintain the transition between these. Initial intention is another prerequisite. reading the Frequently Asked Questions was recommended netiquette for beginners who needed to learn about the history and practises of the group before posting. added on 14/06/1997).Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Collaboration is here seen as the act of shared creation and/or discovery. The observer can contribute by criticising and giving topic-divergent motions. some e-learners do not actively participate but lurk 1 . However.uk/elp_conference07). according to Mercer (1995). the JISC. for example. engage in collaborative learning activities. while the other was likely to be the "reflector" (Blaye et al. create common knowledge. Rather than the facilitator or educator providing rules for learning (Berge & Collins. 1998:495). Within the context of electronic communication..ac. This distinction of passive and active participation appeared in computer-supported tasks. the participant who controls the mouse tended to be "executor". learners took responsibility for their own learning. Wegerif.. lurking is a messaging jargon for activity of one of the "silent majority" in an electronic forum. 1991).d.org. important enough to form a Passive participation and lurking will be used interchangeably in this thesis as a demand for changing the term lurker was observed in the literature as well as online communities and conferences (e. E-mint community. n. Technology & Learning definitions. and follow ground rules that encourage the exchange of relevant ideas and active participation.) Thus shared creativity for new knowledge building is the ultimate collaboration learning target that can also be expanded to online settings. which are not the primary roles of the task-doer. monitoring the situation." Miyake referred to active as well as reflective learning since criticism could support new knowledge production.g. 1 Ph. collaborative learning can take place without members being physically in the same location. Working on shared tasks implies learners’ participation and engagement.

According to Dillenbourg and colleagues (1996).2. it appears that sociability is the basis for collaborative learning supporting both a reflective and active mode. When the group became the unit of analysis. The context of their interaction was seen as a backdrop rather than the focus of research in its own right.1 From the Individual to the Community: a Brief History Dillenbourg and colleagues (1996). Mäkitalo et al. Prerequisites also refer to a joint interactive space for grounding. 2. This idea was further developed in psychology as the principle of the “least collaborative effort” (Clarke & Wilkes-Gibbs. the focus was on comparative processes to establish Ph. During the 70s and early 80s. the focus shifted to the social construction of knowledge. as interactions intended to create common ground (Clark et al.. then it moved to group learning in a more socio-cultural mode. This means that sociability creates the initial conditions for collaborative learning. assumptions or presuppositions (Baker et al. the key to supporting collaboration is to find suitable intermediate variables to describe and support collaborative interactions and their relationships with the conditions that facilitate collaborative learning. then. In terms of empirical research. knowledge. co-creativity as exploration and discovery. In other words. mutual understanding. spontaneous roles and division of labour. said that the development of an understanding of collaborative learning began with the learner as an individual.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review cooperative principle needed before interactions occur (Grice. 1983). it is the e-learners’ shared goals and generated context that make collaborative learning occur. 1986:26). 1994:176). Thesis Centre for Interactive Systems Engineering LSBU Page 18 . and enjoinment of the joint activity are concepts found in collaborative learning. Overall. this was still on the basis of studying individuals.D. as an active concern for the construction of mutual understanding in order to develop a shared focus and a shared narrative (Crook. cooperation and collaboration through dialogue. Therefore grounding is built on sociability. 1999). 1999. beliefs. and finally. Shared goals and activities. 1975). expanded to the community. 2001) or “repairing” misunderstandings (Dillenbourg. research was focused on the individual’s learning processes. however. collaborative e-learning continuity creates the shared narrative that fabricates its history and builds a successful e-learning community..

Socio-cultural learning attached significance to the level of symmetry/asymmetry between the members of a group. more capable peers or other peers. Vygotsky’s work introduced two significant concepts. it was almost impossible to establish causal links between the conditions and the effects of collaboration. Dillenbourg and colleagues indicated the need for new tools and methods for observing and analysing interactions to increase understanding of the collaborative learning social mode. Therefore. and inner dialogue was the medium for self-regulation. 1962.g. the individual could reach a higher level of development with the help of a more capable other. This development of an understanding of learning is briefly presented next. Socio-cultural researchers were able to build on the egocentric thought as the inner dialogue. The research focus was shifted to the causal relationship between social interaction and individual cognitive change stressing the importance of social activity within a group to promote cognitive development. The individual often gets more of a chance to participate actively in critical planning and decision making when interacting with an expert tutor.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review whether and under what circumstances collaborative learning was more effective than learning alone. The second major theoretical foundation that influenced collaborative learning was the socio-cultural approach. Dialogue as social speech was the medium of communication. To Light and Littleton (1999). Because collaborative learning is inherently complex. 1976). Lave & Ph. One of the fundamental concepts that helped collaborative learning to evolve was the socio-cognitive conflict derived from the interaction with other learners as a result of decentralisation. In brief. Egocentrism was the main obstacle to operational thinking and requires its “decentralisation”. 1978). the “zone of proximal development” (zpd) and “scaffolding” (Wood. In Piaget’s early writings (1932). and expand it with the outer dialogue required in socio-cultural contexts. In fact. Tharp and Gallimore (1988) suggested that peer assistance at the same level of asymmetry is required so that peers can help themselves. This is the ability to take into account multiple points of view and multiple covarying factors in a given situation. the potential productivity of peer interaction in relation to cognitive development was related to the achievement of concrete operational modes of thought in the early years.D. Studies on this relationship led to the community knowledge building approach (e. Bruner & Ross. Thesis Centre for Interactive Systems Engineering LSBU Page 19 . it advocated that knowledge acquisition was based on the alignment of asymmetrical interactions between learners and more capable peers (Vygotsky.

CoPs were engaged in the generative processes of producing their own future. support. there were hierarchical levels of engagement depending on many factors. Peck (1987) stated that “If we are going to use the word [community] meaningfully we must restrict it to a group of individuals who have learned how to communicate honestly with each other. Thus. trust. and evolve practices (Wenger." and to "delight in each other. More specifically. define. for example. Thus. committed to the joint creation of meaning. The community focus is next. and change were always mutually present. LPP was a decentred model where a specialist field contained different levels en route for newcomers’ engagement and practice. Learning in communities was configured through the process of becoming a full participant in practice and being able to get involved in new activities. The Ph. the learning process occurred within a larger physical and social context of interactions and culturally constructed tools and meanings. Lave and Wenger (1991) called this learning process Legitimate Peripheral Participation (LLP). whose relationships go deeper than their masks of composure. Thus learning is the process of participating in communities. 2000). 1991). as they become oldtimers. Here." The development of communication channels. Social and individual were not different levels of study but inexorably interconnected. and who have developed some significant commitment to "rejoice together. both external and internal to the participants. Lazlo and Lazlo (1997. and a sense of belonging seemed to be significant to help a community to emerge. must involve conflict between the forces that support processes of learning and those that work against them. It was proposed that this centripetal process of engagement was proposed to be legitimate for all participants. this created two distinct traditions of situated cognition. mourn together. CoP members shared the characteristics of joint enterprise.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Wenger. This was because learning. perform new tasks and functions. One implication of the social reproduction of CoP was that the sustained participation of newcomers. The authors echoed Lave and Wenger in Communities of Practice (CoP) (1991). The peripheral members drifted into the centre as their interests were stirred. one focused on the individual and a second focused on community (Wilson & Myers. make others' conditions our own. There were several community definitions. 1998). transformation. 2000) described the community as a group of two or more individuals with a shared identity and a common purpose. and master new understandings. mutual engagement and shared repertoire to clarify. Thesis Centre for Interactive Systems Engineering LSBU Page 20 .D.

-CoP characteristics: joint enterprise. complexity and responsibility. the key tenets. mutual engagement. the research focus.2. Overall. In fact. Lave and Wenger proposed that LPP was a descriptor of engagement in social practice that entailed learning as an integral constituent. the research methods and results (Table 2.1-1. It appears that the research focus involved the individual. as potential contributors. Research Methods -Inter disciplinary research -Real settings -Ethnographic inputs Results -There are different types of communities. the social as well as their situated context where the individual participates in collaborative practices.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review newcomers were located at the first level. & -tools that enable members’ participation and learning. central and peripheral participation should not be on different levels after all (1991:35).1-1): Table 2. For this reason.D. multidisciplinary methods needed to be evolved to investigate Ph. -Conflicts may allow resolutions. the social and the physical is one research context. -Lurkers are legitimate participants and community members. -The situation needs to be studied as a whole. -Learner’s participation in the community lies in the concept of “becoming part of the community. -The conditions that facilitate participation: -management of the participation process -accessibility to information. The Socially-Shared Cognition Approach: Communities of Practice Communities of Practice (CoP) Research Focus -The individual. -Tools are part of the community’s heritage and cultural life. and were not voiceless or powerless but a vital part of the community. Thesis Centre for Interactive Systems Engineering LSBU Page 21 . shared repertoire -Legitimate Peripheral Participation increases in engagement.2. The next table describes CoP. Research Questions What is learning within a community? Is learning via interactions with other individuals more efficient than learning alone? What are the types of social engagement that facilitate learning? What are the social conditions that facilitate learning? Key Tenets -Learning is situated and grounded in everyday actions. Learning is a process of participating in CoP.

Identity (e. based on the “numeric amount” of posting to the community (number of messages). Thus. lurkers were the participants who did not exhibit any activity. the sense of belonging appeared to be the driving force for collaborative learning enhanced by tools.. 2 Empathy is a “complex psychological inference in which observation. Learning in communities was built on social interactions. The “eyeball of participation” provided a structure to better understand legitimate peripheral participation. the latter was suggested as the basic factor that distinguished a guest from a member in an online community and was initially studied in relation to empathy 2 (e. Donath. In fact. Thesis Centre for Interactive Systems Engineering LSBU Page 22 . memory.2. The participants and key contributors were located on the third and fourth level. 2003) viewed as occurring in separate levels (Figure 2.1-1: The Eyeball of Participation The participants were located in four levels. 2006) and the sense of belonging (Nonnecke.g. McDonald and colleagues stressed the fact that the active members are the ones who “add value” and fill the gaps for all members in order to sustain the community. 1996.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review learning on an individual and social level as well as the tools and the environment of participation. Lee et al. 1997:2) Ph. With a direction from the periphery to the centre. 1999. A project focused on lurking in CoP that accommodated community management.g. explored the meaning of “legitimate peripheral participants” (McDonald.2. On the second level there were the members who occasionally contributed to the community. knowledge and reasoning are combined to yield insights into the thoughts and feelings of others” (Ickes.1-1): Figure 2. Preece. 2000) appeared to be essential concepts in community research.D.

2005b).D. 2001. 2005a.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review 2004. Lambropoulos found that the members who developed empathy became active participants (2005a). Thesis Centre for Interactive Systems Engineering LSBU Page 23 . this was an indirect way to become active contributors. Ph. Lambropoulos. Ramachandran suggested that empathy was a cognitive activity triggered by the mirror neurons (2000) and appeared to be a key in human communication regardless of the medium used. especially when the dialogue engenders “a sense of trust and care”. Nonnecke and Preece (2000:127) found that some lurkers felt a sense of community. Preece & Ghozati.

D. 2000. An online community referred to people who make the community where group dynamics.3 ONLINE & E-LEARNING COMMUNITIES While educators and learners in classroom-based courses have already discovered the benefits of an engaged learning approach to education. On the other hand. 2004). sustainability. e-learning leaders agreed that interaction was the key for their effectiveness (for a review see Conrad & Donaldson. 2000) As for e-learning communities.1953). 2007). Conrad & Donaldson. Nonnecke. and motivations to contribute (e.. Active and passive behaviour is not a new phenomenon. that is people come together for a purpose(s). Hiltz. 2004:ix After Rheingold’s book ‘The Virtual Community’ (1993). of which learning is a corollary. the idea of simulating a community remained popular. and policies that is the behaviour governed by group norms. Researchers had examined a range of phenomena related to online and text-based discussion groups including member contribution patterns. Gulati 2006. software was needed to mediate.g. these explicitly target learning. 2000). being a participant in a community was an essential component of the educational process (Goodfellow 2003): ‘communities of practice’ differ from ‘communities of learners’ in that the latter are reflexively concerned with learning whereas the former are concerned with practice. However.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review 2. and influence or restrict community activity. needs and roles shaped the community. an experiment on active and passive reading was conducted by Janis and King in 1953 (cited in Hovland et al. 3). Butler et al. 1996. Furthermore. support. Thesis Centre for Interactive Systems Engineering LSBU Page 24 . It appears that there is a difference between online CoP and e-learning communities. Wasko & Faraj 2004. Rheingold 1993. rules and sometimes formal policies (Preece. Constant et al. purposes. the power of engagement in online courses is yet to be fully realised. (p. 1985). Research on e-learning communities appeared in the mid 80s (e.g. Barab and Duffy (2000) argued that e-learning environments were practice fields rather than authentic Communities of Practice (CoP) because the activities were not real. Two experimental groups of college students worked as ‘active participants’. who delivered talks in a group Ph. they were educational and not part of the communities’ authentic work. A community is something more than merely an aggregation of users using a collection of communication tools (Typaldos.

but not all the time. with welldeveloped rationales and strategies.. the question is not why it is important to engage the Greek teachers in elearning. calls collaboration and creativity. the Greek teachers remained on the first level of participation. (For a review see Schultz & Beach. Thesis Centre for Interactive Systems Engineering LSBU Page 25 . communities will continue to evolve. The Greek teachers belong to an off-line CoP.D. McDonald. Thus. if the Greek teachers do belong in an offline CoP why is it so important to force them in active participation in collaborative e-learning communities? Participation in CoP involves the use of tools. Nonnecke & Preece. All interviewees lurked. this community is not reflected in an online CoP. but finding ways to enable it. 2004. 2006).1 Passive Participation in Online Communities Some online communities’ members act as invisible observers of synergetic activities. This variance is imperative to be controlled on a pedagogical and operational level. If all passive participants have the potential to become active this requires an effort on their behalf. understanding the technology of practice is more than learning to use the tools. that is passive participation. In the age of ubiquitous computing and the social phenomenon of the Web. however. it is a way to connect with the history of practice and participate more directly in its cultural life (Lave & Wenger. and with the growth of what Berners-Lee (2007). 1953). 1991:100-102). The question that arises is.) Nonnecke and Preece (1999) suggested that ‘lurking is a systematic and idiosyncratic process. The results indicated that all members had the potential to become active participants. 1999. Ph. Gulati. and ‘passive controls’.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review situation. they were located in the grey zone of potential active participation. in other words. Thus.3. This grey zone was characterised by the “sleeper effect” as a change of behaviour after a lapse of time (Hovland et al. 2003.g. 2. For this reason passive participation has been identified as the key variance in this study. Thus. they never seem to cross the threshold of observation and remain in the periphery (e. who read and listened to the source material. artefacts used within a cultural practice carry a substantial portion of that practice's heritage.

in his PhD Thesis (2000). Nonnecke. at a particular time an individual is located within specific external and internal conditions that create reasons that affect his/her online behaviour and performance. Carroll & Rosson (1996) referred to lurkers-to-posters ratios 100:1. Whittaker et al. Fish et al.) Ph. 1995) where interactivity was related to dependency among messages (Rafaeli & Sudweeks. lurking is personal and situated. increased understanding of online lurking by addressing three primary questions: why lurkers lurk. Thesis Centre for Interactive Systems Engineering LSBU Page 26 . Most of the reported reasons fell into the personal sphere. 1999. 1998. Social loafing or free riding (e. Monthienvichienchai. Preece and Nonnecke’s survey response rate was 2.g. not being able to make the software work. Bruckman & Resnick. 1997). Sproull and Faraj (1997) reported an 80% of lurking.. A lurker can generate the sense of belonging but is it enough? Posting and interactivity have come under research scrutiny early in online research (e. (For example. for example counting the views (e.g. Preece and colleagues reported the top five reasons for lurking (2004): not needing to post.D. 2004). needing to find out more about the group before participating. thinking that they were being helpful by not posting. 2003). Wellman & Gulia. This is a limitation on lurkers’ research as free riders might never have responded. 2000) or using proprietary tools that log lurkers’ communicative behaviour (e. et al. what lurkers do. working with a Greek teachers’ online community. Rafaeli. 2005) proposed to be a robust phenomenon that occurred when people work less to achieve some goal when they thought they were working jointly with others.g. The primary reason for passive participation was uncertainty about the community’s goals. Nonnecke & Preece. Figures related to interactivity and passive participation are different. Preece from 46% to 82% (2000) and Lambropoulos. Kollock & Smith.g. Lurkers in one community can actively participate in other communities where they believe they have something important to say (Klemm.. and not liking the group dynamics or the community was a poor fit for them.g. Soroka. depending on the dynamics of the media.” In other words.3%. Passive participation can be tracked approximately using different methods provided that the software supports it. 99% (2002). 1996.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review and several developed a sense of community through their lurking... 1986. and how many lurkers there were at the time of the investigation. 1990. and result in specific lurking behaviours (e. 1998). Ling et al.

Although passive participation is legitimate. responding to specific posts.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Although passive participation is common and normal there is a contribution paradox: even though passive participants may feel a sense of belonging. browsing (passively participating in community’s life). These e-learning managerial Ph. and research methodologies (for a review see Rafaeli et al. 2005). A questionnaire sent via email to 23 "low visibility or "no visibility" learners (p.D.2 Passive Participation in e-Learning Communities Studies of passive participation initially targeted online communities. and 25% said that they were not comfortable in presenting their ideas online. Beaudoin (2002) found that almost 23 out of 55 (42%) of online education master’s degree students. Thesis Centre for Interactive Systems Engineering LSBU Page 27 . 30% did not know what to say as the discussion drifted away from the topic. 2006). In particular. the community. 2. In comparison to online communities. 2000). 2002). it is necessary for the life of the community. or underlying assumptions of e-learning pedagogy (Gulati.. and interacting (responding in a reciprocal manner) (Rafaeli et al. Other observed behaviours were: connecting (visiting the community). writing assignments. the tools for communication. and spent least time on writing comments for online discussions. the situation seemed to be more complex in e-learning if passive participants were beneficiaries of other people’s discussions who did not share their own ideas (Salmon. showed that e-learners were: reading assignments and others’ comments. it appears that the major reason for being a lurker is a lack of knowledge of how to take part effectively. 2004).3. 2004). Beaudoin found that 40% (n=23) were not sure how to articulate their ideas. and their participation is not a prerequisite. the relationship between writing and reading behaviour (Ebner & Holzinger. conducted web searches. 2004). if activity influences learning efficiency (Beaudoin. then e-learning communities. 150).. preferred to read other learners’ messages. 30% did not understand the topic. contributing opinions. attending time durations. Reasons for lurking were related to the individual. Passive participation was observed for discussion forums or blogs (Williams & Jacobs.

g.2-1): Ph. The role of the learner was more concerned with what an individual constructed as an engaging process. the problem lies in their inability to display lurkers’ learning in order to justify their existence. Klemm (1998) advocated that there were psychological and social reasons with roots in the passive conditioning and the “entertain me” mode of mass media.3. Klemm. Furthermore. To Khine and colleagues (2003) there is trainee teachers’ inability to actively participate in online discussions. the lurkers. lack of investigation and questioning that killed conversation in Greek homes (p. The reasons for passive participation are summarised next (Table 2. believed that lurking restricts creative thinking and teachers foster this behaviour with their practices. 2003). She opposed the controlled and structured type of e-learning that considered lurking as dysfunctional and problematic. she suggested that compulsory contributions proved inadequate in increasing collaborative learning since it was connected to Foucault’s (1984) discourse of normalisation through policing.g. She said that the e-tutors do not have a problem with lurking. Williams.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review reasons may be related to reasons inherent in the teaching and learning modes (e. unmeasured learning. E-learners have been found reluctant to criticise each other (Hughes & Daykin. redefining participation in learning allowed an open understanding of adult learning. Thesis Centre for Interactive Systems Engineering LSBU Page 28 . She concluded that there was a need to open the concept of learning and include silent. 2002:222) even if they are “forced” to be active (e. 2002. In other words. working on an online conferences context. Earley & Gibson 2002). Klemm. so ‘visible’ and ‘silent’ roles were personal constructs. 1998. 157-158). the participants were not critical thinkers and failed to sustain interaction. In her PhD research. Oliver & Shaw. and were enacted in order to understand the world and showed how individuals constructed their own perspectives. compulsory contributions tried to police and repress the outlaws. in fact.D. Buhayer (2005) referred to the Greek media as transmitting a model of a reluctance to follow the stories through. In particular. Gulati (2006) suggests that passive participation is an informal mode of learning and an essential part of formal education. Greeks are not different. Both learners and teachers had been exposed to television and traditional classroom teaching that deprived them of critical thinking.

Social reasons were identified as differences in goals. information overload. processes and engagement approaches -access to best practices -organizational obstacles -inconvenient procedures -security considerations -community-knowledge building as a public good -integration of novices into work environments -unite geographically dispersed work units Software limitations -inadequate learning platforms Lack of awareness about informal learning.D. -require a hand-in assignment as a deliverable derived from the discussion.3. such as being shy. -structure the activity. procedures or security problems. -form learning teams.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Table 2. Thesis Centre for Interactive Systems Engineering LSBU Page 29 . -do not settle for just opinions. organizational obstacles. access to best practices and organisational obstacles. Klemm (1998) Ph. researchers developed strategies for active participation (Table 2. This means that appropriate community management can tackle these problems.2-2. processes. Reasons for lurking in e-learning communities REASONS FOR LURKING IN E-LEARNING COMMUNITIES Personal . however. -deep or surface views -different perceptions of what was required -familiarization with the community -learn about a new topic -being shy -information overload -lack of recognition informal learning -differences on goals -differences in processes -different perceptions of what is required from learners Social -differences in learners’ goals.2-2): Table 2. This implies that e-learners were unaware of tools and techniques. engagements approaches.lack of awareness about informal learning.3. In addition.2-1. Active Participation Strategies STRATEGIES TO ACTIVATE PARTICIPATION Authors Active Participation Strategies -require participation.3. -make an activity interesting. inconvenient procedures and security considerations increased passive participation. different levels of understanding and differences in e-learning targets were the main reasons for passive participation. there is nothing that can be done for the organisation obstacles. Based on the reasons for lurking. Rafaeli and colleagues (2004) suggested that personal characteristics.

-valued and dynamic discussion -familiarity with the community -develop a sense of community Strategies such as building e-learning teams were based on specific methodologies to develop a sense of community in a trusted common space for information sharing. Ph. al. The next section will investigate whether current design supports them. -importance attached to the role of the e-tutor. The literature referred to what reduces active participation. Overall. Such tools and techniques need to support investigation for observation. For example. designers and engineers and the organisation hold responsibility for planning and supporting e-learning activities. E-learners. it appears that the key to participation is the shape of collaborative e-learning design supported by suitable tools and methodologies. -employees are trusted. (2004) -e-tutors need to know what they are looking for and involve themselves to help make it happen -peer grading -set of institutional norms promoting trust. -promoting knowledge sharing as a norm of the organization. (2003) Swan et. e-tutors. Since personal and social reasons refer to community and learning management they can be tackled by aligning new tools and evaluation techniques with pedagogical modifications. (2003) Rafaeli et. Thesis Centre for Interactive Systems Engineering LSBU Page 30 .Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Archichvili et al.. designers may provide transparent and usable interfaces. and engineers contribute technical support.D. the e-tutor may structure collaborative activities while the organisation can incorporate standards and rewards for sharing knowledge. -building multiple face-to-face CoP -clear norms -standards for sharing knowledge.. description and analysis of what happens in collaborative e-learning communities. Overall. this section discussed the socio-cultural and pedagogical foundations of participation in collaborative e-learning communities. al. -sharing is a moral obligation. Strategies also referred to community management and members’ roles. -obtain active participation -have transparent interface.

for example. However. the Greek Schools Network (GSN) consist of developers. 2005). 2005).4.D. But to Suchman.1 Socio-Technical & User-Centred Design Socio-technical design (STD) refers to design that is influenced by an organisations’ social structure. will enable users to concentrate on their tasks. design developed strategies and tools insensitive to particular circumstances (1987:121) because plans actually derive after the completion of the course. this is especially so for younger users who might do homework whilst listening to Mp3s and chatting with friends (Dede. this means integrating several levels of functionality which is geared towards the e-tutors rather than considering the e-learning participants. In 1949. interface design might become pivotal and make the difference between a system that is used and one that isn’t. 2. Many design definitions exist. Design may influence participation quality. besides the Greek teachers and e-tutors. Smulders. researchers from the Tavistock Institute for Sociology in London formulated the most important principle for joint Ph. Users in educational settings act as both users and learners and design has to support their dual persona (Smulders. Learning how to be a user is about understanding the technology rather than learning to use the tools (Lave & Wenger. Such a post-hoc design structure challenged the foundation of computational design as a linear process of development. 1991:100-102). one with an easy to use interface. In order to support multi tasking the question of the interface becomes important since a usable system. It appears that the design of new systems is always problematic. 2003). For example. today’s users are typically multi tasking.4 DESIGN FOR LEARNERS AS USERS & USERS AS LEARNERS Online Education is in its infancy and has yet to construct design models that will address the problem of the learner as a user (Wallace. 1999. Additionally. engineers and decision makers. design underpins every form of creation from objects such as chairs to the way we plan and execute our lives (Dini. technologists tend to build techno-centric systems for use by academics. Thesis Centre for Interactive Systems Engineering LSBU Page 31 . Thus. 2003). There is an assumption that the educational systems are easy to use by the educators and learners.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review 2.

Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review

optimisation of social and technical systems: if a technical system is created at the expense of a social system, that is the organisation’s social structure, the results obtained will be sub-optimal (cited in Mumford, 1983). According to Mumford (1983), design should consider the social context and involve users (socio-technical design). Faulkner (2000) suggested that systems should be user-oriented in all aspects of their functionality by adjusting the systems to the users and their natural environments (user-centred design).

Socio-technical design (STD) develops systems for collaborative working environments. STD offered solutions to practical problems by fitting design to its context. STD took the idea of designing for the user and the task a stage further and endeavoured to design within the structure of the organisation and the way in which it operates (Faulkner, 1998:134). Therefore acquisition of the ‘native’s point of view’ was important and was provided by ethnography. Even though STD provided schemes to fit design to its context, most designers were not motivated by incorporating social structures into design (Anderson, 1997). In fact, designers believed that it was user’s inability to cope with the system that was the problem (Mumford, 1983). Nowadays designers acknowledge that the development of interactive technologies relies on an appreciation of social circumstances in which systems are used (e.g. Dourish, 2006). So, STD offered key concepts for systems design such as participation in design, participation in decision making, and the need for evaluation methodologies to support them.

STD considered the organisation as a whole. User-Centred Design (UCD) provided a clear focus on users’ needs. These had to dominate the interface design, and the interface had to dominate the design of the system. This was achieved by a dialogue between the stakeholders and the developers and by their involvement in the early planning stages. This is the purpose of UCD and is what Shakel (1991) described as having the: “…capability in human functional terms to be used easily and effectively by the specified range of users, given specified training and user support, to fulfil the specified range of tasks, with the specified range of environmental scenarios”. He suggested that systems need to be used easily and effectively to support specific users; training and support on the use of the system was central. UCD guidelines were needed to ensure product quality even in the planning stages. Gould and Lewis (1985) proposed four principles for useful and easy ways to create usable computer systems: (i) early focus on users and tasks, (ii) empirical measurement for evaluation, (iii)

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 32

Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review

iterative design, and (iv) integrated design. When systems did not operate as expected then user feedback should be sought and used in the redesign as systems should conform to expectations. User-centred studies provided more coherent frameworks and models for analysis of what is going on and what should be done to ensure users’ satisfaction. Shackel (1991) suggested that usability could be measured by examining learnability, effectiveness, attitude and flexibility; Nielsen (1993) thought it was efficiency, learnability, memorability, errors and whether the system is subjectively pleasing. The International Organisation for Standardisation (ISO 9241-11, 1998) defined usability as a measure of the quality of user’s experience when interacting with a system, in terms of effectiveness, efficiency, and satisfaction. Measurements and assessment procedures had to ensure the product met the purpose of design.

From an educational viewpoint, technology is to enable students to reach their potential rather than technology being the goal itself. Educational designers needed to consider not only the educational system as a whole but also the learners’ dual persona as learners and as users, and facilitate learning without any additional cognitive and physical struggles to use the system. Users of a learning system needed to be free to learn the subject and not have to spend time on learning about the system. In other words, the system must be usable (Faulkner, 2000). Laurillard, in her interview with Neal for the E-learn Magazine (2003) stressed this fact; she also said that there were critical issues for technology enhanced learning and its future. More specifically, there was a need to: adopt pedagogical perspectives; focus on user interface; build on learning activities design; assess performance; and evaluate in the form of checking whether the learning objectives have been met.

These recommendations have translated the user-centred design principles into learner-centred design principles.

2.4.1.1

Towards a Learner-Centred Design

Educational design was built on the integration of the pedagogical and technological levels, and was called Learner-Centred Design (LCD) or Instructional Design. Whereas User-Centred Design focused on making users more effective, LCD focused on making learners more effective. For Norman and Spohrer (1996) LCD had three dimensions: (a) engagement as the result of

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 33

Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review

motivation, (b) effectiveness, and (c) viability of interventions. Engagement was tightly associated with motivation, the provision of rapid compelling interaction and feedback. Feedback was used to achieve effectiveness by measurement and benchmarking. With a social perspective in mind, Squires and Preece (1999) provided the first set of ‘learning with software’ heuristics echoing socio-technical, user-centred, and learner-centred design.

‘Learning with software’ heuristics opened the way to learner-centred usability, or Pedagogical Usability (PU) (e.g. Muir et al., 2003; Nokelainen, 2006). PU derived as part of utility as software has high quality if users can perform their tasks (the software is useful to them, Nielsen 1993). With utility in mind, Finnish researchers (Silius et al, 2003; Nokelainen, 2006) proposed that PU should question whether the tools, contents, interfaces, and tasks provided within the elearning environments that supported e-learners. For Silius and colleagues, PU was tested with a questionnaire linked to the educational website. The questionnaire targeted the suggested tools’ basic use and utility triangulated by logs and data from the discussions anchored in pedagogical usability (Silius et al., 2003a, 2003b). Muir and colleagues (2003) also worked on an e-learning PU pyramid for educational effectiveness and practical efficiency of a course-related website. They stressed the e-learning participants’ involvement in design, evaluation, and decision making. (For a review of pedagogical usability attributes see Nokelainen, 2006.)

Overall, educational systems and their integrated tools constitute the means by which learners are permitted or restricted in the use of their learning capabilities. Instructional design and engineering are design processes to ensure their educational fulfilment.

2.4.2

Instructional Design & Engineering

Instructional Design (ID) is the systematic process of activities to solve an instructional problem with the aid of technologies (Fenrich, in press). There were different approaches in ID models (e.g. Molenda, 1987; Ryder, n.d.; Fenrich, in press). For most instructional designers, the process fell into a framework called ADDIE , that is analyze, design, develop, implement, and evaluate (Bichelmeyer et al., 2005). Fenrich proposed a coherent ID model anchored in Bloom’s taxonomy (1956) with practical guidelines targeted at instructional multimedia

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 34

Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review

solutions. ID begins with initialization and project planning (how the instructional design is carried out), the design and development phase (appropriate strategies and approaches in targeted contexts), and a quality assurance phase focused on deployment, evaluation, and assessment. It also includes easy-to-use checklists and presentation tips offering a comprehensive insight in ID and deployment outside the control room. This interdisciplinary approach involved all stakeholders in design by covering all stakeholders’ benefits defined under specific criteria: roles, skills, characteristics, activities, commitments, and responsibilities. As with the previous design approaches, identification of intents and planning based on stakeholders’ goals, skills, background and characteristics provided the design backbone. Fenrich said that his model works in ideal situations; in reality instructional designers need to adjust the plan to given situations by integrating evaluation results into system requirements and re-design specifications. This means that Fenrich implicitly agreed with Suchman on the post-hoc nature of design.

Instructional design and engineering are interrelated processes to support planning, analysis, design, and delivery of a learning system. Instructional Engineering (IE) integrated the concepts, the processes and the principles of ID, software engineering and cognitive engineering (Paquette, 2004) and explicitly referred to e-learning (Shepherd, 2001; Myrach & Knolmayer, 2005). Paquette (2004) adopted Human-Computer Interaction (HCI) approaches directly into his IE framework. His model described e-learning engineering in relation to: engineering and re-engineering e-learning systems (macro design); producing instructional materials (micro design); delivering training on networks; and reviewing and maintaining e-learning. Paquette provided a complete knowledge presentation system on platforms and portals and described all functions and roles for the stakeholders called actors: the teacher-designer, the learner, the facilitator, the manager, and the platform administrator. His MISA method was an automated system that supported 34 main tasks and about 150 secondary tasks. It has been developed continuously and tested successfully in different contexts for several years.

However, it was suggested that systematic ID models had been accused of not reflecting actual practice and not supporting new competencies for the 21st century and post-industrial societies (Table 2.4.2-1):

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 35

Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review

Table 2.4.2-1. Instructional Design problems INSTRUCTIONAL DESIGN PROBLEMS Authors Visscher-Voerman et al., 1999; van den Akker et al., 2004 Richey et al., 2000; Waters & Gibbons, 2004 Gordon & Zemke, 2000 Gentry, 1994; Berger & Kam, 1996 Seels & Glasgow, 1998; Shambaugh & Magliaro, 2005 Tripp & Bichelmeyer, 1991; Fenrich, in press Schwier, Campbell & Kenny, 2006 Description The ID process itself ID competencies Being cumbersome, ineffective, inefficient The nature of the task Methods to inform practice Costly to implement Not taking of advantages of new technologies

It appears that the major problem is related to the ID process itself with regard to its implementation and flexibility. Some other problems related to e-learning quality and recorded in the Panorama report (Ehlers, et al., 2005) were caused by: e-learning participants’ unfamiliarity with the design process; division between ‘academic’ and ‘corporate’ approaches; lack of awareness of the need for quality standards; the dual identity of the learner as a user was neglected; stakeholders’ engagement in the early stages of design was neglected; and the fact that design in artificial settings makes it vulnerable to the Hawthorne effect, so little evidence existed as to how to use technology effectively.

These problems were relevant to the situated nature of the learning environment as they have a “conclusion-oriented” nature instead of a “decision-oriented” one. In addition, social interactions were partially neglected; with the Web 2.0 signpost, instructional designers and engineers foresee the need of an e-learning 2.0 stage (Downes, 2006; Karrer, 2006; Jennings, 2005). At this point in time, educational design needs to consider the social and collaborative structure of elearning communities for learners’ content creation. The question that arises is not why ID and IE suffer from so many problems, but to what extent designers need a more flexible or a more systematic model. Depending on the organisational target, a managemental educational change suggests a more open model, whereas design for particular systems requires a more coherent model. According to Leigh (1998), instructional designers need to focus either on one aspect of learning and instruction and act as consultants or matter experts. Since the field is becoming too broad for most designers to work with authority,

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 36

Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review

these options give freedom to instructional designers. To answer this question, following Winn (1986:20), “we must have at our disposal a whole battery of methods to deal with the different types of things we need to find out”.

Overall, it appears that systems design can be the catalyst to overcome variances or even crises. This implies that if design is relevant and suitable to situation it can fulfil its purpose; then there is high probability not only of overcoming the variance but reaching a high level of quality. The rest of the literature review explores whether socio-cultural learning principles have been incorporated in tools to support collaborative e-learning communities.

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 37

Ph.. The question that arises is whether collaborative e-learning tools can support the socio-dynamics of an e-learning community.D. email. messaging or interactive. Thesis Centre for Interactive Systems Engineering LSBU Page 38 . 1998).d. n. personal communication via email. visual. these are public or private. Dillenbourg (2000) stressed the social aspect of e-learning as a designed information space where learners are actors i. audio. As for groupware. they co-construct the information space.5 TOOLS FOR COLLABORATIVE E-LEARNING COMMUNITIES For active learning. and the privacy levels.. e-learning applications need to consider social interactions between e-learning participants. such tools can give explicit control of the process and support the type of interactions that were expected to promote it. The computer-based collaborative learning tools are divided into conversation and collaboration tools. see Conrad & Donaldson. asynchronous activities or both. and tools for analysing interactions (Dillenbourg et al.e. 2004. as in synchronous.. Lack of such tools results in lack of understanding collaborative e-learning and vice versa. there are different types of elearning tools categorised under several criteria: temporal. but also sociological: the challenges of social interaction and social organization. the direction of activity as in broadcast. So far. Kommers et al. editor of the British Journal of Education Technology. web-based as offline or online. 06/12/2006). This is evident in current research by a lack of interesting papers (Nick Rushby. what supports collaborative learning is the visibility of the collaborative learning structure. design can impact collaborative learning on the kinds of social interactions that aid learning since aspects of software can modify the socio-dynamics between the learning partners. Jonassen. 1992) Jonassen said there was a chasm to consider between tools that support information provision and groupware. point and click is not enough (Klemm. (For reviews. Thus. For example. Kollock (1998) emphasised the need for such social computing: ‘The key challenges the Internet community will face in the future are not simply technological. tools for observing interactions. simulation and their combinations.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review 2. Designing e-learning tools to influence collaborative e-learning has methodological advantages. 1996). In other words. the medium of communication as in text.’.

The social aspects of learning was scrutinised in Computer-Supported Collaborative Learning (CSCL). 1995). Nonetheless. 1973). The following graph from Google Trends depicts the separate routes of the two terms in the literature (Figure 2. Thesis Centre for Interactive Systems Engineering LSBU Page 39 .5-1. Google trend history for online community and e-learning The separate routes meet just before 2007 suggesting a trend within e-learning communities. (For a detailed view. 2003) or a Time-shared. see Moodle http://docs. Ph. the social aspects of collaborative e-learning were apparent from the early years of the educational technology research. There were early systems in the 70s such as the Programmed Logic for Automated Teaching Operations (PLATO. Van Meer. Figure 2.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review The quest for tools in the e-learning history goes back to the 60’s.D. a room for social collaboration outside home and work (Bruckman & Resnick. They were also excluded from the history of online communities by Ambrozek and colleagues (2004).org/en/Online_Learning_History.5-2) (Tuss.wikipedia. 2001.moodle.5-1): Figure 2. and wikipedia http://en. ComputerControlled Information Television (TICCIT) (Bunderson.) Systems to support e-learning communities were not included in associated reviews because they were considered as a “third place”. It is interesting to observe that the word “collaborative” exists in early designs. Interactive. and had a reference of 33 words in Preece and colleagues (2000).org/wiki/History_of_virtual_learning_environments.

D. Rumble (2001) proposed four models of teaching and learning: the transmission.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Figure 2. related to this research. Today. and the metacognitive model. Moodle developers have synthesized Rumble’s first three concepts in the “social constructionist pedagogy”: The design and development of Moodle is guided by a particular philosophy of learning.5-2. instant messaging.com/?q=node/12 . In his review of e-learning software developments over the last 30 years. Moodle. Following Rumble’s review. the constructivist. remote screen sharing.org/en/Philosophy Last access 20/12/2006 Figure retrieved from the blog http://siliconuser. learning management. current LMS can be located only on the first three levels. http://docs.moodle. Thesis Centre for Interactive Systems Engineering LSBU Page 40 . chat rooms. the socio-cultural. Moodle. a way of thinking that you may see referred to in shorthand as a "social constructionist pedagogy". was based on socio-constructivist theories (http://docs. email. For example. and online games.5-2) pioneered concepts such as online forums. In other words.Posted June 8th.moodle. message boards. higher order thinking is not supported by current LMS.) 3 Ph. PLATO III 3 PLATO (Figure 2. 2007 by Joshua Coventry (Permission to use the image in the Thesis was acquired via email on 06/12/2007. online testing.org/en/Philosophy).

MediaMoo was excluded from the elearning history because it was a shared environment and set of activities for people with the same research interests. said: To create an effective learning community. compared to the pace of emerging technologies of social networks and user-generated context. Kim. you need to develop a social and technical infrastructure that supports the key activity that's happening there . 2001 To Kim. Ph. MediaMoo became publically available in Jan 1993. Kim. in an interview for Elearningpost (February 27. and community management. 2. MediaMoo community management was heavily based on the members’ conversation as the primary activity. MediaMoo targets were explicitly referred to all Rumble’s stages. 1995). the constructivist. Thesis Centre for Interactive Systems Engineering LSBU Page 41 . success is related to both the social and textual infrastructure for situated activity and learner-created context.which can be quite different for various online learning communities.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Despite the fact that the Moodle developers built on sound pedagogical approaches.5. and the metacognitive model and thus. It also seems that more than ten years after MediaMoo. ensuring the quality of participation in a community by design. the socio-cultural. a design to support socio-cultural e-learning is still an issue. it appears that current learning management systems have not evolved significantly. Moodle does not explicitly support collaborative e-learning communities. the transmission. e-learning. When the Online Community Met E-Learning The first e-learning system that supported e-learning communities was MediaMoo (Bruckman & Resnick. MediaMoo developers saw three aspects of an e-learning community: the platform.1. Overall.D. 2001) about the design that ensures a successful e-learning community.

Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review 2.1-1.1.5. this monitoring is undertaken by the community members themselves • A graduated system of sanctions is used • Community members have access to low-cost conflict resolution mechanisms Principles for making virtual communities work: • Use software that promotes good discussion • Don't impose a length limitation on postings • Front-load your system with talkative. diverse people • Let the users resolve their own disputes • Provide institutional memory • Promote continuity • Be host to a particular interest group • Provide places for children • Confront the users with a crisis • Integrate the online environment with the "real" world Axelrod (1984) Ostrom (1990) Godwin (1994) Ph. Design principles for online and e-learning communities DESIGN PRINCIPLES FOR ONLINE & E-LEARNING COMMUNITIES Author Description Requirements for the possibility of cooperation: • Arrange that individuals will meet each other again • They must be able to recognize each other • They must have information about how the other has behaved until now Design principles of successful communities: • Group boundaries are clearly defined • Rules governing the use of collective goods are well matched to local needs and conditions • Most individuals affected by these rules can participate in modifying the rules • The right of community members to devise their own rules is respected by external authorities • A system for monitoring members' behaviour exists.1-1): Table 2.1.5. Thesis Centre for Interactive Systems Engineering LSBU Page 42 .1.D.1. Design Principles for E-Learning Communities Researchers have proposed design principles for online and e-learning communities since the early 80s (Table 2.5.

and socio-technical design were suggested by the writer (Lambropoulos.D. • Providing tools to support governance. 2006). Thesis Centre for Interactive Systems Engineering LSBU Page 43 . Gray and Koch (n. Design factors for collaborative e-learning communities appear to be complex and interrelated.) suggested that the use of familiar settings.d. • Information design riles and • Communications software effective • Finding people & Information communication) • Ensuring readability of • Policies instructions (registration. community management. Other more informal principles to support teachers’ e-learning stressed the need for a socio-cultural basis and came from the teachers’ community TappedIn®. support of social facilitation. and allowing flexible grouping supports e-learning communities. Ph. moderators and other roletrust and players security) Design Principles for e-Learning Communities: Pedagogical Level Operational Level • Intention • Information • Interactivity • Real-time evaluation • Visibility • Control • Support Preece (2000) Lambropoulos (2006) It appears that most principles for online and e-learning communities are related to community management and targeted to promote group coherence and resolve differences as well as supporting members’ individuality. The e-learning community guidelines for synthesizing pedagogy.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Kim (1998) Principles for creating a community: • Define the purpose of the community • Create distinct gathering places • Create member profiles that evolve over time • Promote effective leadership • Define a clear-yet-flexible code of conduct • Organize and promote cyclic events • Provide a range of roles that couple power with responsibility • Facilitate member-created subgroups Guidelines for Online Communities: Sociability Web Usability • Purpose (clear • Navigation definition) • Access • People (access.

and visualisation. SpeakEsay (Hoadley. Therefore. However. Their IRE model was criticised for restricting creativity because of its limited three Ph.D. E-learning communities’ context derives from monological and dialogical sequences so current tools that are directed to simply facilitate these processes are mostly characterised as information-centric media. 2006) criticized such tools as computer-supported intentional learning (Scardamalia & Bereiter. tools need to fill out the “middle spaces” of the continuum. Siegel et al. information provision and groupware. The challenge was to build tools that could help the learner to bridge the “middle space”. This is because contextual information and socially relevant representations were rarely used outside the lab (Hoadley. They often deal with issues of search and retrieval. 1998).Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Kim (2001) suggested that there is no single answer for developing principles for e-learning communities.. 1999. 1999). there was a need to make this structure visible. Thesis Centre for Interactive Systems Engineering LSBU Page 44 . Because monologue derived from dialogue as reflective thought. E-Learning Tools & Interface Design Two types of interfaces were found in the literature. topical and discursive coherence. They proposed the initiate-respond-evaluate (IRE) scheme used in computer-supported collaborative learning argumentation and has reported both positive and negative results (e. that is the transitions for inclusion and centralisation. Information provision interfaces dominate the field and tend to be more data-centric and context-independent.5. 2001. 1995). Shneiderman and Maes (1997) suggested that learning management systems needed to overlap both types. one solution may be that the principles should be open and agile enough so to be incorporated in any situated e-learning interface design. There are no indications that e-learning interfaces have been developed to support both types to a great extend. Hoadley and Enyedy (1999.1. 2. 1991). and the trade off between convergence and divergence (such as reaching a consensus). information presentation.g. Thus. Faulkner. 2000). His & Berman. and SenseMaker (Bell. Mercer and Wegerif. They said that a chasm between internalisation and externalisation in monological and dialogical sequences was reflected in the interface.2. reflections and interactions. visible dialogic interactions could allow the interlocutors to give each other feedback leading to a gradual refinement of partial meanings and construction of increasingly sophisticated approximations of concepts.

. 2000). and then evaluated. 2.d. (For a review on a state of art technology for supporting collaborative learning see Jermann et al.5. 2001. Thesis Centre for Interactive Systems Engineering LSBU Page 45 .5..).2-1): Collaborative Learning Tools (Dillenbourg et al. This was discussed by Scardamalia and colleagues (1996) and Jonassen (n. Jonassen’s constructivist learning environments include tools like listservs. Scardamalia and colleagues worked on knowledge databases so learners’ knowledge could be objectified. social/cognitive. divided in three categories. 1996) Observing Interactions Analysing Interactions social/cognitiv e cognitive/metacognitiv e task/communicativ e Figure 2.5. revised. cognitive/metacognitive. and MOOs (MUDs Objected Oriented. Tools to Support E-Learning Communities It appears that there are two kinds of tools to support human-human and human-computer interactions: tools for observing and tools for analysing interactions. electronic mail. MUDs (multi-user dimensions). the problem appears to be related to a creative interplay between collaborative learning theory and its translation into tools to support collaborative leaning in a social and contextual basis.D.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review levels scheme that did not allow critical thinking (Nassaji & Wells.2-1: Categorisation of collaborative learning tools Ph. examined for gaps and inadequacies. like MediaMoo). bulletin boards.) By studying such initial attempts. and reformulated (1996:201). represented in an overt form. Active and constructive participation in learning proved to be essential but did not happen naturally.2. added to. and task/communicative (Figure 2.

The following section presents existing tools that fit with the collaborative e-learning dialogue scheme. Jonassen (n. Baker (2000) suggested that what was required for tools to structure dialogical sequences was relations between theory. it formed a basis for the design of computational models.2.1-1: Learning and Instructional Activities Learning Activities Exploration Articulation Reflection Instructional Activities Modelling Coaching Scaffolding Jonassen said that each attribute should represent an instructional activity. model and corpus (i.5. The latter is referred to as dialogue management and conversation models drawing from linguistics (for example.5. 1999) triggered efforts on studies to support collaborative learning dialogical sequences by predicting the forms of desirable dialogue as well as interactivity mechanisms. cumulative. Davis & Linn. Thesis Centre for Interactive Systems Engineering LSBU Page 46 . transcriptions of interactions Ph. As for analysing interactions.e.1-1): Table 2. and exploratory talk. Thus. speech types. sentence openers and floor control (I agree. There were different tools supporting collaborative learning discourse such as the collaborative notebook (O’Neil & Gomez. 1994) or the knowledge integration environment (Bell.2. 1995) based on utterances as particular kinds of speech acts (initiate. 1994). managing turn taking or making relevant contributions). 2.2. Another approach that will be explored in detail in the next chapter was introduced by Mercer (1995) and Wegerif and colleagues (1998). continue.1. the authors worked on a collaborative learning model for disputational.D.) introduced instructional activities for an interplay between learning and instruction (Table 2. repair and acknowledge).Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Observing social interactions were the first step built on a learner-generated dialogical context for showing presence and co-presence to make the user visible. argumentation. Could you explain etc) (Traum. Tools to Structure Dialogical Sequences The initiate-respond-evaluate (IRE) scheme (Hoadley & Enyedy. for example exploring and verifying solutions as well as the discrimination between task and communication. tools could capture the collaborative learning cognitive and metacognitive distinction.d.5.

5.interloc.org/about.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review data). the message openers promoted coherent dialogue. 2006). thinking and deep learning which aids in reusable and adaptable learning activity and dialogue game templates. InterLock Interface Message openers remained one of InterLock key features (http://www.2.2.ac. According to its developers. last accessed 21/12/2006). and then design legitimate objects. so the structure of the conversation restricted visibility of dialogical sequences overview.1-1. however. and (e) Java programming cannot be easily integrated in wide use learning management systems. Ravenscroft.uk/ltri/academictalk/).1-1): Figure 2. participated in the new project (e.g. Thesis Centre for Interactive Systems Engineering LSBU Page 47 .5. InterLock. Ph. learners had to think in frames because they were obliged to use the tool and this interrupted the discussion resulting in lack of context depth in reasoning.D. InterLock was the by-product of McAlister’s PhD project Academic Talk (2004) (http://learning. Wegerif.interloc. However. InterLock viewed collaborative e-learning argumentation as a game that promoted engagement and learning (Figure 2.north. and Mercer and Wegerif. As with AcademicTalk. InterLock: (a) was too complicated for most users. Two examples of such models translated into collaborative e-learning tools were InterLock and Messageforum. (b) restricted creativity.org/.londonmet. this was not intended by the developers. Mercer & Hartley. worked on the same attributes to support collaborative e-learning argumentation (http://www. (d) there is no indication that it promoted group-knowledge building. (c) the attributes were distributed on the interface.htm).

ARG = "argument".5.2. MessageForum was built by Jeong (2005) about the same time as InterLock (2006) to support computer-supported collaborative argumentation. elaboration of an argument or challenge.2. a tool to support collaborative e-learning argumentation based on Toulmin (1958). Identifies a reply/message that provides proof or evidence to establish the validity of an argument or challenge. MessageForum Collaborative Learning Attributes MessageForum Collaborative Learning Attributes Symbol + Description of symbol Identifies a message posted by a student assigned to the team supporting the given claim/statement Identifies a message posted by a student assigned to the team opposing the given claim/statement Identifies a message that presents one and only one argument or reason for using or not using chats (instead of threaded discussion forums). Identifies a reply/message that questions or challenges the merits. Number each posted argument by counting the number of arguments already presented by your team.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review The next example had a more flexible nature and structured collaborative elearning in discussion forums.1-2 describes the collaborative e-learning attributes in detail using symbols and attributes rather than message openers: Table 2. ARG# EXPL BUT EVID The translation of the above model in a tool is depicted next (Figure 2.2.1-2): Ph. Identifies a reply/message that provides additional support. Thesis Centre for Interactive Systems Engineering LSBU Page 48 . logic. accuracy or plausibility of a presented argument (ARG) or challenge (BUT). relevancy.1-2. Table 2. This discussion analysis tool created reports in Excel including reports on social interactions built on social network analysis. clarification. validity. Subarguments need not be numbered. that is visualisation of e-learners’ interactions.D. To solve these problems he created the ForumManager. The ForumManager was an MS Excel application for downloading and analyzing messages (and message texts) in Blackboard threaded discussion forums using Internet Explorer browser.5. Jeong suggested that online dialogical argumentation lacked depth and was redundant. explanation.5.

Jeong found that the visibility of the structure helped learners’ reflection. Thesis Centre for Interactive Systems Engineering LSBU Page 49 . 2004).2. Future Learning Environment 3 (FLE3) (Leinonen et al. more replies were elaborated on previous ideas. NegotiationTool (Beers. the potential group performance problems associated with personality traits. The findings about the effects of using message constraints and message labelling were similar to studies on other projects such as ShadowPDForum (Jonassen & Ramirez 2005). and lastly. One of the consequences of not Ph. MessageForum Attributes in a Discussion Topic As Dillenbourg and colleagues had predicted in 1996.. the tool supported the visibility of the mechanisms that facilitated three main areas: the strategic uses of message labels. application of domain content. 2003). one of Jeong’s recommendations for future research was to integrate tools in the forums for realtime feedback to optimize group performance. there were fewer challenges per argument. and methods and tools used with message labelling to diagnose problems.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Figure 2. there were greater gains in knowledge acquisition. greater knowledge of argumentation processes. and convergence towards consensus. Lastly. and ComputerSupported Collaborative Learning Environment (CSILE) (Scardamalia et al. According to Jeong. 1994).5.. there was no difference in knowledge acquisition. there were fewer unsupported claims.1-2.D.

5.g. it seems that Dillenbourg and colleagues (1996) may have predicted another e-learning trend. 2008). and implementation in wide use leaning management systems. Since SNA tools appeared in online community research after 2000. Jeong & Davidson-Shivers.1-3): Table 2. InterLock is a system whereas MessageForum is a tool incorporated in elearning systems such as Blackboard. flexible attributes to allow space for creativity. This will be discussed next. visibility of the communicative actions (interactions). Overall. it appears that any tools to support collaborative e-learning need to aid: • • • • • observing Attributes and analysing human-human and human-computer interactions. Thesis Centre for Interactive Systems Engineering LSBU Page 50 .2. Jeong’s tool also produced social network analysis (SNA) graphs (sociogrammes) via Microsoft Excel (not in real time).2. The differences were connected to interface usability as well as the visibility of attributes structure. Ph. Jeong has presented similar results from other studies (e.5. Jeong.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review having the results in real-time is lack of situated social presence. 2006. The question that arises is whether conceptual frameworks can aid the design process.D. Similarities and differences between MessageForum & InterLock MessageForum + ARG ARG BUT EXPL EVID VS n/a n/a = = = = / InterLock Initial information Agree Question Challenge Reason Reason Maintain The differences as regards the interface are mainly based on the different ways in which the implementation of the main concepts have been carried out.1-3. evaluation to be close to decision making and preferably in real-time. The similarities and differences between InterLock and ForumManager were (Table 2.

6 PROPOSITIONS FOR DESIGN Based on the literature review.1 Collaborative E-Learning Episodes The dialogical context differentiated collaborative learning dialogue from other “simple” dialogues. A collaborative e-learning model is process-based and can be identified in a dialogical context. A CeLE is a communicative discussion episode that has to have a starting point. all attributes function as inputs in a CeLE and are of equal importance. not imposed and easy to use. the Collaborative e-Learning Episodes. which is new to some participants. Ph. In addition. providing a distinction between mere information and knowledge acquisition. and ended with new knowledge construction. A numerical measurement was also considered. and the Sense of the e-Learning Community Index. A dialogical sequence for collaborative learning can be an episode. the scheme needs to be flexible. there are three propositions for design. 2. the Participation Levels. a transition and an end point that indicates a collaborative e-learning cycle. 2002). The basic interaction variables (attributes) can indicate collaborative learning value and reflect pedagogical values in one CeLE. Thesis Centre for Interactive Systems Engineering LSBU Page 51 . Thus. Such a framework was found as a “knowledge sharing episode” (Soller et al.D. These end points are defined by silence or mutual agreement when an idea arises. Based on Gumperz (1982:328-9) for identifying the end point or point for CeLE completion. hereafter called a collaborative e-learning episode (CeLE 4 ). A knowledge sharing episode was initiated when there was new information. however.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review 2. including relational or contextual assessments on how items of information are to be integrated into what we know to expand our knowledge.6. So the proposed process of one CeLE in accordance to its pedagogical values is: 4 Examples of CELEs can be found on Appendix A_VII_13 and Appendix X. collaborative learning appears to be the answer to the initial question on quality in e-learning as these dialogical sequences promote learning. a CeLE inference is involved in a complex series of judgments.

Information-based as one way of communication.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review 1. CeLE can identify the actors and the modes of preferred participation. Evaluation / Feedback. can support the e-learning community’s socio-dynamics. Collaborative Learning – Exploration (Explore. structures data in collaborative e-learning. Thesis Centre for Interactive Systems Engineering LSBU Page 52 . in combination with other tools. 2. raises awareness of possibilities in collaborative dialogue and argumentation. 3. and as such its value in real-time.6. Question).1-1. Ph. as soon as the learner describes the conversation act in an accurate way. This process of progressive dialogue can be depicted as follows (Figure 2. measures the collaborative e-learning value. The Collaborative E-learning Episode (CeLE) The CeLE is proposed to indicate the collaborative e-learning levels. Secondly. and 4. So if a CeLE can be translated into a tool it will support observing and analysing collaborative elearning interactions as it: • • • • • • • makes e-learners aware of active participation prerequisites. that is the type of interactions that can foster the community’s creativity.D. This means that all e-learning participants can instantly be aware of the level and the value of participation. identifies e-learners’ type and thus posting value and quality. CeLE. and facilitates proactive decision making. Interactional as two ways communication (Explain. Reflection). Lastly. provides an overall view of the collaborative e-learning process.1-1): Inform Explain Interaction Inform Explain Question Explore Evaluate Evaluate Question Pedagogical Value Suggestion Elaboration Reflection Elaboration Reflection Explore Figure 2.6.

D.3 Monologue or lecture. 0. Wiley’s reply depth or thread depth will be used to calculate the level of persistence to a topic as discussion depth. So a proposition for measuring e-learners’ participation levels is presented next. Other than evaluating online messages as CeLE.2-1): Table 2. specific prerequisites and attributes were missing.2 and higher Discussion. then measurement. 2. Evaluating participation in multi-thread discussion was conducted by calculating an adjusted mean reply depth (d value) for each participant. Acknowledging this limitation. In an effort to measure the collaborative e-learning value. chit-chat 3.6. Thesis Centre for Interactive Systems Engineering LSBU Page 53 . the value in the depth of discussion was considered on three levels: 1. Levels of participation measurement Authors Beaudoin (2002) Taylor (2002) Oriogun (2006) No Lurkers No Levels of Participation Low Shirkers Low Workers Medium High Ph.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review If CeLE can provide the attributes to observe and analyse interactions in collaborative learning. Wiley (2002) suggested a qualitative classification built on a mathematical approach. 1. Recent approaches for measuring participation levels are not designed to be automated (Table 2.2-1. no discussion 2.3 to 1. The literature review increased understanding on passive and active participation but it did not provide distinct and practical assessment.6. however.2 Simple Q & A.6. participation is of major importance in the search for quality in collaborative e-learning discussions. Multilogue This is an interesting way to calculate value based on internal (monologue) or external levels of discussion (dialogue). 0 to 0. observation and analysis of participation may be feasible.2 Levels of Participation The Greek teachers’ passive participation is a normal and legitimate behaviour and it is not a barrier to developing a sense of belonging.

who participated occasionally but mostly in a read-only mode. Active participation is initiated with the very first message and has Ph. Levels of activity measurement Levels of Activity X 4 3 2 1 Sense of Community High Medium Low Zero Measurement on the overall messages 76-100% 26-75% 1-25% Based on this measurement.6.2-1. the participation eye.6. the bell curve (Herrnstein & Murray. medium and high levels of engagement. Taylor proposed a classification between three groups. make the least collaborative effort.2-2): Table 2.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Beaudoin suggested a distinction between low and no visibility learners. a centripetal process. Participation Levels in Collaborative e-Learning Communities The above levels show the symmetry/asymmetry in participation and also depict the grey zone between passive and active participation characterised by the sleeper effect.6.D. the ones. Furthermore. Thesis Centre for Interactive Systems Engineering LSBU Page 54 . Oriogun considered low. and shirkers. the lurkers. the latter is the white area where the participants decide to make the first step.2-1): Figure 2.6.2-2. To automate a practical measurement of participation. targets to the sense of belonging to the community (Figure 2. there is an area outside the taxonomy and in the middle of the participation eye that refers to the sense of belonging to the community and does not depend on active participation. that followed the minimum requirements. the workers or active participants. 1994) is proposed for calculation on the basis of the total number of messages sent in a forum (Table 2.

medium and high. it minimizes risk as decision making depends on facts rather than assumptions. At the first stages of this research.D.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review three levels. The question here is on suitable research methodologies to inform design towards the development of tools and evaluation techniques for collaborative elearning communities.3 The Sense of E-Learning Community Index Studying the Greek teachers’ participation in collaborative e-learning communities suggested two ways to measure success and quality in e-learning. Thesis Centre for Interactive Systems Engineering LSBU Page 55 . If these levels can be displayed in real-time on a discussion topic level and a course level. This will be explored in the next chapter. 2005c). it is a catalyst for the stakeholders to accomplish their goals faster. So a second approach was developed. This result does not describe the sense of belonging and its interrelated attributes in a coherent way. Based on the framework suggested by Levenson and Ruef (1992:234). the collaborative e-learning episodes (CeLE).6. and (d) whether the tools helped. (b) feeling what another person was feeling when reading the message. part of collaborative learning evaluation was initially anchored in empathy to provide a sense of belonging. then participants in e-learning will be able to be proactive by managing and evaluating participation in real time. Ph. This process will be further elaborated in the research design in Chapter 5. It is important to note that incorporating these attributes in a questionnaire was found the only way to acquire results as all information in online discussions is textual. However. and the sense of community. low. It is costeffective. it is iterative rather than one-off as results are concentrated on the process. Real time online community management and evaluation of participation provides immediate information delivery as needed for all stakeholders. the participants in the initial study had to answer questions on: (a) knowing what another person was feeling when reading the message. the measurement based on empathy was found inadequate to evaluate the sense of community. the Sense of e-Learning Community Index (SeLCI) and empathy was one attribute of the new framework. (c) whether they took any action. the only tangible outcome was that the members who were found to develop empathy were the ones who were very active in the community (Lambropoulos. 2.

Retrieved 17/06/2007. T. Berge. British Journal of Developmental Psychology. Jonassen & S. In P. The Internet and Higher Education. L. R. Thesis Centre for Interactive Systems Engineering LSBU Page 56 . Ph. Using argument representations to make thinking visible for individuals and groups.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review REFERENCES Ambrozek.oise. Anderson. P. NY: McKay. B..edu/~ed626/626. 11.computing. Beaudoin. Berger.A Metaphor for the Lack of Clarity in the Field of IDT. (1956). Theoretical Foundations of Learning Environments (pp. T.indiana. 471-483. (2005). Retrieved 24/05/2005. (2004). (1984). L. Land (Eds. & Kam. S. New York. J. Ethnography and System Design.. 14-21. Baker. T. R. P. Kent & J.html. Learning or Lurking? Tracking the 'Invisible' Online Student. P. The role of grounding in collaborative learning tasks. B. Online Communities in Business: Past Progress. Dillenbourg (Ed. E.). Bundesen. R. P..).. pp. IN. P. Page. (1996). London: Routledge. S. A. Berners-Lee. Z. Joint planning and problem solving on a computer-based task. Barab. (2000). The ADDIE "Model" . A. 20.infonortics.. Canada. 7(1). 64-77.).. 9. The Evolution of Cooperation.. Joiner.d. M.. from http://www. & Wentling. The Roles of Models in Artificial Intelligence and Education Research: A Prospective View. In A. the classification of educational goals – Handbook I: Cognitive Domain. 25-55). & Sheldon. from http://emoderators. E-moderators. from http://www. (2007). Paper presented at the AERA'04 .. Light.umich. Cognitive and computational approaches (pp. San Diego. Bloom.htm. Ca. Paper presented at the Computer Support for Collaborative Learning. L. Axelrod. Journal of Knowledge Management.com/vc/vc04/slides/cothrel. 159183). Paper presented at the CSCL'95 Computer Supported Collaborative Learning Conference.pdf Bell. From Practice Fields to Communities of Practice. NJ: Lawrence Erlhbaum Associates. Mahwah. B. (1999).. Joiner.D. F. (2002). Future Directions. H. Davis.ca/cscl/papers/bell.). Retrieved 25/06/2005. Boshuizen.uk/computing/analysis/2186086/berners-leewarns-changes-ahead. R.com/.. Collaborative learning. Toronto. A. Educational Software Design and Authoring. M. (2000). M. In D. Work. & Cothrel. Berners-Lee warns of changes ahead.. S. Bloomington. The Encyclopedia of Microcomputers (Vol. & Linn. New York. Computing Magazine. (1995). (1997). Computer support for knowledge construction in collaborative learning environments. NY: Marcel Dekker. J. Hansen. International Journal of Artificial Intelligence in Education.edu/idt/shortpapers/documents/aect2004... (2003). J.co. (1991).. A.utoronto. 122-143. (1991). C. Beers. Archichvili. Motivation and Barriers to Participation in Virtual knowledge-sharing Communities of Practice. (2004). Retrieved 15/12/2006. New York. Blaye. P. (n.. from http://www. & Traum.J. A. C. Baker. NY: Basic Books. M. & Kirschner. The knowledge integration environment: Theory and design. D. Bell. M. Amsterdam: Pergamon. Bichelmeyer. Mahwah. from http://www. NJ: Lawrence Erlbaum Associates. Williams (Eds.American Educational Research Association. from http://www.. 31--63). Taxonomy of Educational Objectives. Cooperation: The basis of sociability. Learning in innovative learning environments Symposium. Argyle. & Duffy. IDT Record. V.pdf. G. 147-155. J. T. Retrieved 02/12/2005. (1997). Retrieved 23/04/2005. H. Paper presented at International Conference on Virtual Communities 2004. M. 5(2). & Collins.

5. Oxford: Pergamon Press. & Rosson. S.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Borges. 1(1). S.Culture Smart!: A Quick Guide to Customs and Etiquette. techniques. GamePlay Design. from http://www. S. P.unige. D. Virtual Learning Environments.. Common ground and the understanding of demonstrative reference. Ph. M. Dillenbourg P. F. Sproull. Thesis Centre for Interactive Systems Engineering LSBU Page 57 .). Belgium.).org/periodical/vol_1_2003/borges. A. & Resnick. Retrieved 05/12/2005. Adult Learning: An Overview.edu/papers/convergence. Oblinger (Eds.D. Cognition. The MediaMOO Project: Constructionism and Professional Community. & Duguid. (1995). The social life of information. (1996). 226-247). C.1-19). In A. UK: Routledge... P. Liverpool. Learning in Humans and Machine: Towards an interdisciplinary learning science. In J.. CA: Jossey-Bass. Spada & P. Schreuder. UK. NJ: Lawrence Erlbaum Associates. (2004). London: Random House. (2005). A. J.. 189-211). M. (2005).. (pp. San Francisco. & Donaldson. Clark. Mahwah. Journal of Verbal Learning and Verbal Behavior. Paper presented at the GDTW-2005. Brookfield.. In T. from http://llk. Deliberated evolution: Stalling the View Matcher in design space. Planning for “Neomillennial” Learning Styles: Implications for Investments in Technology and Faculty. Dede. C. D. R. S. & Baranauskas.edu/ir/library/pdf/pub7101o. Adult Education Quarterly.pdf. (1999) What do you mean by collaborative learning?. 7.. A.ch/tecfa/publicat/dil-papers-2/Dil.Learning In The New Millennium: Building New Education Strategies For Schools. Retrieved 25/05/2004.18. In P. Repositioning Ideology Critique in a Critical Theory of Adult Learning. 107-145).educause. H. from http://ifets. 6(1). (2002). Workshop on Virtual Learning Environments. Greece . 1-39. L. H. Mahwah. 245.S. NJ: Lawrence Erlbaum Associates. International Encyclopaedia of Education.-M. Tuinjman (Ed. and Buttrick. H. C. Dillenbourg (Ed) Collaborative-learning: Cognitive and Computational Approaches. Weisband (Ed. Butler. and use (pp. Reiman (Eds. Buhayer. Bruckman. C. M. (2005). R. CO: EDUCAUSE Publishers. (2003). Paper presented at the EUN'00 . Brussels. Dillenbourg. R.. 8-9 November 2005.. & Kiesler.). Community Effort in Online Groups: Who Does the Work and Why? In S. (1996).ieee. (1995). M. Brookfield. M. & O'Malley.media. Boulder.258. Retrieved 20/10/2006. L. Leadership at a Distance: Research in Technologically-Supported Work.html. Boston: Harvard Business School Press. B. M. Organization Science.. CollabSS: a Tool to Help the Facilitator in Promoting Collaboration among Learners. A. Design rationale: Concepts. Kiesler. S. C. (2000). Convergence. (1983). Constant. Referring as a collaborative process.html. & Wilkes-Gibbs. Carroll (Eds. (2000). 22. from http://tecfa. Sproull. Educational Technology & Society. Baker. H. P. D. (2001).. (1996). Oblinger & D. M.. B. Brown. 20. Conrad. Computers and the Collaborative Experience of Learning London. Carroll. Blaye. Dillenburg. (pp. The kindness of strangers: On the usefulness of weak ties for technical advice. In E. 119-135. (1986).7.). (1994).pdf. The evolution of research on collaborative learning.). Crook. P. Dini. 94-109. 7-22. 52(1). P. J. Oxford: Elsevier. Oxford: Elsevier. & Kraut.mit. J. Retrieved 20/12/2005. Game Design and Technology Workshop. Clark. Engaging the Online Learner. Moran & J. Educating the Net Generation (pp. C..

300-311. Introduction to instructional development. X. Training.. A report for the National College for School Leadership. 541-550. (1975). M. USA: Wadsworth Publishing Co. 528-539. R. Syntax and Semantics.pdf. The Foucault Reader (pp. R. C. (2003) Virtual Learning Communities. 28(3). P. 279-282). Inform.org/shared/data/pdf/quality_in_elearning_panorama_en. (1996). Retrieved 26/01/2007. Instructional Design.. Belmont. Lurking: An Underestimated Human-Computer Phenomenon. Goodfellow. British Columbia Campus. B. & Pawlowski J. Milton Keynes. The attack on ISD. London. Downes. Communities in Cyberspace (pp. October 07 – 10. (2006). M. (in press). X. Santa Rosa. W.. (1998). 72-73. S.) Grice. Paper Ph. (1985). R. Free Online Dictionary for Computing. Canada. (2005). Earley. 42-54. Communications of the ACM.06 (June). Foucault. Faulkner. P. Wired. & Chalfonte. New York. eLearn Magazine. 37(4). (2000). S. H. Retrieved 23/03/2004. S. U. Designing for usability: Key principles and what designers think. (2000). 2. Görtz. City University.eun. Faulkner. 12(4). Gould. los Angeles. D. & Holzinger. Logic and Conversation.. from http://www. IEEE MultiMedia. R. L. CA: Morgan Kaufmann Publishers. & Gibson. Mahwah. R. (1984). J. Fenrich. M. C.. C. Fish. Ca. In the Collaborative Book Project 'A Practical Guide to Online Teaching and Learning'. L. 70-75. Ca. E. (Eds.ics. Gordon. Retrieved 02/01/2007. Gentry. Montreal.pdf. (2005). (Reprinted in Baecker. (2006). Ebner. A. & Zemke. B.0. Gulati.uci. Multinational Teams: New Perspectives. B.org. Nine Principles for Making Virtual Communities Work. S.org/subpage. Creating Instructional Multimedia Solutions: Practical Guidelines for the Real World.). from http://www. C.cfm?section=articles&article=291. S. MacMillan.. (1994). E-learning 2. San Mateo. London: Prentice-Hall. The Open University. London: Routledge. Gulati. Fenrich. Ehlers. & Lewis.ACM Conference on Human Factors in Computing Systems: Interact. The VideoWindow system in informal communication. Kraut. (2006). 29-59). The Essence of Human-Computer Interaction. Rabinow (Ed.. In P. The politics of health in the eighteenth century.Quality in European e-learning – PANORAMA report. P. Ca. NY: Pantheon Books. Canada. Retrieved 12/11/2006.elearnmag.. Constructivism and emerging online learning pedagogy: a discussion for formal to acknowledge and promote the informal. PhD Thesis at the Department of Education and Lifelong Learning. (2005). A. J. from http://foldoc. P. Usability Engineering. Inspire. USA: Informing Science.D. (2004).edu/~jpd/publications/2006/implications-chi2006. NY: Palgrave. (1994). Learning during online and blended courses: Unpublished Research. & Buxton. Implications for Design. Identity and Deception in the Virtual Community. (2002).. In P. In press. Thesis Centre for Interactive Systems Engineering LSBU Page 58 . Smith (Eds.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Donath. NJ: Lawrence Earlbaum Associates. Paper presented at the CHI'06 . (1990). 41-58. J. S. 1. 22-27 April 2006.). P. Dourish. Godwin. Kollock & M. from http://cms.) (1987). A study by the European Quality Observatory. M. New York. Institute of Educational Technology. Readings in humancomputer interaction: A multidisciplinary approach. Hildebrandt. Paper presented at the 1990 ACM conference on Computer-Supported Cooperative Work.

& Daykin. Towards Constructivism: Investigating Students’ Perceptions and Learning as a Result of Using an Online Environment. 5–7 April 2004. (1994). NJ: Ablex Publishing Corp. 543-568. The multimedia forum kiosk and SpeakEasy. Varying the number of required postings versus posting days: Promoting student interaction and sustained discourse in online discussions. & Davidson-Shivers. & Enyedy. Hiltz. R. Johnson. (1999). University of Glamorgan. from http://tiger. C.pdf. Hsi. & Remidez. B. Three Dimensions of Learning. D. 5-6 April 2006. (1982). (1994). D. whatever that is. M. from http://alchemi. Hoadley. Online Communities: A Case Study of the Office of the Future. Innovations in Education and Teaching International. Paper presented at the Technology and Learning Symposium. Retrieved 02/12/2005. NJ: Lawrence Erlbaum Associates. 54(6). Hoadley. S. H. A. Empathic accuracy.. (2006). PhD Thesis. New York. N.. Janis.coe. (2002). In J. 363–364. & Berman. Thesis Centre for Interactive Systems Engineering LSBU Page 59 .sp01. International Journal of Knowledge and Learning. J.edu/Research/Proposals/LearnTechSymposium/MethodsTool sAnalyzingInteraction_Jeong2005. CA: ACM. Berkeley. (1995).. R.html.0. J. Hughes. Ph. Illeris.leeds. NY: Guilford.missouri. (2005). from http://dev2244801. A.ac. from http://www.D.). E-learning 2. Gumperz. Centre for Lifelong Learning.. Unpublished research. Hovland.. NY: Free Press. San Francisco. L. (1997).fsu. Palo Alto. 113-129. C. J. C. An Overview of Cooperative Learning. M. Baltimore: Brookes Press. Jeong. (1953). Hillsdale. I. Mapping alternative discourse structures onto computer conference. H.edu/~jonassen/courses/CLE/ Ickes. 1(1-2).Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review presented at the UACE. A. C.d. Scaffolding scientific discussion using socially relevant representations in networked multimedia.htm. New York. New Haven.T: Yale University Press. Palo Alto Ca. Research.. M. Paper presented at the 3rd ACM international conference on multimedia. P. Herrnstein. T. British Journal of Educational Technology. D. Adult Education and Lifelong Learning: Theory and Practice. & Johnson. Retrieved 20/11/2006. (1999). Jonassen.uk/educol/documents/00003562. C. Jeong. Jarvis.. The effects of gender interaction patterns on student participation in computer-supported collaborative argumentation. R. Jeong. (n. Hoadley. CA. Educational Technology. and Development. Creativity and Collaborative Learning. New York.). Jennings. Leicester: NIACE. Inderscience. I. (Ed. Methods and Tools for the Computational Analysis of Group Interaction and Argumentation in Asynchronous Online Discussions. & Murray. Discourse strategies. W.Computer Supported Collaborative Learning 1999.uk/archives/ele/elearning_20_wh. Between information and collaboration: Middle spaces in computer media for learning. (2005). (2008). Communication and Persuasion: psychological studies of opinion change. G. A. UK. W. Retrieved 02/01/2007. D. P. the Annual Conference of the Universities Association for Continuing Education . (2004). Thousand. & Kelley.co. H. NY.. S. University of California. London: RoutledgeFalmer.. H. (2002). Paper presented at the CSCL’99 . MA: Cambridge University Press. K.). Cambridge. 39(3). (2005). Jonassen. Stanford University. Norwood.Regional Futures: Formal and Informal Learning Perspectives. Design of Constructivist Learning Environments(CLEs). C. The bell curve. (1985). Retrieved 10/12/2006. Villa & Nevin (Eds. 217-223. N..

J. (1998). A. Khine & D. Amsterdam. P.html..). Project IRIS: I. Hershey. A. M. A. Learning to become: creating evolutionary learning community through evolutionary systems design. In S. Paradise lost? Primary Empathy in Online Communities of Interest and Ways of Use. M.blogspot. from http://www. New York. Knowles. Kollock. 346-348). User-Centred Design of Online Learning Communities (pp.pdf. Laszlo. Partners in life — Syntony at work.org/ Last access 28/08/2007. Managing the Virtual Commons: Cooperation and Conflict in Computer Communities. Building Successful eLearning Communities. Fisher (Eds. A. Retrieved 15/09/2005. & Laszlo. 21-39). Lambropoulos. Projects for the Hellenic Education in the U. Creating a Technology-Rich Constructivist Learning Environment in a classroom management module. Jonassen. W. from http://elearningtech. (2001).K. PA: Idea Publishing.. Creating Learning Communities. 1-28). Eight Ways to Get Students More Engaged in Online Conferences. NATO ASI Series / Computer and Systems Sciences. Kim. Encyclopaedia of Virtual Communities and Technologies (pp. & Mayes.Jan 1998. Herring (Ed. Hershey. 109-128). & Smith. ComputerMediated Communication: Linguistic. 22-27 July. J. (1992) (Eds). N. Andragogy in Action: Applying Modern Principles of Adult Learning. 15(5). (1998). San Francisco: Jossey-Bass. (2005). Technology-Rich Learning Environments (pp. PA. (2005). 2002. Dasgupta (Ed. P. Ergonomic requirements for office work with visual display terminals (VDTs) -. Kim.). London. Retrieved 26/07/2006. http://www. Klemm. A. K.).). S.net/aklaszlo/syntonyques Ph. P. Design Principles for Online Communities. N.T. Khine. In R. from http://www. Lambropoulos. 62-64. Karrer.elearningpost.C. In S. in the 11th International Conference on Human-Computer Interaction 2005. & Laszlo. (2005). Kommers. Miller (Ed. N. In S. R. M. the Netherlands: John Benjamins. 1-3 September. Presented at the Annual Conference for the Hellenic Education in the U. (2000). K. What is e-learning 2.. Encyclopaedia of Virtual Communities and Technologies (pp. PC Update. In N. 346-348).K. M.intelligenesis. Berlin: Springer-Verlag. New Jersey: World Scientific. PA: Idea Publishing. 26(1). C. Dasgupta (Ed. T.0? Blog entry. Social. N. C. (2003). Online Empathy: Social Consequences for Community Building. Hershey. from http://home. Lambropoulos.html. elearningpost. Paper presented at the 1st Conference on Online Communities and Social Computing. (1997). and Cross-Cultural Perspectives (pp.com/articles/webtechniques. Social Scaffolding: 9 Design Principles for Online CommunityBuilding.interloc.). Vol. NV. User-Centred Design of Online Learning Communities. May 2001.earthlink. Lambropoulos & P. 58-60. (2006). M. (1998).). (1984). (2002). (2006). Web Techniques.com/articles/archives/exclusive_interview_with_ amy_jo_kim/. Online Empathy: Social Consequences for Community Building. (1996).eu/nikiweb/PDF/05/lambropoulosOnlineEmpathy Vegas05Research. In S. Retrieved 05/09/2005. USA: Idea Publishing.. (1998)..Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Interlock. Las Vegas. N. Thesis Centre for Interactive Systems Engineering LSBU Page 60 .Part 11: Guidance on usability. The Higher Education Journal.com/2006/02/what-is-elearning-20. NY: Solomon Press. Laszlo. Retrieved 05/09/2006.D. T. Zaphiris (Eds. Lambropoulos.naima. 81. Kollock. D. Cognitive Tools for Learning. from http://www. International Organisation for Standardisation (ISO) (1998). Lambropoulos.

(2003). R. D.). N. Chang. (1992).. Pietarila. International Society of the Learning Sciences. (2004).. The Guided construction of knowledge: talk amongst teachers and learners. First European Conference on Computer-Supported Collaborative Learning. K. D. article 10. Bloomington IN. P.indiana.com/articles/education/brief%20history. Y. mathematics and culture in everyday life. P. Paper presented at the ICLS '06. Miyake. D. Light (Eds.. P. from http://www..fi/papers/fle3_guide.. & Littleton.). W.. Universiteit Maastricht: Maastricht. P. Hakkarainen (Eds. & K.doc. Ling. (1995). Li. Using social psychology to motivate contributions to online communities.. P. Journal of Computer-Mediated Communication. (1999). Cambridge: Cambridge University Press. Learning with Collaborative Software . Learning with computers. R. Kligyte. and Kraut. & Wegerif. Analyzing productive interaction (pp. Wang.. Häkkinen. K.htm. Lave. from http://fle3. Frankowski. Cognitive Science. Beenen. G. M. Vogel. Learning as Transformation (pp. Social Processes in Children’s Learning. (2000). 63(2). Helsinki. Situated learning: Legitimate peripheral participation... Mercer..-M. X. & Wenger. PhD Thesis at the University College London. J. N. (2005). Retrieved 10/06/2005. J. from http://www. Netherlands. Thesis Centre for Interactive Systems Engineering LSBU Page 61 . Chen. K. Leinonen. Journal of Personality and Social Psychology.) Euro-CSCL European Perspectives on Computer-Supported Collaborative Learning. Eurelings. 79-101). Terveen.). Retrieved 12/10/2005. a. Lee. University of Art and Design. Retrieved 30/04/2006. Big Island. Is 'exploratory talk' productive talk? In K. Levenson. Hawaii. S. & Limayem.org/edu/News/archives/LurkerProjectCoPWorkshop SPring03a. In P. Learning to Think Like an Adult: Core Concepts of Transformation Theory. & Järvelä. (1991). Light..D. Lurking as participation: a community perspective on lurkers' identity and negotiability. L.. R.. June 27-July 1. (1999). J. X. (2002). Cognition in practice: Mind.cpsquare. G. H. M... A.. Retrieved 02/01/2006.pignc-ispi. In J. Rashid. Mercer. Analysing the mechanisms of common ground in collaborative web-based interaction. Constructive Interaction and the Iterative Process of Understanding.. E.. F. H. Hawaii International Conference on System Sciences. M. Adoption of Informatics to Support Virtual Communities. & Ruef. Littleton & P. (2006).. 445-453. D. P. Toikkanen. 234-246. Resnick. Paper No 214. Cambridge: Cambridge University Press. Ph. UK: Cambridge University Press.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Lave. 7-10 January 2002. IEEE Computer Society. London. L. 10. London: Routledge.. (1988).acm. Empathy: A physiological substrate. McDonald.org/ft_gateway. Leigh. M. S.edu/vol10/issue4/ling. Retrieved 02/03/2004. & Jiang.html Mäkitalo. (1986).. from http://jcmc. from http://portal. Indiana University. 10(4). (n. R.-C. Mezirow. Let’s get more positive about the term ‘lurker’. 7th International Conference on Learning sciences. Associates (Ed. Ludford. (2001). J. Monthienvichienchai. T. Media Lab..A guide to Fle3. R. T. 22-24 March 2001. Paper presented at the HICSS 2002.pdf. Cosley. Salo. F.uiah. Computer Support for Vicarious Learning Unpublished Research.. Dillenbourg.. Cambridge. (2003).cfm?id=1150093&type=pdf&coll=GUIDE &dl=GUIDE&CFID=21290811&CFTOKEN=83883790. 3-33). A Brief History of Instructional Design.d. J. A. Lee.-W. 151-177. K.. A. & Dean. San Francisco: Jossey – Bass. N. Clevedon: Multilingual Matters.

pdf. 2000. Moodle (n. 26 February. Oliver. E. & Preece. 56-67. Retrieved 23/05/2005.. K. Retrieved 20/08/2007.iwi. Retrieved 19/12/2005. G. An empirical assessment of pedagogical usability criteria for digital learning material with elementary school students.pdf. Shield. G. Paper presented at the NAISO World Congress on Networked Learning in a Global Environment. from http://opess.indiana. Charlottesville. Norman. from http://moodle. & Preece. Shedding light on Lurkers in Online Communities.unibe. Greece. Distance and eLearning. USA: Academic Press. L.u-net. Q&A With Diana Laurillard. B. T. What’s the use of ‘triadic dialogue’?: An investigation of teacher-student interaction. (2003). USA. C.. D.ifsm..elearnmag. 9(2). Lurking in email-based discussion lists. Netherlands. Participation and Cognitive Engagement within CMC Groups by Cleaning of Transcripts.ch/download/100029-03-TM-024. L. O'Neil. 24-27. & Wells. 21(3). J. Nonnecke. from http://www. from http://www. Thesis Centre for Interactive Systems Engineering LSBU Page 62 . Myrach.cfm?section=articles&article=14-1 Nielsen. (2003).Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Molenda. Atlanta.. B. Mumford. Content Analysis of Online Transcripts: Measuring Quality of Interaction. G. J.. Nokelainen.com/C1book1. D.. P. VA. (1999). Paper presented at the Conference Ed-Media ’94 . & Kukulska-Hulme. Cognitive Science Research Report Sussex. Retrieved 14/11/2006. Berlin. Annual Conference of Association for Educational Communications and Technology.edu/~molpage/Agenda%20Res%20on%20ID. 39(4). The Pyramid of Usability: A Framework for Quality Course Websites. Understanding Explanation. M. (2006). Communications of the ACM. Paper presented at the CHI Conference on Human Factors in Computing Systems.final. Modularity and Integrity in Instructional Engineering.pdf. PhD Thesis at London South Bank University..enid. p.ie.ifsm. (2006). & Knolmayer.org/subpage. J. 15-18 June 2003. Challenges and Solutions for Virtual Education 2002-05. (2005). Muir. Paper presented at the EDEN 12th Annual Conference of the European Distance Education Network. The Collaborative Notebook: A distributed knoowledge building environment for project enhanced learning. NY: ACM Press. (1987). Nassaji. Boston. New York. A. Applied Linguistics. Nonnecke. O'Malley. M. Oriogun. (1983). UK: University of Sussex. H. A.htm. The Quality Dialogue: Integrating Quality Cultures in Flexible.d. International Journal of Instructional Ph. Usability Engineering. 7(1). Designing Human Systems . L. (1993).) Modular Object-Oriented Dynamic Learning Environment. April 1-6.The ETHICS Method. 24-26 January 1999. A. & Shaw. London. Retrieved 22/10/05. Edinburgh..edu/~preece/Papers/SheddingLight. 123-128. from http://www. Lurker demographics: counting the silent. Educational Technology & Society.umbc.umbc. Unpublished Research. (2000). An Agenda for Research on Instructional Development Paper presented at the AECT'87. 178-197. (1987). from http://www.Educational Multimedia and Hypermedia.edu/~preece/paper/13%20Nonnecke&PreeceCHIp aper_ACMnotice. (1996).D.PDF Nonnecke. Neal. Hague. UK. 376-406. & Spohrer. from http://www. (1994). Retrieved 25/06/2004. (2003). C. B.. P. 188-194. (2000). Rhodes. 73 – 80. J. Retrieved 18/11/2004.org. K. Learner-Centered Education. & Gomez. Paper presented at the Conference Ethnographic Studies in Real and Virtual Environments: Inhabited Information Spaces and Connected Communities. (2000). Asynchronous Discussion in Support of Medical Education Journal of Asynchronous Learning Networks. 416-423.

org/3rd_culture/ramachandran06/ramachandran06_inde x. S.. Reading: Addison-Wesley. V. Wegerif. Rheingold. E. Thousand Oaks: Sage Publications Inc. 69. Edge. Preece.computer. UK: John Wiley & Sons.html. D. empathy and trust in communities of practice: Stepping-stones to social capital. N. G. The Virtual Community. The electronic bulletin board: A computer driven mass medium.org/Journal/Mar_06/article03. (1990). M. Nottingham Trent University.. A.. R. Ravid. (2003). E. The Internet and Health Communication: Experience and Expectations (pp. 5-8 January 2004. J. Journal of Universal Computer Science. (2000). In R. What is the difference? a new critique of adult learning and teaching. UK. & Andrews. Preece. from http://www. & M.org/comp/proceedings/hicss/2004/2056/07/2056702 03.sudweeks. (1999). J. Paper presented at HICSS'04. (1997).). C. S. NY: Harcourt. NY: Cambridge University Press. 2(1). Journal of ComputerMediated Communication. (1986). F. 2(3). Thesis Centre for Interactive Systems Engineering LSBU Page 63 . F. Mass: MIT Press. Rafaeli. 37th Hawaii International Conference on System Sciences. Paper presented at the British Journal of Educational Psychology Current Trends Conference. & Soroka. S. Retrieved 15/03/2004.pdf. J. New York. J. A. S. Preece. (2006). (2004). The top 5 reasons for lurking: Improving community experiences for everyone. Computers and the Social Sciences. Network & Netplay: Virtual Groups on the Internet. Ph. G. R. D. V.D. (1993). The Interdisciplinary Journal of Human-Computer Interaction. H. 294-302. S. Retrieved 05/06/2005. Etiquette. Paquette. Sudweeks. (Eds. & Ghozati. The Moral Judgement of the Child. Mirror Neurons and imitation learning as the driving force behind "the great leap forward" in human evolution. Instructional design competencies: The standards (3rd ed. Interacting with Computers.edge. Piaget. Rafaeli. Ravenscroft. (1987). Rogers. (2001). (2004). (2000).. retrieved 09/03/2006.. Reclaiming thinking: dialectics and design for digital learning. Fields. Supporting Sociability. Preece. 237-260).). Performance and Instruction. & Hartley..htm. Online Communities: Designing Usability.. J. IEEE Computer Society. B. Computers in Human Behavior. & Sudweeks. 12(1).. K. S. Nonnecke.). Ramachandran. (2000). The Different Drum: Community making and peace. Governing the Commons: The Evolution of Institutions for Collective Action. J.edu/vol2/issue4/rafaeli. New York: Touchstone.indiana. 2(4). (1998).Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Technology and Distance Learning (March 2006). Empathic communities: Balancing emotional and factual communication. Rafaeli... Richey. New York. from http://csdl2. Observations and Explorations of Empathy Online. C. Leicester: NIACE. 123-136. Cambridge. Katz (Eds. from http://jcmc. from http://www. M. Big Island. (2004). Foxon. Peck. Instructional Engineering in Networked Environments Canada: John Wiley & Sons.. Chichester. (1932). R. R. Preece. Rice & J. J. M. & McLaughlin. Networked Interactivity.html. Retrieved 18/11/2006. Rafaeli. R. Mercer. 63-77.. Brace Jovanovich. NY: ERIC Clearinghouse on Information and Technology and the International Board of Standards for Training. De-lurking in virtual communities: a social communication network approach to measuring the effects of social and cultural capital. 10. Ostrom.. (2003).itdl. Hawaii. Syracuse.

E-moderating: the key to teaching and learning online. Ryder.. 1971-2001. from http://carbon. Direct Manipulation vs Interface Agents. 349-383).Annual Meeting of the American Educational Research Association. Petersburg.. framework. J.. (1991). Resnick. Paper presented at the Ph.). & Bereiter. M. Re-inventing distance education. 8-9 May 2003). (1998). B. K.... P. G. Smulders.au/resources/lurkerstoposters. (2004). R. Shackel & S. The Journal of the Learning Sciences. PA. M. What and how preservice teachers learn: Designing a course that fosters development of useful theoretical knowledge and the assessment methods to capture it. DC: American Psychological Association. Paper presented at PEG’03. Washington. Levine & S. D.. April 10-14. Human Factors for Informatics Usability (pp. D. Handbook of Research on Electronic Collaboration and Organizational Synergy. A. MA: Allyn & Bacon. Retrieved 25/03/2005. International Journal of Lifelong Education. Computer support for knowledge-building communities.htm. Seels. C. London: Kogan Page. Silius. Shambaugh. & Tervakari. B.D.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Rogoff. J. from http://elearnmag. (2003).org/subpage. Z. S. design and evaluation.1 July 2003 . G. Hershey. & Wilson. (2003. Perspectives on socially shared cognition (pp. (1997).pdf. M. B. 31-43. (1994). 42-61. Usability – Context. In M. J. CD-Rom. from http://www. J. Retrieved 06/11/2006. Shackel. & Maes. Shepherd. Instructional Design (an overview). Siegel.). & Pohjolainen.The Evaluation Tool into the Portal of Finnish Virtual University. A. Scardamalia. McGilly (Ed. Bereiter. Pedagogical Usability. S. Classroom lessons: Integrating cognitive theory and classroom practice (pp. PA: Idea Group. Instructional Designers’ Perceptions of Their Agency: Tales of Change and Community. In B.. L. WA.cfm?section=best_practices&article=11-1.fastrak-consulting. A. Upper Saddle River. .html Salmon. N. St. J. A Multidisciplinary Tool for the Evaluation of Usability.co.. 201–228). Accessibility and Informational Quality of Web-based Courses. Seattle. B. M. S.M. from http://flexiblelearning.). Designing for Users. New York. Keppell (Ed. (2005). & Beach. 4(6). eLearn Magazine. (2002). (2008). M. J. Kim. 11th International PEG Conference: Powerful ICT for Teaching and Learning. From Lurkers to Posters (Report): Australian Flexible Learning Framework.). definition. K. J. Shneiderman. Paper presented at the AERA'01 . Thesis Centre for Interactive Systems Engineering LSBU Page 64 . Steinkuehler. excerpts from the debates at IUI 97 and CHI 97 Interactions. (2001).-M.net. Schultz. 265-283. N.. B. (1994).cudenver. In K. 3(3). C.edu/~mryder/itc_data/idmodels. (2006). A. Designing for Learners. M. Derry. Rumble. K.. & Kenny. Teasley (Eds. B. In L. NY: Cambridge University Press. Boston. 21–37).uk/tactix/features/engineer. B. Tervakari. USA: IGI Global Publications. K. Russia. Cambridge MA: MIT Press. & Magliaro. Hershey. Instructional design: A systematic approach for reflective practice.).). (1991). Campbell. Scardamalia. C. Retrieved 23/03/2005. 20(1/2). Engineering e-learning. Retrieved 05/12/2005... & Lamon. NJ: Merrill Prentice Hall. Salmons... Making instructional design decisions (2nd ed. Schwier. (Eds... R. Instructional design: Case studies in communities of practice. (2001). An Evaluation of the Usefulness of Web-based Learning Environments. Social interaction as apprenticeship in thinking: Guidance and participation in spatial planning. The Computer-Supported Intentional Learning Environment (CSILE) Project: Trying to bring the classroom into World 3. 28 June . (2001). & Seymour. Richardson (Eds. & Glasgow. G. Silius. (2000).

Kiesler (Ed.. K. Soller. and databases: The net as a social technology. D. Paper presented at the CSCL'02 . M. Soller. Retrieved 23/04/2006.pdf. International Journal of Artificial Intelligence in Education. Rousing Minds to Life: Teaching. S. (2005). Learning. Atheism. 467-483. Van Der Akker. 65 . S. T. University of Rochester. & Roschelle. L. E. New Jersey. S. Predicting Quality in Educational Software: Evaluating for learning. M..html. NJ.edu/iterations/vanmeer.. D. & Lesgold.fastcompany. Meier. Sproull. 38.unesco. & Rubenfeld. Gustafson. from http://www. J. Iterations: An Interdisciplinary Journal of Software History.D. J.cs.. J. NY. M. Van Meer. In S. S.umn.ht m. The Uses of Argument. Constructing a Joint Problem Space: The Computer as a Tool for Sharing Knowledge. Wiebe. Branch. CO. Ph. A. R. K. A. & Gallimore. In S. 35–51). Squires.Computer-Support for Collaborative Learning. 169-190. A. Jermann. NJ: Lawrence Erlbaum Associates. Tharp. Journal of Technology and Teacher Education. from http://www. 38(1). Traum.ee.). P. 15. Mahwah.. D.). Boulder..org/education/educprog/lwf/doc/portfolio/definitions. 7-11 January 2002. Swan. L.).) Technology & Learning definitions: Collaborative Learning.. UNESCO (n. The Netherlands: Kluwer. Design approaches and tools in education and training. (1990).pitt. D. (2004). Derry (Eds. (2002)..doc. R. MA: Cambridge University Press. Suchman. C. S. & Ur.. 261-290. R..V. Rochester. from http://www. & Preece. Cambridge. & Muehlenbrock. Situated professional development and technology integration: the CATIE mentoring program. & Faraj. Valencia. P. We can see you: a study of communities' invisible people through reachout. (2003). New York: Cambridge University Press. Holmes. L.79). Interview with Katharine Mieszkowski. Interacting with Computers. PLATO: From Computer-Based Education to Corporate Social Responsibility.. (1999). Unpublished research. J..Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review International Conference on Network Universities and E-Learning.. Culture of the Internet (pp. usability and the synergy between them. (1994). Teasley.com/online/38/ideazone. & Bichelmeyer. (1958). sex. Educational Technology Research & Development.. Mones. Jennings. and Schooling in Social Context. Nieveen. Thesis Centre for Interactive Systems Engineering LSBU Page 65 . from http://matriisi. Fast Company. 31-44. E. & Plomp.. J. (2000). Publishers. Spain. M. Retrieved 12/07/2004. (2003)..cbi. Retrieved 02/12/2005. A. (Eds. (1987). (2002).. Toulmin.. Wulf (Eds. V. Soroka. (1993).html. Tripp. A Machine Learning Approach to Assessing Knowledge Sharing During Collaborative Learning Activities. E. Inc.fi/arvo/liitteet/usefulness_of_web. 10(2). Retrieved 19/03/2004. Communities and technologies (pp. PhD Thesis. Lajoie & S.. Vargas. Jacovi. N. In M.. USA: Lawrence Erlbaum Associates. Wenger & V. Boston: Kluwer Academic Publishers.). B. 369. S.. (1988). From Mirroring to Guiding: A Review of State of the Art Technology for Supporting Collaborative Learning. R. Rapid prototyping: An alternative instructional design strategy. (1997). Community Standards.d.. from http://www.edu/~wiebe/pubs/papers/sollerwiebelesgold02. A Computational Theory of Grounding in Natural Language Conversation. J. Retrieved 02/03/2004. A. Typaldos. Deventer. Computers as Cognitive Tools.tut. 11(5). Huysman. A. Plans and Situated Action: The Problem of HumanMachine Communication New York: Cambridge University Press. B.

Enschede. Users as Learners: What’s the Difference? Retrieved 24/05/2005.. Whittaker.edu/~mccrory/pubs/HCIC_LCDboaster. University of Twente. 199-211. (1999). Learning On-line: a review of recent literature in a rapidly expanding field. The role of tutoring in problem solving. Faraj. M. 89-100. & Gulia. Press. 57-88). Australasian Journal of Educational Technology. V. Waters. In B. ACM Press. Paper presented at the CSCW 98. Land (Eds. Design Approaches in Training and Education: A reconstructive study. the Netherlands. Terveen. 52(2). & Gibbons. S. (2004). D.msu. CO: Westview Press. WA. and instructional technology: A case study. (1999). Las Vegas. Situated Cognition in Theoretical and Practical Context. (2002). pp. Seattle. Cherny (1998). Journal of Computer Assisted Learning. B. Retrieved 09/11/2005. Learners as Users. L. Boulder. (2004). B. I. notation systems. Bruner.. M. L. Design languages. (pp. W. S. B.. from http://www. USA: Harvard University Press. Cambridge. 17. Computer-Supported Collaborative Work.. R. (1998). (1976). Cole. 257-264. Wallace. G. Networks in the Global Village: Life in Contemporary Communities.. New York. Williams. The dynamics of mass interaction. Educational Technology Research and Development. & Teigland. NY. Wiley. Wellman (Ed.. Exploring the use of blogs as learning spaces in the higher education sector. R. Press..). A. 26(3). M. Cambridge Mass: M. (1978). Wegerif. Williams. Scribner & E. from http://wiley. & Myers.I.D. & Jacobs. (1962). 5(11). Emerging trends in educational technology research. Wood. 14(3). Thought and Language.usu.). In D. G. S. 57-68. MA. Ph. W. Vygotsky. NV. S. Published originally in Russian in 1934.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 2: Literature Review Visscher-Voerman. A Proposed Measure of Discussion Activity in Threaded Discussion Spaces v0.. N. In M.T. Collective Action and Knowledge Contribution in Electronic Networks of Practice.. A. (1986). Net-Surfers Don’t Ride Alone: Virtual Communities as Communities. (1999). Winn. 1998. Hill & L. & Dawes. Thesis Centre for Interactive Systems Engineering LSBU Page 66 . H. Journal of Further and Higher Education. J. (2000). & Ross. S.edu/docs/discussion09. Vygotsky. Wilson.. 232-247. K. (2002). (2004). Paper presented at AECT'86. Journal of child psychology and psychiatry. Mercer. M. Theoretical Foundations of Learning Environments. S. J. 20(2). D. L.. Journal of the Association of Information Systems. Article 15. C. 263-272. Wellman. S.9. John-Steiner.). R. Software Design to Support Discussion in the Primary Classroom.htm Wasko.ed.pdf. Souberman (Eds. S. Annual conference of the Association for Educational Communications and Technology. J. J. 16-21 January. Jonassen & S. L. Mahwah NJ: Lawrence Erlbaum Associates. Mind in society: The development of higher psychological processes.

Ph.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 3: Research Design 3 • • • • • The Research Design: Research Methods for Investigating Collaborative E-Learning Communities Key Topics Covered in This Chapter: E-Learning research advantages and limitations The need for a systematic e-research design to examine the research context Why and how ethnotechnology can support investigation in collaborative e-learning communities The dual nature of human-human and human-computer interaction methodologies The research coordination and design Chapter 3 introduces the research design. Ethnotechnology was found to be a suitable approach to examine the research context as well as to analyse human-human and humancomputer interactions in e-learning environments.D. time series design and research constraints are presented. Thesis Centre for Interactive Systems Engineering LSBU Page 67 . A review of research methods to support design and e-learning shows advantages and limitations caused by their multidisciplinary nature. The questionnaire design.

that is ethnotechnology. this disadvantage can be tackled by triangulation. Rapid INTRODUCTION technological changes influence communication. n. Ethnotechnology is used alongside human-computer and human-human interaction analysis to shed more light on collaborative e-learning communities by triangulating sides of space and time. information and knowledge management for e-learning. 2003). E-researchers need to balance the conflicts which derive from the implementation of these principles.. 3. a hacker hacked the server where Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 68 . virtual ethnography. there is a risk. Time-series design is suggested as a means of coordinating e-research.). According to the British Psychological Society (http://www.d. Because of its virtual and interdisciplinary nature. data mining. This multifaceted nature is reflected in the lack of a single agreed strategy. to examine the research context. Anderson & Kanuka said that creating and maintaining respectful relationships with e-participants is underpinned by three principles: voluntary informed consent. This chapter builds on situated e-research. research design needs to explore human-human and humancomputer interactions which have learning as their purpose.1. For example. collaboration. Then. Within the context of these new challenges. participant research. confidentiality and anonymity. that is Human-Computer Interaction Education. 2007).bps. however.D.d. privacy. says that “the lack of a single agreed research strategy discipline leaves an individual researcher planning a specific research undertaking out on a limb” (n. The first section discusses issues for elearning research and ethics. the ethical principles for conducting research with human participants include general principles that may differ in e-research. it employs an aspect of ethnography. and recognising the elements of e-research risk.uk/. and ethics (Anderson & Kanuka.1. and Internet use. Sasse. referring to the different aspects of the HCI nature. e-researcher’s skills. E-Learning Research E-Research employs web server analysis.org.). e-research challenges research methods. Nonetheless.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 3: Research Design 3.1. application logs. some times an amalgamation of different research methods fails because of the disengagement between methods and their underlying methodology (Boehner et al.

D. part of the Greek Ministry of Education and Religious Affairs. at the researcher’s request. data can be collected as it occurs naturally. 2003:58). this study was approved by three different organisations: the London South Bank University Ethics Committee. and the Innovations Office. Thesis Centre for Interactive Systems Engineering LSBU Page 69 . it is legitimate to consider such data in order to understand practice in its own terms. the right to withdraw was stressed in the user policy (netiquette) available online as well as sent to the participants prior to the study. On the other hand. The consent was obtained by the participants’ signature on a pro-forma which emphasised that they could withdraw from the study at any time without providing any explanation. these were conducted under all ethical considerations and principles and participants’ consent was obtained for all studies. the system operators collected data including demographic details for GSN’s own use and also for research purposes. on a second occasion. This approach may resolve the problem in research created by the Hawthorn effect. However. in reality “there are no clearly defined criteria for appropriate ethical behaviour for all researchers or all research activity” (Anderson & Kanuka. The participants were allowed to keep a copy. Robinson and colleagues (2007) suggest that a combination of methods is needed. In order to address the ethical issues. working on ethnographically-informed empirical studies. To Robinson and colleagues. Nonetheless. Prior to the transfer of this research to LSBU there had been no approval sought for the two initial studies not presented in this thesis. the Greek Pedagogical Institute.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 3: Research Design the final study was hosted and wiped out all database tables. but in order to avoid any form of control over the participants. Ph. in addition.

Ethnographic research was found to represent a long tradition of studying social processes in real life situations.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 3: Research Design 3. These approaches will be analysed in this chapter as regards their scope.2-1. It has been exploited in computing research.2-1): Table 3. ideas.D. beliefs. see Massey. 1999). Thesis Centre for Interactive Systems Engineering LSBU Page 70 . values.2.1. Research Design RESEARCH DESIGN Research Methodologies 1 2 3 4 HCI-HHI HHI HHI HCI-HHI HCI n/a Ethnotechnology Qualitative Methodologies A Message analysis Social network analysis Quantitative Methodologies A Questionnaires B Log Analysis Time-series design Target Use Coordination Implications for Design Evaluation √ √ √ √ √ √ √ √ √ √ 5 The research design targets methodological triangulation. that is the application and combination of several research methodologies of the same phenomenon to corroborate one set of findings with another. In addition. RESEARCH DESIGN The proposed research design is (Table 3. ethnotechnology can increase understanding of the research context for human-human and human-computer interactions (HHI & HCI). the examination of research design should scrutinise the situation as regards participation in collaborative e-learning communities and determine implications for designing successful collaborative elearning communities. the hope is that two or more sets of findings will converge on a single proposition (for a review of triangulation. 3. and purposes of systems use. especially Ph. principles.2. and thus informing the designers about the setting. time-series design coordinates the research activities on a temporal basis for baselines and interventions. Examining the Research Context For the purpose of this study. and research instruments as well as their advantages and disadvantages for this particular study. used to uncover the knowledge. Initially.

listening to what is said. and the second that sees the engineer conducting the fieldwork. This has created two approaches in the use of ethnography in systems design.1. participation in online communities (Nonnecke. 01/05/2007). watching what happens. and understandings into a more meaningful context. that is descriptive and historical accounts for understanding social settings. personal communication via email. However.1. This is a demanding process. After stressing the importance attached to the use of ethnography for fieldwork analysis in design. Ian Sommerville. but rather the way in which such data are transformed into a written or visual form (Tedlock. and collecting whatever data is available. These provide the explanatory frame and the narrative explaining of “why” and “how” these implications one arrived at. Thesis Centre for Interactive Systems Engineering LSBU Page 71 . It is not simply the production of new information or research data. the engineers and ethnographers.2. asking questions. so some systems designers employed ethnographers to provide description of the context needed for design.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 3: Research Design after Suchman (1987) discussed the profound mismatch between the generic models of work on which IT systems are built at the time and the actual nature of work in which they are used. Many ethnographers advocate the need for a trained ethnographer in order to go beyond simple descriptions (fieldwork). It is an ongoing attempt to place specific encounters. events. to a genuine analytic one (scenic fieldwork) (McGarry. Ph. 3. the ethnographer’s role towards a rich fieldwork is also a disadvantage. One way to utilise ethnography in design is by ethnography’s scenic fieldwork. Ethnography Ethnography was suggested as the preferred approach to study lurkers and thus. 2005:67. Professor of Software Engineering at Lancaster University. However. 2000).D. one that suggests there should be two groups of experts. It involves the ethnographer immersing herself in people’s daily life for an extended period of time. two examples from these traditions will be presented. distinguishing it from positivist research (quantitative). Ethnography is located in the field of naturalistic research (qualitative). The advantages of ethnography in the investigation of tools and evaluation techniques for collaborative e-learning communities are related to the rich description of the research context and the exploratory stance that relies on the researcher’s own ability to provide rich descriptions of data and understand the meaning of social situations. 2000:455).

can immerse herself easily since the prolonged engagement and immersion in the context required in ethnography already exists.2. and how that knowledge is deployed in the ordering and organisation of work. contradicting the definition of the “user” as a passive recipient of technology. she can progressively reconfigure new ways to understand ‘users’ and ‘user contexts’ by understanding work practices within her own culture and society as inspirations and foundations for design activity in order to support new ways of working. Thus. Thus. 2006). 2006) adopted ethnographic perspectives in order to put design to work in particular contexts.g. design is the active process of incorporation and co-evolution of technologies. and observation. 2004.1.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 3: Research Design there was a problem related to finding common ground and communication channels for both the fieldworker and the engineer.D. Hughes et al. Ph. Ethnotechnology: the Virtual Ethnography Ethnography employed in real environments has taken advantage of conversations. 2005. Although this approach has created questions on the engineer’s epistemological validity there were successful implementations (Furnis. and after almost 10 years of research. Thesis Centre for Interactive Systems Engineering LSBU Page 72 . in this case her own community of practice. Hughes (1999) concluded that there are major differences between ethnography and systems design. Despite the efforts to ease their differences (e. practices. Button (2000) suggested that it is important to consider how relevant people do what they do. These were the assumptions that were used in this study. He proposed that this consideration provides the key to understanding the contribution of sociology to engineering and design. they are two different mentalities. adopted and adapted by people in the course of practice. that is the ‘interactional what' of their activities. Sharp et al. 3. A more recent approach considers that an engineer is able to conduct the fieldwork herself. Assumptions on the use of ethnography for the ethnographer’s role as well as the locus of e-research were revisited. This is the explication of members' prior knowledge: what people have to know to do work.g. and settings. If the designer/researcher is located within their domain of study. Studying teamwork in their own agile software development groups. Sharp and colleagues (e. 1997)..2.. written materials. In fact.

2005) and online Communities of Practice (CoP) (Paiva.. has been only recently used to shape a strategy in computing and e-learning research as in collaborative e-learning (Guribye & Wasson.2. Michael Paraskevas. and testing. and analysing online CoP in the ITCOLE EU project (Talamo. 2005. 2005. Talamo & Ligorio. As for the online environment. said that the Greek teachers were not using the tools provided. 2005). Full participation is feasible when she is fully engaged as when they belong to a CoP. 2005. 2006). Talamo.D. Ethnotechnological Methods The GSN Deputy Director. studying the impact of technology raised from the observation of a mismatch among the users’ way of tools’ implementation and the functions for which the developers had planned them. The project aimed at the creation of software tools. His observation brings forward one of the primary aims of ethnotechnology.3. and conducting interviews with individuals from the Greek educational authorities can reveal some aspects of the problem of participation in collaborative e-learning communities.1. based on cycles of software development. The research issues are similar to ethnography. 3. ethnotechnology can be supported by textual analysis and can rely on ethnotechnologist’s empathic understanding of her context. her research role is partly covert.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 3: Research Design Ethnotechnology. built to support collaborative e-learning communities. and treating everything as ‘strange’. however. It is a specific field for studying the impact of technology raised from the observation of a mismatch among the users’ way of tools’ implementation and the functions for which the developers had planned them (Grossen et al. animating. pedagogical best-practices. Ethnotechnology has been recently employed for developing. Participant observation in activities such as seminars and conferences. Thesis Centre for Interactive Systems Engineering LSBU Page 73 . 2002). abandoning her preconceptions. focus groups as well as the quantitative human-computer (logging) and qualitative human-human Ph. 2002. Guribye. Conducting scenic fieldwork within the Greek Community of Practice (CoP) appears to have both a real and virtual locus. Talamo & Ligorio. talking to colleagues. Following Talamo. Talamo and Ligorio combined ethnotechnology and discourse analysis in synchronous and asynchronous communication within the ITCOLE CSCL system. such as the role of the researcher and lack of generalisability since ethnographic approaches are always in principle incomplete (Hine. 2002).

frequently by counting and/or sorting using various criteria (Nonnecke. 2002. are time stamped. privacy issues are raised. permission to conduct the research which is a bureaucratic process in governmental organisations. The advantages are attractive (Nonnecke.g. or details of the computer they are using.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 3: Research Design interaction analysis (messages analysis) can shed more light in understanding the Greek e-learners. one focus group is used to reveal best practices and tools for activating the passive participants and a second to evaluate the tools and evaluation techniques before their implementation. are compatible with ethnography and thus ethnotechnology. 2006): log files can be automated.D. Provided that the participants in a focus group discuss the topic of their specialisation and interest. Laghos & Zaphiris. and there may be inadequacy in answering questions such as “why” and “how”. Examples of what information can be collected include (Laghos & Zaphiris. are useful for finding usage/activity patterns. logging can offer information on the Greek teachers’ passive participation in Moodle@GSN. In this study. server log or log-file. can study users’ behaviour. So in this study. is usually in the form of a text file and is used to track users’ interactions with the computer system they are using. widely used in Human-Computer Interaction (e. patterns of navigation. A log. 2006): when people visited a site. Such information can be translated to: Ph. provide opportunities of quantitative analysis and further research. Suter (2000) worked on the dilemma of their use in ethnography and provided three reasons they should be seriously considered as an alternative way for generating data: (a) access to participants' interaction on topics that are either difficult to observe or rare in occurrence. frequency of visits. the areas they navigated. 2000:21-22). (b) a focus group improves ethnographic practice by providing another option for generating appropriate data. Thesis Centre for Interactive Systems Engineering LSBU Page 74 . Thus ethnotechnology can be utilised in e-learning design. Ethnotechnological research conducted in e-learning environments can acquire information about the users by tracing them. 2000. 2000). Preece et al. a focus group can provide insights in hidden aspects of design with data that could not be acquired otherwise. where they are connected from. the length of the visit. and (c) the method raises important questions of relevance between focus groups and participant observation. The disadvantages include the need for power tools for large amounts of data.. Focus groups. A typical logging operation consists of collecting messages and then performing analysis. also referred to as web-log. Faulkner.

Beyer & Holtzblatt. 2004). However. there should be analytical frameworks in place to facilitate transitioning from ethnographic inputs to designing concepts (Jones. So ethnotechnology can be incorporated in e-learning engineering building on scenic fieldwork. interviews (qualitative data) Figure 3. The key variance of passive participation may not be the cause and this can be detected. if there is no active participation. Reservation exists on the Greek teachers’ passive participation in Moodle@GSN.2.1. DiCoT. that is zero history. Contextual Inquiry. tool usage. Existing analytical frameworks build on ethnographic inputs to provide designers with a series of tools and techniques for understanding social settings and organizing their observations to derive models for design (e. Considering everything as data suggests a data overload and thus.3-1 Ethnotechnology and methods Ph. passive participation levels measured by time spent on the system. ETHNOTECHNOLOGY Thematic Analysis Messages (qualitative data) Social Network Analysis Interactions (quantitative data) Questionnaires Open & closed questions (qualitative & quantitative data) Logging Log files (quantitative data) Fieldwork observation. 2006). b.D. implications for design. Another problem is the danger of amateurisation of ethnography in practice because fieldwork itself is only the first step. there is no narrative. But e-learners do leave traces that appear in the log files.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 3: Research Design a. Thesis Centre for Interactive Systems Engineering LSBU Page 75 . and d. 2005:68). Furniss. the analytical treatment of that data determines whether it is fieldwork or scenic fieldwork (McGarry. c.g. there are some disadvantages. the number of passive and active participants. Logging can also offer opportunities for correlations in order to identify practices that work better than others. and ethnographic inputs. 1999. the depth of discussion threads to measure interactivity.

these were the social cues such as words at the beginning and end of a message (e. simple to understand and remember. Her model provided an initial framework for coding CMC discussions. 3.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 3: Research Design 3. Codes had been developed in a continuum following a cyclic process of sampling.1. Braun & Clarke. Coding needs to be ‘usable’ from other researches for a high inter-rater reliability level. Despite these limitations. This is a process for encoding qualitative information in order to relate the data to prior ideas (Boyatzis. Thesis Centre for Interactive Systems Engineering LSBU Page 76 . greetings and sign offs) and emoticons.2. and Fahy and colleagues’ Transcript Analysis Tool (TAT) (2001.2. In the Search of Quality: Human-Human Interaction Analysis Human-human interaction analysis employs thematic analysis and Social Network Analysis so as to improve understanding of e-learning interactions quality.D. allow quantitative analysis. it has been criticised for poor theoretical support and being strictly a teacher-centred instructional paradigm (Gunawardena & Lowe. there is an advantage in Ph. 1998: 99-127. 1998:143). CeLE analytical framework was found to neglect significant information related to social interactions. thematic analysis has been used in posts analysis searching for Collaborative e-Learning Episodes (CeLEs) as well as in open questions in the final questionnaire. developing themes and codes and validating and using the code scheme.g. it lacked detailed criteria for systematic and robust classification of electronic discourse (Howell-Richardson & Mellar. however. Furthermore. 2002) was used for code development. 1996). Theory and prior-research driven thematic analysis for collaborative learning (see MA dissertation. relevant to the context. 2006). In this study. In addition. Posts: Thematic Analysis Analysis of the posted messages could provide information about active participation and the quality of the Greek teachers’ communication and learning. 1997). 2005).2. and use of a wide variety of information. Consequently. Thus two more analytical frameworks were considered in order to increase understanding of collaborative e-learning.2. Themes and clusters for developing protocols were grouped based on a developmental scale on cognitive complexity (Boyatzis. Henri’s (1992) five dimensions for message analysis. statistical comparison could determine valid differences by reducing large amounts of text into numerical data so as to be analysed statistically.

It appeared that social cues can be related to building social interactions prior to collaborative elearning and can be included in coding sub-units within initiation (Table 3. Then. inter-rater reliability had to be reached to ensure correct coding.1-1): Ph.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 3: Research Design Henri’s proposition. As a result. The classification indicators were rewritten in a clearer and simpler way and a further 10% of the data was coded giving a reliability of 50%. 1991:126). greeting or emoticons for describing emotions (Henri. The schema has been also tested by 2 colleagues using messages from different units of analysis from SKYPE and chats (Lambropoulos et al. 1996) or sociability of online communities (Preece.2.2. A second independent researcher verified the process with a reliability of 93% on a 25% sample. more researchers have attempted to analyze such social effects of conferencing exchange (e. the themes and clusters were discussed and analysed with the independent researcher giving a reliability of 90% on a further 25% of the data. This revealed an initial reliability of 3% on the data which was clearly unacceptable. however. one CeLE was the unit of coding as the most basic meaningful segment. which is the social cues coding for analysing social messages: self-introduction. 1987. Thesis Centre for Interactive Systems Engineering LSBU Page 77 . an analysis of 25% of the data was conducted by an independent researcher in Atlas-ti™ based on the CeLE codes without any prior discussion.g. After Henri. Reliability in the 90%+ range was operationally considered as being acceptable. Rice & Love. Walther. 2000). expression of feeling. The attributes of a CeLE were initially based on conceptual and empirical work.D. Therefore.. 2008).

might. something else. etc. history of something. what about. etc. Level 1 Indicators for classification Information. best practice. worst. additions or superficial amendments. argument. would. should. repetitions. example. bullet points. procedure etc. critical information. solution. Comparison.2. competition of ideas. Hypothesis. therefore. this is why. resource interdependence. images. could.2. example. reasoning. because. but. proposition.D. I know. overall. thus. uncritical information. easiest. I believe. I agree & you are right. assessment. we agreed & finally. think & suggestion. refer-to-a-name. Collaborative e-Learning Episodes Codes Matrix Collaborative e-Learning Episodes LEVELS OF ABSTRACTION Level 3 CeLE Elements 0 1 2 3 3a 4 5 6 Initiation & Social cues Question . confirmation. on the contrary & different. Thesis Centre for Interactive Systems Engineering LSBU Page 78 . help. audio etc Recommendation. etc Agreement. abbreviations. Yes. Disagreement. instruction. comparison. requirements. Strategy. instruction. however.Information Explanation Agreement Disagreement Exploration Evaluation New ideas Co-construction Level 2 Analytical Corroboration Definitions Initiations. I prefer. Question. same. alternative. plan. I think. social cues. plan. it is important. best.. difference. etc. It is very interesting. discrepancy. . method. unfortunately & having no meaning. comparison. Ph. have worked. nice behaviour & suggestion. examples.1-1. argument. flaming.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 3: Research Design Table 3. lexical items. quoting. definition. etc. I tried if. further explanation. I have an idea. opinion. etc. summary. what do you mean. summaries. statement. question. emoticons. corroboration. etc Explanation and self-explanations.

Thesis Centre for Interactive Systems Engineering LSBU Page 79 . several tools were considered for SNA as well as their integration in discussion forums as to support co-presence. however. density and intensity. et al. and "N" was the total number of participants.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 3: Research Design Enhancing social cues required a specific community management framework.2. Intensity in this study referred to the levels of participation and persistence where persistence is the level to which participants pursue topics. A focus group provided a scheme for best community development practices and tools to promote active participation. In addition. The most widely used SNA attributes are nodes (the actors of study). 2005). investigations of social interactions can shed more light in the research context and can triangulate the findings with ethnotechnological fieldwork. only group analysis on cohesion and centrality was found suitable for this study. 3. Interactions: Social Network Analysis Social Network Analysis (SNA) was used to understand e-learning related to the Greek teachers’ sense of belonging to the collaborative e-learning community. Persistence was related to interactivity and discussion depth operationalised by measuring the number of levels of communication in a particular discussion thread from the first posting to the last (depth of discussion threads). where "a" was the number of observed interactions between participants. so larger groups will be likely to exhibit lower density ratios than will smaller groups. Other than the posts’ quantitative and qualitative examination. two relevant concepts were found in Fahy and colleagues’ research on their Transcript Analysis Tool (TAT) (2001. Quantifying qualitative data was essential in thematic analysis.2.D. and centrality (central or isolated person). Density was the ratio of the actual number of connections observed. SNA focuses on complete (or group) and ego networks.2. to the total potential number of possible connections: Density = 2a/N(N-1). The researchers stressed that the measure of density is sensitive to the size of the network. SNA has been used to visualize communication and relationships between people and/or groups through diagrams by depicting social relationships between a set of actors (Baroudi. 1986). Other than counting the words in a post as an indicator of text richness. relations (the strands between actors). Ph.

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Chapter 3: Research Design

Cohesion: Network density for group thickness, reciprocity, cliques, and structural equivalence were used to measure the level of cohesion. Network density is the proportion of possible links in network that actually exist; it was evaluated by the adjacency connection reports. Sent-Received (S-R) number of messages is related to participants’ reciprocity. More specifically, reciprocity is the number of ties that are involved in reciprocal relations relative to the total number of actual ties (Hanneman & Riddle, 2005). A clique is a set of actors with each being connected to each other in smaller groups. Structural equivalence and in particular the CONCOR technique (CONvergence of iterated CORrelations, White, Boorman, Breiger, 1976; Breiger et al., 1975), describes the actors that have similar relations to others in the network with dendrogrammes. So the degree to which two nodes are structurally equivalent can be evaluated by measuring the degree to which their columns are identical: Centrality: Group centrality (Everett, 2005; Freeman, 1979) refers to the distribution of power between the community members and is measured by centrality, closeness and betweenness. In this study it referred to the total number of Sent-Received Messages (direct links), out-degree (replies made) and in-degree (received messages) centrality. Group closeness is defined by the normalised inverse sum of distances from the group to a node outside the group (Everett, 2005:61); in this study closeness was related to reciprocal distances. Betweeness is the number of indirect links in which the actor is required as an intermediary; this characterise the mediator as the controller of the information flow in a network. SNA Tools: SNA has been used for offline (e.g. Breiger, 2004; Bender-deMoll & McFarland, n.d.), off-line and online (e.g. Wellman, 2001), and online educational contexts (e.g. Daniel, 2007; Laghos & Zaphiris, in press). Presenting relationships and perceiving solutions derived from visualisation can assist annotations, consultancy or revision since data visualisation proved to be important for user locus of awareness, control and initiative (Shneiderman, 2000). However, even though SNA tools can depict the activity in online human-human interactions, the use of these tools in educational contexts is offline. Offline SNA tools need the researcher to input the data either based on observation (e.g. Petropoulou, 2006; Laghos & Zaphiris, in press) or extract the data in a file such as excel files (Jeong, 2005). These data are inserted to SNA software for visualisations graphs, however, providing an out-of-date evaluation and decision making.

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 80

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Chapter 3: Research Design

SNA tools could support e-learning discussions in real time. Kreijns and colleagues (2002) have suggested that if a group awareness widget existed, a software tool providing the learner group awareness about the others in the task and in the nontask context can enhance groups’ sociability. Additionally, social presence and copresence can enhance the sense of community by providing a picture of the community (Beer et al., 2005). Social presence was the degree by which a person was perceived as real in an online conversation (Meyer, 2002:59). Real-time SNA has started to support e-research; for example, Microsoft uses SNA tools to visualise users’ clicks on web pages (Milic-Frayling, 2007). To date, some open source offline SNA software used in discussion research are SoNIA (Stanford University; BenderdeMoll & McFarland, 2006), Pajek (University of Ljubljana, Slovenia; Nooy et al., 2005), UCINET (Analytictech, http://www.analytictech.com/download_products.htm) and JUNG (Java Universal Network/Graph Framework, http://jung.sourceforge.net/). JUNG was found to be compatible to Moodle and was integrated in Moodle for the purposes of this study. Other than the SNA online use, an SNA software is essential for data analysis. After studying comparisons between several SNA desktop applications (Laghos, 2007; Huisman & van Duijin, 2005) UCINET (Borgatti et al., 2002) was found most suitable for this study. The main reasons were its usability and the support provided (Everett and Borgatti; personal communication via email, September 2007). Overall, fieldwork and thematic analysis provide the data for qualitative analysis that can be quantified for statistical analysis; logging and Social Network Analysis provide data for quantitative analysis and social networks visualisation. Open and closed questions in a questionnaire could be used for triangulation so as to reveal whether two or more sets of findings could converge in a single proposition.

3.2.3. Questionnaire Design
A questionnaire is a self-reporting technique whereby participants fill in the answers to questions themselves; its purpose is to elicit facts about the respondents, their behaviour and their beliefs/attitudes (Nielsen, 1993). According to Nielsen, there are three types of questions: open-ended, closed, and scaled. Open-ended questions give freedom to the participants to respond, closed, where the participants have to choose from several choices, and scales where the respondents must answer on a pre-determined scale. They have been used for: online communities (Andrews et al., 2003); evaluation of the sense of community (e.g. Brook & Oliver, 2006; Daniel,

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 81

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Chapter 3: Research Design

2007); revealing lurkers’ opinions (e.g. Nonnecke, 2000; Gulati, 2006); or evaluation of evaluation tool’s effectiveness (e.g. Silius et al, 2003). Advantages derived from methodology were: easy access to participants, especially via emails or web-based questionnaires; use of qualitative or quantitative questions or a mix; there can show differences over time. The disadvantages were related to: an idealised version of information; lack of honest and accurate responses; very low response rates from passive participants and less than 20% response from active participants; and difficulty in having an immediate follow up (Mason, 1999). However, these disadvantages can be eliminated if used in conjunction with other methodologies. The questionnaires stages and objectives are (Zaharias, 2004; Kirakowski and Corbett, 1990) (Figure 3.2.3-1):

Figure 3.2.3-1. Questionnaire Design Methodology Three questionnaires were given to the participants, at the beginning, in the middle and in the end of study. The first questionnaire was used to acquire information on demographics, conditions of working and learning on the internet and initial

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 82

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Chapter 3: Research Design

knowledge on collaboration. The second questionnaire was used for e-tutoring purposes and it is not part of this study. The third questionnaire targeted to triangulation via the evaluation of: community evolution, the sense of belonging, empathy, trust, and knowledge acquisition. All questionnaires were given to 3-5 respondents in three phases in both English and Greek languages prior to the study. Lastly, the final section presents the e-research design needed for the e-research coordination.

3.2.4. E-Research Coordination: Time-Short Series
Design
Time-based coordination was used to capture the development and to triangulate sides of space and time of the unit of analysis. The use of quasi experimental time-short-series design was found a suitable approach to set a timeframe (Shadish et al., 2002). Quasi-experimental design is not preferable to other research design such as experimental design, but it is used when an experiment is not feasible, most of the time due to the group selection criteria: the groups were chosen and assigned out of convenience rather than randomization. Most importantly, controlled environments are impossible where participants have access from remote and distributed locations. Also, there is little loss of status or prestige in doing a quasi-experiment instead of a true experiment. Time settings refer to two main sets, defining the baseline(s), and time series. Baseline refers to the observation of behaviour prior to any treatment designed to alter behaviour. As such, the treatment effect is demonstrated by a discontinuity in the pattern of pre-treatment and post-treatment responses. The groups which are going to be used in this study are inactive. The latter suggests a solid baseline for treatments and effects related to causal inference, not affected by threats like history, natural development and maturity for studies mostly observed in children’s research. In time-short-series design aggregation and causal inference are not necessarily affected if a detailed amount of data could be collected. There are three dimensions to be investigated in order to examine the nature of intervention: (a) the form of the effect (the level, slop, variance and cyclicity); (b) its permanence (continuous or discontinuous) and (c) its immediacy (immediate or delayed).

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 83

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Chapter 3: Research Design

Two baselines are initially suggested, before treatments, community management and use of new tools, and after treatments. The time series design is (Table 3.2.4-1): Table 3.2.4-1. Observations, interventions and evaluation Observation
0 1 2 3 4 5 6 7 8 O0 O1 O2 O3 O4 O5 O6 O7 O8

Course Day
Baseline 1 - days 1 3 7 14 21 Baseline 2 28 End of the course

INTERVENTION & EVALUATION
Community Management

√ √ √ √ √ √ √ √ √

Tools n/a n/a n/a n/a n/a n/a

√ √ √

The initial baseline describes the research context before any intervention. Then community management and tools (treatments) support the e-learning community. Limitations and strengths of time-short-series design in this study are (SansonFisher, 2004) (Table 3.2.4-2): Table 3.2.4-2. Limitations and strengths in Time-Short Series Time-short-series Limitations
• • Fewer study units limits generalisability Measures must be suitable for repeated use Depends upon successful, temporal relationship between intervention and measure • • • • • • • •

Strengths
Process-based framework creates close examination of both units and interventions as well as causal inference All units could get intervention if it is effective New theories can be created Flexibility related to individuals, small and large units More intervention research and knowledge Can examine each intervention component Interdisciplinary Research exists Clear research design and data analysis Consisted with decision making processes

Time-short-series strengths add value to research, and limitations are considered as similar to any other research; thus, provision of measures can ensure the validity and reliability. Overall, time series design seems to have the potential to provide the needed time-based coordination of measurements to provide implications for design and e-learning evaluation. The methodological approaches and their attributes are (Table 3.2.4-3):

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 84

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Chapter 3: Research Design

Table 3.2.4-3. Research Methodology

ETHNOTECHNOLOGY

DESCRIPTION
1. Participation in projects, conferences, seminars, online discussions of Greek teachers’ associations and schools in Greece 2. Documentation, manuals and presentations by the Greek educational authorities 3. Personal opinions, conversations, and emails 4. Richness of text 1. Demographics 2. Conditions of working and learning over the Internet 3. Initial knowledge on collaboration 1. Frequency of visits 2. Number of active and passive participants 3. Active participation levels 4. Passive participation levels 1. Collaborative e-learning episodes analysis 2. Sense of e-Learning Community Index • Number of Collaborative e-learning episodes 2. Group Centrality • Centrality 1. Sense of e-Learning Community Index • Community evolution • Sense of belonging to the e-learning community • Empathy • Trust: knowledge exchange, help and support • Collaborative e-learning quality: participants’ opinions 2. Usage and usability of collaborative tools 3. Professional development 1. Examination of the data, missing data, normality. 2. Descriptive Statistics: crosstabulation, frequencies, arithmetic means, standard deviations, and exploration 3. Correlation: Pearson correlation coefficient (r). 4. Inferential statistics: statistical significance, p-value (Cronbach's alpha, α), and null hypothesis. 1. Form of the effect: the level, slop, variance and cyclicity 2. Its permanence (continuous or discontinuous) 3. Its immediacy (immediate or delayed)

1

Fieldwork

2

Pre-Questionnaire

3

Logging

4 5

Thematic Analysis Social Network Analysis

6

Post-Questionnaire

7

Statistical Analysis

8

Intervention Analysis

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 85

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Chapter 3: Research Design

3.2.5. Research Constraints
There were several constraints for the study. First, a number of constraints occurred because the research was conducted completely over the Internet (e-research): data overload; time and knowledge required to access, filter, sort, and combine the quantity of information produced, and endless opportunities. Secondly, additional constraints reflect the nature and circumstances under which the research is carried out including the time frame, having one individual performing all research functions and simultaneously being an active participant in the Greek educational community, use of open source tools, not being an expert ethnographer, and having limited programming skills. Thirdly, there is also a need to acknowledge the Greek teachers’ passive participation; the ratio on questionnaire response is anticipated to be less than 20%. This is not only a phenomenon observed in the Greek teachers’ reality; Cuthell (2005) reported limited activity in a 5 months project on MirandaNet, a British teachers’ network similar to GSN. Another limitation is related to the target population, which is not representative of the total population of the Greek teachers but naturally occurred in the Computer-Mediated-Communication space. Lastly, even though there are recent studies on participation in online communities, there is no similar research for comparison in the Greek and international research community.

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 86

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Chapter 3: Research Design

REFERENCES
Anderson, T., & Kanuka, H. (2003). E-research: Methods, strategies, and issues. Boston, MA: Allyn & Bacon. Andrews, D., Nonnecke, B., & Preece, J. (2003). Electronic survey methodology: A case study in reaching hard to involve Internet Users. International Journal of Human-Computer Interaction, 16(2), 185-210. Baroudi, J. J., Olson, M. H., & Ives, B. (1986). An Empirical Study of the Impact of User Involvement on System Usage and Information Satisfaction. Communications of the ACM, 29(3), 232-238. Beer, M. D., Slack, F., & Armitt, G. (2005). Collaboration and Teamwork: Immersion and Presence in an Online Learning Environment. Information Systems Frontiers, 7(1), 27-37. Bender-deMoll, S., & McFarland, D. A. (2006). The Art and Science of Dynamic Network Visualization. Journal of Social Structure, 7(2). Retrieved 15/12/2006, http://www.stanford.edu/group/sonia/papers/DNV_JOSS.pdf. Bender-deMoll, S., & McFarland, D. (n.d.). Interaction, Time, and Motion: Animating Social Networks with SoNIA. Report, Stanford University. Beyer, H., & Holtzblatt, K. (1999). Contextual design. Interactions, 6(1), 32-42. Retrieved 12/12/2005, from http://www.csc.ncsu.edu/faculty/anton/csc/rep/beyer.pdf. Boehner, K., Vertesi, J., Sengers, P., & Dourish, P. (2007). How HCI Interprets the Probes. Paper presented at the CHI’07 - SIGCHI Conference on Human factors in computing systems. 28 April - 3 May, 2007, San Jose, CA, 10771086. Retrieved 10/07/2007, from http://cemcom.infosci.cornell.edu/uploads/pubs/Boehner_Vertesi_Sengers_D ourish_Probes.pdf. Borgatti, S. P., Everett, M. G., & Freeman, L. C. (2002). Ucinet for Windows: Software for Social Network Analysis. Harvard, MA: Analytic Technologies. Boyatzis, R. E. (1998). Transforming Qualitative Information: Thematic Analysis and Code Development. Thousand Oaks, CA: Sage Publications. Braun, V., & Clarke, V. (2006). Using Thematic Analysis in Psychology. Qualitative

Research in Psychology, 3, 77-101. Breiger, R. L., Boorman, S. A., & Arabie, P. (1975). An algorithm for clusteirng relational data with applications to social network analysis and comparison with multidimensional scaling. Journal of Mathematical Psychology, 12, 328-382.
Breiger, R. L. (2004). The Analysis of Social Networks. In M. Hardy & A. Bryman (Eds.), Handbook of Data Analysis (pp. 505-526). London: Sage Publications. Brook, C., & Oliver, R. (2006). Exploring the influence of instructor actions on community development in online settings. In N. Lambropoulos & P. Zaphiris (Eds.), User-Centered Design of Online Learning Communities (pp. 341-364). Hershey: Idea Group. Button, G. (2000). The ethnographic tradition and design, Design Studies, 21, 319332. Cerulo, K. A. (1997). Reframing Social Concepts for a Brave New (Virtual) World. Sociological Inquiry, 67(1), 48-58. Cuthell, J. P. (2005). What does it take to be active? Teacher participation in online communities. International Journal of Web Based Communities (IJWBC), 1(3). Daniel, B. K. M. (2007). A Bayesian Belief Network Computational Model of Social Capital in Virtual Communities. Unpublished Research, PhD Thesis, University of Saskatchewan, Canada. Retrieved 01/08/2007, from

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 87

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Chapter 3: Research Design

http://library2.usask.ca/theses/available/etd-07132007141903/unrestricted/ben_m.pdf. Dourish, P. (2006). Implications for Design. Paper presented at the CHI'06 - ACM Conference on Human Factors in Computing Systems: Interact. Inform. Inspire. 22-27 April 2006, Montreal, Canada, 541-550. Retrieved 26/01/2007, from http://www.ics.uci.edu/~jpd/publications/2006/implications-chi2006.pdf. Dourish, P., & Button, G. (1998). On "Technomethodology": Foundational Relationships between Ethnomethodology and System Design. HumanComputer Interaction, 13(4), 395-432. Everett, M. (2005). Extending Centrality. In P. J. Carrington, J. Scott & S. Wasserman (Eds.), Models and Methods in Social Network Analysis (pp. 5776). New York, NY: Cambridge University Press. Fahy, P., & Ally, M. (2005). Student Learning Style and Asynchronous ComputerMediated Conferencing (CMC) Interaction. American Journal of Distance Education, 19(1), 5-22. Retrieved 12/12/2005, http://www.leaonline.com/doi/abs/10.1207/s15389286ajde1901_2. Fahy, P. J., Crawford, G., & Ally, M. (2001). Patterns of Interaction in a Computer Conference Transcript. International Review of Research in Open and Distance Learning, 2(1). Retrieved 12/12/2005, from http://www.irrodl.org/content/v2.1/fahy.html. Faulkner, X. (2000). Usability Engineering. New York, NY: Palgrave, MacMillan. Freeman, L. C. (1979). Centrality in Social Networks: Conceptual Clarification. Social Networks, 1, 215-239. Furniss, D. (2004). Codifying Distributed Cognition: A Case Study of Emergency Medical Dispatch. MSc Dissertation at UCL Interaction Centre, London. Retrieved 10/05/2006, from http://www.uclic.ucl.ac.uk/people/d.furniss/Documents/DomProjectFinal.pdf. Grossen, M., Pochin, L. O., Resnick, L. B., Säljö, R., Pontecorvo, C., & Burge, B. (2006). Interactional perspectives on the use of the computer and on the technological development of a new tool : The case of word processing. NATO ASI series. Series F : computer and system sciences (NATO ASI ser., Ser. F : comput. syst. sci.) ISSN 0258-1248 CODEN NASFEG North Atlantic Treaty Organization Advanced Science Institutes series. Gulati, S. (2006). Learning during online and blended courses: Unpublished Research, PhD Thesis at the Department of Education and Lifelong Learning, City University, London. Gunawardena, C., Lowe, C., & Anderson, T. (1997). Analysis of a Global Online Debate and the Development of an Interaction Analysis Model for Examining Social Construction of Knowledge in Computer Conferencing. Journal of Educational Computing Research, 17(4), 397-431. Guribye, F. (2005). Infrastructures for learning: Ethnographic inquiries into the social and technical conditions of education and training. Unpublished Research, PhD Thesis at the Department of Information Science and Media Studies, University of Bergen, Norway. Guribye, F., & Wasson, B. (2002). The Ethnography of Distributed Collaborative Learning. Paper presented at the CSCL'02 - Computer-Supported Collaborative Learning. 7-11 January, Boulder, CO. retrieved 09/05/2006, http://newmedia.colorado.edu/cscl/148.pdf.

Hanneman, R. A., & Riddle, M. (2005). Introduction to social network methods. Riverside, CA: University of California, Riverside.
Henri, F. (1992). Computer Conferencing and Content Analysis. In A. R. Kaye (Ed.), Collaborative learning through computer conferencing: The Najaden Papers (pp. 117-136). Berlin: Springer-Verlag.

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 88

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Chapter 3: Research Design

Hine, C. (2005). Virtual methods and the sociology of Cyber-Social-Scientific knowledge. In C. Hine (Ed.), Virtual Methods: Issues in Social Research on the Internet (pp. 1-13). Oxford: Berg. Howell-Richardson, C., & Mellar, H. (1996). A methodology for the analysis of patterns of participation within computer mediated communication courses. Instructional Science, 24, 47-69. Hughes, J. A. (1999). The general acceptance of ethnography in system design. Paper presented at the Conference Ethnography in systems design, An Industrial Approach to Work Analysis and Software Design. 29 April, 1999, The Conference Centre, Lancaster University, UK. Hughes, J. A., O'Brien, J., Rodden, T., Rouncefield, M., & Blythin, S. (1997). Designing with ethnography: a presentation framework for design. Paper presented at the Symposium on Designing Interactive Systems, In the Proceedings of the conference on Designing interactive systems: processes, practices, methods, and techniques. August 18-20 1997, Amsterdam, The Netherlands. New York, NY: ACM Press 1997, pp. 147 – 158. Huisman, M., & Van Duijn, M. A. J. (2005). Software for social network analysis. In P. J. Carrington, J. Scott & S. Wasserman (Eds.), Models and Methods in Social Network Analysis (pp. 270-316). New York, NY: Cambridge University Press. Jeong, A. (2005). Methods and Tools for the Computational Analysis of Group Interaction and Argumentation in Asynchronous Online Discussions. Paper presented at the Technology and Learning Symposium. 5-6 April 2006, New York, NY. Retrieved 10/12/2006, from http://dev2244801.sp01.fsu.edu/Research/Proposals/LearnTechSymposium/MethodsToolsAn alyzingInteraction_Jeong2005.pdf. Jones, R. (2006). Experience Models: Where Ethnography and Design Meet. Ethnographic Praxis in Industry Conference Proceedings, 1, 82-93. Retrieved 2/03/2007, from http://www.anthrosource.net/doi/pdfplus/10.1525/epic.2006.2006.1.82. Kirakowski, J. and Corbett, M. (1990). Effective Methodology for the Study of HCI. Elsevier. Retrieved 5/10/2005, from http://sumi.ucc.ie/sumipapp.html. Kreijns, K., Kirschner, P. A., & Jochems, W. (2002). The Sociability of ComputerSupported Collaborative Learning Environments. Educational Technology & Society, 5(1). Laghos, A. (2007). Assessing the Evolution of Social Networks in e-Learning. Unpublished doctoral dissertation at Center for Human-Computer Interaction Design, City University, London. Laghos, A., & Zaphiris, P. (In press). Social Network Analysis of Self-Taught eLearning Communities. International Journal of Knowledge and Learning. Laghos, A., & Zaphiris, P. (2007). Investigating Student Roles in Online StudentCentered Learning. Paper presented at the ICEM'07 - International Council for Educational Media Conference. 20-22 September 2007, Intercollege, Nicosia, Cyprus. Laghos, A., & Zaphiris, P. (2006). Sociology of Student-Centred e-Learning Communities: A Network Analysis. Paper presented at the e-Society 2006 Conference. 13–16 July 2006, Dublin, Ireland. Retrieved 22/01/2007, from http://www.soi.city.ac.uk/~zaphiri/Papers/eSociety.pdf. Lambropoulos, N., Kampylis, P., Papadimitriou, S., Vivitsou M., Gkikas, A. Minaoglou, N. & Konetas, D. (2008). Hybrid Synergy for Virtual Knowledge Working. In Janet Salmons & Lynn Wilson (Eds) (2008), Handbook of Research on Electronic Collaboration and Organizational Synergy, Hershey, PA, USA: IGI Global Publications. Lambropoulos, N. (2002). Collaborative and Individualistic Learning in a Situated, Computer - Supported Co-operative Environment for Multimedia Constructions. MA dissertation at the Institute of Education, University of London.

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 89

& Love. In press.veramenezes. Retrieved 14/05/2006.pdf. The Open University. Robinson. Y. (1997). (1999). Thesis Centre for Interactive Systems Engineering LSBU Page 90 . S.co. London. UK: John Wiley & Sons.Π. Korydallos. Preece. University of Queensland. (2006). Chichester. J. 183-197.pdf. d.Ε. Quality in distance education: Focus on on-line learning. NY: John Wiley & Sons. Ethnographically-informed Empirical Studies of Software Practice. Multiple Baseline Designs. Milton Keynes. (1999).. Mrvar.d. Bethesda. Multi-level Health Interventions and Programs. & Sharp. Greece. School of Information Technology and Electrical Engineering. Preece. 3rd Panhellenic Conference EEEP-DTPE "Education and the New Technologies". (2007.uk/htm/triangulation1. Cambridge: Cambridge University Press. A. CA: Jossey-Bass. Paiva. Sanson-Fisher. Studies in Educational Ethnography. MD. N.d. from http://www.open. from http://iet. K.. Lazakidou.D..Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 3: Research Design Mason. (2007). 30 September 1 October 2006. Nielsen. Moodle at the Greek School Network. Explorations in Methodology.gr. (1987). E. from http://www. (2000). Rice. http://e-learning. (2005). Communication Research. D. 2004. 1 May 2007 London Hopper 2007. Nooy. Nonnecke. USA: Academic Press. d. 23-46. Stamford. H. (2000). 4 -5 May. A. Unpublished Research. Online Communities: Designing Usability. (2002). & Retalis. Meyer. G.voicewisdom. Moodle@GSN (n.uk/pp/r. (Translated by Margarida Romero.. Milic-Frayling. (2002).au/archive/00003422/01/McGarry%20PhD%20Thesis%20 -%20Things%20to%20Think%20With. Unpublished Research.gov/Conf_Wkshp/Complex%20Interventions%20Workshop /Sanson-Fisher. Sasse.. V. Supporting Sociability. Retrieved 11/01/05. UK: The Open University.Τ.Ε.sch. Retrieved 15/02/2006.od.htm. 14(1).Ε. & Batagelj.mason/downloads/maeval.. J. British Computer Society. from http://obssr. Interaction Design: Beyond HumanComputer Interaction. New York. Πώς να Αξιολογήσουμε τη Διαδραστικότητα σ’ ένα Τεχνολογικά Υποστηριζόμενο Περιβάλλον Μάθησης. A. Electronic emotion: Socio-emotional content in a computer-mediated communication network. B. "Εκπαίδευση & Νέες Τεχνολογίες". 85-108.nih. Athens.. Rogers. (2005).. (1993). UK. R. San Francisco.cs. M. London. Usability Engineering. J.Π.com/cmc.uq. Boston. Unpublished Research.) Retrieved 22/05/2006.ppt. PhD thesis at the Department of ITEE. (2004). Segal. Retrieved 01/06/2006.sasse/thesis/Frontpage. Calidoscópio. R. Paper presented at the Women in Computing Research: the Hopper Colloquium. Or How To Get Lost Without Being Found Out. Things to Think With: Understanding Interactions with Artefacts in Engineering Design. McGarry. Retrieved 15/12/2005.ac. A pesquisa sobre interação e aprendizagem de línguas mediadas pelo computador (A research on computer-mediated interactions in language learning). A.). from http://www. B. Eliciting and Describing Users’ Models of Computer Systems. e. Methodological Triangulation. R. & Sharp. London.html.ac. 1 May 2007).edu. L.Where Information and Communication Technologies Meet. O. IET’s Masters in Open and Distance Education: What have we learned? CITE Report No 248. J. PhD Thesis. Information and Software Technology. G. Exploratory social network analysis with Pajek. V. JAI Press.ucl. (2005). Ph. Internet . 2.htm. Institute of Educational Technology (IET). University College London. H. Paper presented at the Workshop on Research Designs for Complex.. How can we evaluate interactivity in a technology enhanced learning environemnt? Paper presented at the 3ο Πανελλήνιο Συνέδριο Ε. Inc. H. Lurking in email-based discussion lists. from http://eprint. O. W. 3(1). 5-12.uk/staff/a. Petropoulou. PhD Thesis at London South Bank University. São Leopoldo. Δ. Massey.

ACM Transactions on Computer-Human Interaction. 87.. The Role of Story Cards and the Wall in XP teams: a distributed cognition perspective. (2000). from http://delivery. Boston. Boulder.uk/research/imc/hum/pubs/SharpEtAlCardsAndWallFinalVersion1%20PREPRINT. White. USA.idi. 7(1). Milan.colorado.ntnu.edu/cscl/167.tut. Paper presented at the International Conference on Network Universities and E-Learning. Italy. (2005). community and technical practices. Some social factors of software engineering: the maverick.. Experimental. (1996). Ph.. Handbook of qualitative research (2nd ed) (pp. B. from http://matriisi. 455-486). 711 January. 227-252. Plans and Situated Action: The Problem of Human-Machine Communication New York: Cambridge University Press.acm. (2006). H. Thesis Centre for Interactive Systems Engineering LSBU Page 91 .pdf. Retrieved 21/05/2006.pdf?key1=1083117&key2=8932278711&coll=&dl=&CFID=15151515&C FTOKEN=6184618.dcs... Missouri.D. Talamo. D. & Tervakari. Tedlock. International Journal for Urban and Regional Research. & Quasi-Experimental Designs for Generalized Causal Inference. Developing. T.. American Journal of Sociology. S. W. Zaharias. Segal. A. R. L. Suter.pdf. In N. 23. Unpublished Doctoral Dissertation at Athens University of Economics and Business. (2002). 3-43. Colorado. & Ligorio.qmul. (1987). (1976). 517-547. A.-M. (2003. H. Wellman. Paper presented at the HSSE'05 . Boorman. 114– 138. & Robinson. Suchman. Minnesota. B. J. S. H. & Campbell. B. Louis. Physical Place and Cyber-Place: Changing Portals and the Rise of Networked Individualism. Robinson. Computer-mediated communication: Impersonal. An Evaluation of the Usefulness of Web-based Learning Environments. Thousand Oaks.ee. Paper presented at the Workshop on Learning Communities in the era of Ubiquitous Computing.. Lincoln (Eds. A. Valencia. Animating and Analysing Virtual Communities of Practices.no/~divitini/ubilearn2005/Final/talamo_ubilearn. L. (2002). Sharp.ac. Communication Research. 23-28 July 2006. K.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 3: Research Design Shadish. (2005). D. 25(2). Focus Groups in Ethnography of Communication: Expanding Topics of Inquiry Beyond Participant Observation The Qualitative Report. Blockmodels of roles and positions. Paper presented at the Agile 2006. Retrieved 23/04/2006. Social structure from multiple networks: I. Denzin & Y.org/10. Retrieved 12/05/2006.htm. Talamo. (2000).1145/1090000/1083117/p11sharp. 13 June 2005. & Furniss.). Sharp.The Evaluation Tool into the Portal of Finnish Virtual University. Interpersonal and Hyperpersonal Interaction. C. 16 May 2005. & Breiger. Paper presented at the CSCL'02 Conference. B. H. Creating Creativity: User Interfaces for Supporting Innovation. from http://www. (2001).pdf. Retrieved 21/12/2005.. A. K. Retrieved 12/05/2006. J. Cook. A. B. from http://www. H.fi/arvo/liitteet/usefulness_of_web. 8-9 May 2003). Shneiderman. St. E. USA. A. A Usability Evaluation Method for E-Learning Courses. Walther. (2004). (2000). from http://newmedia. Ethnomethods as resources for developing CVEs in the ITCOLE project. Minneapolis... Spain. P. Ethnography and ethnographic representation. MA: Houghton Mifflin. NJ: Sage.2005 Workshop on Human and Social factors of Software Engineering. M. 5(1 & 2). Silius.

Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context 4 • • • • The Research Context: Ethnotechnological inputs and preliminary studies Key Topics Covered in This Chapter: Understanding the research context The Greek teachers’ background and characteristics The intentions of the Greek educational authorities The Greek School Network and the e-learning environment provided to the Greek teachers Chapter 4 reveals aspects of the research context with the aid of ethnotechnology. background. Thesis Centre for Interactive Systems Engineering LSBU Page 92 . Ph.D. The chapter also sheds light to the Greek teachers’ needs. One of the tools was Moodle@GSN. and tools provided for the Greek teachers’ professional training and development. targets. and characteristics as well as their overall activity at Moodle@GSN. the e-learning platform supported by the Greek School Network. This is connected to the Greek educational authorities’ intentions.

called E-mint (http://emint.1. 3.etutorportal. the Greek English language teachers (PEKADE. E-mint is Ph.primarymusic.gr).gr/). the e-tutors. The directors and the engineers were not Greek teachers. finding best practices for communication and social interaction. and implications for design.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context 4. 2.passionforlearning.net/) and “e-learning fundamentals” (June 2007. and the Greek authorities. and 6. 4. the European projects “E-Tutor” (June 2006.. the Greek teachers with a special interest in music (EEMAPE. The Native’s Point of View. http://www. background and characteristics The participants in e-learning at Moodle@GSN were the directors. http://www. The e-tutors had a twofold role. the e-learners.gr). Because some parts of the jigsaw were not found in Greece. the primary teachers’ training programme conducted by the Greek Pedagogical Institute within the Greek schools (September 2006).http://fecone. observant participation in other groups was considered necessary.1. 5.1 INTRODUCTION The main objectives of this chapter are: • • • understanding the context from the Greek teachers’ viewpoint.eu/). http://pekade.eeep. Thesis Centre for Interactive Systems Engineering LSBU Page 93 . There were no Greek teachers’ online community managers at the time of the research and I was a member of the Online Community Managers Association. the moderator and the e-tutor. the Greek association for the valorisation of ICT in education (EEEP. http://www. Thus the following projects were also used to provide information: 1. Access was provided by the researcher’s colleagues and there were discussions with the organisations’ directors as well as members. the engineers as technical support.D. participation in several conferences with publications in collaboration with other Greek teachers. 4.org/).

The Greek teachers were not familiar with the new technologies: soft skills for online collaboration were almost absent. and absence of professional guidance.gr/forum/). and technical level. and a community management framework (Appendix I). previous experience and familiarization with ICT.pekade. formed in December 2003). training or absence of training on the use of computers.D.) The overall findings suggested changes on a social cognitive/learning. lack of opportunities. Professional help and support was not available as regards e-learning. The main reasons were the lack of training. gender.clab. expensive rates for the internet access in Greece.95%) responded to a questionnaire in July and August 2004.eeep. the ability to incorporate learning derived from discussion forums for educational tasks and working conditions. http://www.edc. I also participated in the Moodle-based Leonardo DaVinci project ‘e-Tutor’ (http://pegasus. soft skills in online environments. 2005b. Thesis Centre for Interactive Systems Engineering LSBU Page 94 . years of teaching. were: (a) organisational. and the potential for professional development via e-learning. the Internet connection from the schools. Moodle modules were not facilitating the communication gaps. Fourteen out of 61 participants (22. and communication was fair. thematic analysis revealed a template for writing online messages. Among other results.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context widely known for members’ genuine interest and creative inputs in the field of online communities. participants’ spare time and time available for training.gr. writing and typing skills. The results revealed that the Greek teachers had difficulties in integrating new practices learned from online seminars in their teaching and learning practice.uoc. The factors that appeared to influence members’ participation. lack of soft skills. related to educational authorities. (b) school-based.gr/ite/) and the Greek English language teachers’ forum (www. (The results of these studies have been published in Lambropoulos 2005a. I acquired the opinions of the ‘Greek Primary Teachers’ Association for the Valorization of ICT in Education’ (EEEP. help and support. Lastly. and lastly. and personal characteristics. (c) personal: age. The aims were to identify: • • • • the Greek teachers’ ability of working online. 2005c. and (d) “the real world”: for example. One interesting observation was the importance of timing regarding 4 spots in a community life-span: Ph.

3.D. for example. One to two days after the registration. Thesis Centre for Interactive Systems Engineering LSBU Page 95 . Maintenance of the community These stages appear to be important. One week after the registration – towards full participation. initiations and interventions need to be applied after the first week in order to allow time for the e-learners to familiarise with the system and the community. Registration – enrolment and initial contact with the community. The next section investigates the context of educational authorities and GSN. 2. and 4. Ph.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context 1.

they teach different subjects and have different roles in the Greek education system. Moodle was used for the elearning service to host and distribute digitized e-lessons. In the case of the Greek School Network (GSN). More particularly. Evidence refers to the first distance education students registered at the British Open University in 1970 Ph. Organisation of the Education System in Greece 2003/04 (Euridice. GSN asked e-tutors and teachers to participate on a voluntary basis. there were problems. The Greek Ministry of Education and Religious Affairs introduced GSN as the educational intranet to provide certified telematic services for all schools and teachers in Greece. secondary state schools. legislation (Law#3328. however. and adult centres (Figure 4. Thesis Centre for Interactive Systems Engineering LSBU Page 96 . For example.D. they could not acquire a valid certificate because associated legislation was absent. 01/04/2005) has prohibited online certification and accreditation provided by educational organisations. GSN initially attempted to provide an online environment for Greek teachers’ training and professional development but the legislation and pedagogical frameworks were not there to support teachers’ involvement.2-1. for teachers with different specialisations working in primary. 2003:6) The Moodle@GSN teachers are located within the black arrow area.2 IDENTIFYING INTENTIONS Intentions determine the stakeholders’ goals and acknowledge the real need and leads towards adaptation of the right attitude to accept and incorporate change. E-Learning at GSN aimed to support Greek teachers’ life-long learning. passive participation may be the real need or a circumstance serving as visible evidence of a different need.2-1): Figure 4.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context 4. unless this was part of their constitution so e-learning was not considered as part of the national policy.

gr/general/history. restrictions in school autonomy. Lack of new capabilities may result in inability to communicate their subjects with the students. a Greek MP. and technical level. the basic characteristics of the Greek educational system and the official educational policy was centralism.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context http://www. there is difficulty apply for a job in a governmental organisation if the degree is acquired via distance learning (personal communication via email with Efthimios Kalaitzidis. representative of the Communist Party.eap. shared vision. MSc graduate with special needs from Manchester University. and legislation.gr/modules/news/article.uk/about/ou/p3. The right attitude. Tsetsilas (2006) reported: • • • • teachers’ training needs at school level were concerned to the social relations and collaborative activities within the school environment. Alekos Alavanos. and • certain parameters in the school environment required training. and legislation to adopt change were absent. there was lack of self-esteem and self-confidence derived from insufficient training (Tsetsilas refers to “insufficient cognitive equipment”. practical application of innovations as part of scientific and professional training. insufficient basic and ineffective professional training.D. conferences. cognitive/learning.ac. Evidence to support this observation came from a recent PhD on investigating Greek teachers’ needs and training design. Unclear criteria to provide rewards have been reported as a blocking factor for e-learning quality (E-Quality project. Even today. teachers needed support and training in collaboration. Tsetsilas found that teachers currently exhibit insufficient professional skills on a social and collaborative. and emailing lists. 08/12/2006). 2006).php?storyid=777. This was an observation in discussions with the teachers in schools. technology.shtml. reported this problem to the Greek Parliament (24/11/2006 - http://www. The first students registered at the Greek Open University in 1998 (http://www. last access 05/06/2007). It is interesting to see that Ph. Tsetsilas proposed that an in-service training program could be planned and organised according to the training needs and suggestions of the Greek teachers themselves. Thesis Centre for Interactive Systems Engineering LSBU Page 97 . 2006:6).syriza. In his speech after the Greek teachers’ 6 weeks strike. UK. he said that everyone involved in Greek education realises the need for change. The Greek teachers talked about absence in governmental planning on pedagogy.open. seminars. and lack of communication and collaboration within their environment.htm).

Therefore a risk analysis was needed (Appendix II). However. Thesis Centre for Interactive Systems Engineering LSBU Page 98 .D. and technical level. there is space for improvement on a social. cognitive. it appears that there were discrepancies between the stakeholders’ goals. Overall. Ph.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context the Greek teachers’ demonstrations continued every Wednesday until this section was written (18/02/2007). common ground and right attitude are not established.

D. http://www.3. 2008). MOODLE@GSN: Moodle at the Greek School Network The Greek School Network (GSN) is the educational intranet similar to the British JANET (Vivitsou et al.1-1): Moodle@GSN Figure 4.sch.3 PLANNING Ethnotechnology can capture relevant information for planning in e-learning engineering.moodle.1-1. evaluation of various open-source learning management systems led its developers to the open source package Moodle (Modular Object Oriented Dynamic Learning Environment.gr). Moodle@GSN is an autonomous. 4.3.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context 4. Thesis Centre for Interactive Systems Engineering LSBU Page 99 . self-organised service for teachers’ distance education training (Figure 4. With regards to the e-learning service. Moodle@GSN research context Ph.1. The Greek teachers played the roles of both the course creators and students on a voluntary basis. The GSN e-learning platform is called Moodle@GSN (http://e-learning. The first section described the Greek teachers’ context..3.org). this section will refer to the e-learning environment.

This means that 19% of the total number of registered users exhibited some kind of activity (e.3. More specifically the categorization was as follows (Figure 4. visiting the online courses.350 members. a baseline study drew the line before and after any implementation in GSN based on descriptive and discourse analysis. there are 853 registered active participants. the categorization of the online course (Table 4. Lambropoulos. Yoga. on this date. The baseline was the 1st of November 2006.1-1. In a total number of 4.1-1): Ph.1. there were 1. Thesis Centre for Interactive Systems Engineering LSBU Page 100 .1. bird watching).1-1): Table 4.3.D. and 25 e-tutors. the offered courses had an open nature and were divided in two major categories.3. 2008).g. Then. Two studies appeared to be significant for its history. 2005 and 26 in 2006. Moodle@GSN started in 2003 with 2 online courses. the structured and well managed and the courses that were designed for a special interest (e. Konetas. Paraskevas & Grigoropoulos.1. posting in chats and discussions) in Moodle@GSN and 3% were the e-tutors. With the subject as the criterion.g. The first study was an overall quantitative evaluation until the 01/11/2006 (Vivitsou.3.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context 4. Courses categories and number of e-learners General Topic Familiarization with Moodle General Interest Greek Schools Network Training Greek Language Pedagogical Approaches IT Training Multimedia Open Office Courses Under Construction Total # Courses 3 6 3 1 2 10 2 4 15 46 #e-learners 16 98 59 57 124 1167 202 124 63 1910 It appears that there were 1.910 users registered online whereas 64 users were registered in the 14 courses that were under construction. Due to the fact that participation was on a voluntary basis. Nine more courses were created in 2004.910 e-learners in 46 Moodle@GSN courses.1 Users and Activity in Moodle@GSN Several observations and studies were conducted at Moodle@GSN as part of time-series design.1. in a total number of 46 courses.

it appears that there is counter analogy to the number of messages and the number of replies. Thesis Centre for Interactive Systems Engineering LSBU Page 101 . It also had material intended to improve the participants’ soft skills for effective communication and about collaborative e-learning principles (Appendix VI). resulting in 132 messages from the e-tutors and 74 from the e-learners There were 94 new topics of discussion launched by 34 e-tutors. and 56.000 messages sent by 138 participants. 16.3. 8 discussion threads started from the e-tutors and 18 from the e-learners themselves. The second intervention was to provide access to the new tools.D.1. Part of the collaborative e-learning intervention was the introduction of community management by the investigator. for example 19 Ph. Lastly.The following table describes the type and the number of messages.1-1.910 Greek teachers having a preference in IT training. The 1st of November 2006 (baseline) was the starting point of interventions.6%. as well as the use of pedagogical approaches. having 1. In 22 courses there were chats. 40 quizzes were activated in 19 courses used by 106 users in 197 efforts to solve them. there were 67 assignments in 18 courses from 47 users in a total of 94 submissions.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context Online Courses in GSN Familiarization w ith Moodle General Interest Greek Schools Netw ork Training Greek Language Pedagogical Approaches IT Training 1400 1200 1000 Learners 800 600 400 200 0 1 2 3 4 Multimedia Open Office Category Graph 4. In addition. From the overall 206 messages sent in these courses.5% were the messages with a reply.7% were the overall replies. Online course categories in Moodle@GSN It appears that in 46 completed and courses under construction there were 1. multimedia. in a total number of 206 messages. In addition. There were two interventions. collaborative e-learning and the use of tools based on the proposed underlying conceptual frameworks. It consisted of material providing advice on how to better use the platform including emoticons. The ratio of single messages 26.

There were 28 words in 6 threads. Ph.1-2.12): Number of messages and replies 30 25 20 15 10 5 0 1 2 3 4 5 6 7 8 9 10 Number of Messages Number of replies Graph 4.3. whereas 1 message produces 27 replies (Graph 4. The intervention stopped in order not to interfere with the study. The messages were sent during the period 26/03/2006 29/03/2006.1.1. The messages in the first course ‘Use of ICT in Religious Education (RE)’ on the discussion topic about the use of Power Point presentation were limited as regards the richness of the text. and the second after. And makes the class more interesting (use of audiovisual material) Participant D: Και δεν κουράζει τους μαθητές. they had to talk about their experiences and the Greek teachers responded enthusiastically. Thesis Centre for Interactive Systems Engineering LSBU Page 102 . The participants (all different individuals) built on each others’ threads and agreed on the importance of collaboration as follows: 1 2 3 4 Participant A: Η παρουσίαση είναι τρόπος παράδοσης αλλά και εξέτασης The presentation (ppt) is a way of delivery and examination. From the total number of 32 courses. One course was built before the intervention.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context messages had 1 one reply. two courses produced the aforementioned messages. Even though the discussion is limited there is evidence of collaborative learning.D.3. Participant B: συμφωνω I agree Participant C: Και κάνει το μάθημα πιο ενδιαφέρον (χρήση οπτικοακουστικού υλικού). Comparison between number of messages and replies This message was initiated to test the e-mint community managers’ suggestions.

1-2): Table 4.09/05/2006) there were 13 messages with no more than 2 sentences in each thread.1-2.. e-Learners’ posts and views in the 6 active courses E-LEARNERS’ POSTS AND VIEWS Online Courses 1 2 3 4 5 6 Initial Page ICT in R.1. built from the researcher with the help of two colleagues. Project Method Project Method ICT in R. A comparison demonstrates the rapid increase of the messages in the course “Project Method” after the initiation of the community management scheme (Table 4. The course under construction for this study appeared to have initiated a significant number of posts. In the second discussion of the same first course (22/03/2006 . they were reluctant to reply since they did not know each other (Participant F: ‘Βάλε φωτογραφία για να δεις βοήθεια. institutionalised by law (Greek Government Gazette 303 & 304/13-3-2003).1. Thesis Centre for Interactive Systems Engineering LSBU Page 103 .Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context 5 6 And is not tiring for the students Participant E: Ενδιαφέρουσα κ΄συλλογική συνεργασία Interesting & collaborative cooperation Participant F: ΠΡΟΠΑΝΤΩΝ ΣΥΝΕΡΓΑΣΙΑ MOST OF ALL COLLABORATION Participant F stressed the importance of collaboration using capital letters. Otherwise you don’t have a chance… (26/03/2006)’. This message triangulates the previous observations on interventions as regards the provision of professional help and the need for collaboration. Έτσι δεν έχεις ελπίδες.3. The number of visits and the number of messages sent were (Table 4. Marianna Vivitsou and Dimitris Konetas..3.3. Project Method Forums GSN News The Use of Ppt.1-3): Ph. It appears that the participants were exploring the environment.E..E.’ Add a photo to find help. e-tutors discussion Initial problems ICT in RE Projects4discussion Discussions GSN community Ppt use Intro What we want ICT in RE Hi everyone Total Users 4576 15 50 50 15 50 4576 Posts 7 5 4 6 7 5 34 Views 653 9 5 18 10 13 708 It appears that 4.. Next. The course “Project Method” was designed to respond to the Greek educators’ need for training on educational project management.D. the participants were invited to the second course on project management.1.. The participants were given educational material (samples exist in Appendix VI) and the e-tutors followed the online community management framework (Appendix I).576 users posted 34 messages with 708 views.

77 sent messages and 13 active participants.1-3): Activity in the Online Course 13.01/11/2006 The e-learners. produced 14 threads with more than 2. there were 8 topics.1.1-3.D. Thesis Centre for Interactive Systems Engineering LSBU Page 104 .06/10/2006 22/03/2006 .19/07/2006 04/06/2006 .3. Project Method GSN News The Use of Ppt. e-Learners’ replies and dates E-LEARNERS’ POSTS AND DATES Online Courses Forums Discussions Replies Dates 1 2 3 4 5 6 Initial Page ICT in R. 6% Number of Topics 28. 10% 8.1-3.3.23/02/2005 26/03/2006 . Project Method Project Method ICT in R. 71% were passive and 29% active participants with an average of 6 messages per poster. Activity in the online course It appears that the there was initial significant activity after the baseline and the first intervention. 28 new discussions.E.09/05/2006 17/10/2006 .29/03/2006 11/06/2006 .Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context Table 4. 22% Number of new discussions Number of sent messages Number of active participants 77. Overall.1. all different individuals.1.01/11/2006 10/03/2004 . e-tutors discussion Initial problems ICT in RE Projects4discussion GSN community Ppt use Intro What we want ICT in RE Hi everyone Total 27 5 7 9 12 14 74 10/03/2004 .000 words in total with the Project Method course among them with 30 replies between 04/06/2006 and 01/11/2006 presenting the following activity in the first week (Graph 4. it Ph.E. 62% Graph 4.3. From the total number of 45 participants.

interactions. were absent on a pedagogical and legislation level.1. The interventions in this research were the support of collaborative e-learning and its social aspects in particular as well as the introduction of a set of new tools to support passive and active participation levels. the studies were as follows (Table 4.D. It is evident that: • • • • • Professional development anchored in e-learning is one of both GSN and the Greek teachers’ targets. Considerations on a socio-cultural. Thus.1-4): Ph. The Greek educational authorities. the use of multimedia. The main areas of training were suggested to be IT training. responsible for planning and support. Thesis Centre for Interactive Systems Engineering LSBU Page 105 . The Greek teachers themselves attached importance to the key variances of collaboration and social interactions evident in their discussions in an explicit and implicit way. and the collaborative e-learning as such.3.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context appears that Moodle@GSN has potential to initiate an active e-learning community. The Greek teachers lacked appropriate e-learning training. and current pedagogical approaches. pedagogical and technological level were directed to supporting social interactions and collaboration in order to activate participation in collaborative e-learning.

D.1-4.3.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context Table 4.Discussion Online course: Introduction to Web Design Online course: Educational Project Management with Collaboration Tools Recommendations for new tools Online course: Educational Project Management with Collaboration Tools Ethnotechnology 1 1/07/2004 -30/08/2004 14/04/2004 -30/06/2004 21/03/2006 07/04/2006 EEEP mailing list E-mint online community managers Greek education mailing lists.Questionnaire Researcher’s Participation in the list . E-Learning Engineering for Moodle@GSN E-Learning Engineering for Moodle@GSN Date Sample Environment Intervention Activity Researcher’s Participation in the list . EFQUEL list Moodle@GSN Greek Moodle developers lists PSD e-mail database Mailing list n/a 2 Mailing list n/a -Learning Management -Tools Learning Management 3 Moodle@GSN & E-mmersion 4 01/11/2006 30/11/2006 06/03/2007 -13/03/2007 01/03/2007 -31/03/2007 Moodle@GSN 5 E-mmersion Tools -Learning Management -Tools 6 Moodle@GSN & E-mmersion Ph.1. Thesis Centre for Interactive Systems Engineering LSBU Page 106 .

PA: Idea Publishing.D. M.intelligenesis. Thesis Centre for Interactive Systems Engineering LSBU Page 107 .O... Shackel. The Journal for Open and Distance Education and Educational Technology. J. 29-56.). Richardson (Eds. Lambropoulos. A. (2005). 21(3). Lambropoulos. . NY: Cambridge University Press. N. Καταγραφή επιµορφωτικών αναγκών των εκπαιδευτικών και σχεδιασµός προγραµµάτων επιµόρφωσης: Η µελέτη περίπτωσης µίας σχολικής µονάδας της Πρωτοβάθµιας Εκπαίδευσης µε το µοντέλο D. P. Collective Information Practice: Exploring Privacy and Security as Social and Cultural Phenomena. Las Vegas. 2. C. In B. Lambropoulos. (1991). Fernandes.N. Vivitsou. design and evaluation. NV. N.. B. Patras. 22-27 July. (2005). 26-39. & Anderson. N. & Montalvo. 597-607. Golafshani. Human-Computer Interaction. Understanding Reliability and Validity in Qualitative Research. 21–37). Greece. In S. "Open Education". D. Lambropoulos.).pdf. (2005). Reporting teachers' training needs and design training schemes: The case study from a primary school using D. Human Factors for Informatics Usability (pp. E. (2003). Retrieved 05/09/2005. N. 319-342.I. The Qualitative Report. E-Quality Synthesis Report: Experience-based Quality in European ODL.O. I. Paradise lost? Primary Empathy in Online Communities of Interest and Ways of Use. 8(4). International Journal of Continuing Engineering Education and Lifelong Learning (IJCEELL). Tsetsilas.. framework. Ph. Paraskevas. 346-348). 18(1). (2006). Usability – Context. EEEP Online Community of Practice: The First to Boldly Go in Greece. The Project Method e-course: the use of tools towards the evolution of the Greek teachers’ online community.. & Grigoropoulos.. Shackel & S.eu/nikiweb/PDF/05/lambropoulosOnlineEmpathyVeg as05Research. New York. (2006). in the 11th International Conference on Human-Computer Interaction 2005. model. Online Empathy: Social Consequences for Community Building. (2008). Konetas.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context REFERENCES Dourish. Encyclopaedia of Virtual Communities and Technologies (pp. N. from http://www. (2006). definition. Hershey. Paper presented at the 1st Conference on Online Communities and Social Computing.N. Dasgupta (Ed. Unpublished research at the Open University. K.I. M.

5: Tools & Evaluation Techniques 5 • • • • • Tools & Evaluation Techniques Supporting Collaborative E-Learning Communities Key Topics Covered in This Chapter: Presentation of the first models for tools and evaluation techniques Initial evaluation of tools and evaluation techniques Applying guidelines and heuristics from feedback for design Implications for e-research design Iterative design and e-research design Chapter 5 presents the initial design and evaluation related to the conceptual frameworks. one with international participation. Thesis Centre for Interactive Systems Engineering LSBU Page 108 .Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. and the research design. and one focus group with Greek teachers who are also Moodle developers.D. Ph. the tools prototypes. one with the Greek teachers. Guidelines and heuristics obtained from feedback on the proposed concepts and tools. the evaluation techniques. Three studies are presented.

1 The tools were developed by Intelligenesis. A chatbot. social networking and e-learning. 3. 5: Tools & Evaluation Techniques 5. a British company specialising in market research. Thesis Centre for Interactive Systems Engineering LSBU Page 109 .D. 2. A keyword search facility that was not used. and 3 tools were found which were either not working for the Greek context or not suitable to be discussed here. that initiated ‘safe’ conversation for passive participants (Lambropoulos. The collaborative e-learning tools to be discussed here are: 1. Participation levels evaluation graphs 3. ASCII was found difficult to be represented in Greek characters. Ph. A real-time pedagogical usability questionnaire that was not particularly used. was built in ASCII (American Standard Code for Information Interchange). 2007). it was a great idea but current systems are not built to support such tools. Tools to structure collaborative e-learning The following sections refer to the design process of these tools. these tools were: 1. Social network analysis tools 2.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. According to one of the participants.1 INTRODUCTION Six tools 1 were initially considered for integration in Moodle.

5: Tools & Evaluation Techniques 5.D.1.2 COLLABORATIVE E-LEARNING TOOLS This section will present the iterative design process from creating the first designs to the actual re-design and implementation.2. 5. Block 2 tested the community management as part of collaborative learning framework. two of them will be discussed here (Table 5.-1. Ph. Iterative Design Blocks ITERATIVE DESIGN Blocks 1 21/03/2006 -07/04/2006 01/11/2006 -30/11/2006 06/03/2007 -13/03/2007 01/03/2007 -31/03/2007 Sample Greek education mailing lists. and will only be used for comparison. Design for initial design: Create prototype testing by the e-learners Simplicity in design helps learners to concentrate on their objectives (Boy. Targeting the sense of community development and socio-cultural learning. the tools were designed to support presence and co-presence awareness. 2007. Alty.2. participation evaluation. This means that one of the targets in e-learning design should be simplicity in order to maintain e-learners’ cognitive stability.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. EFQUEL list Moodle@GSN Greek Moodle developers lists PSD e-mail database Environment Moodle@GSN & E-mmersion Intervention -Learning Management Tools Learning Management -Tools -Learning Management -Tools Activity Online course: Introduction to Web Design Online course: Educational Project Management with Collaboration Tools Recommendations for new tools Online course: Educational Project Management with Collaboration Tools 2 Moodle@GSN 3 E-mmersion Moodle@GSN & E-mmersion 4 Block 1 and 3 will be discussed in this chapter. Block 4 will be discussed in the next chapter as the main study. and collaborative e-learning. Four projects (blocks) were developed. Thesis Centre for Interactive Systems Engineering LSBU Page 110 .-1): Table 5. 1997).2.

The process is as follows (Figure 5.2. Thesis Centre for Interactive Systems Engineering LSBU Page 111 .2. Social interactions can be depicted in SNA Nodes and Centrality windows as Visualisation Interactions tools (VIT) open as Java applets using algorithms (Figure 5. The Java applet uses JUNG library to create a sparse graph.2.2.1.1. 5: Tools & Evaluation Techniques 5.1-2): Figure 5.D.1-2.1-1): (a) (b) Figure 5.1.2. Visualisation Interactions Tools (VIT) production line Ph. Visualisation Interaction Tools (VIT) Nodes and Centrality Each node represents a unique user/learner and the number of messages is indicated as numbers as well as on the interaction lines. JUNG Social Network Analysis (SNA) open source software was found to be suitable to acquire real-time results.1-1. the more the messages. Presence & Co-Presence Awareness: Visualisations Interactions Tools The literature revealed that observing and analysing social interactions can increase collaborative learning understanding. where users are represented as vertices and their relations as edges.1.1.1. the thicker the interaction lines.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch.

whereas GD library was not activated.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch.2. It really gives me the willies when that peculiar X appears in the top lefthand corner’.D. Participant D1 expressed his frustration: ‘It keeps driving me bananas all the time. 5: Tools & Evaluation Techniques Moodle was patched to create a link which passes the forum ID to a Java applet.1. JPEG and GIF images and is commonly used to generate charts on the fly. Participation Awareness: Participation Evaluation Tools Participation awareness is related to presence and co-presence awareness measured by course participation and discussion participation graphs.2. 5. Course and individual participation levels graph Ph.2-1. with all calculations made such as user relations and message count. was not updated. e-tutors and e-learners would be able to observe the sociability of the human network.2-1): Figure 5. Thesis Centre for Interactive Systems Engineering LSBU Page 112 . Not being activated after the upgrade resulted in major problems with the use of these tools. Then XML passed to the JAVA applet and the JAVA applet built a graph based on information generated from the XML file.1.2. The hosting company upgraded PHP4 to PHP5 and constantly reconfigured PHP5 and Apache. The applet executed PHP script that fetched all required information from the Moodle database creating an XML file.1.2. Participation Evaluation Tools (PET) are two graphs showing the participation levels for the overall course activity and the e-learner’s activity in one discussion (Figure 5. GD creates PNG. Despite the problems. the graphing software creator. the images did not display correctly and were represented by an X. JPGraph.

5: Tools & Evaluation Techniques The graphs were built as follows (Figure 5.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. Ph. 5.2. explain.D. explore.1.1. Thesis Centre for Interactive Systems Engineering LSBU Page 113 . The initial tagging was inform. Participation evaluation graphs production line Moodle was patched in order to auto-generate links with the course ID that pointed to PHP scripts in order to enquire users’ database accessing the specified course. parsed information and passed it to JPGraph library in order to perform an output on the screen in real-time.1.2-2.3-1). Then it gathered all statistical information from databases.3. Structuring Collaborative E-Learning: MessageTag The tool to structure collaborative e-learning followed Collaborative eLearning Episode (CeLE).2. and other. drop down menus and message tagging or openers have been designed as an initial requirement (Figure 5.2-2): Figure 5. It also contradicts the existing models and tools.1.2.2. evaluate. was structured onto a drop down menu that was designed and located after the posting text box. This was because posters may not recognise the type of the reply unless they finished their message.

Ph. In addition to the tool. From a research viewpoint..D.2. written in PHP and HTML and can be incorporated in any Post page.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch.1. it was thought that there will be excessive iconic information on the interface and. the use of text would not interrupt their reading but can be part of it. SQL queries could be performed to get different statistical information based on message types or complex statistics where types are only a part of a query (for example keyword and message type that was already implemented but not used). 5: Tools & Evaluation Techniques Figure 5. Overall. However. it is anticipated that the tools can broaden and illuminate the space between human-human interactions so as to open space for reflection for the e-learners. Moodle was patched to allow all messages stored in the database to be tagged. The use of icons was considered instead of text as used in the “vicarious learner” project (Lee et al. Initial Design for Message Tagging The script for such a tool is very simple.3-1. 1999). since the e-learners actually read the title and the actual message when they post. the tools could expand our understanding on the sociocognitive aspects of collaborative e-learning. Thesis Centre for Interactive Systems Engineering LSBU Page 114 .

2. and was announced on EFQUEL front page.2.qualityfoundation.D. Help was sought and found from the European Foundation for Quality in eLearning (EFQUEL) (http://www.1. 19% Portugal Spain 43. 2% 1. The online course in the experimental environment was supplementary to the web design course in Moodle@GSN. 2% 2%1. Participating countries Ph.2-1): Participating Countries 1. 64% Sweden Switzerland USA Turkey Graph 5. The call was also sent to the most popular Greek educational mailing lists. 5: Tools & Evaluation Techniques 5. The remaining 56 participants are shown in the following graph (Graph 5. 2%1. 1. 2% 1. only 5 out of 128 registered users responded to the call (28/01/2006). 2% Greece UK Hungary Germany India 13. EEEP and PEKADE as well as EFQUEL.2.org/).2-1. 2% 2% 1.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 1% 2.2. From the 68 initial subscribers 12 did not enrol. The E-mmersion Block The first block for iterative design to acquire participants’ opinions was conducted from 21/03/2006 to 07/04/2006. Thesis Centre for Interactive Systems Engineering LSBU Page 115 . The aims were: The call for participation was sent initially to registered users in the web design course at Moodle@GSN.

19%). The total code network view was depicted in ATLAS. The demographics. the web design course for the Greeks and a discussion to evaluate the tools for all.D. Also. and questionnaire statistical analysis. Methodology: A questionnaire was distributed and the collaborative e-learning analytical framework was used for messages analysis. Thesis Centre for Interactive Systems Engineering LSBU Page 116 . The suggested research methodology proposed messages analysis. messages quantitative analysis and a collaborative elearning episode example from the web design pool are presented in Appendix VII. 64%) and the UK (n=13.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. however. The participants were divided in two groups and worked in two research pools. English and Greek. Four CeLEs were identified in the Web Design pool with the Greek teachers.ti (highlights added by the author): Ph. The participants were free to communicate in both languages. there was a ‘Living Room’ for all participants. English was proposed in the tools evaluation discussion. logging. 5: Tools & Evaluation Techniques The majority of the participants were from Greece (n=43.

2-1 Total Codes Network in the Web Design pool Ph. 5: Tools & Evaluation Techniques Figure 5.D.2.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. Thesis Centre for Interactive Systems Engineering LSBU Page 117 .

1-1. the tools evaluation pool. The next section presents the results from the second pool relevant to this chapter. it appeared that social cues can be a separate attribute for the CeLE framework. Tools Evaluation Pool In the tools evaluation pool there were 7 low. exploration. evaluation. The two tools were located on Ph. new ideas for co-construction.2.D.2.ti) In addition to the pattern for writing online messages from the E-mint study. disagreement was collapsed with agreement as they are opposite without being separate. a second pattern was identified and used (Appendix A_I_2).2.2. 5: Tools & Evaluation Techniques Figure 5. Participants and number of messages (ATLAS. 2 medium and 1 high activity user and a total of 33 messages were sent. explanation. Additionally. Visualisation Interactions Tools (VITs): There were two graphs providing information about the locus of participants within one discussion in a forum. 5. Thesis Centre for Interactive Systems Engineering LSBU Page 118 . based on Social Network Analysis and the levels of participation.2.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch.1.2-2 shows the code network for the 4 CeLEs in the Web Design pool. and other.2. The codes were: initiation.1-1): Figure 5.) (Figure 5. conflict. (The high activity user A1 was the Swedish Moodle developer. agreement.2. After the analysis.

2.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch.2.2.2.2.2. Location of VIT on the discussion forum Figure 5.2. VIT Centrality Ph.2.1-4.1-3.2. 5: Tools & Evaluation Techniques the top of discussion (Figure 5.1-2) and were VIT Nodes (Figure 5.1-3) and VIT Centrality (Figure 5. Thesis Centre for Interactive Systems Engineering LSBU Page 119 .D.2.1-4): Figure 5.2.1-2.2. VIT Nodes Figure 5.

5: Tools & Evaluation Techniques In the discussion in English language. some applications / tools to be improved (e. GF. E1 expressed her unfamiliarity with the tools as well: ‘I'm not familiar with what they stand for. Thesis Centre for Interactive Systems Engineering LSBU Page 120 . Also.2.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch.’. AM. Also: Some concepts need to be defined (e. so she proposed passive participation to be visible on VIT.g. VIT tools.2. They seem nice. A1 suggested that learners ‘can lurk intensively and actively and learn more than those visibly active’.g. I would use it in me lessons!!’. though!’.g. A1 was initially sceptical on using algorithms and Java compared to PHP applications.2.2. monitor/moderate group work and see to it that the assignments are distributed evenly and that credit is given fairly… [resulting in] better learning. some to be further explored (e. philippa’s repertoire of responses). The participants had to face new concepts and work with new tools without previous experience.g. message tags) and the moodle environment itself needs more elaboration (e. M1 liked VIT as ‘vit nodes also work with me and i agree they can be very useful for class participation statistics (quality measurement)’. what are ‘best learning achievements?”). and D1 expressed the same problem. activity graphs. listening activities?)’.1-1 to 5. ‘through this experience I realised (once again) that there are some many things left for me to ‘learn’ in the LMS management area and the evaluation of technology-based learning. e-/web/online learning etc)’. M1 as an online tutor she ‘would like to know which students are active and participate in the e-lessons and which not. Participant M1 suggested: ‘…i (at least tried to) familiarised with new concepts (quality. However.1-3): Ph. Participation Evaluation Graphs: Two types of graphs were created associated with the online course and the individual e-learner describing the e-learning community progress (Graphs 5. so I'm afraid I can't be of much help here. He was not familiar with sociogrammes but found them interesting and mostly useful by moderators: ‘It would be interesting to see who is active and in what role and what contributions people make.g. I think it is the teacher's job to e. she could not understand the concept of centrality and had to read the instructions.D. M2.

1-1. only two members appeared in high and medium activity during the first days (31/03/2006). Thesis Centre for Interactive Systems Engineering LSBU Page 121 .1-3.g.2. Thus the moderator sent an email suggesting that all participants need to contribute to the discussion.’ To A1 the kind and the quality of the activity is missing ‘or whether anything.2. that there are 5 other users on low level and 1 user. A1 said: ‘they show that e. E1 found difficulty in interpreting them.2. 31/03/2006 – 5/28 e-learners Graph 5.1-2 (04/04/2006).2.2. 5 more participants appeared in the low participation graph 5.2.D.2. This intervention appeared successful.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. my activity level in the Induction course is medium. 04/04/2006 – 10/28 e-learners Graph 5. probably admin with high level. 5: Tools & Evaluation Techniques Graph 5. M1 said that such a tool is not helpful: ‘for a newbie like me passive participation very often proves to be very constructive’ (M1). and if so Ph.2. 08/04/2006 – 10/28 e-learners As viewed in the previous graphs.1-2.

2. may have been learnt.2.D.1-1) (only the participants’ messages were counted): 2 At this point in the study. where bar graphs depict e-learners’ overall activity rather than distinguishing passive and active participation levels.2. the graphs were effective.’. Thesis Centre for Interactive Systems Engineering LSBU Page 122 . M1.’. In addition.1-13).1-5): Figure 5.2. Similar tools exist in other learning management systems such as Blackboard. the first column of participation was unstable. 5: Tools & Evaluation Techniques what. indicating the need for an initial account confirmation to ensure accurate and reliable results. said: ‘how can you.2. passive participants were not shown on the graphs (participation column should have shown 28 participants in Graphs 6. so log files provide more valuable information. Collaborative e-learning tool MessageTag: The participants were encouraged to use a drop-down menu for tagging their messages 2 (Figure 5.2.2. talking about the sense of community.2. Later it appeared as on option in conjunction with agreement Ph. There were some issues due to systems’ instability. from in this way graphically presented statistics. From an e-tutor’s and researcher’s viewpoint.’.1-5. MessageTag The tool was used more in the Evaluation Pool than the Web Design pool as follows (Table 5. it was not possible to retrieve their log files. E1 said: ‘they are more useful to teachers than fellow students.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. Other problems were related to more than one login names registered for each user exhibiting multiple appearances. disagreement was coded under evaluation. can conclude that someone is 'immersed'.

11 (17. and 2 (3. Because the tool was integrated in discussion.’. In the introductory forum (Living Room) 17 out of 35 messages were tagged (48.5%).4) as explorations.2.6 26/47 55.3%) and less in the Greek teachers’ only pool with 20 out of 73 messages tagged (27. Perhaps the text of each tag could be accompanied by an icon. A1 found this tool interesting whereas E1 said that she never saw a tool like this before so she did not know the purpose it served or how it can be used (‘for statistics maybe?’). M1 said that ‘each tag provides a thematic categorisation of the reply.1) as evaluations.1%).6 25. it might prove more Ph.4 11.1 40. 7 (11.D.3 20/73 27. Use of MessageTag USE OF MESSAGETAG Pools Living Room Overall replies 35 CeLE Attributes Inform Question Explore Explain Inform Web Design Question 73 Explore Explain Inform Tools Evaluation Question 47 Explore Explain Evaluate Inform Question Total 155 Explore Explain Evaluate # 9 2 4 2 4 8 5 3 12 6 2 4 2 25 16 11 7 2 63/155 39. Based on the tagged messages. A help file next to "message tag" would be useful. AM and M1 agreed with E1. 5: Tools & Evaluation Techniques Table 5. 16 (25. Thesis Centre for Interactive Systems Engineering LSBU Page 123 .3 17.3 17/35 48. She wondered whether ‘users read messages according to their tagging (from a long list of messages).3) messages were tagged as questions.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch.1 3. the log files are the same with the forum and discussion logs. personally i read all the replies to the topic i'm interested in no matter what the tag is.5 Tagged messages Percentage It appears that MessageTag was mostly used in the tools evaluation pool with 26 out of 47 messages tagged (55.3%).1-1. perhaps because what matters is the content and not the title.2. the inform tag was used more than the other tags (39.6%) and the evaluate tag less (3.1%) as explanations.

D. Lastly. G1 agreed with M1. This will have an impact on the e-learning quality.e. This may result in the increase of low and medium active participation levels as in the previous example.’. A1 recommended some additional tags for consideration: • • • • • • • • • add/develop ask for additional information ask for clarifications confirm approve agree disagree offer conclusions point to resources Overall. then that would certainly be a 'huge step for humanity'’. as such. Nonetheless. can improve tools’ usability and usefulness. He suggested that this tool could be involved into becoming an instrument ‘for learners to enhance their literacy i. In this way the elearners participants can moderate their own messages and use the tool to facilitate their critical thinking skills development. or misunderstand the crucial points. 5: Tools & Evaluation Techniques useful for younger learners' induction to the system though’. becoming better e-readers/listeners and e-writers/speakers. and the participants thought that the tools were more helpful to e-tutors. it appears that some issues influenced MessageTag functionality: the instructions were not helpful. This finding stresses the importance attached to the difference between information provision and knowledge acquisition and that the tool can make this difference visible. the e-tutors can support this process. this meaning that they are likely to miss. it was evident that the use of MessageTag attributes was descending as the level of reflection and higher order thinking was ascending. it was the first time the participants used such tools. Thesis Centre for Interactive Systems Engineering LSBU Page 124 . there were severe technical problems. All learners are not good at 'deep' learning.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. Ph. discussed in the next section. A1 added that the tool would have worked better if it could show the very structure of the discussion as not many people are capable ‘of seeing such structures by themselves. implications for design.

D. and • date and time were added. 5: Tools & Evaluation Techniques 5. Visualisation Interactions Tools (VIT): Following A1’s recommendation the lurkers were added in the VIT centrality that provided more space on the interface. Thesis Centre for Interactive Systems Engineering LSBU Page 125 .2. these messages were not counted in the graphs. A disadvantage is that the Java applet restricts login names’ clickability (Figure 5.2. Application of guidelines and heuristics from feedback in design Based on participants’ recommendations and the researcher’s experience four changes were made on Visualisation Interaction Tools (VITs).2. participation graphs. Ph.2.2. because e-tutors’ messages change the results.2. They can be counted if the e-tutor wanted to appear as one of the participants so the messages were calculated again.2.2-2): • • overall participation column was corrected.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch.2. Lurkers overall view in VIT Centrality (right) Participation Evaluation Tools: The two graphs for overall course participation levels and discussion participation levels had to depict all participation levels for all participants so there were three changes (Figure 5.2.21): Figure 5.2-1. and MessageTag.

grey) Medium participation (red. yellow) The avatars were integrated on the threaded discussion view. blue.2. This means that the participants can compare collaborative e-learning structures and participation levels in one glance. Lastly. the log files will be the same as the discussion forum log files.2-2. the avatar was designed as follows: • • • • Zero participation (grey) Low participation (red.2-3): Ph. conflict was a subcategory used for coding only along with disagreement.2. grey) High participation (red. So the new collaborative e-learning episode analytical framework was redesigned (Figure 5. Agree and Summarise. Thesis Centre for Interactive Systems Engineering LSBU Page 126 .2.2. A1 suggested adding conflict but the decision was made. the avatar appeared to be a good solution. and thus indicates different energy levels in the human body (Isaackson. a new tool thought to depict participation levels and an overall view. 5: Tools & Evaluation Techniques Figure 5.D. 2003). Redesign of participation graphs Participation Avatars: Based on A1’s proposition. After discussing this proposition with the graphic designer MR and supervisors. blue. Based on the notion that the avatar is an Indian concept. two more values were added on MessageTag. MessageTag: First.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch.

D.2. Ph. several changes were made so the questionnaire could be sent via email: • Three sections were elaborated and added: o the initial sense of community identification was based on empathy. and Pedagogical Usability targeted usability and utility in detail. An absence of collaboration with the Greek educational authorities on the implementation of the tools. a Sense of E-Learning Community Index was adapted. In this way the e-learners would be able to see the internal structure of the message. led to the addition of an extra question targeted to Greek teachers’ communication and collaboration with the Greek authorities. o o • the learning section was expanded.2. Thesis Centre for Interactive Systems Engineering LSBU Page 127 . 5.2. 29%). 5: Tools & Evaluation Techniques Figure 5. a messages overall view in the threaded form was adopted. then. Implications for Research Design The findings indicated implications for design and research design: 1.2-3 CeLE MessageTag tool Following A1’s recommendation for overviews. Questionnaire: The e-learners did not use the real-time online questionnaire (N=9/31.2.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch.3.

sense of social presence.D. interaction. 5: Tools & Evaluation Techniques 2. intensity. Daniel. a horizontal and vertical (see figure .). and thus. and the use of social networks. For online communication they stressed the need for common ground. small group activities. and empathy and trust.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. awareness. common ground. a horizontal and interactive direction of learning. The vertical structure exists within the message of the participants and is related to her reflective and evaluation processes. Anchored in the aforementioned recommendations as well as the findings the Sense of e-Learning Community Index (SeLCI) was developed for community evolution. interaction. The horizontal structure referred to interaction between the participants and thus. social network analysis for global cohesion and centrality. Wright (2004) emphasised the dynamics between elements in a Sense of Community Index and suggested that there should be different indexes if the study has an educational purpose. 2003. Daniel (2007). the teaching and learning style as well as the community size. the two new codes “agree” and “summarise” were added in the collaborative e-learning analytical framework. Changes on the Collaborative E-Learning Episode (CeLE): Two structures appeared to occur in a CeLE. a vertical and individualised direction of learning. 2004.2. He also suggested that there are additional elements in e-learning such as social presence. sense of belonging. and commonality of expectations and goals towards the development of the sense of community in e-learning. empathy.. To support the group dynamics online they suggested the use of social networks and reciprocity. Rovai (2002) worked on the role of spirit. Wright. roles. norms and policies. 3. 2007). 2002. A Sense of e-Learning Community Index (SeLCI) was further investigated (Rovai. social protocols. trust. trust. Preece & Maloney-Krichmar provided a framework to support sociability and usability.3-1): Ph. rituals.2. Preece & Maloney-Krichmar. social equality. trust. working on social capital suggested the importance of shared experiences. Thus. Thesis Centre for Interactive Systems Engineering LSBU Page 128 . the research design was adapted as follows (adaptations in grey) (Table 5. group facilitation. Changes on coding: Following changes on MessageTag. collaborative e-learning quality.

2. Usage and usability of collaborative tools 3. Global Cohesion • Density • Reciprocity • Cliques • Structural equivalence 2. Sense of e-Learning Community Index • Persistence: depth of discussion threads • Density: formula 2aN(N-1) • Reciprocity: number of messages sent and received • Intensity: levels of participation 1. Frequency of visits 2. Participation in projects. 5: Tools & Evaluation Techniques Table 5. help and support • Collaborative e-learning quality: participants’ opinions 2. manuals and presentations by the Greek educational authorities 3. Its permanence (continuous or discontinuous) 3. The Sense of e-Learning Community Index • Community evolution • Sense of belonging to the e-learning community • Empathy • Trust: knowledge exchange. Local real-time nodes & centrality 1. Form of the effect: the level. Conditions of working and learning over the Internet 3. Documentation. Inferential statistics: statistical significance. conferences. Initial knowledge on collaboration 1. Research Design ETHNOTECHNOLOGY DESCRIPTION 1. and emails 4. seminars. Professional development 1. 2. Examination of the data. conversations. Its immediacy (immediate or delayed) 1 Scenic Fieldwork 2 Pre-Questionnaire 3 Logging 4 Thematic Analysis 5 Social Network Analysis 6 Post-Questionnaire 7 Statistical Analysis 8 Intervention Analysis Ph. Correlation: Pearson correlation coefficient (r). Collaborative e-learning episodes analysis 2. frequencies. α). Number of active and passive participants 3. Active participation levels 4. slop.D. Descriptive Statistics: crosstabulation. Personal opinions. and null hypothesis.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. Thesis Centre for Interactive Systems Engineering LSBU Page 129 . Global Centrality • Centrality • Closeness • Betweenness 3.2. Richness of text 1. arithmetic means. 4.3-1. standard deviations. online discussions of Greek teachers’ associations and schools in Greece 2. Passive participation levels 5. variance and cyclicity 2. and exploration 3. Sense of e-Learning Community Index • Number of Collaborative e-learning episodes The Sense of e-Learning Community Index 1. p-value (Cronbach's alpha. 1. Demographics 2. missing data. normality.

3-1): Table 5. This perhaps indicates the need for new programming techniques to support systems interoperability. P1 said that ‘self-efficiency is important in e-learning’. training adults using Moodle for 1-5 years.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. Since such programming has not been used yet in available elearning management systems a new hosting company was needed to meet most of the requirements.2. P1 had 1-5 years of professional experience. P2 was using the Internet all day and P3 once or twice a day. being both Greek teachers and Moodle developers. As for the reasons. P2 and P3 had the same working experience. None of them said he knew any tools and evaluation techniques to support participation whereas all of them thought participation necessary. 5. Three of them returned the questionnaire so the demographics are as follows (Table 5.3 Greek teachers Moodle developers Three individuals. The discussion was from 06/03/2007 to 13/03/2007. all working. With the provided time and resources in this study only redesign of the tools was feasible. 5: Tools & Evaluation Techniques Overall.2.D. PHP applications cannot easily work with algorithms and Java. it is difficult for the participants to observe they are missing. participated in a focus group targeting tools redesign before the Greek teachers’ renewed participation.3-1. he was using educational software for 6-10 years. Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 130 . P1 was a teacher. Furthermore. P2 an instructor.2. two between 40-65 years old and one between 30-40 years old and none of them had participated in any of the previous blocks or studies. 11-20 years. and he had been connected on the internet all day. The redesign was tested and developed further with the help of a focus group of Greek teachers Moodle developers. Demographics for the three Greek Teachers / Moodle Developers Demographics Age Gender Occupation Working Experience (years) Use of computer / software (years) Use of Internet Use of Moodle 1 2 3 30-40 Male 40-65 Male 40-65 Male Teacher Instructor Doctor / instructor 1-5 11-20 11-20 6-10 6-10 1-5 All day All day Once twice a day 1-5 years 1-5 years 1-5 years All three were male. if such tools and evaluation techniques do not exist. and P3 a doctor.

The comments about the tools on the provided questionnaire were (Table 5. it may be a good idea to use this image as the VIT Centrality icon (P3)..1-2): Table 5. Thematic analysis was used on the 53 replies to categorise participants’ opinions on the tools. tools. they were quite understandable. o o Simpler names were needed (P3). in addition. The discussion took place in the experimental environment with 11 discussion topics initiated by the researcher with a brief description of each tool. The use of one colour for each participation level may work better ( o ) (P3). Understanding the usefulness of the MessageTag and VIT was not easy without reading the manual. the list with the lurkers’ login names was distractive and perhaps it should be removed. The avatar information was simpler and equally good. whereas P3 replied with a question mark (‘?’). In other words.3-2. Questionnaire open questions Open Questions Results Participation Tools Message Tag Show the quantity but not the quality of the Structures P1 messages. On VIT Centrality. especially for the people who are not familiar with such environments (P3).D. The discussions topics were: an introduction. but not sure VIT Centrality Useful and interesting Other More tools Clickable images Overall. The more information existed on an icon. and suggestions on Greek names. the respondents said the following: • • E-learner’s participation graph was not calculated correctly.είναι λίγο δυσνόητη’). Only discussions on the new ideas and tools are presented here.2. Ph.. 5: Tools & Evaluation Techniques P2 because ‘feeling part of the community’ is important. P2 More information P3 VIT Nodes Not needed.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. new ideas. the new collaborative e-learning scheme was tested for use and usefulness. • Graphics: o Multicoloured avatars are not clear as regards their meaning (P3: ‘Η πολυχρωμία . an image should explain its own use (P2).3. As for the centrality circles. the less the participation. Thesis Centre for Interactive Systems Engineering LSBU Page 131 .

3 Satisfaction = 4 Message Tag VIT Nodes 3.D.3 Participation Tools 4. Tools Pedagogical Usability Scores PEDAGOGICAL USABILITY 1 Participation Tools 4. learnability.3-3): Table 5. This means that the participants were overall satisfied with the tools but made additional recommendations as seen previously. Lastly. there were some final propositions. structuring or not structuring information: ‘I think I understood!!!no need for explaining the use of tags. P3 indicated the problem spotted in the literature.2 out of 5 as regards pedagogical usability.3 Efficiency & Effectiveness μ = 3. 2003) (Table 5.6 VIT Centrality 4.6 VIT Centrality 5 VIT Centrality 3.3 VIT Centrality 4. P2 and P3 also answered the second session of the questionnaire based on a Likert scale 1-5 and focused on efficiency and effectiveness. Thesis Centre for Interactive Systems Engineering LSBU Page 132 .3 Participation Tools 4. an overall users/learners’ view so the moderator can support the learners’ individuality. and a user/learner’s view for self-organisation and self-learning.2 The tools scored 4.9 Message Tag VIT Nodes 3.5 Message Tag VIT Nodes 4 μ 4 4.6 Participation Tools 4. 2004. 1994).3-3.3 μ = 4.6 μ 3 3. 5: Tools & Evaluation Techniques o • The results should be available on the interface without the user/learner having to click on them (P3). Finally.6 Satisfaction μ = 4.3 Imaginative μ = 4.6 μ 2 3. P2 suggested that the interface should provide two views.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch.2. and overall evaluation for imagination and satisfaction (Zaharias.3 Learnability = 4 Message Tag VIT Nodes 4 μ 5 6 Overall Score 4. P1. satisfaction (ISO DIS 9241-11.2.. a user/learner who lost his interest and lurked can be activated in a different team. enjoyability. Silius et al. They would like more tools. But is it necessary to make use of tags? Isn't it easy to understand the nature of the writer's message?’ Ph.6 Enjoyment = 4. In this way. He also suggested the group format for e-learners flexibility.

Two views of the tools. • • • The names of the tools came from social network analysis references. However. It is a bigger project and requires team working as well as time and funding. • The use of one colour for each participation level did not show the user/learner’s potential. 5. Meanwhile work was carried out collaboratively with another programmer to update the tools and she actively participated in the course as an e-tutor. as well as having the tools in a group format could not be implemented in a study such as this one.2. This may partly be because they were not familiar with such tools. Thesis Centre for Interactive Systems Engineering LSBU Page 133 .3. the participants did not provide any alternative names.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. The considerations on design and research design were: • The participants did not immediately understand the usefulness of the tools. The proposed red colour was found inappropriate due to alert signalling of red and usability purposes. Providing all information on the interface was found to be of poor usability because of the information overload on the interface so no additional information was provided. this was a good idea and the high participation graph was changed to yellow ( • • ). this measured was found inadequate to accurately measure utility and usability for new tools. 5: Tools & Evaluation Techniques However. The lurkers’ login names on VIT Centrality were not removed because there was no other way to make them visible on the tool. and partly because of design. • • The pedagogical usability questionnaire was elaborated in more detail.2. The Collaborative e-Learning Episodes (CeLE) was successful and 7 CeLEs were identified. A combination of participation levels and CeLE from a bird’s eye view can provide comparable information (Figure 5.1-1): Ph.1 Application of guidelines and heuristics from feedback in design When this discussion finished the Greek teachers were in the middle of the online course in Moodle@GSN (main study).D.3. The icons for the VIT were changed to icons acquired from a discussion.

Considering users’ unfamiliarity with new tools the utility & usability questionnaire items were developed as follows (Figure 5.2. this will be analysed further in the next chapter. • • • relative items were grouped under one attribute.1-1): Ph. they were transformed. 5: Tools & Evaluation Techniques Figure 5. Discussion on tools’ Greek names Following the graph.2. the aim was to support a utility & usability overview of the tools related to participation in collaborative e-learning communities. since the utility and usability attributes were found inadequate.1-1. the questionnaire was tested twice with 2 groups of different participants with 3 members in each group to ensure that the items were adapted and included appropriately in the questionnaire. Thesis Centre for Interactive Systems Engineering LSBU Page 134 . Moreover. Further correlation analysis suggested the interactivity/engagement and motivation to learn to be combined under ‘motivation to participate’. Due to limited space.3. it appears feasible to compare collaborative e-learning values and levels of participation.D. new attributes were anchored in Zaharias usability questionnaire.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. focused on 5 out of his 13 suggested attributes (Zaharias.3. 2006:198-199) as well as Preece and Maloney-Krichmar’s framework on sociability and usability for online learning communities (2003). Lastly.

the numeric assessment provided by the tools and evaluation techniques was successful.D. all participants agreed that the tools.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch.1-2. addressing learning on an individual and a social level. because she can: direct and control online discussions. the Greek teachers Moodle developers were satisfied with the tools. Pedagogical Usability Attributes Overall. they enjoyed using them. The last part of this research is the main study discussed in the next chapter. and record the discussion. From a researcher’s viewpoint. and asked for more. activate the lurkers with specific questions. 5: Tools & Evaluation Techniques Figure 5. and in particular the Visualisation Interactions Tools (VIT) were useful and important. aid in team building by “bounding” the team.3. Thesis Centre for Interactive Systems Engineering LSBU Page 135 . Ph. discover interlocutors’ weaknesses and strengths. They also said that selfefficiency is important in e-learning and feeling part of the community is what participation is about. Despite the discrepancies with the graphics and difficulty in understanding the meaning of the tools.2. especially for the moderator.

11th International PEG Conference: Powerful ICT for Teaching and Learning. A Bayesian Belief Network Computational Model of Social Capital in Virtual Communities. Dineen. 596-620).). S. (2007). 11 . Vicarious Learning from Educational Dialogue. (2007).1 July 2003 .usask. Dialogic. & Maloney-Krichmar. Publishers.cfm?id=1150283&type=pdf&coll=portal&dl=A CM&CFID=15151515&CFTOKEN=6184618. J.. Boy. Paper presented at PEG’03. L. P. 1999. from http://portal. CD-Rom. C. Handbook of Human-Computer Interaction (pp. London: Element Books.Computer Supported Co-operative Learning. PhD Thesis. Ergonomicrequirements for offi ce work with visual display terminals. Hershey. Online Communities. D. K. Tervakari. Zaharias. A. & Mayes.ca/theses/available/etd-07132007141903/unrestricted/ben_m. Beijing. Stanford University. Multimedia. McKendree. In A.. & Pohjolainen.. Online Learning Idea Book. Paper presented at the HCII'07 . (2003). Shank (Ed. G. Accessibility and Informational Quality of Web-based Courses. (1997).. The Way of Yoga. Daniel. M. R. Pedagogical Usability. Paper presented at the CSCL'99 . (2004). 1994. Petersburg. Wegerif. Thesis Centre for Interactive Systems Engineering LSBU Page 136 . (2003). (2004). J. (1999)... Jacko & A. Tucker (Ed. K. Boca Raton. Wright. University of Saskatchewan.org/ft_gateway. Educational and Technology: Expanding the Space of Learning.12 December.). (in press). Silius.HCI International 2007 Conference. Exploring Psychological Sense of Community in Living-Learning Programs. J. PhD Thesis. International Standards Organization. Preece. University of Maryland. Unpublished research. London: International Standards Organization Lambropoulos. from http://library2. FL: CRC Press. F. China.. B.pdf. Preparing the On-line Learners: Information Provision and Intention Setting by Chatbots. 22 – 27 July 2007.D. A. The Computer Science and Engineering Handbook (pp. Perceived Complexity and Cognitive Stability in Human-Centered Design.. N. Unpublished Doctoral Dissertation at Athens University of Economics and Business Ph. (2003). A Usability Evaluation Method for E-Learning Courses. (in press). T.M. S. In P. Lee. Stenning. 5: Tools & Evaluation Techniques REFERENCES Alty.). P. Russia. B. Mahwah: NJ: Lawrence Erlbaum Associates Inc. PA: Idea Publishing. K. Sears (Eds. St. Cox. In J. Retrieved 01/08/2007. A Multidisciplinary Tool for the Evaluation of Usability. J. 28 June . 1551-1570). R. New York: Springer-Verlag. Canada.Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. Retrieved 12/12/2005. Isaacson.Part 11: Guidance on usability (ISO DIS 924111).acm. Unpublished Research.

The study also investigates whether a collaborative e-learning community exists using the sense of e-learning community index. pedagogical usability and utility values provide information about the Greek teachers’ human-computer interactions. Then the levels of participation reveal their passive and active presence in the course in comparison to the experimental environment. The comparison continues with the quest for quality in their discussion under the lens of the analytical framework of collaborative e-learning episodes. findings. Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 137 .D. There is an initial examination of the Greek teachers’ human-human interactions for working and learning online. the intervention analysis and comparison with current research are presented in this chapter. Finally. and discussion of the main study. interventions.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study 6 • • • • • • • • Main Study: The quasi experimental study on Moodle@GSN and the research pool Key Topics Covered in This Chapter: How initial difficulties were resolved in the main study Demographics and initial knowledge or lack of it Tracing active and passive participation The collaborative e-learning episodes in the service of e-learning quality Unfolding the Greek teachers’ sense of community How pedagogical usability and usability informed human-computer interactions How and why the intervention was successful Comparison with other contemporary studies Chapter 6 presents the preparations. In addition.

it explores the attributes of the Sense of e-Learning Community Index: community evolution.1-1): Figure 6. conditions of working and learning over the Internet. Moodle@GSN and the experimental environment. in addition. Lastly. I participated as the 5th e-tutor (Figure 6. the next section in this chapter describes demographics.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study 6. trust. these tools were incorporated in an experimental Moodle. e-learning quality. The evaluation of the new collaborative tools is conducted by utility and usability evaluation. The e-research methodologies used to aid e-learning engineering were centred on ethnotechnology: fieldwork. sense of belonging.1. and questionnaires. thematic and social network analysis.D. intensity. and examples of Collaborative e-Learning Episodes.5 as with Moodle@GSN. The e-tutors in the online course A link in Moodle@GSN in the last week led the participants to the research pool. Because the Moodle@GSN technicians were reluctant to implement the new tools. logging.1 INTRODUCTION The aim of this study is to address the Greek teachers’ passive participation in the e-learning environment provided by the Greek School Network (Moodle@GSN). Ph.1-1. started on 01/03/2007 and finished on 31/03/2007. empathy. The online course was on the ‘Use of New Technologies for Educational Project Management’. More specifically. Thesis Centre for Interactive Systems Engineering LSBU Page 138 . Thus comparison was feasible between the two contexts. Then.4. global and local social network analysis will aid in investigating group cohesion and the importance of social networking and collaborative e-learning. intervention analysis examines the overall success and compares the findings with other contemporary research. version 1.

1. Thesis Centre for Interactive Systems Engineering LSBU Page 139 . Only 40 questionnaires from the participants who returned the first and third questionnaire and appeared on both Moodle@GSN and the research pool were accepted for analysis (Table 6. Initial activities The initial email call for participation in the online course attracted insufficient response and another call was issued. Q3. the participants were given gmail accounts. project management. blog. wiki. Q3 or Moodle@GSN and the research pool 40 (67. create their own context using the tool and evaluate its use. It had 4 sections. To improve communication flexibility.1%).Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study 6.1. the response ratio compared to the initial expression of interest (117) was satisfactory (34. Ph. and videoconference. 1993) are interlinked. a decision was made to change from an instructional (e-tutoring) to a student-centred )collaborative elearning) approach. The course had to be expanded by one more week than initially planned.1. The questionnaires selection process Accepted / Rejected Questionnaires Number of Questionnaires Total Accepted Rejected Accepted for analysis Value 59 34 19 Justification Returning Q1. Two questionnaires were sent: one at the beginning of the course to gather information about demographics and some background and one at the end with questions related to the exploratory research questions. One hundred and seventeen (117) participants returned the first and 59 the third questionnaire.D. Because interface and pedagogical modifications (Delich. 2006) as well as methods and tasks (Draper. Considering the total absence of activity and the ratio referred to in the literature (20% lurkers’ response).7%) Forty questionnaires were selected for analysis using the above criteria and were given a unique number. The participants had to study the educational resources.1-1. participation in Moodle@GSN and the research pool Participation in all previous activities Not participating in either Q1. this resulted in having the experimental session the week before Easter 2007. The evaluation of the videoconference was conducted in the research pool after modifications to the interface based on previous research carried out before hand into the effectiveness of the system.1. Q2.1-1): Table 6.

1-2. Normality overview for tools pedagogical usability and utility in HCE The responses from the questionnaires were also screened for normality skewness and kurtosis in HCE (e. Hierarchical Clustering Explorer 3. 1998) to determine whether the variables are “normal” enough to be analysed.umd.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Even though 2.cs. When participants had failed to supply a response this was treated as a no response rather than using the mean.D.1.0 ∗ (HCE) (Seo. this was a low percentage and did not impose any serious problems with the subsequent analysis (Tabachnick & Fidell.edu/hcil/hce/). Hair et al.. These results. ∗ Ph.) HCE is a visualization tool for interactive exploration of multidimensional datasets to help users explore and understand multidimensional datasets by maximizing the human perceptual skills (http://www.g. the small initial sample in the trials and previous absence of variables justification in the literature suggested that the questionnaire items could remain the same. 2005) was used for the examination of the data (Figure 6. skewness and kurtosis suggested that original variables can to be used. (Tests of significance for skewness and kurtosis test the obtained value against a null hypothesis of zero for a normal distribution. Thesis Centre for Interactive Systems Engineering LSBU Page 140 . 1996).03% of the overall responses in the questionnaires items were missing. Correlation analysis was used to suggest the highly correlated factors in order to integrate them in one variable.1.1-2): Figure 6.

2004).1-1): Ph. Who are the Greek Teachers? The analysis showed 1 participant between 20-30 years old (n=1.2 DESCRIPTION OF THE RESEARCH CONTEXT From the initially registered 177 teachers. and 95 participants produced usable profiles. 6. The age range in the closed questions was created anchored in the way teachers are hired and previous studies (e. 49 never fully enrolled.D. Hlapanis & Dimitrakopoulou. and IT secondary school teachers (n=22.2. 36%). It was hoped that this would create a co-presence feeling by getting to know each other before their attempts for active participation. They participated from different parts of Greece. 3%). 16 from Athens (Figure 6. 36 created illegible profiles due to Greek language encoding. and 24 more than 40 years old (n=24.2. Thesis Centre for Interactive Systems Engineering LSBU Page 141 .5%) Greek teachers.1.5%) and 31 male (n=31. Most of the participants with Moodle profiles were primary school teachers (n=36. 36%). Of the remaining 128 participants. The participants were asked to create their profiles immediately after their registration. 28%). 14 between 30-40 years old (n=14. and was successfully tested in the preliminary studies. 77. 61%) (1 missing). Based on the initial questionnaires (N=40) there were 9 female (n=9.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study 6.g. 22.

originally retrieved 03/09/2007.edu/maps/europe/greece_div96.D.2. 7. 10 teachers working for 6-10 years (n=10.1-1.2. The participants have used computers in education: 3 for 1-5 years (N=3.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Figure 6. 37.5%). and 7 for more than 20 years (n=7.5%). 5%).lib.1-1.utexas. as well as their experience in computers and Learning Management Systems (LMS) were (see data in Appendix VIII): ICT in Education: Most of the participants have worked in the Greek education system for more than 6 years (N=40).2. their familiarity with Learning Management Systems (LMS) is relatively low. 17. Participants’ locations in Greece This selection process assigned a representative sample of the teachers with a special interest in the use of ICT in education in Greece (Figure 7. 22. 35%). 45%) and 9 more than 20 years (n=9. Conditions of Working and Learning Online The participants’ years in education. 15 for 1120 years (n=15. However. 25%).5%). There were 2 new teachers (n=2. the majority Ph.5%). 18 working for 11-20 years (n=18.2. from http://www.jpg): 6. Thesis Centre for Interactive Systems Engineering LSBU Page 142 . 14 for 6-10 years (n=14.

20%). from http://docs. retrieved 17/07/2007. most of the participants did not attend courses in higher education but outside the formal Greek education system (n=29. 72. This result may be related to the short time LMS in education has been available.2.2. 65%). for example Moodle@GSN was available to the public in August 2002.5%). one of the respondents had 6-10 years and 2 had 11-20 years in education. The use of computers is in parallel with the years of employment. Ph. and Learning Management Systems (LMS) The previous graph presents that the use of LMS starts in early employment. As they were not recent graduates. The correlations between these factors are (Chart 6.org/en/Online_Learning_History). 32. 7. 7. Thesis Centre for Interactive Systems Engineering LSBU Page 143 . 8 learned to use Moodle in an ICT course (n=8.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study have not used LMS at all (N=22.e. use of computers. Correlations between time in education. Use of Computers and LMS 100 Number of Participants 22 10 13 14 10 18 15 9 7 Years in employment Years using computers Years using LMS 2 1 0 years Months 3 2 1 6-10 years 1-5 years 11-20 years 20+ years Time Scale Graph 6.5%). The vast majority did not have any training in LMS (n=26.2-1): Correlations between Time in Education.2-1.D. This response may not be reliable since Moodle was only made available to the public in November 2001 (Moodle e-learning history.5%) whereas one participants said that she was using LMS for more than 6 years (i.moodle. Lack of training and launching online courses as initiatives on an individual basis has been reported as an e-learning quality blocking factor (Fernandes & Montalvo. this means that their course was part of a life-long learning course whereas LMS were not part of the curriculum. and 3 within the official Greek education system in graduate or postgraduate courses (n=3.5%). less than 6 years ago). 55%) and 3 for 1 to 6 months (n=3. 2006). Only 13 participant have been using LMS more than one year (n=13. As for training on Information and Communication Technologies (ICT) in education.

22–32. self-instruction. On the significance of co-presence and e-learning management.5%).5%) and 2 every 2 or 3 days (n=2. formal pedagogical training on ICT and LMS is lacking.2. These results are also reflected to the next findings.5%) and the vast majority believe in their importance in elearning (n=26. and their presentation on the web from a life long learning perspective.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Familiarisation with the social aspects of the Internet: The majority of the participants used the Internet on a daily basis: 13 are on-line all day (n=13. 57. 21 participants have used forums (n=21. 32. 52. Despite the fact that the participants use the Internet.D. Previous Knowledge of Collaborative E-Learning Techniques & Participation The Greek teachers replied to the question on collaborative e-learning techniques and participation as follows (Table 6.2.5%).3. 10-22. In addition.3-1): Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 144 . Before the study. Their opinion was checked in the final questionnaire and appeared to explicitly correlate communication and the creation of profiles. 65%). 23 once or twice a day (n=23. 45%). Hartley (2007) suggested the need for improving British teachers’ social and learning skills. relatively high or very high: 14 participants have used profiles before (n=14. they think that profiles and discussions forums are of average importance. the respondents seemed to think that the use and importance of profiles and discussion forums are significant in e-learning. 6. a small ratio responded on the Likert scale as low and very low although the ratio of not answering these questions was relatively high (n=8-13. Lastly. it appears that they might not know ways to represent themselves and talk in online discussions. and because the Internet is mostly based on textual communication. Most of the respondents considered the use and importance of profiles and forums is of average importance (n=4-9. the Greek teachers use the Internet regularly and participate in discussion forums. lack of opportunities and professional training incorporating current pedagogical approaches.5%). 35%) whereas 18 think they are important in e-learning (n=18. 5%). Overall. These results were similar to my observations and Tsetsilas’ findings (2006) on Greek teachers’ lack of soft skills.

however. The majority replied that participation in e-learning is necessary (n=31. As for their participation. 9 14 9 % 22.5 65.5 The previous table presents that only 11 participants knew some e-learning collaborative techniques (n=11. Gulati.2. most of them said that their participation is useful (n=26.5%).D. and 9 participants did not answer the question (n=9. despite the fact that the Greek teachers did not know any particular collaborative e-learning techniques. most of them either said they did not know or did not answer (n=29. 20 0 0 % 50. For this reason. Greek teachers’ knowledge and attitudes on collaboration and participation Collaboration & Participation Yes Freq. a significant number (n=14. 22. they believed that their own and other e-learners’ active participation is of great importance. This result is similar to other results (e. This was found to be the major difference between the online communities and the e-learning communities.5 No Freq. the use of emoticons.5%). Overall. 72. On the question about collaboration with the Greek educational authorities everyone agreed on the absence of communication channels (Graph 6.g. Do you know. 1 collaborative ways for e-learning? Do you think that… 2 your participation is useful? 3 active participation is necessary? 11 26 31 % 27.0 0.5%). 2006) and this lack of knowledge of how to work and learn together is inherent in the teaching and learning modes for teachers’ training found in the literature review.. 77.0 22.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Table 6.5%). 2. 27. 65%).5%). 2002.0 N/A Freq. 35%) did not answer the question. how to write online messages based on results from this project’s preliminary studies.5 35. as the e-learning participants have a specific purpose: to learn. Beaudoin. The N/A response on collaboration and participation as well as on the use and importance of profiles and forums is relatively high compared to the response on questions about personal details (n=1-3..2.0 0. the e-learners were given explicit guidelines on: netiquette.5-7. none denied that participation is not necessary. and the structure of the collaborative e-learning episode (see Appendix X).0 77.3-1. Thesis Centre for Interactive Systems Engineering LSBU Page 145 . This implies a gap between knowing the importance of participation but not being able to act upon it because of lack of know how.3-1): Ph.

A reason for this lack of consideration of the age of employment comes from Katsaros and Karageorgiou on their study on the absence of the Greeks primary teachers training for the environment in their area between 2001-2006 (2006). 4050 (47 participants) and over 50 years old (21 participants). how to represent themselves online as well as techniques to collaborate are all lacking. Some participants (average 16. basic knowledge of the Internet as a tool for pedagogical activities. despite my efforts it was impossible to find reports on the workforce so no assumptions can be derived from this result. However.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study The Greek educational authorities' communciation with participants 35 30 25 20 15 10 5 5 1 3% 0 Some times Very few times Never 13% 5 13% 2 5% 34 84% 82% 33 Pedagogical Institute (N=40) Ministry of Education (N=40) Graph 6. the little communication with the Greek educational authorities implies lack of participation in any major changes introduced in the Greek education system as regards the use of ICT in education.3-1. In addition. Communication with the educational authorities The communication channels with the educational authorities are not open as most participants have never been contacted by the Pedagogical Institute or the Ministry of Education (average 83%). The majority use the Internet regularly and have an active online life.D.5%) were approached a few times by either authorities. Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 146 . Overall. only two young Greek teachers took part in this study. the age range had three parameters: up to 40 years old (22 participants).2.

Since particular modules to facilitate logging in Moodle do not exist.1%) whereas ‘forum add forum/post was limited’ (N=175. this was deliberately encouraged so that the e-learners could know more about each other before participation started (Table 6. only the messages with at least one reply discussion depth were considered appropriate for analysis.555 175 43.3.D.4%).597 1.193 7.1-1.3. These messages were 175 from Moodle@GSN (N=616) and 80 from the research pool (N=98). most other messages were either announcements or members’ introductions without seeking a response from other learners.5 0. as well as the number of active and passive participants. 0.985 Average Percent per day on total views 671 393 247 51 50 6 1.418 47.1 Frequency of Visits This study targets participation in collaborative e-learning communities.799 12.): Table 6.4 100 3 4 5 6 Almost double the clicks were related to viewing other e-learner’s profiles and then discussions and forums (45.3.6 3.) 6.4 27. So logging will present a quantitative view on the frequency of visits. Views VS Posts: The activity in the two environments was (Table 6.3 TRACING PARTICIPATION Logging for e-learning environments provides a description of e-learners’ activities.4 3.666 1.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study 6. therefore. Moodle@GSN forums and users view log files MOODLE@GSN LOGS (01/03-31/03/2007 – 31 days) Type of Activity 1 User view 2 View individual discussion View individual forum View all users View all forums Add forum/post Total Activity 20. Thesis Centre for Interactive Systems Engineering LSBU Page 147 . (My messages as the observer participant in the study were not counted unless indicated otherwise. the mdl_log file was extracted to create an Excel compatible Comma Separated Value (CSV) file format.1-2): Ph.3.7 17. These were general comments in Moodle@GSN and initiations to discussions in the experimental environment.1-1. As stated before.

8%). 175 (28.0 100.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Table 6.1-3): Table 6.8 messages for Moodle@GSN and 13.3 24.5 13.8 5.0 2 3 4 5 6 Forum view discussion Forum view forum View forums Forum add forum/post User view User view all Total The logs related to forums (view discussion. and posts for analysis appeared as follows (the total number of posts including my posts is in the parenthesis) (Table 6.7%) whereas the users view was limited (69 logs.1-3. 90.4%) were suitable for analysis. view forum.461 Average per day 114 61 12 11 6 4 208 Percent on total views 55.6 for the research pool.2%) were analysed.0 (31 days) 8.3. Thesis Centre for Interactive Systems Engineering LSBU Page 148 . This also means that the social reasons for passive participation were almost eliminated at the beginning of the course.8 5.2 35. 80 (84.4 23.6 14 11. Total number of posts NUMBER OF POSTS Frequency Percentage (N=714) 86.8 2.D.7 Average per day 19. the total number of posts. whereas from the 95 messages in the research pool.5 2. Forums and users view log files in the research pool RESEARCH POOL LOGS (25/03-31/04/2007 – 7 days) Type of Activity 1 Activity 799 427 86 80 39 30 1.3.1-2.2 5.3.7 11.2 (31 days) 1 (31 days) 2 Moodle@GSN Message for Analysis Research pool (7 days) Message for Analysis Total Total for Analysis 616(850) 175 98 80 714 255 The previous table presents that the overall average posting per day was 19.7 29. 4. A comparison is presented (Graph 6.312 logs. Next. and view forums) in the research pool were of high priority (1.1-1): Ph. From the overall 616 posts in Moodle@GSN.3.

the proposed tools and evaluation techniques are suggested to be catalysts for active participation and collaborative elearning.3.987 07/03/2007 2.1-4): Table 6. Temporal overview of all activities Observation Course Day 0 1 2 3 4 5 6 7 8 9 10 O0 O1 O2 O3 O4 O5 O6 O7 O8 INTERVENTION & EVALUATION Tools n/a n/a n/a n/a n/a n/a On date Until date 501 501 387 3.553 42. Comparison between sent messages and messages for analysis The number of selected messages for analysis is higher in Moodle@GSN: there were 175 messages (24.771 30/03/2007 505 46. Temporal View: A temporal overview of all activities can provide an in-depth view (Table 6.969 Baseline 3 Ph.243 30.4 80 Moodle@GSN Experimental Environment 175 Graph 6.793 31/03/2007 176 46. However.088 10.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Comparison: Number of Messages & Messages for Analysis 200 180 160 Number of Messages 140 120 100 80 60 40 20 0 Analysed Messages Percentage on total number of messages (N=714) Average per Day (N=31) 24.2 5.796 364 4.821 6.312 Baseline 2 26/03/2007 365 44. Active participation in Moodle@GSN was richer as regards the discussion depth.990 14/03/2007 1.6 11.160 Community Management Moodle@GSN Baseline 1 On date Until date 01/03/2007 3.427 20.5 11.D. Thesis Centre for Interactive Systems Engineering LSBU Page 149 .3.1-1.5%) in Moodle@GSN and 80 (11.1-4.4 messages in the research pool posted per day.6 messages in Moodle@GSN and 11. It appears that about half of the messages were posted in the experimental environment in a quarter of the time compared to Moodle@GSN.007 22/03/2007 1.3.2%) in the research pool with an average of 5.808 03/03/2007 2.

Posted messages appear as follows (Table 6.088 2. the posted messages decreased towards the end of the course with some more messages in the research pool.000 Logs 100 38 33 10 1 01/03/07 03/03/07 05/03/07 07/03/07 09/03/07 11/03/07 13/03/07 15/03/07 17/03/07 19/03/07 21/03/07 23/03/07 25/03/07 27/03/07 29/03/07 31/03/07 32 11 11 18 34 15 Activity Posted messages 3.3.D. Thesis Centre for Interactive Systems Engineering LSBU Page 150 . there were 16 messages on the first day and 29 messages on the last day of the course. From almost 4.553 866 892 540 Observation dates Graph 6.1-5): Table 6.821 2.1-5.243 1. There were 38 messages on the first day of the course and 5 messages on the last day of the course in Moodle@GSN.3.3.427 1.000 activities on the first day of the course. Temporal overview of posted messages (add post/forum) Observation Course Day 0 1 2 3 4 5 6 7 8 9 10 O0 O1 O2 O3 O4 O5 O6 O7 O8 INTERVENTION & EVALUATION Tools n/a n/a n/a n/a n/a n/a On date Until date 16 16 9 69 29 98 Community Management Moodle@GSN Baseline 1 On date Until date 76 01/03/2007 38 03/03/2007 33 152 07/03/2007 32 313 14/03/2007 11 416 22/03/2007 11 567 Baseline 2 26/03/2007 2 585 30/03/2007 6 601 31/03/2007 5 606 Baseline 3 Similar to activities.1-2. As the course continued in the research pool. A comparison between activity and posted messages reveals the following (Graph 6.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study The previous table presents that the number of all activities in Moodle@GSN is decreased as the course progresses.1-2): Overall activity VS posting 10. there were 176 activities in the end of the course.3.000 1. Logs of overall activity VS posting Ph.

0 From the 59 active participants in Moodle@GSN.0 35. it might indicate that the e-learners had acquired enough knowledge to finish the course with a good rate of quality posts.3. 34 of them participated in the research pool. 6. Overall.2 On Participation This section discusses active and passive participation. online community management was initially focused on enhancing individual actions (one way of communication) and social relationships (interactions) in order to increase the feeling of presence. 6. Number of active and passive participants NUMBER OF ACTIVE & PASSIVE PARTICIPANTS Participants Moodle@GSN Participants in both Research pool Active Passive 59 36 34 6 26 14 Total 95 40 40 Percentage Active Passive 62.3. This seemed to work. the posts on the last day (34 messages) were almost as many as on the first day of the course (38 messages).1. Overall. Because the messages were related to each topic and were not on social networking (e.2.1 37.1-1): Table 6. saying goodbye or staying in touch after the course).0 15. The calculation of active participants was conducted by counting the members who posted at least one message. Active & Passive Participants An overview of participants’ active and passive participation shows (Table 6.D. In fact. Lastly.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Activity was significantly decreasing whereas posting was slightly rising in the research pool.2.2.g.1-1. there were 26 active participants in the research pool. as for Ph.9 85. the logs revealed more activity on viewing the profiles during the first days of the course that were diminished in the end whereas posting was relatively stable and the messages quality was increasing. Another explanation may be the Hawthorn effect. Thesis Centre for Interactive Systems Engineering LSBU Page 151 . the next section examines participation. Then it was focused on enhancing participation quality.3.0 65.3.

2-1. Active Participation Levels There were two phases in determining active participation levels as the initial calculation was found not to be practical on a large scale.2-1): Table 6.2-2): Ph.D. as previously. the calculation was anchored in the bell curve to determine low. Thesis Centre for Interactive Systems Engineering LSBU Page 152 .2. medium and high participation levels (Table 6.3.3.2.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study the participants in both environments there were about 20% (N=14) more active in Moodle@GSN. medium and high participation. based on the total number of sent messages (Table 6. Active participation Levels II: A second attempt was made to calculate the messages based on the highest respondent’s posts.3. 6.2.2. The next sections will try to shed more light in understanding differences in active and passive participation as mere calculation may not provide a coherent view. LEVELS OF ACTIVE PARTICIPATION (TOTAL NUMBER OF POSTS) Total Posts Moodle@GSN Participants Research pool Active Participation Levels (%) (posts) Low (1-25%) (1-154) 616 (1-83) 333 (1-24) 98 Medium (26-75%) (155-462) (84-250) (24-71) High (76-100%) (463-616) (251-333) (72-95) 100 Low 100 Low 100 Low Percentage Medium High 616 333 98 Medium Medium High High The previous table presents that the proposed model calculated on the total number of sent messages was not functional on a large scale as all participants appeared to be on the low activity level. Active Participation Levels I: The levels of active participation were calculated according to the proposed model on low.2. Active participation levels (Initial proposition) A.3.

Once more it appears that the initial ‘social’ kick on the participants and instructional modifications to adjust e-learning behaviour to the new collaborative tools worked. active participation was e-learners directed.0 Medium 3. Since there was no e-tutoring in the research pool.3.D. elearner P50 was the highest poster in the research pool. ACTIVE PARTICIPATION LEVELS (HIGHEST POSTER) Highest Poster Moodle@GSN P52:152 Highest Poster P50: 12 Active Participation Levels (% & highest participant’s posts) Low (1-25%) (1-38) Medium (26-75%) (39-76) High (76-100%) (77-152) Low Percentage Medium High 32 participants Low (1-25%) (1-3) 54(P58) 1 participant Medium (26-75%) (4-9) 152(P52) 1 participant High (76-100%) (10-12) 94.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Table 6. There were 32 low.2-2. 1 medium and 1 active participants in Moodle@GSN. With 12 posts. The highest poster sent 152 messages in Moodle@GSN (P52). Thesis Centre for Interactive Systems Engineering LSBU Page 153 . medium and high) based on the days of viewing. and 16 low. zero level is when a participant only subscribed to the 2 environments and returned the 2 questionnaires. However.0 High Research pool 16 participants 5 participants 5 participants 62. 5 medium and 5 high participants in the research pool. P50 posted almost the same number of messages in 31 days at Moodle@GSN and in 6 days at the research pool. high passive Ph.2. Both medium (P58) and high (P52) participants were e-tutors in Moodle@GSN. Passive Participation Levels Calculation of passive participation was conducted on four levels (zero.3. Active participation levels (Second proposition) B. P58 posted 1 message and P52 did not post any messages in the research pool. 6.2. low.0 19. Lastly. This means that P50’s posting activity was accelerated from low to high activity in a week. it appears that Moodle@GSN required significantly more e-tutoring than the research pool and this was reflected in e-tutors and e-learners’ behaviour. and 12 posts in the research pool (P50). the e-learner-generated text provided the context to some participants to express themselves.0 Low 3.0 19. whereas he was a low activity poster in Moodle@GSN sending 14 messages. Overall.3.0 The second attempt provided better practical results.

3. An overview of the activated passive participants was as follows (Table 6. this means 33. and 2 in the research pool (5%). P43: 3 days 9 participants.5%).3.2. most participants were located in low passive participation. P19: 3 days 15%) P1: 4 days P7: 8 days Total 1 5 Percentage 17 83 Zero Low (0 days) (1-2 days) P51: 1 day P59: 2 days Research pool P16: 1 day (7 days. 4 (33%) on medium.2. this means an average of 10 viewing days for each level. The course lasted 31 days. P51 was the only participant who remained in zero participation.5%) P38: 2 days P1: 2 days Total 7 Percentage 0 59 Medium (11-20 days) High (-31 days) Total 0 0 Medium (3-4 days) P21: 7 days P40: 7 days P49: 4 days P19: 3 days 0 0 6 100 High Total (5-7 days) P56: 6 days 4 33 1 8 12 100 There were 6 passive participants in Moodle@GSN (15%) in 31 days and 9 participants in the research pool in 7 days.3-1): Table 6.D. so all participants’ logs needed to be checked and verified from the individual logs. P1. these were the activated lurkers.3. There were different passive participants in the two environments.) (Table 6. P38: 7 days 6 participants. 7 participants were active in Moodle@GSN (17. P19. Passive Participation Levels PASSIVE PARTICIPATION LEVELS (VIEWS IN DAYS) Zero Low (0 days) (1-10 days) P47: 3 days Moodle@GSN P51: 0 (31 days.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study participation is when a participant appears to visit the course but never crossed the threshold of participation.3-2): Ph. All 5 of the Moodle@GSN e-learners remained on the low level of passive participation (83%) whereas 7 (59%) were on low. and 1 on high passive participation. Thesis Centre for Interactive Systems Engineering LSBU Page 154 .3% of the passive participants. Overall. and P51 remained passive in both environments (10%). (There is a need to note that the graph for the activity overview in Moodle was misleading. P29: 2 days 22.2.3-1. Tracking the exact hours and minutes spent on the system was found to be extremely difficult. not exceeding the 10 viewing days. P38.

3.2. Thesis Centre for Interactive Systems Engineering LSBU Page 155 . The tendency of passive and active participation can be depicted in Graph 6.2.3.3-2.D.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Table 6. Active & Passive Participation locus from the same participants Ph. There were 3 low. 2 medium and 3 high passive participants who posted in Moodle@GSN but not in the research pool.3.2.3-1. Posts from passive participants PASSIVE PARTICIPANTS’ POSTING BEHAVIOUR To: Moodle@GSN Participants From: Moodle@GSN P59: Low P21: High P43: Medium P29: Low P16: Low P40: High P49: Medium P56: High 1 2 2 3 3 6 11 25 Posts To: Research pool Participants P47: Low P7: High Posts 1 1 From: Research pool There were 2 Moodle@GSN low and high passive participants who posted 1 message each in the research pool.3-1: Graph 6.

2. I couldn’t get into the course and I was wondering in the classes.”. he sent a message addressed to me in Youtube and said the following: “Professional work. I contacted her and tried to solve the technical problem (28/03/2007). I couldn’t follow either because I couldn’t find the specific link or. and he got scared of the environment and the immediate responses of co-learners. Although O2 also quit. P1. however. 01/08/2007). however. a person who observes a lot of participation is highly likely to be an active participant in a different forum provided favourable circumstances to the e-learner allow the change of behaviour. I did dedicate many hours to build my profile and in the end I didn’t make it. I was angry because the Greek educational community does not take advantage of its talents and knowledge of technology in many fields.1). the week of participation in the research pool was the week before Easter.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study The previous graph presents that low. Because the result of the increased passive participation in the research pool was unexpected. and P51 reported that they did not have any spare time. It also depicts the idiosyncratic character of the individual as found in Nonnecke and Preece (1999). when I could find the link. I couldn’t participate in any of the videoconferences due to the Internet connection. she was not trained. he said he needed to follow at own pace. O1 quit because of GSN technical problems. It appears that there were different reasons. P1 complained that time was limited for him (personal communication via email 27/03/2007). Looking into P38’s first questionnaire. it appears that she was using computers for 5 years. The online course was extended by one week. P19. Thesis Centre for Interactive Systems Engineering LSBU Page 156 . P49.D. I contacted the passive participants and asked them why they did not participate. This gap was found as the sleeper effect in the literature review (Chapter 2. and never worked with LMS. positive thinking was beyond any expectations. an active participant in Moodle@GSN.” (Personal communication via email. and stresses the importance of investigating the space between passive and active participation if more participation in the e-learning communities is desirable. Once I was asked to prepare a text and I wrote about the water. appeared to have serious login problems in the research pool and the log showed continuous failed logins. I was proud to be a Greek teacher. kindness. P38 said: “I had extreme difficulty with communication. P51 visited the research pool for one day and never looked at Moodle@GSN. this meant that participants were preparing for Easter or as in the case of P16 were in Ph. In other words. I did receive the newsletters from time to time explaining exactly what I had to do. medium and high level passive participants remained in the same level of active participation.

2. numeric analysis found inadequate for in-depth messages analysis and as a means to determine their quality. These participants were not in fact passive. and being unable to participate as much as they wanted (4. Ph. Overall. 30%).3. One way or another. technical problems (10. Active and passive levels of participation provided a technique for coherent and accurate measurement that can be used by the e-tutors to support the e-learners. Thesis Centre for Interactive Systems Engineering LSBU Page 157 . most said lack of time (N=40. This was however feasible with the Collaborative e-Learning Episodes analysed in the next section. the overall participation was decreasing and increased slightly at the end of the course. 10%). the main reasons were lack of knowledge (both personal and technical) as to how to collaborate online (Table 2.D.2-1). the messages for analysis in the research pool were found to have more replies than the ones in Moodle@GSN. This section discussed active and passive participation. Furthermore.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Europe on Comenius projects. and perhaps this prevented the participants from engaging in the way they wanted. When the participants were asked about the worst thing in the course. 25%). 12.

) 6.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study 6. Quantitative Variables Comparison of the messages quantitative analysis in the two environments (see data in Appendix XI) led to the following results (Graph 6.1. (My messages were not counted unless indicated otherwise. All images and Urls were considered in the analysis and counted as one item. find correlations between passive active participation and collaborative e-learning. The quantitative variables are: richness of text. The proposed CeLE analytical framework considers 4 variables. and density. and evaluate the CeLE updated version. discussions depth.1-1): Ph.4 THE COLLABORATIVE E-LEARNING EPISODES Messages analysis was used to determine the number and quality of the Collaborative e-Learning Episodes (from now on CeLE) and aimed to: • • • • distinguish mere provision of information and collaborative e-learning as one measure of e-learning quality.D.4. The CeLE structure as such is the qualitative variable. Thesis Centre for Interactive Systems Engineering LSBU Page 158 . All messages from both Moodle@GSN and the research pool were analysed. provide some evaluation towards the pedagogical usability and use of the new collaborative tools.4. 3 quantitative and 1 qualitative.

richness of text. Richness of Text. & Discussion Depth Graph 6. and discussion depth Page 159 Chapter 6: Main Study /2 00 8/ 03 7) ct s( sA /2 01 00 rc -0 7) hi 8/ ve 05 /2 ( 00 B l 0107 og 3) /0 s( 3/ 01 To -3 200 Bl 1/ ols og 0 3 7) (0 /2 & 300 HT 12 7) M /0 L 3/ Pr (0 20 ob 07 lem 5-1 ) 2/ s( 03 12 /2 De 00 -2 si g 3/ 7) n Pr 03 (1 ac /2 700 tic 18 7) ali /0 ty 3/ (1 20 Gr Te 07 o u 3 -1 8 ch ) ps ni c /0 3/ (1 al 2 8Pr 24 0 0 7 ob ) /0 lem 3/ 20 s 07 G r (1 Pr ) ou 6-2 oje 2/ ps ct 03 (1 id 8.D.Ph./ 07 ea 23 ) s /0 (0 3/ 1Ot 07 h e 0 6/ ) 03 r( /2 01 VC 0 N e 18/ 07) in 03 Ews /2 lea (2 00 rn 8in 30 7) g /0 (2 3/ 8/ 07 03 ) -0 1/ 04 /0 7) Messages for Analysis Richness of Text Discussion Depth (E-Tutors) Discussion Depth (Participants) .1-1. Thesis In t ro du cti on (0 1et ab So Ti m 10000 cia 1000 ln le 100 10 1 et wo 1 – 28 /0 3/ 20 07 ) ) 07 4/ 03 -2 -0 rk ing dl (0 1 (0 1 e 3/ 20 M oo lem (0 21 /0 Centre for Interactive Systems Engineering LSBU Pr oje Pr ob Subjects & Dates Tools and Evaluation Techniques for Collaborative E-Learning Communities Correlations between Messages for Analysis.4. Comparison between messages for analysis.

175 in Moodle@GSN and 80 in the research pool. Qualitative Variables E-learning quality in this study was measured by the Collaborative e-Learning Episode (CeLE) analytical framework. explorations.4.4. N=number of participants).782(82) 526(9) C Blogs 1 61 Total 3 1 Ph. 2a/N(N-1) (a=interactions. A productive CeLE occurs when the participants introduce a new idea. Fahy’s formula does not consider interaction time. However. agreements. richness of text as well as the discussion depth related to both e-learners and all participants (elearners and e-tutors). All these variables were found to be interrelated. Messages density was almost double in Moodle@GSN (0. when the text was rich. and is highly sensitive to the size of the group. There were 255 messages suitable for analysis. information and questions). Quantitative variables can give a specific picture of the research context.1-1 provides a comparison between messages for analysis. Density was calculated with Fahy and colleagues’ density formula. middle (explanations. and evaluations).4.4. Thesis Centre for Interactive Systems Engineering LSBU Page 160 .2-1: Collaborative E-Learning Episodes Overview COLLABORATIVE E-LEARNING EPISODES CeLEs in Moodle@GSN Section A Introduction Total 1 5 Forum # Introduction 2 Blogs Threads 58 I II III IV V VI CeLEs # Thread #18/ 10(3)-PX2 #25/ 8(1)-PX1 #29/ 7(1)–P18 #46/ 5(1)-P2 #50/ 18(2)-P52 #54/ 6(1)-P15 # Words 601(8) 623(11) 133(6) 276(93) 1. 6. in other words. Richness of text and discussion depth involved counting the messages and the words included. there is a need to investigate its quality. The analysis suggested thirteen CeLEs (Table 6. A CeLE is an argumentation cycle with starting points (social cues. however.D. and ending points (summaries.1). social cues.19) compared with the research pool (0.2. and silence). These messages were categorized and inserted in Atlas-ti™ in order to find the messages that indicated at least one CeLE.2-1): Table 6. then there were more messages for analysis this means having more than one replies and the higher the discussion depth. disagreements.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Graph 6.

with 3. Similarly.378(140) 3 10(1) 7 6 17(1) 5(1) 4(1) 3 10 3 3 4 4 5 CeLEs in the research pool were more than twice as rich as in Moodle@GSN.378 words (70%) in the research pool. Thesis Centre for Interactive Systems Engineering LSBU Page 161 .941(209) 725 2.2-2. I initiated 4 discussions in the research pool and none in Moodle@GSN.4. there was 1 CeLE in the news forum and 6 in the videoconferencing discussion in the research pool.941 words (30%) in Moodle@GSN and 9. It appears that the richest CeLEs last on average 4 days and in 2 occasions the richest CeLEs were open 10 and 11 days rather than the average of 4 days. They become Ph.990(86) 474(27) 153(18) 2 9 Total 2 It appears that in regard to Moodle@GSN. this means that the richest in words CeLEs were initiated by e-tutors.4. CeLEs overview in the two environments is presented next in relation to individual posters and duration in days (researcher’s posts in parentheses) (Table 6. P52 was also an etutor. and 5 in the blogs discussion.782(82) 526(9) 3.213(9) 882 941 3.990(86) 474(27) 153(18) 9. 1 CeLE was found in the introduction forum.D.2-2): Table 6.213(9) 882 941 3. Collaborative e-Learning Episodes selected for analysis FINAL CeLEs Moodle@GSN CeLE # I II III IV V VI Total VII VIII IX X XI XII XIII Total # Individual Posters 4(1) 6(1) 5(1) 5(1) 9(1) 4(1) Dates 01-05/03/2007 06-06/03/2007 05-05/03/2007 03-03/03/2007 18-22/11/2006 & 25/02-02/03/2007 28-28/02/2006 Research pool 27-29/03/2007 21-31/03/2007 29-31/03/2007 27-29/03/2007 26-29/03/2007 26-29/03/2007 24-28/03/2007 Duration (days) 5 1 1 1 11 1 # Words 601(8) 623(11) 133(6) 276(93) 1.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study CeLEs in the Research pool 1 G VC in ELearning News VC in Elearning 8 VII VIII IX X XI XII XIII #4/ 3-P13 #3/ 18-R #4/ 10-P37 #6/ 8-P50 #7/ 22-R #8/ 5-R #9/ 6-R 725 2.

Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study richer as the course is unfolding related to the number of days and the number of words. threads and individual posters can also provide a posting timeline (Graph 6. III. Collaborative e-Learning Episodes temporal overview Observation Course Day 0 1 2 3 4 5 6 7 8 O0 O1 O2 O3 O4 O5 O6 O7 O8 01/03/2007 03/03/2007 07/03/2007 14/03/2007 21/03/2007 28/03/2007 31/03/2007 INTERVENTION & EVALUATION CeLE in Moodle@GSN Tools & CeLEs Baseline 1 n/a n/a n/a n/a n/a VII.2-3): Table 6. the CeLEs stopped on 06/03/2007 and appeared again on 18/03/2007. For example. For now. XI. Threads & Indivudual Posters 1. The CeLEs’ temporal overview is presented next (Table 6. II. XIII VIII.204 18 9 3 882 10 7 3 941 8 6 4 3. These first CeLEs were on getting to know each other as well as technical problems.700 18 8 1 517 6 3 3 725 3 3 10 2.904 447 135 8 5 1 7 4 1 5 4 1 6 3 1 10 7 3 8 6 3 CeLE-II CeLE-III CeLE-IV CeLE-V CeLE-VI CeLE-VII CeLE-VIII CeLE-IX CeLE-X CeLE-XI CeLE-XII CeLE-XIII 1 612 8 5 1 127 7 4 1 183 5 4 11 1. an overall comparison on duration in days.2-3.4. XII. For example CeLE III had three chunks of smaller enquiries that led to answers to questions unfolding within the CeLE. The first four CeLEs were initiated in the first week and then a 2 weeks gap appears. It is interesting to note that there is a gap after the first week.904 22 16 4 447 5 4 5 135 6 3 Graph 6.2-1): 10000 Number of words 1000 100 10 1 CeLE-I Duration Words Threads Posters 5 593 10 4 10 5 4 CeLEs: Comparison between Duration (days) and number of Words.4. X. CeLEs factors’ comparison graph Ph.4.2-1. IV V & VI Baseline 2 Most of the Collaborative e-Learning Episodes in Moodle@GSN were technical enquiries and most of the time the discussions were developed in small chunks.D.4.700 593 612 127 183 18 11 8 18 10 9 3 22 16 4 5 4 6 5 3 517 725 2.204 882 941 3. the last ones were of more quality. and more CeLEs appeared towards the end of the course. number of words. Thesis Centre for Interactive Systems Engineering LSBU Page 162 . IX I. the CeLE-I lasted 5 days and had 601 words.

In other words. However.246 1. threads and individual posters reach a peak.056 384 142 2. and idea generation reaches a peak.D.389 89% 0 0 167 36 786 0 0 989 11% 3 10 5 5 15 5 4 Research pool 3 10 7 6 17 5 4 3 10 3 3 4 4 5 It is evident that the e-learners were more active in the research pool.2-4. when duration is in days. idea generation is found to be related to all factors. This difference is significant as illustrated in the next graph (Graph 6.695 53% 47% 725 2. the aim of collaborative e-learning is e-learners’ generated text. there were 8. Since a CeLE is related to co-creativity and new knowledge construction.2-4): Table 6. Thesis Centre for Interactive Systems Engineering LSBU Page 163 .695 (47%) from the e-tutors.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study The previous graph presents that there are interrelations between the factors in a successful CeLE.389 words from the elearners (89%) and 989 (11%) words from the e-tutors. number of words. In Moodle@GSN there were 2.4.246 words from the e-learners (53%) and 1. CeLEs: e-learners and e-tutors’ contributions FINAL CeLEs CeLE # I II III IV V VI Total Percentage VII VIII IX X XI XII XIII Total Percentage 7 8 9 10 11 12 13 1 2 3 4 5 6 Moodle@GSN # E-learners – Duration E-tutors Unique # posts (days) 2 3 2 2 5 2 4 6 5 5 9 4 5 1 1 1 11 1 # Words E-learners E-tutors 396 205 455 168 93 40 192 84 726 1.204 474 153 8. As for the research pool.213 715 905 3.4. for this reason the same analysis is presented with and without e-tutors’ contributions (Table 6.2-1): Ph.4.

These were expressed both as monologues as well as threaded dialogue.D.2-2. Comparison for number of words posted by e-learners and e-tutors The previous graph presents that the e-learners’ ratio of words in posting was increased by 32% in the research pool CeLEs. CeLE-VIII seems to be interesting for investigation since there were 10 messages sent by 10 different individuals. information exchange in CeLE III had a linear structure which unfolded as a problem solving activity on a technical problem. from Moodle@GSN and the research pool. In addition. one from each environment. In summary. all being low active participants. To conclude the section on the qualitative variable. 2 CeLEs were selected. CeLE-III from Moodle@GSN and CeLE-IX from the research pool. In CeLE IX there were more agreements. The two environments were viewed as complementary so selection was based on the most interesting and representative examples. CeLE-II and CeLE-IX found to be similar as regards the number of words as well as the number of e-learners and posters. On the other hand. The last CeLE detailed analysis is presented in Appendix X (A_X_1-4). and exploration of each other’s ideas. These processes can be seen as knowledge internalisation and externalisation through monologue and dialogue which promoted participants’ critical and creative thinking (Appendices A_X_5 and A_X_7). arguments.4. disagreements. CeLE IX did not have a linear structure and referred to the use of specific e-learning tools in building a project.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Number of words posted in CeLEs by E-learners & E-tutors 9000 8000 7000 Number of Words 6000 5000 4000 3000 2000 1000 0 1 Moodle@GSN 2 Experimental Environment 57% 2246 E-learners E-tutors 43% 1395 11% 989 89% 8389 Graph 6. Thesis Centre for Interactive Systems Engineering LSBU Page 164 . Ph.

however. can be indicated by the fact that previous knowledge existed in Moodle@GSN. the CeLEs were not as rich. Ph. 2006). Thesis Centre for Interactive Systems Engineering LSBU Page 165 . The use of tools to reveal these structures even on a limited level. Jeong & Davidson-Shivers. It appeared that there were transitions between internalisation and externalisation of learning and this means that design should explicitly support dialogical sequences by broadening and deepening this space. the appearance of these structures with the aid of previous knowledge on collaborative e-learning and even the limited use of MessageTag may be reasons for tackling passive participation (e. Klemm.. Khine et al.g. viewing. 1998. Knowing. varied and consistent as in the research pool.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study As there was only one participant in a medium activity level and seven participants on a low activity level. 2003.D. and carefully using the collaborative e-learning technique can give confidence to and encourage e-learners in their participation.

Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study 6. The next sections will explore the SeLCI starting with the community evolution. and structural equivalence. and 5. social network analysis was aided by UCINET (Borgatti et al. Ph. Finally. 7. Quantitative.D. Thematic analysis was conducted for the open questions and the collaborative e-learning episodes using Atlas-ti™. 6. and social network analysis were employed within the ethnotechnological framework to evaluate the SeLCI. Sense of belonging to the e-learning community. help and support. Thesis Centre for Interactive Systems Engineering LSBU Page 166 . 2. knowing about the community. and Local Real-Time Nodes and Centrality: tools results. The SeLCI attributes are: 1. the questionnaires from 40 participants (N=40) were analysed. 3. shared interests and values. As before. Intensity: levels of passive and active participation. new members’ contribution. thematic. Social Network Analysis o Global cohesion: density. and persistence. and the collaborative tools. reciprocity.. cliques. Trust: knowledge exchange. Empathy as a representation of what co-learners know and feel. 2002).5) and Microsoft Excel. and the number of collaborative e-learning episodes. closeness and betweenness.5 THE SENSE OF E-LEARNING COMMUNITY INDEX The Sense of e-Learning Community Index (SeLCI) aims to provide in-depth insights to human-human interactions within e-learning communities. 4. The data for quantitative analysis were inserted in the Statistical Package for the Social Sciences (SPSS 11. Collaborative e-learning quality: participants’ opinions. Community evolution: initial opinions on the community evolution. o o Global centrality: centrality degree.

2 responses were N/A (not applicable) so there was not an opposing opinion.1-1): Community Evolution Number of responses (N=62) Increasing interactivity / help Communication outside and after the course Common interests / goals Online communication 0 2 4 6 8 10 19% 12 12 14 Affective elements Quick familiarisation Increasing number of participants Communication – general Collaborative atmosphere E-learners’ participation in planning Sense of belonging Personal messages / experiences / first name New colleagues Photos Visiting others’ web pages Knew nobody initially Participation quality / quantity 13% Communtiy evolution elements 11% 10% 6% 5% 1 3% 2% 1 1 1 1 1 2 2 2 2 3 4 4 6 6 7 8 Graph 6. Community evolution elements Ph. 95%) thought that the community was developing.5. As regards the time needed for community evolution. new members’ contribution. this means 1 to 2 weeks is necessary for developing a community and collaborative e-learning where the sleeper effect is considered to occur.1-1. In other words. knowing about the community.5%) 1 week.5. the first week the participants explore and familiarise themselves with the system and the other learners and make decisions upon passive or active participation. Evolution: Almost all participants (n=38. If collaborative e-learning started developing after the first week. most of them (n=17. 10 about 45 days (10%) and the rest 6 (15%) 1-3 days.D.1 Community Evolution The first SeLCI attribute is related to understanding the community evolution.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study 6. and the community roles. Participants were asked to suggest elements that showed community evolution on an open question (Graph 6. Thesis Centre for Interactive Systems Engineering LSBU Page 167 . 42. 13 (32.5. shared interests and values.5%) said it needs some time to develop: 2 weeks.

some people did not know anyone initially. participation in course planning as adjustment to particular circumstances. 19%) as well as increasing communication outside and after the course (e. Ph. This means that community evolution can be moderated by supporting communication and participation in collaborative activities by helping elearners becoming passive and then active participants.10%). blog.D. blogs. the level of communicative activities (n=3. Facebook. Thesis Centre for Interactive Systems Engineering LSBU Page 168 .g. Other elements (1 response. however. 2%) were on meeting new colleagues. 24 said yes (60%). This may also suggest that different people have different desires when it comes to communication modes. the increasing number of participants (n=4. For example. nobody said no. 10%). 3%). exchange of personal messages. and Facebook) other than the tools inside the e-learning system has been found crucial for the community maintenance (Boase et al. a Facebook group be notified if a new course will start. and the quantity and quality of participation. phone.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study The previous graph presents that the most important community evolution elements were the increasing participation based on mutual help (n=12. The use of other communications means (e. n=8. The increasing interactivity and participation indicated that the community was an evolving organic entity. Willingness to keep the community going was also evident on their demand for a blog. 6%). and indicates a healthy practice in this study. the number of photos in profiles.g. Having common targets and discovering similar interests creates a tendency known as selfdisclosure reciprocity as the participants exchange personal information and experience (Wallace. experiences. 1999). affective elements such as trust and support (n=6. and address of e-learners by their first name (n=2. visiting each other’s personal web pages and adding each other’s links. SKYPE. and from 2 responses (3%) the collaborative atmosphere. then shared goals and interests (n=7. the quick familiarisation with the e-learning environment (n=4. phone. 2006). 11%) communication during the course (n=6. so this request was not feasible. 6%). there were 16 N/A and missing responses (40%). 5%). However.g. 13%). the use of profiles and the initial social interactions to enhance trust and empathy supported elearners’ quick familiarisation with each other as well as with the LMS. When they were asked whether they would continue their collaboration outside the course. SKYPE. the sense of belonging and expressions of familiarisation e. because this is not organised by GSN and the online course was part of this research.

Following the literature (e.5 35 32. 82.1-1.5 5 12. 28 (70%) said they developed online relationships. Most of the participants thought that they knew about the netiquette (n=31.5 40 35 15 52.5 6 4 5 5 8 5 5 15 10 12.5) and the new tools significantly helped in this process (n=33. The LMS provided the platform for the community to exist (n=35.g. 17.5).5 17. Knowing the community: The participants said the following about the community (Table 6. most participants replied to the closed question that they had same interests (n=35.5 20 12.5 12.5%) and 7 were N/A (17. 27. Preece. 87.5%).5 14 13 16 14 6 21 20 35 32. they were positive on collaboration and the Ph. Most also said they had shared values (n=32.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Shared Interests & Values: Shared interest was found to be the third community evolution element. In this study these results were more than 80% which means that there was a strong common ground from the participants’ viewpoint.5 30 42. 82.5%). 87. although 1 denied (2..5 37.5%) and were expressing themselves freely (n=29. shared values and interests is the most important indicator to identify whether a community existed or not.5 27. Although the majority of the participants did not have experience in e-learning communities and online collaboration. half of the participants knew nobody before the course and the other half knew a few. an essential element for collaborative e-learning.5%) as well as the community (n=25.5. On freedom of expression the results were lower (n=11. 1 2 3 4 5 6 7 know the netiquette? know the kind of community? like working together? express yourself freely? participate actively? think that the LMS helped the community? think that the new tools helped the community? 1 Very small 2 3 Neither 4 5 N/A 2 5 1 11 2 5 2.D. Also.5%). Moreover.5.1-1): Table 6. Knowing the community THE COMMUNITY: Did you. 77. 72. 2000).5%) as well as active participation (n=7.5 50 2 5 1 2. 2 denied (5%) and 2 were N/A (5%). 80%)..5 32.5 12.5%).5 6 15 7 The respondents liked working together (n=33.5 17 12 17 15 13 14 13 42. Thesis Centre for Interactive Systems Engineering LSBU Page 169 . 10 (25%) did not (2 were N/A). 62.5%).

5.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study only thing that they were not happy about was not to be able to participate as much as they wanted. bringing new perspectives and abilities into the community. The most important element is their enthusiasm (n=5. 3%) are freedom of expression. Other responses refer to the need for training (n=3. 9%). 5%). 6%). Using tools in e-learning was important to them. Thematic analysis on the e open question also revealed the following (Graph 6.5. 6%) as they bring up questions (n=4. Ph. from two responses each (n=2. 6 (15%) were N/A and nobody opposed. Thesis Centre for Interactive Systems Engineering LSBU Page 170 . exploration with propositions and criticism. 8%). quality and life-long learning are the last elements on new members’ contribution. Shared interest. provide feedback for older members. 24%) to share with the older members (n=9.D. 5%) and to keep the community going (n=3.1-2): Graph 6. 14%) via active interaction and participation in the community (n=6. The participants believe that heterogeneous groups function better (n=4.1-2. knowing what the community was about and developing relationships was a strong determinant in SeLCI. New members’ contributions The respondents believed that the new members can regenerate the community by bringing new ideas (n=16. whereas willingness to learn and collaborate is essential as well as the use of new technologies. New members’ contributions: The results from the closed question on new members’ contribution were in favour of new members: 34 (85%) supported their contribution.

the lack of training in the use of ICT in Education and knowledge of collaborative techniques may be due to absence of teachers’ life-long learning courses in Greece (Tsetsilas 2006). and 11 (27.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Moreover. Thesis Centre for Interactive Systems Engineering LSBU Page 171 . According to the closed question. 6 (15%) there were not.1-3): Ph. ‘old members can identify the new knowledge coming from newbies and help in its transfer in positive ways’. it is evident that new ideas are of major importance to keep the community going. This means that the Greek teachers even as newcomers have not recently graduated so there is a gap in their professional training especially when there is a demand for faster training cycles in recent years. thematic analysis on the open question revealed the following (6. In addition. In other words. there were older newcomers.5%) thought there were roles.5. this is another collaborative elearning beneficial factor to community evolution.’. more experienced members can elaborate and analyse the coming data to transform the elearning context and future actions as interaction that can only bring positive results. Roles: The participants said that there were roles assigned to the members. two issues were revealed with regard to the new members in an elearning community.5%) thought there were no specific roles. Based on their questions. 23 (57. P7 and P9 made interesting suggestions. However.D. however. Additionally. To P9. Despite the evidence on the importance of newcomers bringing new ideas to regenerate the community. P7 said that new members ‘…contribute their own knowledge that most experienced members can take advantage of depending on e-learning targets. there was only one young participant in the study.

students (n=6. As for the extent of provision of help that can define roles between community members.1-3. 13 (32. 3%) were also included. Roles in the e-learning community Participants’ responses were on community management. The moderator’s role was found important (n=7. There were also teachers (n=5. and technical support (n=2. community management. 21 (52.5. Based on the assigned role of the moderator and motivator. active participants (n=2. 6. 18%) along with motivators (n=5. Overall. Thesis Centre for Interactive Systems Engineering LSBU Page 172 . 14%) and leaders (n=4. 14%). e-learning and participation were themes for roles which could be assigned.5%). 1 said it was not strong Ph. the assignment of roles was not found to be a very strong element in community evolution and collaborative e-learning. Equality in participation was also indicated by the fact that e-tutoring and moderating was evident in Moodle@GSN and did not exist in the research pool. In other words.5%) very great.D. e-learning.5%) said that was neither great nor small. 6%) and observers (n=1. it appears that organisation of the community and elearning were of equal importance. 11%).2 Sense of Belonging On the question of bonding or togetherness 16 respondents said it was strong (47. 17%).Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Graph 6.5. 18 said it was neither strong nor weak (45%). 6%). 4 great (10%) and 2 (5%) to a small extent. and participation.

Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study (2. to P39 there was a ‘desire to keep the community going’.D. P14 said that concerning the use of profiles that the ‘photos created a climate of familiarisation with each other and helped in developing a sense of belonging’. P11 believed that something that shows the sense of community is active participation despite the technical problems. participation as such and effective learning (n=3. Other suggestions (n=1. collaborative learning for projects development and problem solving (n=2. enthusiasm.5%). affective elements such as mutual help and trust. More responses were: desire for success and communication (n=4. and equally e-tutors (n=5. 7%). Furthermore. and 2 were N/A (5%).2-1): Graph 6. 3%). 7%) and collaborative tools (n=5. 30%).2-1. there was ‘willingness to collaborate beyond and after the online course’. 16%).5. to P35 there was an ‘incredible increase of active participation’. Thesis Centre for Interactive Systems Engineering LSBU Page 173 . In the question on the factors that kept the community together they said (N=40) (Graph 6. understanding. 5%). 1%) were the immediate success and feedback as well as reliability. Participants’ opinions on e-learning community elements More than half of the participants (n=22) said that sharing the same goals and interests was the main reason for holding the e-learning community together (n=22. Ph. and to P38 and P47 there was mutual help. and trust. 4%). the subject.5. then curiosity for new knowledge (n=12. willingness to participation. to P22.

5. In other words. 6.5 27. then e-learning (35%). The mediators in this process are the e-tutors and the tools. Comparison of themes and community elements Community management is considered the most important factor in this study (59%).3-1.5 Action Number 11 7 9 11 Percent 27.3 Empathy Table 6. Empathy factors EMPATHY FACTORS EXTEND Very Small Small Neither Great Know what Other was feeling when reading a message Number 1 8 17 13 Percent 2. new knowledge is the collaborative e-learning outcome.2-2): Correlations between Codes and Resposnes 100 48 21 10 10 6 5 Codes Responses 1 Community Management (59%) e-Learning (35%) 1 Technology (6%) (Percentage on codes) Graph 6. Learning is an important element of community evolution and vice versa.5 32. Also.5.5 Ph.5.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Overall.5 17. It is interesting to see an overview of the importance the participants applied to the observed clusters grouped in three themes (Graph 6. it seems that the same factors that defined the sense of belonging are the factors that contribute to the e-learning community evolution: shared interests and values.2-2.5. meditated by artefacts.D. Thesis Centre for Interactive Systems Engineering LSBU Page 174 . community management facilitates community evolution and thus allows e-learning to take place.5 20 42. and third the tools (6%).5 22.5 Feel what Other was feeling when reading a message Number 2 6 15 15 Percent 5 15 37. in this case e-learning tools (Lave & Wenger 1991).5 37.

so significance evaluation was conducted with Pearson's Correlation Coefficient (Table 6.5.5%) and one response (2.5%) they knew to a great extent. A scatter plot provided a more detailed view (Graph 6. It appears that the results reach a peak in “knowing what the other person was feeling”.9 100 2 0 40 5 0 0.5%) was N/A.5.3-1.5%) replied they more or less knew what the other person was feeling when reading a message and 13 (32. 1 very small (2. 37. Thesis Centre for Interactive Systems Engineering LSBU Page 175 .5. Scatter plot for empathy Convergence of the three empathy parameters when reading a poster’s message. 8 (20%) said they knew to a small extent. these are “knowing what Other was feeling”. 11 respondents (27. they are more distributed on “feeling what the other person was feeling”. 7 (17.5) equally said to a very small and great extent. “feeling what the Other was feeling” and “action taking” seem to be on both a small extent and a great extent. 6 (15%) said they could feel other’s feeling to a small extent.5%). As for whether they took any action. Plotting suggested a linear relationship between the three variables.D.3-2): Ph.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Very Great N/A SD Total 0 1 40 0 2.3-1): Graph 6. the responses were equally distributed between neither great or small and great extent (n=15. and equally 2 (5%) were on the very small and very great scale. and they reach a down peak in “action taking”.3 100 On the question of knowing what someone else was feeling (42. On “feeling what the Other was feeling”.5) to a small extent and 2 (5%) took action to a very great extent.5 0.9 100 2 0 40 5 0 1.

347(*) .028 40 Feel what Other was feeling when reading a message . Lanzetta and Englis.001 .028 .5.D. A moderate association (r=0. “Feel Other” and “Action” association was the weakest (r=0. 2007). In this study.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Table 6. (2-tailed) N 1 . as a coefficient of reliability and consistency between the variables found α=0. in regard to action taking.000 40 Action .347(*) .527(**) . 1989) it seems that it can occur online and it is influenced by the properties of different communication media (Preece. There is a need to note that empathy has been related to the mirror neurons.g.01 level (2-tailed). (2-tailed) Pearson Correlation Sig.001 1 . which is on the limit of acceptance in social research (0.527) is between “Feel what Other was feeling when reading a message” and “Action” with correlation coefficient p≤0. .05.489(**) . this means that the e-learners in their profiles provided information to the mirror neurons to build representations of the other learners and their actions (Goleman. investigation of such correlations was not part of this study.347) with correlation coefficient p<0. people may need time to actively participate in activities (the sleeper effect) before consciously decide to work with others (the least collaborative effort).01. Although empathy has been related to gaze and body language (e.05 level (2-tailed). . Thesis Centre for Interactive Systems Engineering LSBU Page 176 . Ph. It was not possible to extract explicit relationships between the results of this study and others as the frameworks for investigating online empathy were different. Cronbach's alpha.000 1 . Correlations for Empathy factors Correlations for Empathy Factors Feel Other when reading a message Know what Other was feeling when reading a message Feel what Other was feeling when reading a message Action Pearson Correlation Sig.3-2. 2004). the respondents appeared to know what the other person was feeling when posting more than actually feel the poster.70). However. * Correlation is significant at the 0. The strength of association between the three variables is positive but it does not suggest a strong linear relationship. Nonetheless.527(**) . 40 ** Correlation is significant at the 0. (2-tailed) Pearson Correlation Sig.698.489(**) .

5%) could trust them more or less.5 5 5 The contradictory closed questions on trust suggest high level of participants’ reliability.5.4-2.5. I have to be very careful as some take advantage of others. Yes 37 Percent 92. the results indicate trust to a great (n=16. the majority (over 90%) responded that they could trust the other learners as most were trying to help.5%).5 N/A 2 3 2 2 Percent 5 7.5 22. 30%) whereas 12 (30%) could trust them more or less.5 40 5 Very great 12 12 30 15 30 30 75 37. So trust was evident to a significant extent.5) neither small or great extent. Trust development towards individuals TRUST Individuals who had a similar specialisation Individuals who had similar writing skills E-tutors Experienced individuals You can’t trust anyone 1 Very small 2 Small 3 Neither 11 12 1 9 32 80 6 15 1 27. as for individuals who had similar writing skills.5 2. The results on trusting the etutors were the most positive with 39 (97.5 4 Great 16 16 9 16 40 40 22. 15 to a very great extent (37. 40%) and very great extent (n=12.5%) and 9 (22. Thesis Centre for Interactive Systems Engineering LSBU Page 177 .Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study 6.5 92. Nobody can trust anyone else. As for trusting the experienced individuals the results were: 16 (40%) to a great extend. 30%) whereas 11 (27.4 Trust The participants responded to statements on trust (Table 6.D.5 30 2.5.4-1): Table 6. More specifically (Table 6. Trust levels TRUST 1 2 3 4 Statements I can trust most of the participants. The question to ensure that the Ph. Most are trying to help. 40%) and very great extent (n=12.5.4-2): Table 6.4-1.5.5 1 2 3 4 5 It appears that most of the participants could trust other e-learners with a similar specialisation to a great (n=16.5 1 2.5%) to trust them on a great or very great extend and 1 neither a small or a great extent (2.5 No 1 37 38 95 38 95 Percent 2.5 1 N/A 2.

such as personal information.5 Intensity Intensity refers to the participation levels and persistence. demographic backgrounds.5. Passive & Active Participation Process Ph. 72. Participation Levels: Active and passive participation levels were found to vary in the two environments. Almost all participants felt that they could trust the e-tutors and their co-learners. 2007:124). presence on the community. Trust is also related to the mental models people develop when they first meet as well as the content of their conversation and tend to develop very quickly (Norman. Trust has been reported to be related to different forms of awareness. 6. here. capabilities and skills in performing specific tasks (Daniel.D. However. trust was linked to reciprocity and levels of participation to allow participants to work freely together.5%).5-1. Seeing these levels as a process from low passive to high active provides a different overview for intensity (Graph 6.5.5%). evident in the freedom of expression to a great extent (n=29. 97.5-1): Passive & Active Participation Process 35 30 Number of participants 25 20 15 10 5 0 1 Zero Low Passive Participation 7 5 4 1 5 1 5 1 16 A: Moodle@GSN B: Research Pool 32 Medium High Passive Low Active Medium High Active Passive Participation Participation Active Participation Participation Participation Graph 6.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study participants did not respond randomly revealed that the majority (n=32. all but 1 participants reported that the level of trust had risen (n=39. the latter is the level to which participants pursue topics. Thesis Centre for Interactive Systems Engineering LSBU Page 178 .5. 80%) could trust other people. these suggestions were not explicitly investigated in this study. 1988). Lastly.

Line B corresponds to the research pool and seems more broadly distributed showing higher activity than Line A. Persistence: The intensity element of persistence is the level to which participants pursue topics in order to evaluate the emergence of a clear focus.D. Persistence was located on two levels. In other words.5. Lastly. (Note that the e-tutors used the ‘split forum’ facility to facilitate the flow of the conversation and intervened in the persistence ratio in a positive manner. from an activity viewpoint.21/03/2007) Timetable (01 – 28/03/2007) Social networking (01-24/03/2007) Moodle (01-08/03/2007) Problems (01-08/05/2003) Total B Project Management Total C Blogs Total D Wikis Total E F Videoconferencing Total Internet Cafe 2 Project ideas (01-06/03/2007) 4 Technical Problems (16-22/03/07) Groups (18-23/03/07) 3 Problems (12-23/03/2007) Design (17-18/03/2007) Practicality (13-18/03/2007) Groups (18-24/03/2007) 1 Blogs (01-31/03/2007) Tools (03-12/03/2007) Blog & HTML (05-12/03/2007) 5 Projects Archive (01-07/03/2007) Depth of persistence Forums 43/49 7/7 1/1 2/2 6/22 59/81 1/1 1/1 42/43 3/3 1/1 46/47 9/9 0/2 2/2 6/7 17/20 1 1 2/2 1/1 Messages 66/231 127/127 30/30 7/7 117/127 347/522 6/6 6/6 187/198 9/10 3/3 199/211 37/37 0/6 10/18 31/31 78/92 12/12 16/16 28/28 3/3 A Introduction Ph. As described in the previous sections.5-2): Table 6. Persistence in Moodle@GSN PERSISTENCE IN MOODLE@GSN Section Forums Introduction (01. high low active participation and one e-learner in medium and active participation. intensity in the research pool seemed to be in greater balance.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Line A corresponds to Moodle@GSN and is interrupted. Thesis Centre for Interactive Systems Engineering LSBU Page 179 . discussion topic and thread.5-2. this may be because medium and high activity users were e-tutors whereas there were no etutors in the research pool.) Both the number of initiations and replies with more than two messages were calculated (Table 6.5. the participants were located in zero and low passive participation. it is evident that participation increased from passive to active levels on a social and temporal basis.

The forums overall percentage was 83. Two similar changes of focus occurred in the blog/blogs discussion when a problem on the suggested url appeared and 5 messages were on finding the correct url. sometimes coming from the same e-learner.1 6/6 9/9 667/868 76. to serve the audience. the more the replies the more the probability of lack of persistence.) Both forums on ‘wiki design’ were irrelevant. a large number of forums indicated a small number of replies so the level of persistence was maximum for these forums. in other words.8%) with 667 out of 868 messages to follow the discussion topic. as in videoconferencing persistence was 100%. without doing it intentionally. shift of focus was justified. for example. both or something else?’ As for levels of persistence in replies. As for the rest of the messages. Thesis Centre for Interactive Systems Engineering LSBU Page 180 . 5 out of 6 irrelevant messages in the ‘Problems’ forum came from P48. I am P50. for example. this perhaps was because of the title forum: ‘What are the main characteristics in wiki design? the goal to match. there were a small number of them irrelevant to the topic.8 80 The previous table presents that 59 out of 81 forums and 347 out of 522 messages were following the subject of discussion in the introductory section. Ph. in wikis the persistence depth was 17 out of 20 in forums and 78 out of 92 for messages. 2 in 2 forums and 28 out of 28 messages. in blogs. in the project all messages were on topic these are 1 out of 1 forums and 6 out of six messages. in Wiki topic.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Other (01-18/03/2007) Total Overall Overall Percentage 2 16 Average Percentage 2/2 3/3 128/154 83. the same rate appeared in the internet café with 3 in 3 forums and 9 out of 9 messages. the focus was shifted in the practicality discussion when one of the members found to have posted in another person’s wiki without realising it (Reply 5): “Hi O3. you created a webpage post in my wiki…”. Some times there was a reason for shifting the discussion focus. More persistence was observed in messages with 2-3 replies.1% with 128 out of 154 forums persistence depth which meant that the forums were relevant to the forum topics. Lastly. The overall persistence depth in Moodle@GSN was 80%. Overall. they were events announcements. the persistence depth was found to be relevant to the thread depth. persistence was strong as 46 out of 47 forums and 199 out of 211 messages were on topic. (Note that there were no members’ announcements area in the online course. Slightly lower was the messages overall percentage (76.D.

The overall persistence in the research pool was 95. 2003). and the last two messages partly brought the focus back to the use of tools in projects. as the participants gained knowledge and experience of collaborative techniques based on information provision and observation of the etutors. This is done by checking whether a conversation partner has heard and correctly understood what is being said (Preece & Maloney-Krichmar. their behaviour changed in the research pool. Lastly. Thesis Centre for Interactive Systems Engineering LSBU Page 181 . More intensity found in the research pool: the process of participation was more stable and coherent. with 90.5.6% percentage.D. there were 68 out of 75 messages exhibiting persistence. It appears that initial online dialogical argumentation lacked depth and was redundant as participants failed to sustain interaction. it Ph.3% comparing to 80% in Moodle@GSN even though the number of messages was low. also found to be evident in the research by Khine and colleagues (2003). Persistence in the research pool PERSISTENCE IN MOODLE@GSN Section A Introduction Total Percentage Average 2 Forums News (28-30/03/07) VC in E-learning (28/03-01/04/07) Depth of persistence Forums 2/2 8/8 10/10 100 Replies 5/5 63/70 68/75 90. whereas 1 message produces 27 replies (Chapter 4.6 95. with 10 out of 10 messages following the topic.5. after the 13th reply the focus changed to how the Ministry of Education supports the teachers on the projects and what happens when the project finishes.2). Then the focus shifted to the Greek education system and teachers’ training.5-3.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Similar results on persistence appear in the research pool (Table 6.3%.6%.5-3): Table 6.3.3 The forums had 100% persistence. and there was an overall persistence of 95. It is interesting to note that for 3 years there were 19 messages with 1 one reply. and persistence was found 90. Intensity appeared to be higher in the research pool: the participation levels were more coherent and stable in the research pool having all levels from low passive to high active participation. however. As for the messages.1. Intensity also provides evidence for establishing common ground. In depth analysis revealed that the discussion on re-using videos in a project in the videoconferencing forum produced 21 replies.

5%).6-1.5%) and less from their co-learners (n=3. if the e-learners learned to collaborate. one participant provided more than one suggestion) (Graph 6.5.6 E-Learning Quality The Collaborative e-Learning Episodes analysis provided a qualitative view in online discussions. 6 in Moodle@GSN and 7 in the research pool. There were 13 Collaborative e-Learning Episodes. 92.D. Thesis Centre for Interactive Systems Engineering LSBU Page 182 . 92. The respondents said that they asked help from the e-tutors (n=37. aiming to assess quality in e-learning interactions.5%).Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study is evident that intensity can depict any imbalance or equilibrium between passive and active participation and a level of persistence as a SeLCI determinant.6-1): Collaborative e-Learning 0 2 Number of responses (N=41) 4 6 8 10 12 Information & knowledge exchange Collaborative activities Dialogue development Mobility of ideas Learn to communicate Vicarious learning New skills acquisition Number of created projects Elements for community evolution 37% 27% 17% 7% 1 3% 1 1 1 1 2 5 8 11 Graph 6. how they worked together. and 2 N/A (5%). Elements that show community evolution: e-learning Ph. However. it was considered important to acquire the e-learners’ viewpoint: whether or not the e-learners asked for help and who helped them the most. they suggested ways to achieve collaborative e-learning (N=41. and to determine how the tools aided their learning.5%). The majority replied that they actually learned ways for collaborative learning (n=37.5. More specifically. 7. 6.5. 1 did not (2.

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Chapter 6: Main Study

Information and knowledge exchange were the most important factors for collaborative e-learning (37%); then collaborative activities (27%), dialogue development (17%), and mobility of ideas (7%). One suggestion each (3%) referred to learning to communicate, vicarious learning, new skills acquisition, and the number of created projects. Only 5 responses were on tools, 4 on the new collaborative tools (80%) and 1 on profiles (20%). It appears that mere information was essential to trigger collaboration but it was the collaborative techniques and the use of tools that transformed information to collaborative e-learning by supporting different learning styles. This result indicates that progressive discourse can be the outcome of increasing participation in collaborative e-learning communities. The next graph shows more specific opinions with regard to whom they learned from (Graph 6.5.6-3):

Whom did you learn from? (N=35)

3, 6%

3, 6%

E-tutors Otherlearners 17, 32% More experienced learners 30, 56% Own work

Graph 6.5.6-3. The e-learning facilitators The previous graph presents that the participants learnt from the e-tutors (n=30, 56%), other learners (n=17, 32%) and equally more experienced learners (n=3, 6%) and on their own (n=3, 6%). It appears that passive participation and vicarious learning was one of the learning styles; P21 also said that she was ‘watching how other people were working’. This result is in agreement with the centrality scores and the responses on passive participants. Overall, thematic analysis pointed at three themes, community management, elearning and technology (Graph 6.5.6-2):

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 183

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Chapter 6: Main Study

Collaborative e-Learning
100 62 41

17 10 Codes 8 5 Responses

2 1 Community Management (63%) e-Learning (30%) Technology (7%) (Percentage on codes)

Graph 6.5.6-2. Correlations between codes on collaborative e-learning quality The importance attached to the factors on collaborative e-learning quality is in favour of community management, then e-learning and finally, technology is the last one. It is interesting to see that this attached importance is even higher for collaborative elearning than e-learning community (see Graph 6.5.2-2): the percentage is increased as regards community management (63% from 59%), decreases for e-learning (30% from 35%), and slightly increases for technology (7% from 6%). These results stress the importance of the social aspect of collaborative e-learning. Overall, quality in collaborative e-learning can be a Sense of e-Learning Community Index (SeLCI) attribute as it provides a clear indication of quality in textual online interaction by the number of collaborative e-learning episodes and participants’ viewpoint on their learning. Three distinct learning styles were revealed, instructional, collaborative and vicarious learning. As before, community management was the most important job for the e-tutors, then e-learning activities and lastly, the technology. Some results can be triangulated by social network analysis in order to provide a different viewpoint towards SeLCI. These will be discussed in the next section.

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 184

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Chapter 6: Main Study

6.5.7

Global Social Network Analysis

Social Network Analysis (SNA) can describe Greek teachers’ interactions as well as triangulating previous findings. This is feasible because the relationships based on text and words only have limited capacity to represent the social network; in other words, photos provide a different view of a group of people than the script of what they said. SNA focuses on global (found also as complete or group) and ego networks. Global cohesion and centrality were investigated using UCINET (Borgatti et al., 2002); cohesion can represent the interactions’ weight (density), participants’ preferences (reciprocity), any small groups (cliques), and similar behaviour (structural equivalence); centrality can depict interaction direction (in-out degree centrality), speed (closeness), and control (betweenness). Two adjacency matrices were produced one 64*64 for GSN (Figure 6.5.5-1) that included the main participants and others who had been interacting but not on a regular basis and did not fit the criteria of participants’ selection. Also another 41*41 matrix was produced for the research pool that included my interactions (Valued links and passive participants zeros are depicted; all other zeros were eliminated to make the matrix more legible) (Figure 6.5.7-1):

Figure 6.5.7-1. GSN adjacency matrix in UCINET

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 185

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Chapter 6: Main Study

Several problems occurred in producing the matrices: one was the discrepancy between the system reply and the actual address of reply and the other was the participation of two e-tutors and myself. First, the function ‘Reply’ in Moodle was referred to the previous message whereas the actual reply could be addressed to one or more e-learners sometimes higher up the forum. So in this study, where apparent, the input in SNA was the actual addressed reply as conceived from the message rather than as depicted on the system; this indicated a limitation in the forum application in Moodle. For example, someone may send a post later in the week by replying to the last message; however, the message may have been addressed to somebody else. Messages that were replied to everyone as when the participants were introducing themselves were linked to themselves on the adjacency matrix. Then, two new sets of adjacency matrices were analysed, one with the researcher and the 2 e-tutors as the most active participants in GSN, and one without the 3 actors, these were two 38*38 matrices. This decision was made because this study focuses more on collaborative e-learning for e-learners-generated text rather than the e-tutors-generated-text.

6.5.7.1. Global Cohesion
The level of global cohesion was measured by assessing network density, reciprocity, cliques, and structural equivalence. Density is the proportion of possible links in network as it is the ratio of the number of links present in the network, to the maximum possible links. Density was evaluated by the adjacency connection reports in UCINET (Table 6.5.7.1-1): Table 6.5.7.1-1. Group Network Cohesion: Density & Reciprocity GROUP NETWORK COHESION: DENSITY & RECIPROCITY
All Total nodes Density (matrix average) Standard deviation Reciprocity (Hybrid) GSN E-learners 698 122 1.0872 0.0256 2.0167 0.1819 0.3618 0.2222 Research Pool All E-learners 81 73 0.0470 0.0418 0.2376 0.2226 0.1852 0.2174

E-learners’ density is rather low, 0.0256 in Moodle@GSN, however stable in the research pool (0.0418). This means that 2.6% in Moodle@GSN and 4.7% in the research pool of all possible links were present; however, there was an increase in

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 186

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Chapter 6: Main Study

density. The participants actually recognised their limited participation; they said they were not as active as they wanted to be. In addition, the 2 highest posters in Moodle@GSN influenced the groups’ density level; this means that the actual increase in participation was almost doubled (0.0418 - 0.0256= 0.0162). This was also evident in the collaborative e-learning episodes text richness, as it was doubled in the research pool (see Table 6.4.2-2). Reciprocity in SNA is the number of ties that are involved in reciprocal relations relative to the total number of actual ties (Hanneman & Riddle, 2005). Reciprocity appears higher within the e-learners (Graph 6.5.7.1-1):

(a)

(b) Graph 6.5.7.1-1. Reciprocal ties in GSN (a) & the research pool (b)

There were 4 reciprocal ties in GSN (28.6%) and 10 (71.4%) in the research pool. However, due to the e-tutors role in GSN, it appears more as an evolutionary process. The increase of reciprocal ties is another indication of evolution in discussion from monological to dialogical sequences between two participants. Strong and weak reciprocal ties can also define strong or weak relationships within an e-learning community. Therefore, reciprocal ties can maintain a strong social network; thus, they are important for knowledge exchange and community knowledge building as it means members’ constant by give and take within a community (Preece, 2004). Reciprocity is also related to the social exchange theory (Blau, 1964); it posits that individuals engage in social interaction based on expectations or the benefits active participants can get from active participation, for example some sort of personal gain

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 187

Tools and Evaluation Techniques for Collaborative E-Learning Communities

Chapter 6: Main Study

or status. However, reciprocity can also be triggered by intangible returns in the forms of intrinsic satisfaction and self-actualisation (Äkkinen, 2005). Based on participants’ opinions for feeling guilty because of inadequate participation in the course as well as their initial voluntary involvement in the GSN courses, it appears that their target was learning, and thus, only implicitly related to the social exchange theory. This also means that social loafing was not evident in this study as they rather had shared interests and values. A clique is a subgroup, a set of actors with each being connected to each other as a maximal complete subgraph of three or more nodes (members) adjacent to each other and there are no other nodes in the network that are also adjacent to all of the members of the clique (Laghos, 2007). Cliques may overlap, that is a forum member (node) can be a member of more than one clique (Bock & Husain, 1950). The results presented in the following table are cumulative and refer to cliques created by 3, 4, 5 and 6 participants (Table 6.5.7.1-2): Table 6.5.7.1-2. Cliques CLIQUES
Minimum set size of participants 6 5 4 3 Moodle@GSN All 2 33 58 68 E-learners 0 0 0 4 Research Pool All 0 0 2 15 E-learners 0 0 1 12

Most cliques were created by 3 participants in both environments. The e-tutors dominated the cliques gathering up to 6 participants. The cliques were developed without any intervention by any of the participants, e-tutors or myself. It is interesting to note that the top scorers had inter-clique connections. When the cliques increase, the social network remains active and thriving, especially if e-learners interact with other e-learners who did not appear in a clique before; these are the activated lurkers. In other words, the absence of cliques could have indicated a lack of clustering that would have reflected the prevalence of weak ties. As most the participants did not know each other before the study and were more skilled in the research environment, the cliques were the glue for forums. However, what fostered the cliques was not investigated in this study.

Ph.D. Thesis

Centre for Interactive Systems Engineering LSBU

Page 188

So the degree to which two nodes are structurally equivalent can be evaluated by measuring the degree to which their columns are identical. 1976) uses dendrogrammes (tree-diagrammes) for hierarchical clustering whereas other techniques use algorithms to calculate network members’ individual behaviour (e. "equivalent” (Hanneman. 2005). Thesis Centre for Interactive Systems Engineering LSBU Page 189 . Interpretation of dendrogrammes for network clustering is as follows: the labels of the actors are given on the left in UCINET. the column in the middle is the row number in the UCINET matrix for the network. 1993)..1-2): Ph. It presents a different clustering view within a human network. the network positions appear as lines. the numbers at the top are the clustering levels. resulting in failing to identify observed clusters. Equivalence is important for generalizations about social behavior and social structure.D. it begins with one group and then divides it up so the dendrogramme looks like an inverted tree.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Structural equivalence describes the actors who have similar patterns of relations to others in the network and exhibit similar communication behaviour. Everett & Borgatti. The CONCOR technique calculates Pearson’s correlation coefficient between columns and depicts whether two nodes are structurally equivalent if the corresponding rows and columns of the adjacency matrix are identical. actors must not be thought about as unique persons. It is computed by the Euclidean distance of tie-value from and to all other nodes (Lorrain & White. Two actors (nodes) are said to be structurally equivalent if they have identical ties with themselves.. White et al. 2001). indicating the number of clusters at the level of sharing at least 3 ties. 1971).g.7. but as examples of categories (sets of actors) who are in some way.5. each other and all other vertices (de Nooy et al. CONCOR is a divisive top-down clustering technique. (Dividing clusters of 3 or less individuals is not preferable as correlations get very unstable) (Graph 6. The CONCOR technique (CONvergence of iterated CORrelations. This structure is calculated and thus artificial.

This means that 7 and out of them 3 actors had exhibited similar behaviour (Graph 6.1-3): Graph 6. the CONCOR dendrogramme reveals 7 splits on the first level and 2 splits on a second level with four actors participating in all groups.7.7.7.5.1-2. Structural equivalence dendrogrammes in the research pool (all) In the research pool 3 participants were active in 2 second level groups and 5 first level groups.5.5.1-4): Ph.5.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Graph 6. Structural equivalence dendrogrammes in GSN (all) If each line represents a participant.1-3. Thesis Centre for Interactive Systems Engineering LSBU Page 190 . The next CONCOR dendrogramme reveals 7 first and 4 second level participants (Graph 6.7.D.

7.7. Lastly.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Graph 6. Structural equivalence dendrogramme in the research pool (e-learners) Ph. the overall structure of the groups remain the same.7.1-5): Graph 6. however. Structural equivalence dendrogramme in GSN (e-learners) Here. the next dendrogramme refers to the e-learners in the research pool (Graph 6.5.1-4.5.5.D. the ties are less than the one with the researcher and the 2 high participation e-tutors. Thesis Centre for Interactive Systems Engineering LSBU Page 191 .1-5.

In other words.D.1-3. Overall. This means that observation had a positive effect in replicating behaviour active participation especially if the participants did not have previous knowledge of working and learning online. 6.5. This was in accordance to participants’ comments on watching what the e-tutors were doing and learning vicariously.7. if grouping actors with equivalent behaviour.7.5. In conclusion. more similar behaviours (structural equivalence) were observed in Moodle@GSN rather than the research pool.2. more e-learners were imitating e-tutors’ and other elearners’ behaviour. Thesis Centre for Interactive Systems Engineering LSBU Page 192 .5. Equivalent e-learners EQUIVALENT E-LEARNERS Moodle@GSN All 1st level 2nd level 7 3 E-learners 7 4 Research Pool All 5 3 E-learners 5 2 Structural equivalence seems to be steady in the two e-learning environments. the social network analysis attributes of the Sense of e-Learning Community Index (SeLCI) indicated that the interactions’ weight (density) was doubled in the research pool.7.1-3): Table 6.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study This graph reveals 5 first and 2 second level multi-actor positions with one solo-actor position. and thus passive behaviour was decreasing. participants’ preferences (reciprocity) were also significantly increased. the results from the previous dendrogrammes appear as follows (Table 6. and there were some small groups (cliques) that remained almost the same throughout the study. There was one more e-learner with second level equivalence if e-tutors were excluded (43=1) whereas it was the opposite situation in the research pool with one less elearner (3-2=1). Global Centrality Global centrality investigates the communication nodes between the members of a network and is characterised by direction and strength and refers to Ph. There were 7 and 5 participants with first level equivalence in the two research pools.

431 Research Pool All E-learners 8. Even though Borgatti (2005:70) suggests that Freeman’s centrality has been misapplied.7(P52) 5(P24) 1.590 2.8 (P18) 7.316 2.53% 5.328 0.30% 20.581 11. it is widely used because there are no other suggestions for coherent results on global centrality.590 2.751% 5.852% 10.173 0.436 Network Centralization (Out-Degree) Mean Standard deviation Network Centralization (In-Degree) Mean Standard deviation In-closeness (Network in-Centralization) Mean Standard deviation Out-closeness (Network out-Centralization) Mean Standard deviation Betweenness (top 3 participants) Mean Standard deviation *Data matrix dichotomized for closeness.655 32.885 5.089 1.7.321 2. such that Xij > 0 was recoded to 1 Degree centrality refers to a directed network (where the direction of the communication is important).235 49.7.565 4.459 73.974 20. Centrality is measured by the portion of nodes that are adjacent to each node.885 5.7 (P24) 0.2 (P50) 11.5.751 3.667 1.606% 9.D. 2005:70).447 1.the ability to influence others directly or in one time period (Borgatti.565 4. Ph.859 55. the sum of each row in the adjacency matrix representing the network.68% 1.83% 5.999 1.97% 20.154 10.2-1.088 16.950 3.5.328 0.974 13. Thesis Centre for Interactive Systems Engineering LSBU Page 193 .49% 5. in-degree centrality (received messages) as well as group in and out closeness and betweenness. Degree centrality was preferred rather than eigenvector centrality as a measure of immediate influence . and out-degree centrality is the portion of nodes that are adjacent from each node (Freeman.762 78.7I (R) 18.9(P37) 0.826% 8.2-1): Table 6.2(P50) 11 (P18) 6.671% 1.983 49.254 4.387 32.581 10.74% 5. The nodes with the highest degree scores are the ones which are more central (powerful) in the network.04% 1.940% 6.626 16. Group Centrality GROUP CENTRALITY SCORES GSN All E-learners 13.447 2. the in-degree centrality is the portion of nodes that are adjacent to each node.3 (P24) 1.532 2(P32) 1.250 6. This was because the subject of investigation was the levels of activity and thus the centrality and peripherality degree of the members.372% 1.147 42.7 (P24) 1.667 1.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study out-degree centrality (replies made). The centrality scores were (Table 6.919 42. 1979).425% 8.

This means that the e-tutors had the power in the information flow in GSN and the e-learners in the research pool in a more distributed manner.425 (SD=1.5.089). with the e-tutors the scores were: out-degree=8. The out. in-degree=6. Standard deviation dropped in the e-learners centrality scores in the research pool which means that the differences between the e-learners were diminishing.49% (SD=2. the next 3 active participants appear first in out-degree centrality (sent messages) in GSN.degree centrality appear to have great differences depicted mostly in the standard deviation rather than centrality itself. In a more in depth analysis. in-degree=73.671% (SD=1. and P58) are omitted. For the e-tutors the scores were: out-degree=13. There was an increase in in-degree centrality and a decrease in out-degree centrality. the e-tutors’ role was apparent in Moodle@GSN. indegree=49. Top 10 Scorers in Out-Degree Centrality OUT-DEGREE CENTRALITY: TOP 10 GSN All Participant 1 2 3 4 5 6 7 8 9 10 P52 R P58 P18 P37 P32 O2 P56 O4 O9 OutDegree 120 108 47 25 20 19 13 10 9 9 E-learners OutParticipant Degree P32 13 P18 4 P48 P37 P24 P9 P13 P2 P50 P6 3 2 2 2 1 1 1 1 All Participant P50 P22 P18 P14 P48 P37 P6 P2 P12 P13 Research Pool E-learners OutOutParticipant Degree Degree 12 P50 12 6 P18 6 6 4 4 4 4 3 3 3 P22 P48 P37 P24 P14 P6 P12 P13 5 4 3 3 3 3 3 2 If the 3 e-tutors (P52.97% (SD=13. Thesis Centre for Interactive Systems Engineering LSBU Page 194 .321).5.7. the top 10 centrality scorers in out.983).606% (SD=2.826% (SD=2.5.2-3): Ph. P18 and P37 from GSN appear to be among the top scorers in the research pool.852% (SD=20. The top scorers for in-degree centrality were (Table 6.and in. For the e-learners the scores were very limited: out-degree=10.459). The e-learners had more stable scores in the research pool.2-2.372% (SD=2250).D.7.and in.99).154).7.2-2) were: Table 6. R.316). without the e-tutors the scores were: outdegree=9.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study In this study.degree centrality (Table 6. indegree=5.

In other words. In a way.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Table 6. a node has high closeness centrality if it has very short communication paths to the others. the same participants appear to have incoming messages posted specifically for them. In other words. Closeness: Nodes with low closeness scores have short distances from the others. the responses were originated from a group of members that was larger than the group that received the messages. An example for its use is the following (Borgatti. While degree centrality measures use only direct and local connectivity information. ‘In-closeness centrality’ is measured as a function of the minimum geodesic distance from all other nodes to the selected node. 2005:59): organizations with low closeness in an R&D Ph. it indicated a movement from a powerful group of e-tutors to a group that was working more and more collaboratively to increase their learning. the active participants became more democratic in their communication instead of maintaining their status.D. 2004:25). while ‘out-closeness centrality’ is measured as a function of the minimum geodesic distance linking that node to the other nodes.7.5. the increase of in-degree centrality indicates that they were more communicative and received more messages. the simultaneous slight decrease in the out-degree messages indicates that they lost some of their power and this power was distributed to the other e-learners. closeness centrality measures also use indirect connectivity information (Braha & Bar-Yam. Top 10 Scorers in In-Degree Centrality IN-DEGREE CENTRALITY: TOP 10 GSN All Participant 1 2 3 4 5 6 7 8 9 10 R P52 P37 P9 P18 O2 P56 O9 P58 P15 InDegree 95 52 31 25 19 19 19 18 17 15 E-learners Participant P9 P24 P13 P18 P37 P2 P6 P32 P48 P3 InDegree 7 4 4 3 3 2 2 1 1 1 All Participant P50 P18 P37 P24 P6 P13 P22 P12 P2 P33 Research Pool E-learners InDegree 10 5 5 5 4 4 3 3 3 3 Participant P50 P18 P37 P24 P6 P13 P22 P12 P2 P33 InDegree 9 5 5 5 4 4 3 3 3 3 As for the received messages (in-degree centrality). In addition. this is more apparent in the research pool where exactly the same e-learners were in the top 10 as there were central students controlling in-coming and out-coming connections.2-3. Thesis Centre for Interactive Systems Engineering LSBU Page 195 .

As with degree centrality.387). there were major differences between the e-tutors and e-learners in the two environments.97% (SD=10. ‘in-closeness centrality’ represents the speed of interaction from all other nodes to the selected node. without the e-tutors in Moodle@GSN.3 (P24). this might be affected by how long they were on-line. A reason may be that e-tutors were connected more times than the e-learners and this affected their interaction speed.closeness. As for betweenness. Thesis Centre for Interactive Systems Engineering LSBU Page 196 . the in-closeness score was 32. For example.2 from P52. P24 scored 5 and others 2 (P32). There was a significant difference between Ph. with regard to in.7 (P24) and 0. so in Moodle@GSN this speed was high when the etutors were included. This was indicated by the closeness score. the standard deviation was high as well.71) and the e-tutor 52 (18.30% (SD=11. The scores remained on the same level in the research pool with and without the e-tutors: 16.68% (SD=1. Overall. this score and standard deviation was also high in Moodle@GSN. the scores without the e-tutors and in the research pool were more equally distributed and the standard deviation was very low. a virus can sooner infect the members with low closeness. including the e-tutors in Moodle@GSN the in-closeness score was 73.7) in Moodle@GSN.919) and the out-closeness score 42. therefore playing an important role in the network.53% (SD=4.626).9 (P37).147) and the out-closeness score 42. These scores indicated that the e-tutors were controlling the speed of information flow. and 6. without the e-tutors the in-closeness score was 49. Betweennes measures the node’s prominence according to its position in the network as an intermediary measuring the volume of traffic moving from each node to every other node that would pass through a given node.8 from P18. Some active participants act as “brokers” or “gatekeepers” between groups of nodes. 11 and 11.762) and the out-closeness score 78. e-learners stayed significantly behind me (32.532).7 and 7. 1. indicating a more stable interaction speed between the e-learners.D.088).74% (SD=5. the means and standard deviation. ‘Out-closeness centrality’ represents the speed of interaction from one node to the other nodes.83% (SD=4.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study technology-sharing network can develop products sooner than others. In the research pool the results are more equally distributed and distance of communication was reduced: with the e-tutors the in-closeness score was 49. However. It was evident that the highest values came from the e-tutors in Moodle@GSN and the e-learners in the research pool. As before.49% (SD=5.04% (SD=2.and out.859) and the outcloseness score 55.

Figure 6. Figure 6. 6. Thesis Centre for Interactive Systems Engineering LSBU Page 197 . two SNA tools running in real time aimed to provide a different viewpoint of the discussion and triangulate the events of active participation and collaborative e-learning.and out-degree centrality) the in-degree centrality increase and the out-degree centrality decrease indicated that the e-tutors had the power in the information flow in Moodle@GSN and the e-learners in the research pool in a more distributed manner. The Visualisation Interaction Tools Nodes and Centrality (VIT Nodes and VIT Centrality) supported local SNA nodes and centrality in real time.7. e-tutors and e-learners. they can be classified as low active participants or.3-2) were integrated in Moodle in the research pool.3-1) and Centrality (VIT Centrality. or with no mediation power (Willging. A different abstract representation was given with regard to interaction density (weight). Information control (betweenness) could also be observed. 2005:51).5. there was no single participant who ranked high in all the centrality measures in both environments including the e-tutors and myself. Lastly. the centralisation indexes were: for interaction direction (in. to a small extent. so too. Moreover.and out-degree centrality (direction). This was also evident in the interaction speed (closeness) and control (betweenness) that showed that the e-tutors hold the network power in Moodle@GSN whereas this power was quite evenly distributed in the research pool and the discussions have not been monopolised. The participants with the lowest betweenness values could be considered outsiders in the conversation. Local Nodes and Centrality in Real-Time Just as a photo and a recording give a different ‘picture’ of two people discussing something. Visualisation Interactions Nodes (VIT Nodes.5. reciprocity (preferences) as well as in.7. These can be located in the low active participation level. (Note that the Ph.3.D. in SNA. Overall. local SNA describes the human and information network in a particular situation. Other than global Social Network Analysis (SNA).7. Closeness as the interaction speed was represented on both graphs as the geodesic distances indicate the temporal distance between the messages. isolates.5.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study the information gatekeepers as regulators of the information flow in the network within the two environments despite the fact that the participants did not see any particular roles or. there was no active participant who could control the information flow in both environments.

5.7. the direction of the messages is indicated by an arrow and the number represents the number of messages (Graph 6.3-2): Ph.D. She also responded to her own message a couple of hours later after the argument with O2. the discussion was a collaborative activity between 7 individuals.3-1): Graph 6. It is interesting that this CeLE was developed by different individuals with only two interlocutors exchanging 2 messages.5. In other words. The reciprocal tie with O2 was an argument.7.) In VIT Nodes the individuals are represented as circles (nodes).5. Thesis Centre for Interactive Systems Engineering LSBU Page 198 .7. Most participants were replying to P37 and two of them talked to each other.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study participants were not given specific information on the exact use of these tools as the tools should indicate their own use (usability). VIT Centrality provided a different viewpoint (Graph 6.3-1: VIT Nodes in CeLE IX P37 was the information broker in this CeLE.

their co-learners and on their own.e. As a central connector and information broker she moved the knowledge around leading to a new proposition by taking into account her colearners responses even though they appeared as low activity e-learners (i.5. it appears that after the initial knowledge acquisition and information exchange. only O2 was an e-tutor). Being self-aware corresponds to self-organised learning and development. It was also suggested that knowledge awareness plays a major part in the creation of opportunities for Ph. Overall. VIT centrality also indicates the response time space related to geodesic distances between the participants. As they did not have any previous knowledge of collaborative learning and techniques they acquired this knowledge for community knowledge building. the Visualisation Interaction Tools Nodes and Centrality provided opportunities to the e-learning participants to observe their personal styles and performance within the e-learning community as well as observe small groups created within their discussions. To sum up. the collaborative techniques and tools helped participants to learn from the e-tutors.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Graph 6.3-2. 2000). Thesis Centre for Interactive Systems Engineering LSBU Page 199 . This means that e-tutors have a complex job that incorporates moderating as well as e-tutoring (Salmon. Information organisation roles can be unofficially assigned to e-tutors to support the collaborative e-learning development.7. VIT Centrality in CeLE IX In VIT centrality P37 is clearly located in the middle of the e-learning social network.D.

This means that there were more clues as opportunities for critical engagement in dialogue in the research pool caused by two events. add to the evidence for the e-learning quality. Moreover. CeLE was found to build on progressive discourse and fill the middle space between internalisation and externalisation in the form of monological to dialogical sequences. with regard to collaborative e-learning quality. the initial need for familiarisation with collaborative e-learning in the early stages of the online course and the sleeper effect. Almost the same number of collaborative e-learning episodes. 2000) which leads to the fact that different learning styles are supplementary to each other in e-learning environments. Thesis Centre for Interactive Systems Engineering LSBU Page 200 . the other e-learners (collaborative learning) as well as on their own how to work together and how to use the new tools and new technologies (vicarious and self-organised learning).Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study efficient and effective collaborative opportunities (Ogata & Yano. In a way. Ph. Lastly. 6 in Moodle@GSN and 7 in the research pool.D. e-learners were progressively adopting a two-way communication. correcting communication gaps between the collaborative learning discussion stages is feasible for the e-learning participants. The effectiveness of the Sense of e-Learning Community Index will be discussed in the interventions section at the end of this chapter. the participants learned from the e-tutors (instructional learning). This was evident not only within one CeLE but in the participation process as monological postings stand alone without open clues for dialogue as in information provision. and the use of MessageTag that revealed the collaborative e-learning structure.

However.0-2.6 PEDAGOGICAL USABILITY The participants’ responses in the closed question revealed that the new tools supported the e-learning community development (n=33.4 3. The tools were: (a) Participation graphs and avatars.2).7 3.0-1.7 3. Pedagogical usability values PEDAGOGICAL USABILITY VALUES 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Instructions Frequency of use Alignment with educational goals Support collaborative e-learning Learnability Accessibility Originality Motivation to participate Information overload Tool failure Functionality Graphics Attractiveness Fast response Overall satisfaction Overall 3.1 (5. Thesis Centre for Interactive Systems Engineering LSBU Page 201 .4 3.6. Nonetheless.4 3.D. The law learnability rating may explain why the tools were infrequently used (2.4). (c) Visualisation Interaction Tool for Nodes (VIT Nodes). the use of the new tools was the third best thing in the project after collaboration and feeling a sense of contributing to something great (Appendix A_XII_4).7).5%).6 2.4) and learnability were low (2.8 (5.1-1): Table 6. Most of the pedagogical usability scores for the new collaborative tools were satisfactory. the pedagogical usability values were (Table 6.6.1 Pedagogical Usability The participants responded to 15 closed questions on a Likert scale 1-5.4 3. 6. In sum.2 Values 9 & 10 were inverted for reasons of compatibility and clarity The overall score for pedagogical usability was found to be just higher than the average of 1 to 5 Likert scale (3. and (d) Visualisation Interaction Tool for Centrality (VIT Centrality).6.5 3.5 2.0 3.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study 6.9) 3.1-1. The overall results are presented in Appendix XII. Ph. This result was similar to the preliminary studies although there the tools were being used by the developers who perceived the tools differently.2) 3.4 3. 82. (b) MessageTag. according to the participants.5 2. originality (2.6 3.

6.1-2): Table 6. 20 out of 79 messages were tagged with an average 27. He said that most users will not notice them and he gave examples on MessageTag and Visualization Interactions Tools nodes and centrality: “it is not easy to actually see the links that lead to them because the MessageTag is at the end of the message and the images for the VIT are invisible”. get familiarised quickly or be keen to experiment with new tools.D.3 20/70 28.1-2. Thesis Centre for Interactive Systems Engineering LSBU Page 202 . Involving users with different levels of experience or filtering of users based on their profile for basic and advanced user testing can provide different results. advanced users might know uncommon ways to overcome problems on the interface.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study The messages were tagged with MessageTag (Table 6.6.3% use. Interestingly. Use of MessageTag in the Research pool USE OF MESSAGETAG IN THE RESEARCH POOL Forums CeLE Attributes Inform Question Explain News Explore Agree Evaluate Summarise Other Inform Question Explain VC in E-learning Explore Agree Evaluate Summarise Other # use 0 0 0 0 0 0 0 0 7 3 3 2 2 2 0 1 Total 20/79 25. P6 sent a message to the forum saying that the new tools are invisible and lost in the interface. The results were very similar to the e-mmersion block. Comparing the results with the developers’ results (55.6 0/9 0 Tagged messages Percentage The tool was not used in the introductory news section and the participants started using it in the main discussion. In addition. his message indicated frustration concerning their use which is in accordance to the low learnability score: “At last! I managed to find a way with the VIT nodes and Ph.3%) a great difference appears to be between the way average users’ utilisation of the new tools and the more advanced users.

In addition. for example. From an HCI viewpoint. and VIT. P6’s messages raises questions on the decisions made about the images selected after the developers’ recommendations. He also said that usability measurement of one task cannot be applied in exactly the same way for all users. This also means that the ‘real’ users are more reluctant to use the tools. Thesis Centre for Interactive Systems Engineering LSBU Page 203 . new products development requires cycles of communication between the organisational stakeholders (Safayeni. A developer needs to listen to users’ opinions. different social network applications on Facebook.com). The participation graphs and avatars as well as the MessageTag achieved more scores than the Visualisation Interactions Tools (VIT) in all pedagogical usability metrics. because the design process is technical and social. this was the relationship between the task and the method. Lastly. her decision making is based on her own expertise and usability tests in usability laboratories. and no differences appeared between the participation and collaborative e-learning tools. http://www. the relevance of the tool to the particular task had a positive effect. The next section refers to the most noticeable correlations and crosstabulations between the pedagogical usability and utility factors as appeared in the questionnaires. their utilisation will be more obvious and the users will be more familiar with their practicality. et al. 2008). Ph. As tools to represent social networks have now reached the average user (e. in comparison to the previous usability evaluation in the two preliminary studies with e-learners and developers. all previous scores were much higher than the average (>4). the use of visualisation interactions tools was not apparent without the aid of the literature.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study centrality…” (“Επιτέλους! Έβγαλα μια άκρη με τα VIT nodes & centrality. this is in contradiction to usability rules as the tools should be easy to use. In other words.g.”). (There were 3 Greek teachers who were Moodle developers participating in tools’ testing. the tools. as Draper (1993) suggested. However. the actual e-learners scored significantly less.) It also appears that new tools need more than one week of use and evaluation.. This section discussed results from the questionnaires.facebook. The alignment with the educational goals and support for collaborative elearning was more than the average. especially when the developers’ sample is relatively small. however.D. Overall. he expressed his enthusiasm about the tools and suggested that their use will be determined by time. and the participants’ comments.

Thesis Centre for Interactive Systems Engineering LSBU Page 204 . The information visualisation provided a simultaneous correlations overview.2-1. In HCE. N/A were viewed as missing data. the data were normalized row by row in a scale 0. Correlations analysis in HCE 3.2-1): Figure 6. scatter plotting. and all data were initially filtered with SD≤1. and in depth analysis (Figure 6. SPSS was used to find all correlations as well as the ones related to specific issues. In more detail. in order to increase similarities and reduce differences between the data.D.6.0 to 1 using the equation . HCE provided an overview of correlations focused on the most important positive and negative correlations.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study 6. Then Pearson’s r correlation coefficient was used to identify 1. in SPSS (Appendix A_XII_1-2) and in Hierarchical Clustering Explorer (HCE) (Appendix A_XII_3).2 Correlations & Crosstabulations Two ways were found suitable to measure and analyse inter-correlations between the participants’ responses in the final questionnaire. Twenty out of 40 rows remained and then.747 and highest score 0.6.981 (scores<1).0 Ph. the matrix with the data was exported from SPSS to Excel and then inserted in the HCE to distil them.6.770 correlations variables with lower score -0.

the practicality of the first was more explicit than the latter. fast response (r=0. As in the previous findings.71). Originality was found to be an issue only for VIT Nodes.948). satisfaction (r=0. motivated participation. Centrality failed (r=0.=1.81). In Nodes alone functionality was also found relevant to satisfaction (r=0. functionality (r=0. The latter means that clicking on the participants (top left) located them on the scatter plot. In Centrality alone correlations were found between: satisfaction and functionality (r=0.784). and for MessageTag 2.7 (SD=1.721). the variables with r>0. and the e-learners expressed their satisfaction. Not all 1. Thesis Centre for Interactive Systems Engineering LSBU Page 205 . satisfaction and Ph. In other words.2).600>r<-0. accessibility (r=0. This can be explained under the rubric of functionality.782). the two groups of tools supported different practices in discussions. The results show that there were problems concerning tools failure and accessibility.771). VIT Centrality) used for social network analysis. related to: educational goals (r=0.801).792). motivation to participate (r=0. additionally there was a visual summary (top right). the first to structure and analyse collaborative e-learning.770 variables are presented here.747). satisfaction and support CeL (r=0.7 (0.710).751).7 (SD. it appeared that correlations exist between the two groups of tools the collaborative e-learning and participation tools (Participation graphs and avatars.600 (-0. and satisfaction and graphics in visual design (r=0. and MessageTag) as well as the Visualisation Interactions Tools Nodes and Centrality (VIT Nodes. and support Collaborative eLearning (CeL) (r=0.977).712).961).Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study HCE provided a correlations overview from the questionnaires (bottom left).712). educational goals (r=0. scatter plotting (bottom right). graphics (r=0.977<r>0.703) and r>-0. The positive correlations between the VIT were: where Nodes failed. satisfaction and support CeL (r=0.D. there were in accordance to educational goals. The positive correlations between participation graphs and avatars.3).944).930). motivation to participate and support CeL (r=0. and satisfaction and educational goals in graphs and avatars alone (r=0. This preference was also evident in the average use (Likert scale 1-5): for graphs and avatars 2. where Nodes functioned. satisfaction (r=0. and MessageTag were: failure (r=0. and the latter to support visibility in social networks. however.788). and in-depth analysis of the results.883). Centrality functioned (r=0.752).955) and motivation to participate was related (r=0. educational goals and functionality in graphs and avatars alone (r=0.748).

1 (SD=1) and VIT centrality 2 (SD=1).6. The difference in the standard deviation on the use of tools showed that some elearners used the tools more frequently than others. and graphics and attractiveness in visual design (r=0.2-1. Thesis Centre for Interactive Systems Engineering LSBU Page 206 . and graphics and attractiveness (r=0.629). information overload and attractiveness (r=-0.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study graphics in visual design (r=0. motivation to information overload (r=-0.777).729).628). functionality and graphics (r=0. functionality and attractiveness (r=0. For VIT Nodes the results were: satisfaction was negatively related to failure (r=-0.961).712). avatars and MessageTag: VIT (Likert scale 1-5): VIT Nodes 2.718). For VIT Centrality the results were: information overload and motivation for Centrality (-0. Time using LMS * Frequency of Use CROSSTABULATIONS: TIME USING LMS & THE FREQUENCY OF USE Tools Time Using LMS 1-3 1 4-6 2 Months Missing 3 Graphs/avatars # 2 1 3 40 22 13 1 3 1 40 MessageTag # 2 1 3 40 22 13 1 3 1 40 VIT Nodes # 2 1 3 40 22 13 1 3 1 40 VIT Centrality # 2 1 3 40 22 13 1 3 1 40 Mean 2 1 3 40 22 13 1 3 1 40 4 5 6 7 Years Total 0 1-5 6-10 Missing N/A Total Ph. Less strong negative correlations appear only for the VIT. information overload and attractiveness (r=-0.6. It appears that information overload was the main and learnability the second pedagogical usability problem that affected attractiveness. it seemed that the e-learners who did not have any experience of LMS used the new tools more than the experienced e-learners (Table 6. After crosstabulating the first and final questionnaires.718).729).632).2-1): Table 6. More meaningful correlations between Nodes and Centrality were on: instructions (r=0.D.722). satisfaction and information overload (r=-0.6776). learnability and attractiveness (r=-0.763). accessibility and instructions (r=0. motivation to participate and the overall satisfaction.0609). VIT were not used as much as graphs. and accessibility and learnability (r=-0.747).

D.2-2. no cell to have expected values less than one.6.g. less the ones attended an ICT course and even less the ones who had proper training within universities. Although there are arguments in the research community against its use (e. In addition. Similarly. the visibility of active participation levels structure in the graphs and avatars may have helped e-learners’ reflection on their participation. the participants with no training at all used the tools more than the others. Training on LMS * Frequency of Use CROSSTABULATIONS: TRAINING ON LMS & THE FREQUENCY OF USE Tools Train LMS Graphs avatars # χ2 11 26 1 2 MessageTag # 11 26 1 2 χ2 VIT Nodes # χ2 11 26 1 2 VIT Centrality # χ2 11 26 1 2 1 2 3 4 Yes No Other Missing Total 40 40 40 40 As before. It seems that the first were found more practical and relevant to the specific practice. Yates’ correction of continuity was performed instead. Another explanation may be that the users who were familiar with Moodle hardly noticed the new tools in the research pool and acted by habit. the e-learners who did not have any training on LMS used the new tools more than the trained elearners (Table 6. Thesis Centre for Interactive Systems Engineering LSBU Page 207 .Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study The participants gave similar results in regard to the tools. Overall. 2007). The variables were checked for being ordinal or categorical. Pearson’s correlations were also calculated for the two tables. chi-square becomes unreliable. it appears that the graphs and avatars as well as MessageTag received higher scores than VIT Nodes and Centrality. because in the two rows and two columns (two-by-two tables) case of the second table. This may mean that the more experienced the e-learners the more reluctant they are in the use of new tools. 2003). In particular. Their effectiveness is a device for information Ph. and no more than 20% of the cells have expected values less than five (Muij. Yates’ correction of continuity is automatically calculated in SPSS statistical output to prevent overestimation of statistical significance for small data. Haviland. the responses are similar.2-2): Table 6.6.

for participants lacking certain skills.D.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study retrieval as the visual objects (graphs and avatars) provide users with the elements to explore and reflect. training and experience in association to using new tools may have been a disadvantage. Accordingly. E-learning tools need to address such participation trajectories (Suthers. collaborative e-learning. if compared to the results in the previous studies. Ph. However. the visibility of the collaborative learning structure in MessageTag seemed to help e-learners’ reflection and increased challenges per argument. 2006). opening interaction spaces in participation. Thesis Centre for Interactive Systems Engineering LSBU Page 208 . Nonetheless. and social networks seemed to be essential.

7 INTERVENTION ANALYSIS The intervention analysis can be depicted as follows (Table 6. as in level. and cyclicity.7-1. More specifically the overall discussion on the suggested tools and evaluation techniques is as follows: E-learning engineering was found to require five corresponding changes: • • • a fundamental change in perspective.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study 6.7-1): Table 6. evaluation.7-2): Ph. slope. the coherence of action as an emergent property of moment-by-moment interactions between the actors and between the actors and their environment. The treatment was immediate and its permanence was continuous to this point in time. was successful. • • unification of design. Intervention Analysis Target Intervention Analysis A. and building interventions in ‘time-boxes’ to achieve coherency of action. and use under the rubric of community and real-time research. a commitment to situated approaches. Form of the effect Level Slope Variance Cyclicity B.D. the suggested model for e-learning engineering is (Table 6. Permanence Continuous Discontinuous C. Immediacy Immediate Delayed Adequate √ Adequate Participation in the Greek teachers’ collaborative elearning community √ - √ - The target in this study was to investigate participation in the Greek teachers’ elearning community and provide tools and evaluation techniques to support their development. In the intervention analysis the form of the effect. Thesis Centre for Interactive Systems Engineering LSBU Page 209 . Thus.

Such approaches have been found in the history of engineering design as nowadays design requires flexibility between various ways of thinking (McGarry. Explain. exploration. Ph. Redesign and evaluate with userlearner to redesign 9. Documentary review 4. Their attributes. Goals: stakeholders’ goals. 1987) and evolved over time integrating design. Evaluate. evaluation. 1994). Prerequisites: real need & right attitude 2. Background studies: description of stakeholders background & characteristics 5. design was used as a problem solving activity in which the problem was broken down into a number of sub-problems. use and evaluation (Bannon. explanation. This structure allowed e-learners to externalise their thoughts in order to reflect upon them. Evaluation & feedback Ethnotechnology R Scenic fieldwork E A L Ethnographic inputs T I M E E V Implications for design A L U A T I Scenic fieldwork O N B PLANNING C ITERATIVE DESIGN D EVALUATION & USE Initially. In order to do this. Create one prototype for user-learner testing • Evaluation to acquire feedback for redesign 8.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Table 6. co-construction. solved independently and reintegrated to produce a design solution. initiation. the value of unarticulated expertise and tacit knowledge was acknowledged. Agree.D. Design for initial design • Apply guidelines from feedback for design 7. requirements & variances 3. Explore. Thesis Centre for Interactive Systems Engineering LSBU Page 210 . E-learning engineering E-LEARNING ENGINEERING Design Stages A INTENT 1. Collaborative e-Learning Episodes found to be a successful analytical framework in regards to e-learners’ use as well as in their in-depth analysis.7-2. Question. Design was found to be a reflective process having a post hoc nature (Suchman. Interdisciplinary research • Evaluation for the targeted pedagogical approach (intentional variance) • Evaluation for the targeted technical problem (operational variance) • Pedagogical Usability guidelines 6. Implementation in situ 10. 2005). were visualised in MessageTag as Inform.

iterative design made the tool more accessible to the e-learners and it became the most successful tool in this study. Active and passive participation levels in collaborative e-learning were found to be related to social interactions and thus grouping as the participants’ interactions were more and more intimate. The tools aimed at reducing misleading information and interruptions by triangulating the otherwise separated information context and e-learners’ network. their working methods. and Other. the participants were unfamiliar with them and it was difficult to interpret their use and incorporate it in their practice. Another explanation may be the absence of a direct match between the task and the method (Draper. Visualisation Interactions Tools were VIT Nodes and VIT Centrality. The tools based on the participation were successful. this means that SNA methods were not explicitly connected to the Greek teachers’ educational practices and thus. As they were built on SNA. representation of the progressive dialogue and relationships between the messages can provide a more coherent view of the research context. graphs and avatars were found to be in accordance to their purpose. In other words. They depicted the local networks in a particular discussion and were found to be less important for collaborative e-learning. however. learning emerges from passive and active participation in collaborative e-learning suggesting intentional rather than reactive behaviour. This also means that e-learning participants can manipulate information. missing their log files because of their failure without having time to repair them prevented triangulation from the results. The Sense of e-Learning Community Index (SeLCI) provided a different viewpoint to describe the e-learning community and processes in depth (only the e-learners’ results were considered in social network analysis) (Table 6. However. and interactions as interrelated areas of one situated context in order to achieve their goals. Translating a theoretical framework into a tool was found difficult and it was not easy to create a tool that could be used without instructions. knowledge. Thesis Centre for Interactive Systems Engineering LSBU Page 211 . 1993).Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Summarise. The interactions depended on the learning styles and the personal learning goals as active and passive participation were found to have an idiosyncratic character. in fact some participants complained about the simplicity of the forum when they got back to Moodle@GSN.7-3): Ph. The Moodle discussion application was found to be rigid.D. E-learners’ interpersonal and intrapersonal skills were found to control the dynamics of the collaborative e-learning networks.

425% 10.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Table 6. 6. Sharing their interests and values.826% 32. Establishing sociability and common ground was the first level towards a progressive course in collaborative e-learning. It is also interesting to note that the sense of community was evident in the participants’ communication outside the course.4 participants with identical behaviour 5. When I asked them via Ph. Time Overall Moodle@GSN Research Pool √ √ Interrupted 80% Coherent 95.671% 9.2174 – 10(71. the e-tutors other than myself met for the first time in a conference in Athens (7 October 2007).8 (P18) 7.04% 2(P32) 1. Intensity was above the average level and more coherent in the research pool.53% 16.3 (P24) 5 √ √ 6 31 days 7 6 days 7 √ SeLCI was found to be successful in describing.68% 55.2 2nd level groups 1 solo-actor positions. sense of belonging.7-3.0256 0.2222 – 4(28%) 4/3 actors .3 second level groups . the participants responded in a positive manner about community evolution. For example. evaluating and triangulating the research context.4%) 12/3 actors 1/4 actors .7 (P24) 0.2 (P50) 11. The Sense of e-Learning Community Index Checklist THE SENSE OF E-LEARNING COMMUNITY INDEX Attributes 1 2 3 4 Community evolution Sense of belonging Intensity Levels of participation Persistence Empathy Social Network Analysis Group Cohesion Density Reciprocity – reciprocal ties Cliques Structural equivalence Group Centrality In-Degree Out-Degree In-closeness Out-closeness Betweenness 6 Trust Collaborative e-learning quality Participants’ opinions Number of collaborative elearning episodes.9(P37) 0.0418 0.5 1st level groups . empathy and trust.372% 49.3% √ 0. Thesis Centre for Interactive Systems Engineering LSBU Page 212 .83% 42.D.

He also measured learning as knowledge awareness depending on information about other e-learners’ activities. Thesis Centre for Interactive Systems Engineering LSBU Page 213 . additionally. Ph. The participants experienced innovative collaborative e-learning tools and techniques and stressed the importance of incorporating them in the Greek teachers’ training cycles. he measured reciprocity by assessing participants’ frequency of sharing class related resources.111-126). There was a high ratio of shared goals and values (p. There was an interesting observation from P35 about activities and communication: “Many people were collaborating at the same time I saw that I can change things with my participation both as a student and as a facilitator.D. tested 11 dimensions to sense of community within 15 undergraduate students using an e-learning environment.113). He didn’t expect to have the same strange feeling when something like a live conference is over. There were differences in the number. The formal educational authorities do not actually know what this is about. he related trust to the levels of awareness and co-presence. Hope new technologies will change things”.111). There were high rates of social networking. Comparison with recent studies: In the recent research. The feeling of belonging to the community was high (p. as he used a questionnaire and message analysis to obtain his results there were several differences and similarities (p. it is a programme in parallel with the official one (παρα-πρόγραμμα). the respondents’ comments in Other on the final questionnaire were in accordance to the checklist.) The similarities were: • • • • • • Participants continued their networking outside and beyond the time frame of the class (p. familiarisation with the other participants and the interface was immediate even though it was the first time he was participating in an online course. Peer support was evident and reinforced members’ sense of belonging (p. P14 said he was looking forward to login. that was what individual knew (competence awareness) and what they could do (capability awareness) (p.126-127). age. On participation and social protocols 67% of the participants said they were aware of them. For example.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study electronic communication whether they could describe the feeling they had they said that it was like being old schoolmates. (Note that Daniel did not always describe his findings using numeric representation. For example. Daniel in his PhD Thesis (2007).116).120). profession and experience of the participants. In addition.

(Note that there are no page numbers as the thesis was not on a PDF format and there were differences on the page numbers in the chapters. (Να ερευνώ από δω και πέρα περισσότερο γιατι υπάρχουν τελικά και άλλοι που ψάχνουν το ίδιο και μπορούν να φανούν χρήσιμοι πάντα.. and the use of new technologies (9 responses).Πιστεύω ότι σιγά . They felt part of the group.. (. having common interests and targets (6 responses).) Ph. Time was limited. these were: their participation in collaborative activities (11 responses). since there are others who look for the same thing and can be useful’.. (For all results see A_XII_4). Online communication skills were crucial to enable participation. knowledge construction and passive participation for nurses participating in an online course. (.124-125). P47 expressed her ‘satisfaction that there are many colleagues interested in the further development of educational practices’. Thesis Centre for Interactive Systems Engineering LSBU Page 214 . feeling of belonging to something greater (7 responses). They shared same interests and responsibilities.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study • • • Most participants did not know each other before the course (56% said they knew nobody. To finalise. And People discovered they had similar problems and interests.ικανοποίηση ότι υπάρχουν αρκετοι συνάδελφοι που ενδιαφέρονται για την βελτίωση εκπαιδευτικών πρακτικών. They exhibited desire to be connected. Gulati (2006) found that there were different forms of learning and different types of e-learners depending on how sociable they were.) P45 said that ‘the still waters start to move in the area’.σιγά αρχίζουν να κινούνται τα λιμνάζοντα ύδατα στο χώρο. p. P2 said that she needs ‘to look for people more. There were access problems..).) Her investigation was mainly qualitative and the similarities to this study were: • • • • • • • The participants enjoyed learning together. none of the participants knew the specific aims and objectives of this project when they replied on the open question about the best things they experienced in the course. In another study on online social identity. E-learners’ knowledge awareness bred trust to the community.D.

Journal of Mathematical Psychology. The Internet and Higher Education. (2006). D. & Freeman..html. Braha. 413-439. & Husain.. Learning or Lurking? Tracking the 'Invisible' Online Student. Wellman. (2002). M. In D.. International Journal of Computer Supported Collaborative Learning. 1993.pdf.com/borgatti/papers/centflow. & Bar-Yam. G.pdf. from http://helecon3. Social Networks. Social Networks.pewinternet. L. Exploratory Social Network Analysis with Pajek. Boorman. & Rainie.Steps towards an Integration. & Montalvo. Delich.. (2005). N.fi/pdf/wp/w387. Z. Conceptual Foundations of Online Communities (Working Paper W-387). 12. P. (1975). Published doctoral dissertation.com/pqdlink?Ver=1&Exp=09-092012&FMT=7&DID=1144195631&RQT=309&attempt=1. Information flow structure in large-scale product development organizational networks. Retrieved 23/11/2006.org/pdfs/PIP_Internet_ties. from http://www. The Netherlands. S. P. 147-155. Centrality and Network Flow.Holland. Draper. Retrieved 12/02/2006. Borgatti.umi.ca/theses/available/etd-07132007141903/unrestricted/ben_m. 12). Beaudoin. Tauber & R. D. V. Freeman. Retrieved 10/09/2007. 5(2). A. Breiger. The Netherlands: North . W. Boase. from http://www. from http://proquest. K. Retrieved 09/05/2006. & Batagelj.htm.. (2007). 146-153. J. 27(1). J. C. NY: John Wiley & Sons.. L. De Nooy. E-Quality Synthesis Report: Experience-based Quality in European ODL. Enyedy. PhD Thesis. B. Amsterdam. S. A. S. Goleman. DC: Pew Internet & American Life Project. Thesis Centre for Interactive Systems Engineering LSBU Page 215 . New York: Bantam Book. L. C. 19. Centrality in Social Networks: Conceptual Clarification. Sociometry. An adaptation of Holzinger's B-coefficients for the analysis of Sciometric data. (2002). C. A. J. The Design of ComputerSupported Cooperative Work and Groupware Systems (Vol. (2006). Malibu.ac. Helsinki: Helsinki School of Economics. CA. Exchange And Power In Social Life. S. P. Retrieved 14/02/2006. P. C..psy. M. Shapiro. Ucinet for Windows: Software for Social Network Analysis. Mrvar. Retrieved 01/08/2007. 55-71. Daniel. Bock.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study REFERENCES Äkkinen.hkkk. Borgatti. 1(4). Use. M. New York. The notion of task in HCI. Design & Evaluation . Social intelligence: The New Science of Human Relationships.uk/~steve/task. Everett. 13.D. The strength of internet ties.. University of Saskatchewan. W.. (2007). from http://library2. S. Retrieved 02/04/2006. Traunmueller (Eds. M. 1. A Bayesian Belief Network Computational Model of Social Capital in Virtual Communities. Amsterdam. 244–253. Paper presented at the Interchi'93. Y. R. in the Adjunct proceedings.analytictech. Pedagogical and interface modifications: What instructors change after teaching online. (2005).pdf. B. Canada.usask. Blau. from http://vlado. Harvard. L.. Journal of Information Technology.gla. MA: Analytic Technologies. Fernandes. from http://www. & Hoadley. F. (1979). (2006).si/pub/networks/data/esna/default. 328382.. (2005). M. Horrigan. (2006). (1964).unilj.. R.. (1993). (1994). L. (1950). Ph. An algorithm for clustering relational data with applications to social network analysis and comparison with multidimensional scaling. Pepperdine University. 215-239.fmf. M. (2004). P. Washington. D.pdf.). From dialogue to monologue and back: Middle spaces in computer-mediated learning. Bannon. April 24 – 29. Unpublished Research. & Arabie.

"Open Education". Retrieved 24/04/2007. Hartley. M. Klemm. Lambropoulos. F. Retrieved 14/05/2006. B. Khine. T..pdf. F. S. PhD Thesis at the Department of Education and Lifelong Learning. R. 56. & Karageorgiou. & White. Lorrain. 363 . R.367. 729736. M. 49-80. P. (1998). R. from http://eprint. The effects of gender interaction patterns on student participation in computer-supported collaborative argumentation. (2005). 54(6). C. E. Riverside. Hannemann. 42-62.. Yates's correction for continuity and the analysis of 2 × 2 contingency tables. R. E. Situated Learning : Legitimate Peripheral Participation. New Jersey: World Scientific. Technology-Rich Learning Environments (pp. from http://kpe-kastor.). 1. CA: University of California..gr/kpe/yliko/sppe2/oral/PDFs/327334_oral. Upper Saddle river. 15-17 December 2006. (2004). (1971). London. 38(1). Athens. Learning during online and blended courses: Unpublished Research. Thesis Centre for Interactive Systems Engineering LSBU Page 216 . Greece.. (2006). Αποχή των εκπαιδευτικών της 4ης Περιφέρειας Πρωτοβάθμιας Εκπαίδευσης του Ν.D. (1998). Introduction to Social Network Methods. London: Sage. The Journal for Open and Distance Education and Educational Technology.sch. Journal of Personality and Social Psychology. H. (2001). and Black.. A. 26(1). J. 2. Teaching. R. The Higher Education Journal. Haviland.htm. A. Lanzetta. New York. G. Things to Think With: Understanding Interactions with Artefacts in Engineering Design. W. The primary school teachers absence from their environmental education training in five years.kas. & Englis. Eight Ways to Get Students More Engaged in Online Conferences. PhD thesis at the Department of ITEE. 543-554. D.Ε.pdf. City University. EEEP Online Community of Practice: The First to Boldly Go in Greece. Statistics in Medicine. Greek Teachers' Training via the Internet: Presentation of the Greek Online Community KME from the University of the Aegean. A. J. and Development. Φθιώτιδας από την εφαρμογή προγραμμάτων Π. L. J. NY. Muij. Journal of Mathematical Sociology. Research. 29 September – 3 October 2004. Ph. (2005). G. (2005).. D. (2003). Multivariate data analysis. A. 543-568.G. (2006). 29-56. Paper presented at the 2nd National Conference on Environmental Education. 21-39).Διερεύνηση αιτιών.. In S. E.ucr. Tatham. Katsaros. McGarry. Fisher (Eds. New York: Cambridge University Press.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Gulati. M. Introduction to social network methods. Educational Technology. κατά την πενταετία 20012006 . S. 62-64. Jeong. Riverside. Norman.edu/%7Ehanneman/SOC157/TEXT/TextIndex.. Hlapanis. M. 2001-2006 in the 4th Sector of Fthiotida Prefecture. & Riddle.. Doing Quantitative Research in Education with SPSS. & Davidson-Shivers. & Wenger. The psychology of everyday things. (2007). New Jersey. E. Expectations of cooperation and competition and their effects on observers' vicarious emotional responses.au/archive/00003422/01/McGarry%20PhD%20Thesis%20 -%20Things%20to%20Think%20With. Hair.edu. learning and new technology: a review for teachers. Structural Equivalence of individuals in social networks. A. W. (1988). School of Information Technology and Electrical Engineering. & Dimitrakopoulou. Anderson. Khine & D. British Journal of Educational Technology. N. (2007). Lave. (2006). (1989). J. Retrieved 14/02/2006 from http://faculty.. Hanneman. USA: Basic Books Inc. C. Cambridge [England] . (1991).uq. University of Queensland.Investigating the reasons. Prentice-Hall Inc. Paper presented at the 4th Conference in ICT in Education. Athens. Unpublished Research. Creating a Technology-Rich Constructivist Learning Environment in a classroom management module. (2003). T. B. 9(4).. M.

"Contingency table involving small numbers and the χ2 test". Suthers. C.. (1987). A. (2000). 315–337. 517-547. Greece. Etiquette.. Social structure from multiple networks: I. D. model.. Harper Collins College. L. L. Cambridge: Cambridge University Press. empathy and trust in communities of practice: Steppingstones to social capital. E-moderating: the key to teaching and learning online. Online Communities. Poile. B. and Fidell. International Journal of Computer-Supported Collaborative Learning.umd. & Breiger. R. Yates. Sears (Eds. (2006). R. Hierarchical Clustering Explorer 3. Thesis Centre for Interactive Systems Engineering LSBU Page 217 . C.). White. A.cs. Seo. P.0.O. B.. D. Ph. 596-620). J. University of Maryland.edu/hcil/hce/.N. Patras. Technology affordances for intersubjective meaning making: A research agenda for CSCL. In J. (2005). Καταγραφή επιµορφωτικών αναγκών των εκπαιδευτικών και σχεδιασµός προγραµµάτων επιµόρφωσης: Η µελέτη περίπτωσης µίας σχολικής µονάδας της Πρωτοβάθµιας Εκπαίδευσης µε το µοντέλο D. G. Journal of Universal Computer Science. Mahwah: NJ: Lawrence Erlbaum Associates Inc. Requirements Engineering in New Product Development: How Effective are the Socio-Technical Interactions? Communications of ACM. (1976). Duimering. The Psychology of the Internet. 217-235. & Ran. (2005).Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Preece. (2004). S. F (1934). (1996). Wallace. Preece.. J.10)..I. PhD dissertation at the Greek Open University.D. (2008). Using Social Network Analysis Techniques to Examine Online Interactions. K.N. 51(3). 10. Available at http://www. Tsetsilas. Plans and Situated Action: The Problem of Human-Machine Communication New York: Cambridge University Press. H. Human-Computer Interaction Lab. D.O.. (1999). 46-56. Zheng. P. A. Journal of the Royal Statistical Society (Supplement) 1. Handbook of Human-Computer Interaction (pp. Salmon. Jacko & A. J. Publishers. L. I. S. P. US-China Education Review. Derbentseva. College Park. 1(3).. Reporting teachers' training needs and design training schemes: The case study from a primary school using D.. 87.. 2(9) (Serial No. Using Multivariate Statistics. 77-82. Willging. Boorman. London: Kogan Page. (2003). Safayeni. Tabachnick.I. American Journal of Sociology. Blockmodels of roles and positions. & Maloney-Krichmar. G. (2006). 294-302. N. Suchman. F.

the implications from the findings. the limitations.D. Thesis Centre for Interactive Systems Engineering LSBU Page 218 .Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions 7 • • • • • • Conclusions: Summary of the thesis. contributions and recommendations Key Topics covered in this chapter: How the thesis reached this point Exploring the contributions to existing knowledge How this study is limited to this particular context in time and elearning space Recommendations to the participants in the e-learning design Suggestions for future directions The conclusion derived from this study Chapter 7 provides a summary of the thesis and explores its contributions to knowledge. Ph. the conclusions as well as some future research directions. it presents the main blocking factors for the Greek teachers’ passive participation. In addition.

and externalisation of knowledge as active learning from active participation. the conclusions are presented next. scrutinize the strengths and limitations of the work and suggest implications for findings and directions for the future investigations might take.D. Thesis Centre for Interactive Systems Engineering LSBU Page 219 . These two levels are found to be brought together in collaborative learning as a socially shared cognition problem solving and evaluation technique. this study set out to answer a number of exploratory questions. Collaborative learning research is multi-disciplinary.2 SCOPE OF FINDINGS In this section the scope of the research and the overall findings are described in relation to the aim and objectives. from the study of the individual to the study of groups and communities. Cognition is a complex social phenomenon based on the individual’s re-arrangements of knowledge influenced by social interactions. This study aimed to carry out the following with regard to Collaborative e-Learning Communities: Thus.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions 7. This exact process is also reflected in the structure of the current educational systems that fail to follow the Ph. This process was reflected in the literature review in a progressive way. Social interactions provide the common ground for discussion that occurs on two levels: internalisation of knowledge as reflection from passive participation.1 INTRODUCTION The aim of this chapter is to examine the scope of the findings. to put them into the context. 7.

The second signpost was the development of a social and technical infrastructure that supports the key activity. if educational design could understand the technology of collaborative practice. she acts as both a user and a learner. Thesis Centre for Interactive Systems Engineering LSBU Page 220 .Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions last step towards community knowledge building and thus. Thus. In other words. educational design and in particular e-learning design neglected the dual and situated persona of the learner. it is legitimate for the participant to be in any participation level. learning. finding suitable variables to describe and support increasing participation was part of the problem. in Ph. So. these two trends have evolved almost separately. This means that situated design needs to follow the evolution of its context. the e-learning systems were found to be information-based mainly supporting monologue instead of being communication-based towards dialogue. the collaborative e-learning community. in this case. e-learning quality could be improved. However. if the e-learners were adequately supported by tools and evaluation techniques then passive participation would drop to the minimum. The first signpost was the radical view that design is planning with a post hoc nature. In other words. However. resulting in the e-learning passive mode. In addition. For the past 50 years two main trends have been observed in education. all Greek teachers never crossed the threshold of passive participation for 3 years because of personal. For this reason they fail to support e-learners’ transition between internalisation to externalisation and becoming active participants. available learning management systems do not support users’ in co-creating the e-learners-generated context. educational or instructional design is the systematic processing of activities to solve an instructional problem with the aid of technologies.D. As a result. social. There have not been many attempts to support collaborative e-learning from a community viewpoint. No coherent frameworks were found to support increasing participation in collaborative e-learning communities. Additionally. the sociocultural focus and the use of technology. Design is planning. Socio-technical design and user-centred design were planning approaches aiming to acknowledge that the development of interactive technologies increasingly relies on an appreciation of the social circumstances in which systems are used. design was found to be inadequate to carry the social interactions. mere provision of information points to poor e-learning quality. institutional or technical obstacles.

Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions this study. and technical level. Therefore the answers to the exploratory questions led to thesis contributions. Thesis Centre for Interactive Systems Engineering LSBU Page 221 . learning. No tools and techniques were found to evaluate the process of increasing participation in collaborative e-learning communities as progressive discourse. However. collaborative e-learning. none of the projects targeted to the creation and development of an active collaborative e-learning community.D. These guidelines suggested ethnotechnology as the methodology to inform design on a social. The third signpost was the real-time setting of the key activity and its evaluation. Several projects investigated the use of progressive dialogue to support collaborative leaning. The last signpost was the involvement of e-learning participants in the design. Ph.

and is informed by ethnotechnology and real-time evaluation tools. evaluate.3. reflection. The Sense of e-Learning Community Index (SeLCI) was developed and successfully tested based on collaborative e-learning community sociodynamics and collaborative e-learning. In addition.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions 7. critical thinking. 7. this involves all e-learning participants in design. Thesis Centre for Interactive Systems Engineering LSBU Page 222 . agree.3 CONTRIBUTIONS The second set of questions suggested specific schemes to tackle passive participation and poor e-learning quality. medium. trust. These determinants were analysed on a temporal basis towards the evolution of the collaborative e-learning community. empathy. and social network analysis based on: global cohesion anchored in density. a coherent and measurable framework was developed upon the increasing participation process: zero. medium. and local nodes and centrality in real time. the transition between passive and active participation was called the sleeper effect. and use. and high passive participation. explore. summarise. and high active participation. low. explain. These were the key contributions of the thesis. and co-creativity. sense of belonging. question.1 Key Contributions The Collaborative e-Learning Episode (CeLE) was proposed as a coherent and cyclical analytical framework to identify. cliques and structural equivalence. Ph. The main attributes were found to be: inform. reciprocity. The post hoc structure for E-Learning Engineering was revealed integrating design. SeLCI consists of indicators to understand how a community functions as well as being determinants for its success. and evaluate contributions in Collaborative e-Learning Communities (CeLC).D. The CeLE was found to support Greek teachers’ progressive discourse. and low. intensity characterised by e-learners’ levels of participation and persistence on posting. evaluation. global centrality derived from in.and out-degree centrality and closeness. there were also secondary contributions. collaborative e-learning quality measured by the number of CeLEs and participants’ comments on their learning. analyse. Lastly. Nevertheless. and other. These were: community evolution.

Although it is unadvisable for Ethnotechnology to be conducted by one person being ethnographer. that was collaborative e-learning. The collaboration and interaction between the Greek teachers ii. it appears that their use aided the participation of the Greek teachers in their e-learning community. A conscious effort to tackle some of them was based on the suggested tools and evaluation techniques found to have solved the initial problem of passive participation and e-leaning quality. 7.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions Quality by design in e-learning appeared to be related to a complex web of factors. MessageTag was the tool built on the CeLE analytical framework. The Visualisation Interactions Tools Nodes and Centrality were found to be less supportive in the e-learner’s endeavour and had the most problems from a pedagogical usability viewpoint. However. 1. The role of external factors such as lack of legislation and slow Internet connections iii.2 Secondary Contributions The secondary contributions refer to e-research methods and pedagogical usability. The participation graphs and avatars that depicted active participation were found to have supported increasing participation in collaborative e-learning. As with the participation tools. Ethnotechnology was the exploitation of ethnography in design aiming at better understanding of the context under investigation. Some of these factors have been investigated and determined in each of the previous exploratory questions. it successfully supported e-learners because of its relevance to the key activity. The use of tools by users as learners Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 223 . e-tutor and researcher it was unavoidable in this study. The management of human-human interactions b. The use of ethnotechnology provided insights by: a. designer. Facilitating investigation on relations within a situated context for i. This was due to the fact that they were built on the technology of participants’ real practice. Facilitating investigation on human-computer interaction for i.D.3.

needs. targets. Rising levels of expectations and enthusiasm iv. and developers d. The use of tools by learners as users c. Supporting community’s continuity and Greek researchers’ professional learning v. Advancing best practices in researcher’s own community 2. Investigating e-learning participants’ roles. Supporting feedback for design on i. The dual persona of the learner as a learner and a user b. The emergence of alternative options and perspectives d. The measurement of pedagogical usability of new tools as related to: a. Triangulating qualitative and quantitative data with social network analysis increased: a. Increasing understanding for tasks and associated practices for i. The design for new tools c. Thesis Centre for Interactive Systems Engineering LSBU Page 224 . The evaluation by distinct groups of users. The translation of tasks into tools ii. The validity and reliability in the study b.D. Ph. The researcher’s self-reflection These contributions need to be considered within the limitations that follow. The derivation of new concepts 3. teachers. The context for diversity of perspectives c. The emergence of alternative perspectives d.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions ii. and visions ii. Transferring of knowledge within the organisation iii.

Also. the sample was not representative but rather based on participants’ special interest in Information and Communications Technology in education. In addition. From a design viewpoint. and future research directions.D. A network of institutional. the implications from the findings. 7. instructional. Lastly. on time. I had too many roles and limited programming skills resulting in not being able to tackle a problem on the log files for three of the tools. the circumstances under this study was carried out including time frame and unwillingness of the Greek educational authorities to cooperate. The problem of e-learning quality was found to be related to the Greek teachers’ passive participation. In addition. the Hawthorn effect was evident.4 THESIS LIMITATIONS There were several research constraints: the nature of e-research. there was no usability testing of the tools in a usability laboratory. the suggested tools and evaluation techniques need to be further tested and developed in different contexts to ensure their validity and reliability. Thesis Centre for Interactive Systems Engineering LSBU Page 225 . and limited research in collaborative elearning communities. one participant explicitly said that she wanted to contribute to this project. technical. the tools were not implemented in the real e-learning environment for the Greek teachers future use.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions 7.5 CONCLUSIONS This final section presents the overall blocking factors responsible for the Greek teachers’ passive participation. and personal obstacles acted as blocking factors on a micro and macro level: • • • • Diversity on the concept of e-learning quality in current research Traditional low value of collaborative e-learning reflected on lack of soft skills and training in online collaboration Lack of national policies and coordinated activities on behalf of educational authorities Lack of a variety of e-learning evaluation tools and techniques Ph.

improvement of soft skills and communication techniques.D. Brief description of the findings. the tools were built to facilitate them. and the associated tools aimed to bridge the social and learning gap. the recommendations for researchers. collaborative elearning and the introduction of new tools. e-learning engineers. 2006) and were also identified in my participant observation in 2 European e-learning projects. elearning practitioners. and legislation. The failure for e-readiness is evident in the great discrepancy between different ranks in Greece (The Economist Intelligence Unit & IMB Corporation. the participants’ prior knowledge and ability to interact was enhanced (Yang. there was also lack of governmental planning for elearning pedagogy.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions • • • • E-learning in Greece is still an unofficial practice based on isolated initiatives Lack of time for the Greek teachers to be engaged in life-long learning Lack of technical support Lack of perceiving the direction in education in the 21st century. the problem of elearning quality in the Greek School Network was not different or more intensive than in other European countries. Within the context of the study. Ph. their implications and recent related studies will be discussed next. the Sense of e-Learning Community Index (SeLCI). the Collaborative e-Learning Episode (CeLE) analytical framework. The implications from the first finding suggest the need for training the Greek teachers in current pedagogical collaborative approaches and new technologies. Implications from the findings The findings in this study demonstrate the significance of social. This also means that although this study was designed to be strictly situated. on a scale from 1 to 10. and the Greek educational authorities may have an impact on e-learning quality on a broader level (see recommendations in Appendix XIII). access to professional help and support. the first was related to community management and collaborative e-learning. Thus.80 was the government rank.g. Two interventions were made. technology. 8. 2008). these blocking factors have been reported on a European level in the recent E-quality report (Fernandes & Montalvo. 2003:16). learning. and technical aspects of e-learning. 2002). The conceptual frameworks of the participation levels. These implications point to the problem of e-readiness for e-learning (e. Kaminsky & Currie. Thesis Centre for Interactive Systems Engineering LSBU Page 226 . In fact. development of shared vision and goals.

They are related to the participants’ idiosyncratic character and influence participation. An implication is that in such favourable circumstances. the gap between the previous two levels. experiences. goals. Thurmond. From a community management viewpoint. as a key component in assessing the effectiveness of e-learning (e. The interaction speed was high when the e-tutors were involved. trace solutions as processes. Observing other e-learners’ behaviour and develop reciprocity empathy and trust are interpersonal and intrapersonal skills with an impact on participants’ motivation.g. Zhao et al. and educate and support everyone involved in elearning. involvement. 1991. Community development was built on shared background. empathy and trust. These implications stress the impact of e-learners’ control in their own process of engagement as learners are usually not knowledgeable enough to make effective decisions (Dron. Thus. Sharing personal information and experiences through the profiles and developing a sense of identification and co-presence between the e-learners can foster community building. time and activities management was anchored in the passive and active participation levels.. Increasing participation from passive to active fostered interaction flow and continuity and thus.66 the society rank.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions 5. This was to enhance interactivity. 2004). these levels confirmed the concept of learning as participation (Lave & Wenger. In fact.87 the industry rank. passive participants are highly likely to get engaged. Tutors-learners and learners-learners interactions were investigated using social network analysis on a macro level across the network and on a micro level within small groups. Kanes & Lerman. and 5. Squires (1999) suggested the need to design for freedom and flexibility so that educational software can adapt to their idiosyncratic needs and styles. innovation and the classification of useful ideas (Wu et al. Thesis Centre for Interactive Systems Engineering LSBU Page 227 . and visions. the implications is that organisations and business need to understand and correctly judge real situations. This is because the differences between the participants are minimised.. In social networks. 2008). information flow is relevant to issues of productivity. Controlling and delegating interactions can be Ph. Implications from this study suggest that interactivity has to be increased prior to e-learning activities and facilitated by reciprocity. 2005. and learning. 2003). An implication is to delegate control in interactions to e-learners in order to stabilise the interaction speed. as well as the “sleeper effect”.D. 2007). indicating control of the speed of the information flow.

since interaction density was directly linked to text richness. An implication is linked to the collaborative learning strong socio-cultural nature. 2007:63). They were evident in the more complex discussions at the end of the course comparing to monologues as redundant messages at the beginning of the course.g. 2007) even though it was mentioned in Thorndike (1920) as knowledge to manage social situations and ‘act wisely in human relations’. In addition..com). social emotions require self-consciousness (Goleman. richness is a measure of the interactive learning process (Stahl. Another implication is related to the new model for distributed leadership implemented as public consultancy from governmental organisations and business. avatars and other tools aiming to enhance presence and co-presence. For example. an implication refers to simulating (cloning) e-tutoring via vicarious learning. Decentralising the power in a network increases the possibilities for co-creativity and innovation within an e-learning community. Pickering et al. http://www. social awareness and the development of the sense of belonging to a community were only recently related to social intelligence. Ph. 2002). The development of collaborative learning networks and shared practice has been found to be themes for continuing professional development (e.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions connected to controlling or delegating e-learning (Dron. Thesis Centre for Interactive Systems Engineering LSBU Page 228 . Dell. First. An implication is that increasing reciprocity can facilitate the transition from monologues to dialogues as there are increasing clues as opportunities for critical engagement in dialogue. thus social intelligence can be a significant factor for successful e-learning communities. The latter has been accepted as an important soft skill (Goleman. voting and discussing ideas (e. as the e-tutors created groups with similar behaviour around them. Lastly. 2007). Therefore. communication and problem-solving skills were improved by the elearners’ interactions. Overall. anyone can participate in decision making by sharing. The e-tutors should adopt different learning and interaction levels and styles based on e-learners’ idiosyncratic character aiming for all learners to achieve their goals.g. 2007:131) that can be with the creation of profiles. The analytical framework of collaborative e-learning episodes (CeLE) and the associated tool MessageTag managed to bridge methods and tasks and provide simple to use and reliable assessment of the e-learning quality. technologies need to be able to adapt to individuals’ changing needs and situations.dellideastorm.D. A second implication is related to the e-tutors’ ability to guide the e-learners into the journey of critical thinking and knowledge co-construction and then leave them on their own capabilities.

Thesis Centre for Interactive Systems Engineering LSBU Page 229 . discover e-learners’ weaknesses and strengths. However. aid in team building by “bonding” the team. The direct implication leads to the different learning and interaction styles needed in e-learning (Cooze & Barbour. there is a need to re-think e-learning processes and methodologies and make an effort to engage participants in the learning process (Rodgers. Hendler and colleagues stress that ‘today’s applications are very early social machines. intensity. Anderson. 2007). formative and summative evaluation. it created an opposing dynamic in the Greek educational system by suggesting that the community needs to move forward and acquire the new competencies and technologies for the 21st century. E-tutors can direct and control online discussions.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions Overall. small groups and networks or as Anderson (2008) suggested. From a design viewpoint.D. 2007. 2000) as e-learners as well as developers and e-tutors. in the recent Communication of the ACM journal. several groups need to evaluate learning technologies and applications anchored in their different personas such a novice and expert users (Faulkner. Such tools need to function on multiple levels supporting interaction for individuals. The implications are related to the need for numerical and graphical evaluation tools that demonstrate their purpose of use. Educators and designers need to re-visit teaching and learning strategies and their relationship to the level of e-learners’ engagement taking under consideration the strong idiosyncratic character of e-learners’ interaction and engagement. and record the discussion. This is because the social and learning aspect was anchored in legitimate peripheral participation as in Community of Practice Type 1 (CPT1). limited by the fact that they are largely isolated from another’ (2008:65). conflict and discontinuity of practice and Ph. and facilitate e-learners increase control as in self-emerged collaborative learning (Dron. This implication depicts the limitations of current interactive applications. Designers need to seriously consider the social aspects of learning. A pedagogical usability evaluation framework can be used to evaluate usability and utility of new tools. and persistence in social and learning interactions. activate the lurkers with specific questions. 2008): ‘CPT2 is built around tension. The interventions built the types of communities of practice discussed by Kanes and Lerman (2008). 2007). The use of new tools was found to support collaborative e-learning to an adequate extent. 2007). This creates a conflict within the educational system as with Community of Practice Type 2 (CPT2) (Kanes & Lerman. Such tools can increase elearners’ focus. employ social software.

learning. • Research o Multidisciplinary research on the social. • E-Learning o o E-learning management as a social as well as a learning process. The impact of the relationships between the Collaborative e-Learning Episode’s attributes. Future research directions Some future directions are proposed on research. 3 different research perspectives can open possibilities beyond initial propositions and hypotheses. An implication is to consider conflict as changing behaviour that can facilitate the community’s movement to another level. Overall. o o Soft skills as part of teachers’ training and professional development. A final implication for research was connected to triangulating the data via quantitative. The process of e-learning and the ways different learning styles influence it. o o The e-learning styles in relation to social intelligence.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions production’. e-learning communities. • E-learning readiness o Institutions and organisations assessment for e-readiness on a technical and pedagogical level. Thesis Centre for Interactive Systems Engineering LSBU Page 230 . qualitative and social network analysis. Design for learning objects and their re-contextualisation based on the different critical thinking levels in Collaborative e-Learning Episodes. and technical aspects of e-learning activity. o Ethnotechnology as a methodology to advance community members’ own practices. Ph.D. useful and reliable tools and evaluation techniques to assess different levels and types of participation and critical thinking in collaborative e-learning. Resistance to change as a longitude study of e-learning communities. there is room for research on usable. and associated design. o The relationships between personalised and collaborative e-learning and their impact to e-learning quality. e-learning. o o Design for each of the Collaborative e-Learning Episode’s attributes. e-learning readiness for organisations.

within groups and on an individual level. indicate that collaboration is part of the human nature. o The role of decision making and leadership in e-learning communities. evaluation and development. Tools based on numeric and graphical representation for formative and summative evaluation. and the attributes that most facilitate e-learning to occur. o Specific pedagogical frameworks and design on associated tools. Ph.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions • E-Learning Community o Community design. management. o Design to support personalised and community e-learning environments. as well as this study. The discovery of the mirror neurons as the neurons that enable the representation of other humans in the brain. The impact of interactions in e-learning across a learning network. o The role of pedagogical usability in facilitating e-learning. Thesis Centre for Interactive Systems Engineering LSBU Page 231 . • Design o Multimodal interaction and interoperability between educational technologies. collaboration is written into our DNA.D. o o Design to support community management and evaluation. Building on collaborative e-learning communities can ensure e-learning quality. o The impact of the relationships between the Collaborative e-Learning Episode’s attributes. o Design for each of the Sense of e-Learning Community Index’s attributes. current e-learning research. the new communication technologies and social software. social intelligence. o o Evaluation of participation and its impact on students’ performance.

Thorndike. Rodgers. (Cscl 2002 Proceedings). Hall. Mahwah.) (2002) Computer support for collaborative learning: foundations for a CSCL community. J. Stahl G. T. M. Education for a Digital World: Advice. Examination of Interaction Variables as Predictors of Students' Satisfaction and Willingness to Enroll in Future Web-Based Courses while Controlling for Student Characteristics. 15(1). 10(3). Educational software and learning: Subversive use and volatile design. (2007). J. Available at http://cde. 221-241. Contributions to a theoretical framework for CSCL.)(2008). New York. (1999). New York: Bantam Book. and E..col. 51(7). Available at: http://www. Social Software to Support Distance Education Learners. Pickering.. NY: Palgrave. S. D. (Ed. Learning Styles: A Focus upon E-Learning Practices and their Implications for Successful Instructional Design. New Designs for Teachers’ Professional Learning. Shadbolt. A. Daly. and Currie. Situated Learning: Legitimate Peripheral Participation. C. Seattle: IBM. Education Economics. 45. (2007. Book Review – Control and constraint in E-learning: Choosing when to choose. Analysing Concepts Of Community Of Practice. 1-3.com/jaet/JAET4-1_Cooze.. D. Lerman (2008). Athabasca University: AU Press. D. Last access 21/07/08. Stahl (Ed. J. & Pachler. New Directions for Situated Cognition in Mathematics Education. NY: Cambridge University Press. W. Available at http://csdl2. Fernandes.computer. D. Ph. T. The 2003 E-learning Readiness Rankings. Faulkner. C. Educational Technology & Society. 8(2). (1920). 227–235. N. Berners-Lee. (2002). (2007). Measuring Value Added in Higher Education: a Proposed Methodology for Developing a Performance Indicator Based on Economic Value Added to Graduates. & Barbour.org/comp/proceedings/hicss/199/0001/01/00011079. Harper's Magazine. Web science: An Interdisciplinary Approach to Understanding the Web. British Columbia. Cooze. Wenger (1991). Available at http://www. Anderson. Kanes. M.pdf. T. 140. In Hirtz. Last access 20/01/2008. J. Intelligence and its use. (2007). (2008). (2006). NJ: Lawrence Erlbaum Associates. (2003). T. 60-71. Thurmond.D.athabascau.pdf Last access 15/08/07. L. J. Hendler. 55-74.org/colweb/site/pid/5312. The Economist Intelligence Unit and IMB Corporation (2003). MacMillan. Vancouver. While Paper. 60-69). E-Quality Synthesis Report: Experience-based Quality in European ODL. In Anderson T. The Theory and Practice of Online Learning. Thesis Centre for Interactive Systems Engineering LSBU Page 232 . E. (2008). US. Lave. In G. 191-211. Goleman.. N. International Review of Research in Open Distance Learning. Last access 21/07/08. Squires. Usability Engineering. Dron. (2008). Doctoral Dissertation. June). and Effective Practice from Around the Globe. University of Kansas. Journal of Applied Educational Technology. S & Harper. 4(1). & Montalvo. New York. Kaminski. UoL. Social intelligence: The New Science of Human Relationships.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions REFERENCES Anderson. (2007). CD-Rom. (2007). Guidelines.eduquery. Communications of the ACM. V. Bedford Way Papers. (Series: Mathematics Education Library): 301-326. X. Planning Your Online Course.. and S. C. (2000). Designing the undesignable: Social software and control. & Weitzner. (2008). London: Institute of Education. Canada: BCcampus and Commonwealth of Learning.ca/online_book/.

com/research/idl/papers/flow/flow. Teachers College Record. (2004).. 327-335. Huberman. B. Zhao.pdf Last access 15/03/2008.. Physica A.hpl. What makes the difference? A practical analysis of research on the effectiveness of distance education.A. J. A case study for promoting collaboration on online project-based learning. C.. (2005). 107. Information Flow in Social Groups... 1836-1884. L. Y.A. F. S. 2107-2112. Y. Lei. Adamic. H. Ph. S.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions Yang.. & Tan.D. USA. 337. Thesis Centre for Interactive Systems Engineering LSBU Page 233 .hp. Lai. (2002). Proceedings of the World Conference on Educational Multimedia. B.R. Yan. Wu. Available at http://www. Hypermedia & Telecommunications. & Tyler J.

Interactivity Management 1. Induction and training: An initial meeting for using Moodle might bring issues of usability of the system and suggest the problems students have on using the system on site (if any). An initial authentic reply and warm welcoming will encourage students to continue communication as well as shy students to send a message. only samples of the data are presented. 6. Profiles: The students need to be encouraged to construct their profiles. 2. Ph. information will be provided on ways for writing. − − Informative first page: Ensure that the community’s first page provides all needed information to encourage and inspire the students to engage. Additionally. surveys. Welcome note: A welcome note inspires and encourages the students to participate as well as giving additional information on technical and management issues (e.g. First message: The first of each student might define later behaviour. polls.D. APPENDIX I: ONLINE COMMUNITY MANAGEMENT A_I_1. votes. 9. 3. activities.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices This is the English version of the Appendixes. 5. Initial one-way communication: Introduction of the students (research interests. experiences etc) as well as tools (e. 8. Subgroups: Based on students’ research interests as well as hobbies sub-groups will create initial locus of interactions. Profiles provide a feeling of co-presence and enhance the sense of belonging. newsletters) can break the ice and give the necessary information to move to two ways of communication and productive interactions. Information. Registration system: Facilitate the registration process if necessary. Moderator’s Responsibilities . inappropriate behaviour etc. Newsletters inform members for news. Introduction of themes in collaboration with the students and the Students Administration Office. 7.g.) 4. replying and form an online message. software description. Discussion highlights as newsletters: A monthly newsletter that will provide a summary of the discussions and any additional information. Thesis Centre for Interactive Systems Engineering LSBU Page 234 . personal information that would like to share. make members aware of the previous issues and develop a felling of belonging to a community. In some occasions. hobbies.

12. Use of the expert: Students might need more formal or expert advice in addition to confidentiality that a community cannot provide. usually with an agreement with a previous message. stress of interesting points. an example to support the previous suggestions 4. arguments and points of view. According to Jacob Nielsen Internet user only skim webpages for headlines and marked keywords and they do it very fast. Then you get longer texts. Help and Support: The students need to feel that there is always someone there on a 24/7 basis. 2. and 5.1%) appeared to have a pattern: an initial introduction as a response to the selected message.D. A detailed description is following: 1. IMO this makes online communication and collaboration on sophisticated issues more or less impossible. 53. a greeting or an interesting quote used to ‘sign’ the message. A_I_2. A second suggestion from Participant A1 in the e-mmersion study is as follows: ‘The lines in text on paper should. 3. signing out. Introduction. On the screen lines should probably be even shorter. more suggestions. Thesis Centre for Interactive Systems Engineering LSBU Page 235 . 13. Suggestions for Writing Online Messages Twenty five (25) messages (N=47. Give the members ownership of their learning and learning outcomes (work behind the scene). Monitoring and control: Subjects and discussions moderation could enhance or prevent specific issues to be brought on the surface. Motivation of silent participants: Moderatos need to ‘kick’ the students to contribute to the community. for the sake of readability not exceed 55-60 characters. 11.’ Ph.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices 10. an extensive explanation and justification of their point was made. an example was making suggestions very clear and lastly. 14.

their contracts have not been renewed which indicates a negative motivation to conduct extra work. it cannot ensure that the stakeholders can take actions to support this study. it seems that this is not possible to be done under the specific circumstances. In other words. • The Greek teachers can acquire knowledge on the pedagogical use of new technologies. Quantification of Risk Ph. used for their professional development. In addition. Nature of risk 4. Even though it is usually a continuous and developing process.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices APPENDIX II: RISK MANAGEMENT The British Institute of Risk Management (IRM). however.D. • The Greek teachers’ absence of participation in collaborative e-learning activities does not mean that they will not cooperate in the study. it was only used for the specific purpose of completing the course under the desirable circumstances. fieldwork suggested that if they were given the adequate help and support they will. and estimation as well as risk reporting and decision making. Risk can be defined as the combination of the probability of an event and its consequences (ISO/IEC Guide 73). Risk assessment found to be important for this study after the initial findings on the absence of cooperation with the Greek educational authorities as well as the absence of the Greek teachers’ participation in collaborative activities. Two certificates can be provided by London South Bank University and the Greek School Network. Risk Description • Absence of cooperation between the Greek educational authorities • Absence of participation in collaborative e-learning for 3 years • The Greek educational authorities may have provided a document to facilitate the conduction of the research. Stakeholders 5. Risk management increases the probability of success and also reduces the probability of failure and uncertainty. Risk description was based on risk analysis. Name of risk 2. • Situated implementation in a real environment is the ultimate goal for this study. • Operational • Knowledge management • The computers engineers that provide the technical support do not have any additional benefits if they implement the proposed tools other than some publications in their CV. However. description. Scope of risk 3. In 1. a shift to a different research context was not desirable. identification. the Association of Insurance and Risk Managers (AIRMIC) and the National Forum for Risk Management in the Public Sector (ALARM) (2002) have published a standard risk management procedure. Thesis Centre for Interactive Systems Engineering LSBU Page 236 .

D. In other words. • Not participating in collaborative e-learning will be a failure of the suggested propositions in total. Despite the fact that the Deputy Director of the Greek School Network Technical Support said that the tools will be implemented. it appears that in the first case of implementing the tools in the real e-learning environment developed for the Greek teachers the risk is very high. The advantage of having an inactive community for 3 years is the increase of reliability and validity of the study. Having a Ph. this study has a national Greek character and this is another advantage for the study as well as for me as being a Greek teacher. • A server is needed to host the research pool • Initial work on instructional learning is acquired by producing documents on the use of the e-learning environment as well as collaborative e-learning. (Studies on the internet are highly unlikely to be completely controlled. the Hawthorn effect is always a threat in research either in real or experimental environments. the risk estimation is 95%. (A high risk is usually more than 25%). Thesis Centre for Interactive Systems Engineering LSBU Page 237 . • The Greek teachers can be taught and supported towards collaborative e-learning. 7. Potential action for improvement 9. • An alternative environment linked to the e-learning environment at the Greek School Network can be a middle solution. The study conducted in a research pool cannot not be completely experimental if linked to the real one.) • Theory and fieldwork suggested that if appropriate conditions are in place then absence of participation can be eliminated. Risk treatment and control mechanisms 8. However. 6. In addition. Strategy and Policy Development Based on the risk description. This is because the chance of enabling participation is considered to be more than 75%. in the second occasion of supporting Greek teachers in e-learning the risk is very low. This is because cooperation with the Greek educational authorities found to be 0% on more than one occasion. • There is nothing that can be done to change the situation for the computer engineers as this project is not part of their everyday work.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices addition. another is the Internet nature. it is not certain that finding a different environment can solve this problem. Continuous help and support can reduce participants’ uncertainty. The above description and risk estimation suggest that a shift to the study from a real implementation to a quasi experimental one cannot completely alter the results and significantly affect reliability and validity. • Collaborative e-learning is the pedagogical goal of this study and it is highly likely to achieve it. less that 2%. Risk Tolerance • Not implementing the tools in a real and situated environment can turn the study to quasi experimental.

provision of adequate help and support based on initial instructional learning and then shift to collaborative e-learning can be the risk treatment for the second risk of absence of participation.D.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices linked research pool can be the implemented measure to tackle the first risk. Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 238 .

D. The usability section in the Initial Questionnaire Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 239 .Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices APPENDIX III: INITIAL QUESTIONNAIRE (SAMPLE) A_III_1.

Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices APPENDIX IV: QUESTIONNAIRE MAIN STUDY I (SAMPLE) A_IV_1. Thesis Centre for Interactive Systems Engineering LSBU Page 240 . Demographic data Ph.D.

Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices APPENDIX V: QUESTIONNAIRE MAIN STUDY II Dear colleague user. Niki Lambropoulos Ph. there was an effort to be easy to answer. The e-learning community Participation in the e-learning community Learning in the e-learning community New tools’ use and usability Professional development The questionnaires should be sent to Niki Lambropoulos until the 15th of April 2007 the latest. Thank you very much for your participation. The information you will provide will provide a process for the Greek teachers’ active participation in the online community and will reduce the time of engagement. your opinions and propositions will present existing e-learning problems. Β. C. Despite the fact that the questionnaire is rather long. Best regards. The information you will provide are strictly confidential and will only be used for research purposes. at the email nikilambropoulos@gmail. late April 2007 the latest.com You can use this text box to write your name and address in Greece in order to send you the certificates of participation from London South Bank University and the Greek School Network. They will be sent after all questionnaires are sent. Ε. Thesis Centre for Interactive Systems Engineering LSBU Page 241 . In addition. The results from this study will be available to you after its completion (September 2007). with accuracy and speed in 30 minutes. Your contribution to this research is more valuable.D. It is consisted of the following 5 sections: Α. D.

In your own opinion. Thesis Centre for Interactive Systems Engineering LSBU Page 242 . You can fill it in the way you prefer as there are not right and wrong answers –only different pinions-. can you describe some elements that prove it? 2bi.D. If yes. In your own opinion. The questionnaire was based on Word Processor form function.Was it comfortable or uncomfortable to talk to an online environment? 3). most of the participants shared their values. How did the discussion forums help/restrict the development of a sense of belonging to the e-learning community? 2ci. THE E-LEARNING COMMUNITY 1). (a) Yes (b) No (c) N/A 7). In your own opinion. Do you think there was community development in time on the online course? (a) Yes (b) No (c) N/A 2ai. Did you develop any relationships. therefore some questions are not very clear. either friendly or professional with the other co-learners? (a) Yes (b) No (c) N/A 3ai. The selection Don’t know / Don’t answer is referred as N/A. you simply click in the grey areas and select one of the answers (unless stated otherwise) on a scale from 1 (low) to 5 (very much) or simply write. How much do you think that the members were helping each other in the online course? Ph. If you think community development was evident. if any? 5). most of the participants had common interests and goals. How many people did you know in person before the online course? (a) Nobody (b) A few (c) Almost everyone (d) N/A 2). Α.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices FINAL QUESTIONNAIRE You are invited to dedicate some of your precious time to fill in the following questionnaire. (a) Yes (b) No (c) N/A 6). do you think they will continue outside the online course? (a) Yes (b) No (c) N/A 4). what was the element that hold the community members together.

you knew what another person was feeling when a you were reading her/his message? .. How long (in days) do you think it took for the community to emerge – develop a sense of working together? 9).. Was the sense of being together strong? (a) Yes (b) No (c) neither yes or no (d) N/A 10).you took an action upon it? 1 2 3 4 5 11)... I can trust most of the participants. (a) Yes (b) No (c) N/A 1 Very Small extend 12). Do you agree or disagree with following statements: 11i.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices (a) Very (e) N/A (b) Not much (c) A little (d) Almost no help 8). where 1 means very little and 5 a lot.. (a) Yes (b) No (c) N/A 11ii. (a) Yes (b) No (c) N/A 11iv. a b In a scale from 1-5. how much do you thin the participants trusted each other in the online course? Individuals who had similar specialisation Individuals who had similar writing skills 2 Small exten d 3 Neit her 4 Great exten d 5 Very great extend Ph..you could feel what a person was feeling when b you were reading her/his message? c . .. Nobody trusted no one as regards knowledge exchange and contribution. Most of the participants were trying to help.D. Thesis Centre for Interactive Systems Engineering LSBU Page 243 . Do you think that.. (a) Yes (b) No (c) N/A 11iii. Somebody has to be very careful in the online course because some participants wanted to take advantage of people and situations.

Do you think that YOUR participation was useful to the e-learning community? (a) Yes (b) No (c) N/A 15ai.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices c d e E-tutors Individuals who seemed to have advanced knowledge on the subject You cant trust anyone 13)... how did you learn issues that were of your interest? Can you describe some ways? 17).you participated actively in the discussions? . a b c d e f g Do you think that.... the e-learning management system helped the e-learning community? . If NO. Thesis Centre for Interactive Systems Engineering LSBU Page 244 . In general.you knew the kind of community you were participating? . do you think that the level of trust during the online course: (a) Increased (b) Decreased (c) Remained the same (d) N/A Β. what were these roles (please give an example)? 18. from whom did you learn issues that were of your interest? 16a After your registration to the online course.. PARTICIPATION IN THE E-LEARNING COMMUNITY 14).. can you describe why not? 16. If YES.the new tools helped the e-learning community? 1 2 3 4 5 15).you liked to work together with the other members? . After your registration to the online course.... can you describe why? 15bi...you felt free to express yourself? .. If YES.. ..D. Do you think that active participation is necessary? Ph.you knew the community’s netiquette for proper interaction? .. Do you thin that there were roles developed between the members? (a) Yes (b) No (c) N/A 17ai.

18bi. What were the most important examples of collaborative learning for you? Ph.g. Do you think that the members who DID NOT participate actively had to remain in the course? (a) Yes (b) No (c) N/A 20). why not? 22). in what ways? 20β. why not? 19. programmers. The participants were specialised in different areas. etc) is necessary? (a) Yes (b) No (c) N/A 21a. why?.. Do you think that cooperation with other professional communities (e. Do you think that this helped collaboration in the online course? (a) Very (b) Not much (c) A little (d) No (e) N/A C. If YES. Did you learn ways to enhance collaboration using new tools in the online course? (a) Yes (b) No (c) N/A 23a. LEARNING IN THE COLLABORATIVE ELEARNING COMMUNITY 23). If YES. how did you learn them? 24).D. If NO.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices (a) Yes (b) No (c) N/A 18ai. why not? 21). If NO. If NO. If YES. If YES. why? 21b. Do you think that new members in the e-learning community can contribute to existing knowledge? (a) Yes (b) No (c) N/A 20a. Thesis Centre for Interactive Systems Engineering LSBU Page 245 . multimedia developers.

Who did you MOSTLY turn for help when you faced some problems? (a) to the e-tutors (c) nobody (b) to other e-learners (d) to a different resource (e) N/A D. NEW TOOLS’ USE & USABILITY 26 a b c d Frequency for using the tool in Moodle.D... Thesis Centre for Interactive Systems Engineering LSBU Page 246 .Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices 25). Participation – participation graphs and Avatar Kind of reply – message tag in reply Sociogramme VIT Nodes Sociogramme VIT Centrality Did you think that the tools was attractive? Participation – participation graphs and Avatar Kind of reply – message tag in reply Sociogramme VIT Nodes Sociogramme VIT Centrality Was the additional information with the new tools on Moodle tiring? Participation – participation graphs and Avatar Kind of reply – message tag in reply Sociogramme VIT Nodes Sociogramme VIT Centrality How would rate the quality of the graphics? Participation – participation graphs and Avatar Kind of reply – message tag in reply Sociogramme VIT Nodes Sociogramme VIT Centrality Did you think that the tool design was functional (easy 1 to use)? Participation – participation graphs and Avatar Kind of reply – message tag in reply Sociogramme VIT Nodes Sociogramme VIT Centrality How would you rate the originality of the tool? Participation – participation graphs and Avatar 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 27 a b c d 28 a b c d 1 2 3 4 5 29 a b c d 30 a b c d 2 3 4 5 31 a Ph.

D. Thesis Centre for Interactive Systems Engineering LSBU Page 247 .Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices b c d Kind of reply – message tag in reply Sociogramme VIT Nodes Sociogramme VIT Centrality Were the instructions on the tools satisfactory? Participation – participation graphs and Avatar Kind of reply – message tag in reply Sociogramme VIT Nodes Sociogramme VIT Centrality Did you think that learning how to use the tool is time consuming and tiring? Participation – participation graphs and Avatar Kind of reply – message tag in reply Sociogramme VIT Nodes Sociogramme VIT Centrality Did you think that accessibility of the tool was easy? Participation – participation graphs and Avatar Kind of reply – message tag in reply Sociogramme VIT Nodes Sociogramme VIT Centrality Were there any failures to complete an activity because of tool failure? Participation – participation graphs and Avatar Kind of reply – message tag in reply Sociogramme VIT Nodes Sociogramme VIT Centrality Did you think that the tool responded relatively quickly? Participation – participation graphs and Avatar Kind of reply – message tag in reply Sociogramme VIT Nodes Sociogramme VIT Centrality Did you think that the tool responded to the educational goals? Participation – participation graphs and Avatar Kind of reply – message tag in reply Sociogramme VIT Nodes Sociogramme VIT Centrality Did you think that the tool enhanced motivation for 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 32 a b c d 33 a b c d 1 2 3 4 5 34 a b c d 35 a b c d 1 2 3 4 5 36 a b c d 1 2 3 4 5 37 a b c d 1 2 3 4 5 38 Ph.

D.. PROFESSIONAL DEVELOPMENT WITHIN CoP 41. can you describe what is was? 42. The worst thing(s) in my participation in the study was… 44.. How many times have the educational authorities from the Greek Ministry of Education contacted you to ask your opinion on learning technology issues? Ph. The best thing(s) in my participation in the study was… 43.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices communication and collaboration? a b c d Participation – participation graphs and Avatar Kind of reply – message tag in reply Sociogramme VIT Nodes Sociogramme VIT Centrality Did you think that the tool was effective to facilitate collaborative learning? Participation – participation graphs and Avatar Kind of reply – message tag in reply Sociogramme VIT Nodes Sociogramme VIT Centrality 39 a b c d 1 2 3 4 5 40 a b c d How satisfied were you using. Which one of the statements describes the common space between discussions and your own educational practice? Please choose accordingly (more than one answers are allowed): (a) I brought ideas/information from my job to the discussions (b) Some e-learners provided information that made me think on my educational practice (c) Some e-learners provided information that triggered ideas for my educational practice (d) The discussions will not change my educational practice (e) I found ideas/information I can use in my educational practice (f) Other If Other. Participation – participation graphs and Avatar Kind of reply – message tag in reply Sociogramme VIT Nodes Sociogramme VIT Centrality 1 2 3 4 5 1) Ε. Thesis Centre for Interactive Systems Engineering LSBU Page 248 .

com * niki@intelligenesis.ac. London. United Kingdom http://www.D.K. Thesis Centre for Interactive Systems Engineering LSBU Page 249 . Thank you very much for your participation in the study.homechoice. Niki Lambropoulos Research Student. Do you have any other comments? Did you have any problems with the questionnaire? You can use this space for more comments and observations: OBSERVATION REPORT Name (Optional) No.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices (a) Many times (b) Some times (c) A couple of times (d) Never 45. How many times have the educational authorities from the Greek Pedagogical institute contacted you to ask your opinion on learning technology issues? (a) Many times (b) Some times (c) A couple of times (d) Never 46. Centre for Interactive Systems Engineering Faculty of Business. Computing and Information Management London South Bank University. Comments Please check whether you have replied to all questions. Tel: +44(0)2084465909 Email: nikilambropoulos@gmail.lambropoulos Ph.co.lsbu.uk/bcim/research/cise/ * http://nikilambropoulos. Description of activity 1 2 3 4 .org & 1 Dale Grove London N12 8EE U..uk Skype: niki.

Netiquette Ph. these were the following: A_VI_1.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices APPENDIX VI: PARTICIPANTS’ DOCUMENTS Only samples from the participants’ documents are presented here. Thesis Centre for Interactive Systems Engineering LSBU Page 250 .D. Invitation to the study A_VI_2.

Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices A_VI_3.D. Instructions of use Moodle and the Research Pool Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 251 .

7 12.92) Missing Ph.2 54.6 32.5 3.9 9.6 6.5 3.7 0 0 25.6 61.2 0 39 61 0 35.5 12.9 0 38.3 9.8 16.d: 0.7 51.8 35.4 12.7 0 1 17 6 4 3 0 0 13 8 5 5 0 1 21 4 2 3 0 0 8 8 11 4 0 12 16 2 1 7 19 2 2 1 0 0 3. Thesis Centre for Interactive Systems Engineering LSBU Page 252 .2 67.8 25.7 0 0 41.5 9.software 5 Use of new technologies in class: h/w 6 Use of new technologies for education: h/w 7 Use of Internet in class: h/w 8 Use of Internet for education: h/w 9 Use of LMS 20-30 30-40 40-65 Missing Female Male Missing 1-5 6-10 11-20 40+ Missing 1-5 6-10 11-20 20+ Missing No use 1 -2 h 3-5 h 6-12 h 12+ h Missing No use 1 -2 h 3-5 h 6-12 h 12+ h Missing No use 1 -2 h 3-5 h 6-12 h 12+ h Missing No use 1 -2 h 3-5 h 6-12 h 12+ h Missing 0 1-5 6-10 Missing Simple OK Difficult N/A Other 7 10 14 0 12 19 0 11 5 11 4 22.9 6.5 16.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices APPENDIX VII: THE E-MMERSION DATA ANALYSIS A_VII_1.2 22.9 25.9 45.8 19.1 16.2 32.5 6.3 45.1 35. Demographics Demographics Variables Values # Participants Percent 1 Age 2 Gender 3 Working Experience 4 Use of computer .5 12.2 0 10 Moodle (std.3 6.9 0 4 14 10 3 0 12.D.1 0 3.

9 12. Crosstabulation: Moodle Use * Time using LMS Time using LMS (years) 0 Simple OK Moodle Use Difficult Don't Know Other Total 4 6 1 1 0 12 1-5 2 11 1 1 1 16 6-10 1 1 0 0 0 2 nil 0 1 0 0 0 1 7 19 2 2 1 31 Total E-Learners' perseptions on Moodle usability level 2.2 12. 7% 7. Training in Educational Technologies Training in Educational Technologies Training locus 1 Old academies 2 Universities 3 Postgraduate studies 4 Courses 5 N/A 6 Other Total # Participants 1 4 4 14 4 4 31 Percent 3. 7% 2. Thesis Centre for Interactive Systems Engineering LSBU Page 253 .0 Ph. Moodle Usability A_VII_5.2 12.9 100. 62% A_VII_4.9 12. Crosstabulation: Internet Use in class * Use for educational purposes Internet use for educational purposes (h/w) 1 -2 h no use Internet use in class (h/w) 1 -2 h 3-5 h 6-12 h 12+ h Total 0 8 0 0 0 8 3-5 h 1 4 2 0 1 8 6-12 h 0 8 1 1 1 11 12+ h 0 1 1 1 1 4 Total 1 21 4 2 3 31 A_VII_3.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices A_VII_2.D.9 45. 24% Easy OK Difficult Don't Know Missing: 1 18.

Reasons for participating in e-learning communities A_VII_7. Use of e-learning tools Web Design Pool: A_VII_8.D. Messages Quantitative Analysis MESSAGES QUANTITATIVE ANALYSIS Forums 1 Ask the expert Topics 3 Threads Depth (P) 4/5 (P) 2 (P) 2 4/8 (P) 10 2 6/7 5/5 6/6 Richness of text (# words) 522 302 233 59 838 48 558 282 247 Date 30/03-11/04/06 27/03-09/04/06 30/03-09/04/06 27/03-07/04/06 03-05/04/06 27/03-04/04/06 29/03-10/04/06 29/03-03/04/06 27-31/03/06 2 3 4 5 6 7 8 Problems Quality in Education Hypertext .Url Files management Files characteristics Navigation Menu 3/6 1/5 2 1 1 1 1 4/4 3/3 863 431 28/03-03/04/06 28/03-15/04/06 Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 254 .Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices A_VII_6.

1 9 88.4 6.158 324.1/d Appendices 28/03-12/04/06 28/03-12/04/06 27/03-04/04/06 28/03-12/04/06 27/03-15/04/07 29/03-12/04/06 03-12/04/06 27/03-15/04/06 19 days A_VII_9.8 3.8/d 495 138 235 146 108 115 538 6.D.7/d 1 1 2 1 3 1 23 1.2/d 4/4 4/4 (P) 3/3 (P) 1 2/2 (P) 3/3 3/3 2 6 73 3.9 2.493) 30. Participation in e-learning Communities A.689 Average per day 39.9 3. Time to Develop a Sense of Community Time 1 2 3 4 1-2 days 3-4 days 5-7 days Up to 2 weeks Responses 18 2 3 3 Percent 58 6 10 10 Ph.2 29.2 B.4 # Responses 29 2 Percent 100 6.7 2 3 4 5 6 Forum view discussion Forum view forums Forum view forum Forum add post User view all User view Total A_VII_10.9 67.4 2. Thesis Centre for Interactive Systems Engineering LSBU Page 255 . Forums and users view log files LOG FILES (27/03-15/04/2006 – 19 days) Type of View 1 Views 758 61 567 73 59 171 1.9 Percent on total views (2. Active Participation in E-Learning Communities Importance 1 Yes Missing Reasons 1 2 3 4 5 6 7 8 9 10 Knowledge exchange – active learning The community can exist Responsibility for own learning & self-esteem Learning is more interesting Problem solving Achieving learning targets Collaborative learning Win-win situation Expands learning further N/A Missing 8 6 3 1 1 3 5 1 2 1 4 27 20 10 3 3 10 17 3 7 3.Tools and Evaluation Techniques for Collaborative E-Learning Communities 9 10 11 12 13 14 Total Per day Website Plan > left-right click Layout Colour palette What to avoid Hosting 14 0.2 12.8 3.4 22.7 2.

Tools and Evaluation Techniques for Collaborative E-Learning Communities 5 I don’t feel part of the community Missing 5 0 16 0 Appendices A_VII_11.D... Thesis Centre for Interactive Systems Engineering LSBU Page 256 . 2 The worst thing was… 3 More comments Ph. Online course experience Online Course Experience Reasons Responses Learning in e-learning Interesting & useful experience The new tools Interaction and participation The course was in Greek e-learning community Time issues for course organisation New ideas Participation in e-learning research My limitations in the use of ICT Total Unclear instructions Lack of time Technical problems The course was short Fear of unknown Total Exciting experience Life-long learning e-learning is the future New approaches Low implementation More e-learning initiatives More evaluation & feedback Better implementation of the tools Better e-learning quality in Greece More chat use More Moodle usability Total # Responses 6 4 3 12 1 1 1 2 2 1 33 (28/31) 2 7 2 2 1 14 (14/31) 1 2 1 1 1 2 4 4 3 1 2 22 (6/31) Percent 18 12 9 37 3 3 3 6 6 3 100 14 51 14 14 7 100 5 9 5 5 5 17 17 14 5 100 1 The best thing was.

unfortunately. Co-construction Other TOTAL 2 1 30 7 25 376 Ph. bold etc) #References 57 19 32 24 20 11 9 7 5 1 28 23 3 3 9 7 1 7 18 4 3 2 23 15 7 3 3 4 3 25 1 Initiation: Question . therefore Help Yes. same Refer-to-a-name for agreement It is very interesting But. on the contrary Disagreement. Messages Analysis: Collaborative E-Learning Episode I COLLABORATIVE E-LEARNING EPISODE I CeLE parameters #Codes Levels of Abstraction Indicators Info. definition Social cues (nice behaviour. what do you mean alternative Best. Thesis Centre for Interactive Systems Engineering LSBU Page 257 . I agree. finally Emphasis (colour. should. statement. I believe Instruction I know. have worked Image Individual solution Example. think Suggestion I have an idea. something else. thanks. different If. could. this is why.Information 9 185 2 Explanation 4 57 3 3a Agreement Conflict 3 2 17 7 4 Exploration 4 27 5 Evaluation 5 51 6 7 Ideas. would. no meaning I prefer Easiest Mutual solution Overall. might. you are right. further explanation Because. however. we agreed.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices A_VII_12. thus. greetings) Question Url Problem Bullet points I think.D. it is important Comparison Worst.

9 April 2006. CeLE 1 (AIa-1:stanzas1-25) Introduction A I Ask the expert! Sophi Danis answers your questions a How we can create a website in two or more languages? 1. 11:02 PM Re: How we can create a website in two or more languages? by Antoniou Konstantinos .Sunday. Thesis Centre for Interactive Systems Engineering LSBU Page 258 .D. 09:37 AM Re: How we can create a website in two or more languages? by Damian Damianopoulos . 11 Apr 2006. 09:51 PM > Re: How we can create a website in two or more languages? by Mary Frentzou . 09:10 PM > Re: How we can create a website in two or more languages? by Sophi Danis – Friday. 31 March 2006.Thursday. 11:02 PM 30 80 122 22 194 96 544 A I a 6 (1+5) Ph.Friday. 11:02 PM Spyros Papadakis 5 Tue. Spyros Papadakis 5 Tue. 07:26 PM Re: How we can create a website in two or more languages? by Niki Lambropoulos .Tuesday. 11 April 2006. 30 March 2006. 11 Apr 2006. 31 March 2006.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices A_VII_13.

Which techiques are the best in this case. I prefer this way. Spyros There were five responses from AK1. DD1 & MF1. Other web sites offer templates in html format. .D. The answer from Participant AK1 was agreement based on the same interest and provision of more information as the initiation of his message (AIa-2:stanza18).… Participant SD1 agrees with a previous message and refers to the name of the Ph. This is a very interesting question. because I don't have to code or design my webpages from scratch.Are there any templates available? Then the first message ended with participant’s name (social cue) (AIa-1:stanza8). An alternative solution initiated the aim of the next verse by replying explicitly to SP1 (you) (AIa-2:stanza20) If. Response 1 (AK1). SD1. now we cooperate across countires. but you ought to have some experience with Photoshop to edit them. The first was an explanation with an example (AIa-1:stanza14).Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Participant SP1 started with a question as the title of the discussion (AIa-1:stanza3). Then there was a reference to AK1’s experience and evaluation took place by referring to best practice and justification of the evaluation (AIa-1:stanza14). you want to design a webisite can be both Greek and English… Based on the previous Participant’s SP1 message explores the issue further (AIa2:stanza20). Thesis Centre for Interactive Systems Engineering LSBU Page 259 . tehn you will ahve to resort the translation of teh website and use. A conflict stressed a comparison and initiated justification for best practice (AIa1:stanza15). NL1. so I prefer Photoshop file format Participant AK1 closes his message with an emoticon declaring satisfaction (social cue) (AIa-1:stanza15). especially working on EU educational projects. Sometimes international websites contain a cookie that will search and detect the regional settings your computer is using and they will present the version that is compatible with those settings. It was followed by 2 questions (AIa-1:stanzas5-6). Discussion Title: How we can create a website in two or more languages? Participant SP1 elaborated the subject and indicated the aim of the discussion (AIa1:stanza4). . This site (website) has great templates. including the country (regional settings)…. but I'm a little bit lazy and want to get the best results with little effort. A second initiation took place providing further information (AIa-1:stanza15). I want to make a website in Greek and English (or and more) Languages. however. The only way I know is having a template and then editing-translating the content and the buttons. Response 2 (SD1).

that is to group the materials which are needed for a page can have a common characteristic at the beginning. If you use FrontPage you can develop the corresponding pages in parallel.ker.developertutorials. SD1 justifies and evaluates the proposition with a comparison referring to practical implementation (AIa-2:stanza20).gr/> If does not work > (pic available) Response 4 (DD1) DD1 started his message by greeting the person who sent the initial question (social cue) (AIa-3:stanza30). site above). In order to produce an identical site in two languages DD1 compared and evaluated the suggestions (AIa-3:stanza32).europe. He agreed with a previous message and provided his own individual solution (AIa3:stanza31). The researcher tried to get involve in the discussions as minimum as possible. A different approach is the bilingual one.sch. as mentioned by K. You can even perform exactly the same steps in the construction of the pages by toggling from one page to the other clicking on the relevant tab and perform the identical jobs. but in the different languages if the graphics involve speech (cf.sch. or alternatively. templates. A good example is my personal home page (<http://users. This will make your job easire. and use one css template for both. After exploring and explaining his suggestions. you can build your website and translae it.D. DD1 made an evaluation on a topic as regards the design (AIa-3:stanza33).gr/geoker>) Then he went into an exploration of SD1’s suggestions (AIa-3:stanza32). I built the Greek School of London website here <http://dim-lon. SD1 closes the message by providing more information as well as evaluating the given information (AIa-2:stanzas22-24) More on CSS can be found here> http://www. SD1 provides a different solution (AIa-2:stanza20). explaining her solution on a project (AIa-3:stanzas27-28).w3.org/Style/CSS/ this is basic and some more http://www. It is best to give names to your files which facilitate grouping Then he provided a justification of this evaluation (AIa-3:stanza33).… Additionally. You must also use identical graphics. Yes you can make a website in two or more languages.com/css-2/cover. S. it is best to produce the two templates from a single one. He explored and explained the suggestions further (AIa-3:stanza32). Hi.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study person (social cue) (AIa-2:stanza20). Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 260 .html Response 3 (NL1) The Researcher replied by exploring the issue further.

MF1 evaluated his suggestions (AIa-3:stanza34). <corfu. If you want to have both languages on the same page if you use frontpage (at least the older versions) you copy and paste <meta http-equiv="Content-Type" content="text.jpg>. he offered more help if needed (AIa-3:stanzas34). indicating empathic feelings reflected in her writing. I tried to upload it as an attachment but it was impossible. if you see what I mean. Finally.htm> for the HTML page. D. Hi everybody Then she agreed with DD1 referring to his name (social cue). Response 5 (MF1) MF1 adopted DD1’s style of structuring the message. she explored an alternative option to everything said up to now (AIa3:stanza43). MF1 she finished her message with a greeting and providing her name (social cues) (AIa-3:stanzas45-46). The various graphics would have names like <corfu.html.D. MF1 refers to a person from a previous message and ends the message in the same way.mainframe. For instance if I want to produce a page connected with Corfu I would give names like <corfu.bottombanner.jpg>. The he finished his message with a greeting and providing his name (social cues) (AIa-3:stanzas35-36). <corfu. Referring to previous participants’ solutions MF1 explored them by adding her suggestion (AIa-3:stanza41).banner. Thesis Centre for Interactive Systems Engineering LSBU Page 261 . Then she explored DD1’s solution with an example (AIa-3:stanza41). If you use dreamweaver you change the page properties to Greek (English is recognized automatically) Imitating DD1. the participant who sent the previous message (AIa-3:stanza39). Lastly. The message has a greeting as an initiation.sidebarmenu. Frontpage 2003 seems to recognize the languages automatically although I am not pretty sure beccause I haven't worked with it. <corfu.jpg>. MF1 started her message by greeting everyone (social cue) (AIa-3:stanza38).Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study He gave an example of best practice for further exploration (AIa-3:stanza33). charset=iso-8859-7"> you insert it after the <head>.htm>. She tried to provide additional material but her attempt failed due to technical problems (AIa-3:stanza41). I should be glad to help more if you need any more specific details. The way he describes is the easiest. Cheers. cheers M Ph. I agree with D.

Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study A_VII_14.D. Participant SP1: Thought Processes AIM – Q1 Q2 (AK1 ) explanation Example Evaluation (AK1 experience) Justification (reasons) Further Information Conflict to justify Aim Evaluation Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 262 .

5 1 1 All day 13 32. % 1 2.0 0 0.5 2 5.5 5. % 1-5 years Freq.0 Train in Moodle ICT course 29 72.0 12 30.5 Freq.0 14 1 35.0 % 5.5 25.5 8 20.5 6-10 years Freq.0 2 5.0 Postgraduate 4 10.0 3 13 7.0 University 2 5.D.0 % 32.5 7.0 0 0. % 2 5.5 1 2.5 20.0 2.5 0 years Freq.5 1 2.0 22.0 6 15. 2 Very High 8 20.0 1 2.5 11-20 years Freq.0 1-3 4-6 2 1 5.5 22.5 32. 13 10 9 8 2. % 9 22.0 Train in ICT 1 No 3 7.5 6 15.5 % 2.5 Freq.0 9 22.5 1 3 37 Other 1 2.5 10.0 13 32.Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study APPENDIX VIII: CONDITIONS OF WORKING AND LEARNING ONLINE TIME IN EDUCATION AND THE USE OF ICT IN ONLINE EDUCATION Time 1 Employed 2 Use Computers 3 Use LMS 22 55.5 2 5. Thesis Centre for Interactive Systems Engineering LSBU Page 263 .5 7 17.0 Neither 9 9 4 5 High 6 15.5 26 65.5 N/A Freq.5 1 2.5 Very Low 3 7.0 8 20.5 USE & IMPORTANCE OF PROFILES AND FORUMS (LIKERT SCALE) 1 Use Profiles Importance 2 Use Forums Importance 22. % 10 25.5 Ph. % Months Freq.0 15 37. 1 2 Freq.0 12.5 20+ years Freq.0 Time on the Internet Once–twice a day Every 3 days 23 57.5 17 42.5 92.0 2. % 18 45.0 Low 1 2.

Thesis Centre for Interactive Systems Engineering LSBU Page 264 . Post-retreat opinions on e-learners’ participation in the project POST-RETREAT OPINIONS ON E-LEARNERS’ PARTICIPATION (N=31) LEVELS Sub-levels On this community (personal involvement) Responses Collaborative activities Active participation Dialogue Communication Contribution to the project Motivation Provision of help Own participation Target to learn Following instructions New challenges Sharing abilities Participation in activities # 4 1 1 1 1 1 4 2 1 1 1 1 1 On any community Responses Positive thinking Active participation Transformation Continuity # 1 1 1 1 Management Social activities Contribution Learning activities Management Everyone helps Small stone to community building 2 1 Ph.D. Post-retreat opinions on participation POST-RETREAT OPINIONS ON PARTICIPATION IN COMMUNITIES (N=34) LEVELS Sub-levels Responses Willing to participate Collaboration / interactions Sense of belonging / bonding Situated problem solving Evolution Implementation of theory & methodology Common goals / problems Dynamic grouping Visible in public Provision of help Coordination Viability Encouragement Information exchange Quality More interesting learning Collaborative learning Shared work experience New knowledge building Vicarious learning Discovery Common questions Skills acquisition New pedagogical approaches New technologies Functionality # 5 4 4 3 3 2 2 1 1 1 1 1 1 3 3 3 2 2 2 1 1 1 1 1 1 1 Management Social & Learning Activities Knowledge Tools Use A_IX_2.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study APPENDIX IX: THEMATIC ANALYSIS IN THE MAIN STUDY A_IX_1.

Learning Communities Knowledge building Tools Use Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 265 .D.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Knowledge Problem solving Learn something interesting Learn something useful on a personal level Learn something useful on a community level Implementation of new experience Implementation of new knowledge Participation in planning Presentation of own experience/work Opinions Questions Information Propositions Discussions Exploration Comments Advice Suggestions Answers Use new tools 1 1 1 1 1 1 1 3 3 2 2 2 2 1 1 1 1 1 1 Discussions Ideas Opinions 1 1 1 A_IX_3. Post-retreat opinions on new members’ contribution Post-Retreat Opinions on New Members’ Contribution LEVELS Sub-levels Responses Active participation Different problems / heterogeneous group Interaction Enthusiasm Exploration Viability / regenerates the community New perspectives / abilities Willingness to cooperate Willingness to learn Shared interest Training Frees the community / expression Feedback to older members Shared ideas / knowledge Fresh knowledge / ideas / experience Knowledge transfer Quality Questions Propositions Criticism Life-long learning Knowledge building Training in new technologies # 4 4 2 5 1 3 2 1 1 1 3 2 2 5 16 3 1 4 1 1 1 1 2 Management Collaborative e.

D.reliability Provision of directions Multiple perspectives Everybody wins Effective problem solving Users / teachers’ engagement . Post-retreat opinions on other communities POST-RETREAT OPINIONS ON OTHER COMMUNITIES (N=34) LEVELS Responses Knowledge contribution / specialisation . Post-retreat opinions on communities POST-RETREAT OPINIONS ON COMMUNITY EVOLUTION LEVELS Evolution elements (N=38) Responses Increasing interactivity / help Communication outside and after the course Common interests / goals Online communication Affective elements Quick familiarisation Increasing number of participants Social Level Communication – general Collaborative atmosphere E-learners’ participation in planning Sense of belonging Personal messages / experiences / first name New colleagues Photos Visiting others’ web pages Knew nobody initially Participation quality / quantity Information & knowledge exchange Collaborative activities Dialogue development Mobility of ideas Learn to communicate Vicarious learning New skills acquisition Number of created projects Collaborative tools Profile # 12 8 7 6 6 4 4 3 2 2 2 2 1 1 1 1 1 11 8 5 2 1 1 1 1 4 1 New knowledge / curiosity Effective learning Projects development Collaborative learning Willingness to learn Immediate feedback Problem solving New technologies Collaborative tools 12 3 2 2 1 1 1 3 2 What hold the community together? (N=40) Responses Common interests / goals E-tutors Desire for success Communication Enthusiasm / being positive Willingness to collaborate Active participation / atmosphere Subject Mutual help / trust Reliability Immediate success # 22 5 4 4 3 3 3 2 2 1 1 Learning Level Tools A_IX_5. Thesis Centre for Interactive Systems Engineering LSBU Page 266 .pedagogical issues Individual development Collaboration Further development Quality Knowledge dissemination Time saving # 12 1 9 1 2 4 8 1 3 1 2 1 Positive Ph.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study A_IX_4.

Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study Negative Tools Simple interfaces are usable from all users so programmers are not needed Not following pedagogical approaches Need for technical infrastructure 3 2 A_IX_6. Thesis Centre for Interactive Systems Engineering LSBU Page 267 .Learning Communities Knowledge building Tools Use A_IX_7. Post-retreat opinions on learning POST-RETREAT OPINIONS ON LEARNING LEVELS Whom (N=36) Responses E-tutors Other learners Own work More experienced members Forums / chats Emails / newsletters Educational material E-learning platform Internet Tools Search engines Moderators E-learners E-tutors Motivators Leaders Supporters Technical support Active participants Observers # 30 17 3 3 17 16 10 6 4 4 3 7 6 5 5 4 4 2 2 1 How (N=37) Roles (N=22) Ph. Post-retreat opinions on new members’ contribution Post-Retreat Opinions on New Members’ Contribution (N=32) LEVELS Sub-levels Responses Active participation / interaction Enthusiasm Heterogeneous group Training Viability / regenerates the community Frees the community / expression New perspectives / abilities Feedback to older members Willingness to learn & cooperate Shared interest Exploration Fresh knowledge / ideas / experience Shared ideas for knowledge building Questions Propositions Criticism Quality Life-long learning Training in new technologies # 6 5 4 3 3 2 2 2 2 1 1 16 9 4 1 1 1 1 2 Management Collaborative e.D.

login name: nikilambropoulso@gmail.com for username? > E-tutor P32 explained what he had to do (stanza11): you simply write the user name you have from gmail > Then. that was creating a blog (stanza 5): I have created a blog in Pathfinder Then P18 describes her aim (stanza 5): but I would like to comment on Blogger.com password: nbn3vb4325rb43wqbrliqwyfiuQGFLCBAf > E-tutor P52 asked more clarifying questions (stanza 17-18): Which blog do you want to use to comment? Are you allowed to post as administrator? Are the login name and password incorrect? > Then P52 asked for a following up on the process (stanza 18): Keep us updated on the process… > He finished his message with a social cue (stanza 18): pleaaaaaaaaase! Ph.D.g.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study APPENDIX X: COLLABORATIVE E-LEARNING EPISODES (EXAMPLES FROM THE MAIN STUDY) A_X_1. R provided an example based on P24 and P32’s suggestion (stanza 14): e. > P18 initiated the discussion with a statement about the exercise they had. The steps P18 took (stanza 5): I created an account in Google P18 refers to the problem (stanza 5): but every time I log out [the system] it does not recognise the login name and password.) CeLE-III Discussion Title: A different kind of question. Actions to solve the problem (stanza 5): I ended up having 3 accounts Result from actions: and I still have the same problem Question (stanza 5): I am doing something wrong. problem based on her colleagues’ suggestions (CeLE-CIII: Stanzas 4-23). but what? Asking for help: Please help! > P24 asked P18 a question to clarify the problem (stanza 8): Do you add yourname@gmail. Thesis Centre for Interactive Systems Engineering LSBU Page 268 . (Numbers on stanzas occurred automatically in Atlas-ti™. Collaborative e-Learning Episode III CeLE-III was completed in one day (05/03/2007) and described the stages with which P18 found the solution to a registration.

not the user name and this resulted the system to ask for this address as a user name. CeLE-III Locus (CeLE-CIII: Stanzas 4-23) COLLABORATIVE E-LEARNING EPISODE III: LOCUS Section Forum Discussion Replies Thread A different kind of question . Collaborative e-Learning Episode III: Analysis A_X_2. > He found what the problem was (stanza 20): I found what was wrong with it. The problem was solved (stanza 22): Now I can login. 5 March 2007.D.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study > P18 replied with a social cue (stanza 20): Thank you all for your help. 07:07 PM >Re: A different kind of question – P52 Monday. 5 March 2007. Thesis Centre for Interactive Systems Engineering LSBU Page 269 . 5 March 2007.P24 Monday. 5 March 2007. 06:25 PM >> Re: A different kind of question – R Monday. 11:48 PM # Words 43 Total 6 10 6 133 21 C I: Blogs in Education #29 7 (1+6) 38 9 Ph. 5 March 2007. 5 March 2007.P18 Monday. > P52 provided feedback (stanza 23): Bravo mate P18!! P52 sent greetings and thanked for the discussion (stanza 23): be well. 11:12 PM >>> Re: A different kind of question – P52 Monday. 5 March 2007. thank you for the message. 01:59 PM Re: A different kind of question . 04:54 PM > Re: A different kind of question – P32 Monday. He explained what the problem was (stanza 20-22): When I first got the login name and password I wrote the mail address. 10:57 PM >> Re: A different kind of question – P18 Monday.

Evaluation. statement. Exploration. CeLE-III Analysis COLLABORATIVE E-LEARNING EPISODES CeLE parameters Levels of Abstraction #Codes Indicators Info. greetings) Social cue .D. CeLE-III Code Network: Initiation. Thesis Centre for Interactive Systems Engineering LSBU Page 270 . Explanation. Coconstruction TOTAL 1 2 1 2 12 1 5 1 2 21 A_X_4. thanks.help Question Aim Problem Example Suggestion Question Ask for result Result Bravo #References 3 4 1 1 1 2 1 2 3 1 1 1 12 1 Initiation: Question Information 6 2 4 5 6 Explanation Exploration Evaluation Ideas. New ideas & Social cues Ph. aim Social cues (nice behaviour. definition.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study A_X_3.

The discussion started with a message that initiated the aim and the problem to solve in the discussion. Thesis Centre for Interactive Systems Engineering LSBU Page 271 .Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study A_X_5. Persistence for measuring CeLE’s intensity seemed to be on a high level. Collaborative e-Learning Episode IX CeLE-IX was completed in 3 days (29-31/03/2007) and referred to the use of specific e-learning tools in the project creation process. This was verified by P52’s social cues for feedback (bravo). The agreement and conflict stage was missed.D. and managed to locate the problem and found the solution. It appears that P18 was based on these suggestions. (Numbers on stanzas occurred automatically in Atlas-ti™. the discussion was focused on the particular problem. A social sue attached an emotional level to the post (P18 seemed desperate). A_X_6. > P37 initiated the discussion by presenting a proposition worked and agreed with P22 (social cue) (stanzas 2-9): After a conversation with P22 about the way e-learning tools can be implemented in the project method stages. The following list represents the discussion in a linear form as a collaborative problem solving process: Initial post (P18): Statement – Aim – Problem – Question – Social cue (Help) (P24 ) Exploration (P32) Exploration: suggestion (R) Explanation: example (P52) Exploration – Initiation (P18) Idea – Social cue (P18) Evaluation: result (P52) Co-construction . three exploration levels were built on participants’ questions. The discussion was initiated outside the online course between P37 and P22. Internalisation and externalisation thought process CeLE-III is relatively small (133 words) with more information on initiation and social cues.Social cue This discussion is a Collaborative e-Learning Episode for a technical problem. we propose: Ph.) CeLE-IX Discussion Title: Implementing e-learning tools in Project Method. then the participants shared their proposition with the rest of the group (CeLE-GIX: Stanzas 2-84). Three interlocutors located the problem correctly using exploratory questions as well as an example to help P18.

I don’t do the search via a blog. stanzas 10-16). and Evaluation was related to videoconferencing and a wiki. whereas the blog is somehow still – unless the presentation is very short. Thesis Centre for Interactive Systems Engineering LSBU Page 272 . situations etc. He replied the same day and tagged the message as INFORM (29/03/2007. Research – information search and Elaboration was related to a blog. Obviously. > P37 replied the next day (30/03/2007. Presentation was related to videoconferencing and a wiki. In ‘Presentation’ it is not the wiki but the blog which I present my complete project.D. Synthesis – production was related to a wiki. Ph.0 tools: Free subject selection and initial design was related to videoconferencing.] > O2 was an e-tutor.0 and Web 1. Then she explored the topic further and provided alternative points using bullet points (stanzas 22-27): The presentation in the World Wide Web using a wiki was [suggested] for the following reasons: The whole tam will create the context and its contents The whole team will follow the stages before the final product The whole team will evaluate [the project] and will be evaluated And lastly it is recommended for navigation via the links. She explained the reasons for this decision with an example and stressed her opinion by highlighting and bolding the main points of her argument (stanza 21): recording and commenting of the data gathered and focused on events and situations for their elaboration. His points were directly disagreeing with P37 (stanza 12): I disagree with ‘Research – information search and Elaboration’ and with ‘Presentation’ as regards the use of tools. stanzas 17-27). He provided two justifications (stanzas 13-16): In ‘Research – information search and Elaboration’ it is not the blog because I don’t conduct search with a blog or information elaboration but I record events. She agreed (stanza 21): But this is exactly what is going to happen in a blog.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study [The attached ppt image described an example using Web 2.

D. He agreed with P58 referring to his name(social cue) (stanza 40): I also agree with P58 He provided his own opinion as a co-constructive idea (stanza 41): And I believe that chat can be added in the first stage which is the friendliest.0 and Web 1. the most known in the community and its use could boost participants’ self esteem. a wiki and a website. Search and data collection is conducted by the usual methods: Web. and Evaluation was related to videoconferencing and a blog.. He signed off with his name (social cue) (stanza 43): P50 Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 273 .. [The attached ppt image described an example using Web 2. discussions with specialists etc.] > P50 tagged his message as EXPLAIN (30/03/2007.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study > After 4 and half hours.. educational visits. Presentation was related to videoconferencing. Synthesis – production was related to a wiki. Research – information search and Elaboration was related to WWW. stanzas 38-43). if we want to have more multi-media elements. literature review and a blog. In addition. she made a new proposition as continuing co-construction based on previous arguments (30/03/2007.0 tools: Free subject selection and initial design was related to videoconferencing. the presentation can be done in a web-site. stanzas 28-37): [This is] The advanced diagramme after the dialogue with O2: The blog is used only for data recording and commenting as regards events and situations gathered for further development... literature review.

se/frameuk.eupro. After this can be proceeded then videoconferencing can be part of the design if needed. this environment is very attractive for them. this was a great job.D. As for the medium of communication there could be emails and chat.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study > (The following message was not on the logical order for the argument as reply to messages can be interrupted in discussion forums. There. He continued with an explanation and a co-constructive idea: I propose the division of the first stage in two sub-stages: Expression of interest and then on a separate basis the Design. stanzas 59-74).) P58 tagged his message as QUESTION (30/03/2007. stanzas 53-58). it is very difficult to start the first stage with videoconferencing. P37. Have a look on http://partbase. He was defensive on the initial ideal as the expression of an under construction proposition (stanza 64): 2) This diagramme presents only a proposition for the implementation and use of tools. especially web 2. He explored P37’s argument referring to his name (social cue) (stanza 56): P37. He explored the argument further and provided and example based on individual experience (stanza 63): I just need to stress two parameters: 1) Our students are too familiarised with the new technologies. He agreed with P58 referring to his name (social cue) (stanza 56): I also agree with P58. we worked on and not all of existing tools. Ideas to be project or having the potential to be projects can be announced via a database. He agreed with the previous message (stanza 56): As for the rest of the stages I agree. on the contrary. he signed off with his initials (social cue) (stanza 50): P58 > P13 tagged his message as INFORM (30/03/2007. He agreed with the previous messages (stanza 62): I agree with all previous speakers. > P37 tagged his message as EXPLORE and wrote its Greek translation in capital letters (ΔΙΕΡΕΥΝΗΣΗ) (30/03/2007.0. stanzas 48-50). He evaluated P37’s work referring to his name (social cue) (stanza 56): Anyhow. This has been tested fro many years in Comenius and I think recently in etwinning with great success. Lastly he finished the message with his first name (social cue) (stanza 58): P13. They wont face any problem at all with videoconferencing. Ph.htm as well. anybody or a class or a school can write about their interests and using a search engine finding others with common interests. Thesis Centre for Interactive Systems Engineering LSBU Page 274 . He asked a question for all to agree (stanza 48): Do you agree that it is difficult to have videoconferencing on the first stage if we refer to a technologically illiterate audience? He explained his point with an example (stanza 49): Even this online course had to have videoconferencing in the third week… He provided an evaluation by exploring its negative aspects (stanza 49): It is not prohibiting but it can cause problems in the implementation… Lastly.

He further explored the subject by questioning and evaluating previous statements (stanza 69): Fine. could you send it to me via email?? Thank you. develop and complete his work. He finished the message with his full name (social cue) (stanza 74): P14 > P22 replied the next day and tagged his message as EXPLAIN. school. He explored the argument further with bullet points (stanzas 81-84): . and in many occasions. the teachers and the technology allow it. infrastructure. this is just a proposition based on the tools we used in the online course. tools that most of our students can easily use. would it be ‘very erratic’ if we thought for example that as regards an issue in a Blog or wiki our students knowing and implementing brainstorming presented their ideas?For me.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study > P14 kept the EXPLORE tag (30/03/2007.D. and this is the essence of the online course. As you wrote in your magic phrase ‘it is just a proposition’. He thanked (social cue) P37 in advance (stanza 72): Since you created this wonderful presentation. this depends on the situation (students’ age. He evaluated and agreed with P37’s work (stanza 68): Very good job P37. (It appeared that he was defensive in his message. stanzas 75-84). It is accepted and implemented if the situations. we hardly knew them and for sure we had never used them. stanzas 66-74). including skype and msn. level. . how. students’ mood. the issue is to start playing in the ‘field’ they like that is internet and new technologies. it is not that insignificant that a month ago most of us did not know or heard about these tools. Ph. etc.The order we propose with the tools and its implementation it is not.) (31/03/2007. Thesis Centre for Interactive Systems Engineering LSBU Page 275 . He quoted the first part of P37’s message and stressed the fact that it was an initial idea by changing the size of the font as well as making it bold (stanza 79): After a conversation with P22 about the way e-learning tools can be implemented in the project method stages. Each one said [that] it depends on what. neither strict nor unique. better than many of our colleagues.The positive thing is that the previous slide created an argument that lead to a creative dialogue. we propose: He agreed with P37 (social cue) (stanza 80): . the important thing is the ways we are going to implement and taking advantage of them. etc) . He summarised (stanza 69): We learned what Blogs and wikis are and how to use them.On the question which tools somebody will use to design. Then he co-constructed an idea based on previous statements (stanza 70): In other words.At is appears in P37’s first post. and it can’t be in any way.

– P37 – Thursday 29 March 2007.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study A_X_7. 09:31 AM >> Implementing e-learning tools in Project Method – P37 Friday. Internalisation and externalisation thought process The following thought processes represent the discussion in a linear form as a collaborative problem solving process: Initial post (P37): Statement – Aim – Proposition (text and image) (O2 ) Disagreement (P37) Agreement – Explanation – Exploration (P37) New proposition / idea (P50) Explanation .New proposition / idea (P58) Question (P13) Agreement – Initiation – Explanation . CeLE-IX Locus (CeLE-GIX: Stanzas 2-84) COLLABORATIVE E-LEARNING EPISODE III: LOCUS Section Forum II: VC in G ELearning Discussion #4 Replies Thread Implementing e-learning tools in Project Method. 01:48 PM >>> Re [EXPLAIN]: Implementing e-learning # Words Total 7 67 921 60 96 67 41 Ph.New proposition / idea – Social cue (P22) Agreement – Exploration Collaborative E-Learning Episode IX: Analysis A_X_8. 30 March 2007. 08:53 AM Re [INFORM]: Implementing e-learning tools in Project Method – O2 Thursday. Thesis Centre for Interactive Systems Engineering LSBU Page 276 . 10:51 PM > Re: Implementing elearning tools in Project Method – P37 Friday. 29 March 2007.D. 30 March 2007.New proposition / idea – Evaluation – Social cue (P37) Explore (P14) Evaluation – Agreement – Summarise – Exploration .

Thesis Centre for Interactive Systems Engineering LSBU Page 277 . 10:34 PM >>> Re [EXPLAIN]: Implementing e-learning tools in Project Method – P22 Saturday. names) Question Links Aim Quoting previous message Images MESSAGETAGs Example Agree Agree to all previous interlocutors Agreement referring to a name Disagree Disagree with justification MESSAGETAGs Question Suggestion. 30 March 2007. greetings.Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study tools in Project Method – P50 Friday. thanks. 30 March 2007. alternative solution Individual experience Bullet points Evaluation #References 3 2 8(+4)∗ 2 1 1 1 2 2 3 3 1 4 1 3 2 2 4 1 1 5 21/3 1 Initiation: Question Information 7 2 3 3a Explanation Agreement Disagreement (Conflict) 1 3/2 12 4 Exploration 8/2 5 Evaluation 10 Ph. 30 March 2007. 30 March 2007. 31 March 2007. 01:38 PM 40 117 63 128 233 A_X_9. 30 March 2007. CeLE-IX Analysis COLLABORATIVE E-LEARNING EPISODE IX CeLE parameters Levels of Abstraction #Codes Indicators MESSAGETAGs Propositions Social cues (nice behaviour. 08:48 PM >> Re [EXPLORE]: Implementing e-learning tools in Project Method – P37 Friday. 09:40 PM >>> Re [EXPLORE]: Implementing e-learning tools in Project Method – P14 Friday. 04:33 PM Re: [QUESTION] Implementing e-learning tools in Project Method – P58 Friday.D. 02:47 PM > Re [INFORM]: Implementing e-learning tools in Project Method – P13 Friday.

and Agreement/Disagreement CeLE-IX appears to be rich in arguments (921 words). Ph. Both CeLEs diagrammes depict the process inside and outside the individual as a personal monologue and simultaneously a dialogue with the other colearners. CeLE-IX Code Network: Initiation. New ideas.D. New ideas and knowledge construction seemed to be related with all CeLE stages as explanation and evaluation were relatively rich. bold etc) Chapter 6: Main Study 1 4 1 4 2 15 Tagged 9. Other. especially on initial information. it cultivated two series of arguments that were mostly based on explanations and explorations in order to evaluate the previous comments and reach new knowledge constructions. Even though the discussion started as a disagreement on a previous agreed proposition between two participations. Coconstruction Other TOTAL 1 7 15 76/7 ∗The number in parenthesis refer to a reply to a specific person indicated by her name A_X_10.2% 6 7 Ideas. Evaluation. Exploration. Thesis Centre for Interactive Systems Engineering LSBU Page 278 . Explanation.Tools and Evaluation Techniques for Collaborative E-Learning Communities Negative evaluation Best practice & justification Result New idea Summary Emphasis (colour. exploration as well as agreements and disagreements.

Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study It is interesting to note that the interlocutors were different individuals except P37 who appears to be on the medium activity level. The participation in the discussion appeared as follows: Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 279 .D.

Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices Appendix XI: Messages Quantitative Analysis A_XI_1.1(5.5 7.6 698 22.2 Ph.1 3 171 5.9 283.055 116 856 748 2.21/03/2007) Timetable (01 – 28/03/2007) Social networking (01-24/03/2007) Moodle (01-08/03/2007) Problems (01-08/05/2003) Total 5 Projects Archive (01-07/03/2007) 59 45 6 3 47 160 6 6 61 8 3 72 15 2 4 9 30 8 1 9 4 4 8 # Initiations (add forum topic) Analysis 49 15 5 2 26 97 1 1 42 3 1 46 10 2 2 7 21 2 1 3 1 2 3 Replies (add post) Total 182 112 25 5 101 425 5 5 156 7 2 165 27 4 16 25 72 15 10 25 2 4 6 Discussion Depth 231 127 30 7 127 522 6 6 198 10 3 211 37 6 18 31 92 12 16 28 3 6 9 Participants 49 7 1 2 22 81 1 1 43 3 1 47 9 2 2 7 20 0 0 0 1 2 3 Participants 142 (40) 80 (32) 16(9) 3(2) 70(31) 311(114) 4(1) 4(1) 127(29) 7(0) 2(0) 136(29) 24(3) 4(0) 13(3) 22(3) 63(9) 7(8) 7(3) 14(11) 3(0) 2(2) 5(2) Richness of text (# words) Total 9.8 152 4.97 01/03 – 31/03/2007 – 31 days 95 participants 285 9.1 26.113 15.6 868 28 9.3) 5.654 354 260 4.110 498 61 7. Thesis Centre for Interactive Systems Engineering LSBU Page 280 .9 1.669 1.775 303 94 397 236 232 468 B Project Method Total Blogs Total 1 Blogs (01-31/03/2007) Tools (03-12/03/2007) Blog & HTML (05-12/03/2007) 3 Problems (12-23/03/2007) Design (17-18/03/2007) Practicality (13-18/03/2007) Groups (18-24/03/2007) C D Wikis Total 4 E Videoconferencing Total 2 Technical Problems (16-22/03/07) Groups (18-23/03/07) Project ideas (01-06/03/2007) Other (01-18/03/2007) F Internet Cafe Total 2 Overall Percentage per day Percentage 16 0.D.5 1.3 533(166) 17. Messages Quantitative Analysis in Moodle@GSN Post Message Analysis in Moodle@GSN (01/03 – 31/03/2007 – 31 days) Section Forums Total A Introduction Introduction (01.907 867.456 142 142 7.075 1.

1 * My initiations were included as some participants’ messages were produced from these initiations. The first column describes the online course section. Thesis Centre for Interactive Systems Engineering LSBU Page 281 . depth of discussions.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices per participant Messages Density 2a/N(N-1) 2X868/95(95-1)=0.2 0.2 1.2) 1. Ph.941 G VC in E-Learning Overall 7 9 16 Percentage per day Percentage per participant Messages Density 0.988. Messages Quantitative Analysis in the Research Pool Research Pool (25 – 31/03/2007 – 6 days) Section Forums News (28-30/03/07) VC in E-learning (28/03-01/04/07) 2 # Initiations (add forum) Total Analysis 2 8 10 # Replies (add post) Total 6 70 76 Participants 5 4 9 Participants 5(1) 70(0) 75(1) Overall (Analysis+ Participants) 7 78 85 Richness of text (# words) Total 999 8. found in logs as “add posts”. and the actual replies.19 A_XI_2.) These tables describe the results in Moodle@GSN and the research pool. the messages for analysis as well as participants’ only messages.6 2X85/42(42-1)=0. the second column the forums and the dates these forums were introduced and completed.8 0. (My posts are in parentheses.2 1. From these messages. The last two columns show the overall number of messages provided the initiations for analysis including my messages.8 15(0.4 2 0.79 17 2 1. richness of text.942 9.D. the sub-columns present the total number of messages. and the richness of text. and the participants’ messages.2 15.4 25 – 31/03/2007 – 5 days 42 participants 2a/N(N-1) 3. and messages density.2 236. The tables present the first three quantitative variables. Then the replies are divided into initiations found in logs as “add forum”.

5 15 17.9 1.5 35 20 27.9 3.9 1.3 0.9 0.5 7.8 3.725 3.2 1.5 2.5 35 32.5 5 40 32.5 35 35 30 Frequency 10 11 8 8 5 4 1 1 8 9 7 7 8 8 5 5 3 % 25 27.5 17.5 20 22.2 3.2 2.5 Mean 3.D.3 1.3 3.5 7.5 Frequency 17 17 14 14 7 6 1 2 16 13 10 10 17 17 14 14 12 % 42.5 27. PEDAGOGICAL USABILITY – UTILITY RESULTS Frequency Graphs & Avatars MessageTag VIT Nodes VIT Centrality Graphs & Avatars MessageTag VIT Nodes VIT Centrality Graphs & Avatars MessageTag VIT Nodes VIT Centrality Graphs & Avatars MessageTag VIT Nodes VIT Centrality Graphs & Avatars 2 1 4 4 7 14 16 1 1 4 4 1 1 7 7 6 % 5 2.7 3.8 3.4 3.5 35 40 40 40 32.5 12.5 42.5 35 40 2.5 42.5 10 10 2.4 2.5 7.625 1 Instructions 2 Frequency of use Alignment with educational goals 3 4 5 Support collaborative e-learning Learnability Ph.5 25 Frequency 8 8 8 7 7 14 13 9 14 16 16 16 13 13 11 11 9 % 20 20 20 17.7 3.5 2.5 2.2 1.5 10 10 17.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices APPENDIX XII: MAIN STUDY DATA & REPORTS A_XII_1.2 1.5 7.3 1.1 0. Thesis Centre for Interactive Systems Engineering LSBU Page 282 .5 17.5 32.2 1 1.5 15 Frequency 3 3 6 7 14 8 11 12 1 1 3 3 1 1 3 3 10 % 7.5 17.5 17.5 2.5 2.1 2 3.5 35 35 17.5 2.8 0.5 St.5 15 2.5 17.5 20 20 12.5 22.5 10 2.5 20 20 12.7 2.3 1.475 3.5 30 2. 1 1 1.5 7.5 22.5 7.5 2.7 3.3 3.5 25 25 42.8 2.1 N/A 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2.4 3.D.5 27.

5 5 5 10 7.4 3 3 3.5 7.5 27.38 3.2 2.5 7.5 12.7 3.8 0.4 3.75 3.5 15 15 11 10 11 2 2 4 3 3 27.5 27.5 Ph.5 45 25 7 5 12.5 30 1 3 3 12 13 10 10 6 2.5 37.2 1.675 2.2 1.5 12.5 10 9 8 10 8 9 9 20 25 22.5 40 27.5 25 25 25 8 10 10 14 17.5 5 2.5 13 11 11 10 50 32.5 5 12.5 25 27.5 12.2 3.3 1.47 3.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices 6 Accessibility 7 Originality 8 Motivation to participate 9 10 Information overload Functionality MessageTag VIT Nodes VIT Centrality Graphs & Avatars MessageTag VIT Nodes VIT Centrality Graphs & Avatars MessageTag VIT Nodes VIT Centrality Graphs & Avatars MessageTag VIT Nodes VIT Centrality Graphs & Avatars MessageTag VIT Nodes VIT Centrality Graphs & Avatars MessageTag VIT Nodes 11 6 6 1 1 6 6 1 2 4 4 2 1 6 6 15 16 14 14 1 1 5 27.2 1.2 1.5 30 30 16 11 11 4 37.5 2.5 20 25 25 4 5 5 19 10 10 12.5 16 10 11 15 25 40 25 27.1 1.5 18 10 47.5 20 2.5 10 12.65 3.1 1.5 2 1 1 5 2.5 2.5 7.35 3.5 15 15 2.4 3.2 1 1 1.5 12.2 1.2 1 1 1.5 2.5 14 15 35 35 37.2 2.5 6 12 12 15 16 11 12 10 15 30 30 37.5 10 6 7 3 25 25 15 17.5 2.5 27.5 5 10 10 4 5 5 3 7.5 7.9 3.5 40 27.8 1.2 2.175 3.5 17.5 20 25 20 22.9 1 1.9 4 3.5 4 9 8 10 5 2.1 1 0 0 0 0 0 0 0 1 1 1 0 1 0 0 0 0 0 0 0 0 0 3.5 25 25 15 10 22.4 3.5 40 35 35 10 10 10 1 27.D. Thesis Centre for Interactive Systems Engineering LSBU Page 283 .1 2.5 10 9 12 12 7 25 22.1 1.2 1.375 2.3 0.5 22.5 30 32.4 3.5 15 15 3 5 4 11 7.1 0.5 12.5 3.2 3.

3 3.5 10 7.5 2.65 3.5 15 15 8 9 6 6 20 22.5 37.55 0.5 5 6 7 7 6 7 4 4 15 17.1 1.9 1 1.5 5 12.5 12.5 42.5 2.5 25 25 32.1 3.2 1.5 15 15 5 10 15 15 47.5 32.5 3.5 2.5 35 32.5 6 15 2 1 4 4 5 2 8 7 14 12 13 12 35 30 32.5 37.5 15 15 3.5 2.5 22.5 22.5 7.5 10 10 35 40 32.5 7.3 0.7 3.8 0.5 7.2 1 0 0 0 3.5 1 1 5 5 1 1 5 6 19 22 18 19 2 4 6 6 1 1 6 6 2.2 0.5 2.7 2 1.65 3 2.3 0.2 1.5 42.1 0 2 2 10 11 6 6 5 5 1 1 2 3 3 4 2.2 3.5 Ph.5 10 15 17 15 32.8 1.5 17.5 12.1 3.4 3.2 1.5 8 20 13 17 9 9 14 13 10 10 3 3 3 3 16 16 12 12 16 12 8 9 40 30 20 22.8 3.5 37.5 40 40 30 30 7.9 25 27.3875 3 1.0875 1 0 0 3.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices 11 VD: Graphics 12 VD: Attractiveness 13 Tool failure 14 Fast response 15 Overall satisfaction VIT Centrality Graphs & Avatars MessageTag VIT Nodes VIT Centrality Graphs & Avatars MessageTag VIT Nodes VIT Centrality Graphs & Avatars MessageTag VIT Nodes VIT Centrality Graphs & Avatars MessageTag VIT Nodes VIT Centrality Graphs & Avatars MessageTag VIT Nodes VIT Centrality 5 12.5 7.2 0 0 0 0 3.5 15 2.5 10 10 3.8 1.5 10 10 16 40 18 15 15 15 14 16 13 13 4 3 4 4 12 9 15 15 13 5 7.1 0 0 0 0 1.5 37.5 30 22.5 55 45 47.5 37.2 0.5 5 20 17.5 45 37.5 3.8 3.1 1.2 0 0 0 0 3.6 3.5 5 2.5 12.D.9 1 1. Thesis Centre for Interactive Systems Engineering LSBU Page 284 .9 0.9 1.1 1.85 12.5 15 17.5 30 12.375 1.9 1.5 37.

Thesis Centre for Interactive Systems Engineering LSBU Page 285 .D. CORRELATIONS IN SPSS Ph.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices A_XII_2.

792 0.729 0.784 0.740 0.737 0.737 0.748 0.788 0.977 0.784 0.961 0.763 0.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices A_XII_3.961 0.729 0.777 0.782 0.733 0.712 0.771 0.710 Pearson’s’ r Ph.752 0.D.718 0.721 0.977 0. MOST IMPORTANT CORRELATIONS IN THE HIERARCHICAL CLUSTERING EXPLORER USABILITY – UTILITY CORRELATIONS Rank X axis Y axis Positive Correlations 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 VD: Attractiveness VIT Centrality Accessibility VIT Centrality Tool failure VIT Centrality Functionality VIT Centrality Instructions VIT Centrality Satisfaction VIT Centrality Info overload MessageTag Failure MessageTag Motivation VIT Centrality Accessibility MessageTag Education goals MessageTag Satisfaction MessageTag Education goals VIT Nodes Education goals VIT Centrality Satisfaction VIT Centrality Satisfaction VIT Centrality Satisfaction VIT Nodes Support CeL VIT Nodes Support CeL VIT Centrality Satisfaction VIT Centrality Satisfaction VIT Centrality Education goals Graphs/avatars Satisfaction VIT Centrality Satisfaction VIT Centrality Originality VIT Nodes Accessibility VIT Centrality Originality VIT Nodes Originality VIT Nodes Satisfaction VIT Nodes Fast response MessageTag Satisfaction VIT Nodes Satisfaction VIT Nodes Originality VIT Nodes Learnability VIT Nodes Learnability VIT Centrality Accessibility VIT Nodes Functionality VIT Centrality Functionality VIT Centrality Accessibility VIT Centrality Functionality VIT Centrality Satisfaction VIT Nodes VD: Graphics VIT Nodes VD: Graphics VIT Centrality Satisfaction Graphs/avatars Support CeL VIT Nodes Support CeL Centrality Satisfaction VIT Nodes Satisfaction VIT Nodes Motivation MessageTag VD: Attractiveness VIT Nodes Accessibility VIT Nodes Tool failure VIT Nodes Functionality VIT Nodes Instructions VIT Nodes Satisfaction VIT Nodes Info overload Graphs/avatars Failure Graphs/avatars Motivation VIT Nodes Accessibility Graphs/avatars Education goals Graphs/avatars Functionality VIT Nodes Originality VIT Nodes Originality VIT Nodes Functionality VIT Centrality Originality VIT Nodes Functionality VIT Nodes Motivation VIT Nodes Motivation VIT Nodes Support CeL VIT Nodes Support CeL VIT Centrality Functionality Graphs/avatars VD: Graphics VIT Nodes VD: Graphics VIT Centrality Functionality VIT Nodes Instructions VIT Centrality VD: Graphics VIT Nodes VD: Graphics VIT Centrality Functionality VIT Centrality Fast response Graphs/avatars Support CeL VIT Nodes Support CeL VIT Centrality Functionality VIT Centrality Learnability Graphs/avatars Learnability Graphs/avatars Instructions VIT Centrality VD: Graphics VIT Nodes VD: Graphics VIT Centrality Instructions VIT Nodes VD: Attractiveness VIT Centrality Originality VIT Nodes VD: Attractiveness VIT Centrality VD: Attractiveness VIT Centrality Education goals Graphs/avatars Originality VIT Nodes Originality VIT Nodes VD: Graphics VIT Nodes VD: Graphics VIT Centrality Motivation Graphs/avatars 0.722 0.930 0.810 0.810 0.751 0.918 0.777 0.710 0. Thesis Centre for Interactive Systems Engineering LSBU Page 286 .712 0.718 0.752 0.723 0.883 0.955 0.843 0.788 0.748 0.981 0.946 0.751 0.944 0.801 0.712 0.801 0.

641 -0.642 -0.676 -0.719 -0.D.632 -0.603 -0.620 -0.605 -0.629 -0.621 -0.657 -0.612 -0.687 -0.628 -0.619 -0.688 -0.747 -0. Thesis Centre for Interactive Systems Engineering LSBU Page 287 .605 -0.630 -0.689 -0.607 -0.638 -0.633 -0.631 -0.729 -0.629 -0.712 -0.676 -0.620 -0.672 -0.665 -0.624 -0.690 -0.638 -0.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 Negative Correlations Satisfaction VIT Nodes Failure VIT Centrality Learnability MessageTag VD: Graphics VIT Nodes Accessibility VIT Nodes VD: Graphics MessageTag Support CeL Graphs/avatars Functionality VIT Nodes Satisfaction VIT Nodes Info overload MessageTag Satisfaction VIT Nodes Info overload Graphs/avatars Motivation VIT Centrality Info overload MessageTag Satisfaction VIT Nodes Failure VIT Nodes Motivation VIT Nodes VD: Graphics MessageTag Satisfaction VIT Centrality Accessibility Graphs/avatars Instructions VIT Centrality Frequency Graphs/avatars Originality VIT Nodes VD: Graphics MessageTag Motivation VIT Nodes Info overload MessageTag Satisfaction Graphs/avatars Functionality VIT Centrality Education goals MessageTag Accessibility VIT Nodes Education goals MessageTag Accessibility VIT Centrality Education goals Graphs/avatars Instructions VIT Nodes Satisfaction VIT Nodes Accessibility Graphs/avatars Motivation VIT Centrality Info overload VIT Nodes Motivation VIT Centrality Info overload VIT Centrality Satisfaction VIT Centrality Info overload VIT Nodes Satisfaction VIT Centrality Info overload VIT Centrality Education goals VIT Nodes Info overload MessageTag Education goals VIT Centrality Info overload MessageTag Support CeL Graphs/avatars Functionality VIT Centrality Motivation VIT Nodes Info overload VIT Nodes Motivation VIT Nodes Info overload VIT Centrality Education goals Graphs/avatars Instructions VIT Centrality Education goals Graphs/avatars Accessibility VIT Centrality Support CeL VIT Nodes VD: Graphics MessageTag Support CeL VIT Centrality VD: Graphics MessageTag Instructions VIT Centrality VD: Graphics MessageTag Education goals Graphs/avatars Accessibility VIT Nodes Learnability VIT Nodes VD: Attractiveness VIT Centrality Learnability VIT Centrality VD: Attractiveness VIT Centrality Originality VIT Nodes Info overload MessageTag Functionality MessageTag VD: Attractiveness VIT Nodes Support CeL VIT Nodes Fast response Graphs/avatars Support CeL VIT Centrality Fast response Graphs/avatars Satisfaction VIT Nodes Failure Graphs/avatars Instructions VIT Centrality VD: Graphics MessageTag Info overload VIT Nodes VD: Attractiveness VIT Nodes Info overload VIT Centrality VD: Attractiveness VIT Nodes Accessibility VIT Centrality Learnability VIT Nodes Accessibility VIT Centrality Learnability VIT Centrality Education goals VIT Nodes Info overload Graphs/avatars Education goals VIT Centrality Info overload Graphs/avatars Accessibility VIT Centrality VD: Graphics MessageTag Info overload MessageTag VD: Attractiveness VIT Nodes Originality VIT Nodes Info overload Graphs/avatars Learnability VIT Nodes VD: Attractiveness VIT Nodes Learnability VIT Centrality VD: Attractiveness VIT Nodes Info overload MessageTag VD: Attractiveness VIT Centrality Accessibility VIT Nodes Learnability VIT Nodes Accessibility VIT Nodes Learnability VIT Centrality Info overload VIT Nodes VD: Attractiveness VIT Centrality Info overload VIT Centrality VD: Attractiveness VIT Centrality -0.729 -0.611 -0.653 -0.600 -0.630 -0.628 -0.608 -0.712 -0.688 -0.632 -0.657 -0.747 Ph.621 -0.603 -0.698 -0.657 -0.624 -0.653 -0.636 -0.609 -0.628 -0.687 -0.655 -0.

THE BEST THING IN THE PROJECT THE BEST THING IN THE PROJECT WAS… (N=39) LEVELS Sub-levels E-learners Responses Common interests / goals Communication / participation / work together Sense of belonging to something greater Exchange of opinions Implementation of acquired knowledge Presentation of own experience / work Educational material Professional training Feedback New interests New ideas / knowledge New skills New technologies Working with new technologies Real-time evaluation Excitement Successful contribution # 6 11 7 1 2 1 1 2 1 1 1 1 9 1 2 1 1 Community Management Management Learning Knowledge building Tools Study Use Hawthorn effect Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 288 .Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices A_XII_4.D.

and techniques Recommendations for e-learning engineers • • • • • • • • • • • • • • Open communication channels Get involved in multidisciplinary teams Acknowledge the post hoc nature of design in evolving situations Design for learners as users and users as learners Design for pedagogical usability and utility Design tools relevant to professional practice Offer pragmatic solutions Use learner-centred design sensitive to its context on a macro and micro level Evaluate design in real-time as circumstances evolve Involve all e-learning participants in design Acquire opinions from different levels of practitioners’ expertise Integrate design. Thesis Centre for Interactive Systems Engineering LSBU Page 289 . just-in-time. and reactive decision making Use time-series design for formative and summative evaluation Ensure scalability of methods. use and evaluation in design Make visible and support social and temporal structures of social interaction Design for collaborative e-learning tools and activities to o o o o • observe and analyse human-human and human-computer interactions support persistence and depth of discussions support increasing participation provide detailed and accurate reports for each e-learner Modify the tools in this study Ph. tools.D.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices APPENDIX XIII: RECOMMENDATIONS Recommendations for researchers • • • • • • • • Open communication channels Work on multidisciplinary frameworks to acquire coherent views of the research context Use ethnotechnology to inform design Target human-human and human-computer interaction in e-research Use social network analysis to triangulate results from qualitative and quantitative data analysis Use real-time evaluation for proactive.

D. just-in-time. and reactive in decision making Shorten decision making periods using new technologies Enhance participatory decision-making Ph. Thesis Centre for Interactive Systems Engineering LSBU Page 290 .Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices o Calculation of participation levels based on the highest poster’s responses o o • Enable quoting Semantic and temporal arrangement of the “Reply” function in discussion forums Use different methods for pedagogical usability testing specifically designed for new tools Recommendations for e-learning practitioners • Plan for E-learning o o o o o o Acknowledge the strategic use of collaborative e-learning Facilitate and moderate progressive discourse Tackle performance problems associated with personality traits Obtain a balance on personalised and social learning Support self-organised e-learning Plan and routinise collaborative activities as a process of increasing participation o • Use available technologies effectively Plan for E-Learning Communities o o o o o o o o o o Open communication channels Define stakeholders’ intentions and goals Create highly targeted and interactive courses to engage e-learners Use community assessment Enhance social networking to create learning opportunities Train e-learners how to work together Work towards the different levels of participation Allow time for getting-to-know activities Focus on development and maintenance of empathy and trust Keep the enthusiasm going Recommendations for the Greek educational authorities • Open communication channels o Create an inclusive environment to support cooperation between national institutions and teachers o o o Be proactive.

Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices o • Create groups for research and development beyond administration Create policies to support e-learning quality o o o Create a central organisation to ensure quality in e-learning Train e-tutors and e-learners Provide quality short training cycles relevant to the Greek teachers’ profession on a life-long learning basis o Provide opportunities for Ministry chairs to learn firsthand about the benefits of e-learning and the changing nature of 21st century learning o Incorporate e-learning in institutions to support employees’ life-long learning o Create opportunities for e-learning experts and e-learners Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 291 .

Thesis Centre for Interactive Systems Engineering LSBU Page 292 .D.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices Glossary Ph.

Thesis Centre for Interactive Systems Engineering LSBU Page 293 . knowledge.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices Baseline: The observation of behaviour prior to any treatment designed to alter behaviour. Heuristics: A form of usability inspection where usability specialists judge whether each element of a user interface follows a list of established evaluation variables. synchronous activity as a result of a continued attempt to construct and maintain a shared conception of a problem. Instructional Engineering: The systematic process of activities to solve an online instructional problem anchored in Human-Computer Interaction. assumptions or repairing misunderstandings in a group. memory. knowledge and reasoning are combined to yield insights into the thoughts and feelings of others. Decentralisation: The ability to take into account viewpoints of a given situation Empathy: A complex psychological inference in which observation. Community: A group of people who consciously share a sense of belonging anchored in common interests and enhanced by social interactions. Collaboration: A coordinated. Collaborative e-Learning Communities (CeLC): Social aggregations that emerge in online courses when enough people carry on progressive dialogues for the purpose of learning. planning and design of what happens when humans interact with computers. Collaborative E-Learning Episode (CeLE): A communicative discussion episode with a starting point. Ph. Human Computer Interaction (HCI): The study. E-Research: The research in online environments. Instructional Design: The systematic process of activities to solve an instructional problem with the aid of technologies. a transition and an end point that indicates a collaborative elearning cycle. Grounding: Interactions intended to create mutual understanding. Ethnotechnology: An ethnographic field for studying design in real environments. An online community is a community where social interactions are facilitated by information and communication technologies. Ethnography: The branch of anthropology that provides scientific description of individual human societies. Informal learning: The unofficial and unstructured way of learning. Collaborative Learning: The type of learning that takes place when learners work in groups on the same task using progressive dialogue for co-creativity. Communities of Practice (CoP): Groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly.D. beliefs.

Legitimate Peripheral Participation (LPP): The process of social learning that occurs in Communities of Practice containing different levels en route for members’ engagement and practice. Mirror Neurons: The premotor neurons which fire both when an animal acts and when the animal observes the same action performed by another (especially of the same species) animal. Sense of e-Learning Community Index (SeLCI): The index to measure the sense of belonging in an e-learning community. Centrality: Measures who is central (powerful) or isolated in networks. SNA glossary follows: Betweennes: The measurement of the node’s prominence according to its position in the network.Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices Learner-Centred Design (LCD): The design that considers the learner/user as the center of instructional design. Social Network Analysis (SNA): The mapping and measuring of relationships and flows between people. Learning Management System (LMS): A software package designed to manage learning interventions in technology enhanced learning. Social Computing: The incorporation of sociological understandings into interface design aiming at building systems that fit more easily into the ways we communicate and work. Online Learning: A planned teaching and learning experience that uses a wide spectrum of technologies to reach learners at a distance. It supports that cognitive development is based on the negotiation of meaning that originates from individuals’ actual relationships. Online Communities: Online social aggregations that emerge when enough people carry on those public discussions long enough to form relationships. Socio-cultural learning: A theoretical framework which emphasises the role of social interaction in the development of cognition. organizations. groups. computers or other information/knowledge processing entities. Lurking (Passive Participation): The activity of one of the "silent majority" in an electronic forum that involves posting occasionally or not at all but reading the group's postings.D. Ph. Clique: A subgroup where actors are connected to each other as a maximal complete subgraph of three or more nodes (members) adjacent to each other. Closeness: The measurement of the distance between one node and other node in a network as the number of other nodes divided by the sum of all distances between the node and all others. Thesis Centre for Interactive Systems Engineering LSBU Page 294 . Pedagogical Usability: A quality attribute that assesses how easy learner/user interfaces are to use with a purpose of learning.

User-Centered Design (UCD): The design that considers the user into the center of software design. Usability: A quality attribute that assesses how easy user interfaces are to use. Thesis Centre for Interactive Systems Engineering LSBU Page 295 . network where the direction of the Ego Network: Consists of a focal node and a set of alter nodes adjacent to or from the focal node. Ph. the underlying theory or to make forecasts. participants’ preferences (reciprocity). Intensity: The participation levels and persistence in online learning. Structural Equivalence: The role-set structure of a network based on the similarity of tie-profiles among its nodes and is computed by the Euclidean distance of tie-value from and to all other nodes. Out-degree Centrality: The number of lines that are incident from a node. Nodes: The actors or subjects of study. Isolates: Nodes whose degree equals 0. Time-Short Series Design: A sequence of data points spaced and measured at short time intervals using methods to understand such time series. retaining and replicating novel behavior executed by others. Density: The number of actual ties in a network compared to the total amount of ties that the network can theoretically support. Thematic analysis: The process for encoding qualitative information in order to relate the data to prior ideas.D. Reciprocity: The number of ties that are involved in reciprocal relations relative to the total number of actual ties. Vicarious Learning: It is the type of learning that occurs as a function of observing. In-degree centrality: The number of lines that are incident to a node. subgroups (cliques).Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices Cohesion: The representation of interactions’ weight (density). Global centrality: The communication nodes between the members of a network characterised by direction and strength. and similar behaviour (structural equivalence). Equivalence: A description of the actors who have similar patterns of relations to others in the network and exhibit similar communication behaviour. Degree Centrality: A directed communication is important. Socio-Technical Design (STD): The design that is influenced by an organisations’ social structure.

Sign up to vote on this title
UsefulNot useful

Master Your Semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master Your Semester with a Special Offer from Scribd & The New York Times

Cancel anytime.