Supporting Small-Group Learning Using Multiple Web 2.0 Tools: A Case Study in The Higher Education Context

ACCEPTED MANUSCRIPT
Jari Laru, Piia Nykki, Sanna Jrvel, Supporting small-group learning using multiple Web 2.0 tools: A case study in the higher education context, The Internet and Higher Education, Available online 28 August 2011, ISSN 1096-7516, 10.1016/j.iheduc.2011.08.004. (http://www.sciencedirect.com/science/article/pii/S1096751611000546) Keywords: Case study; Cloud-based social software; Explorative analysis; Higher education; Small-group collaboration
Supporting small-group learning using multiple Web 2.0 tools: A case study in the higher education context
JARI LARU* Faculty of Education, University of Oulu, Snellmania, Oulu, P.O. Box 2000, 90014 University of Oulu, Finland jari.laru@oulu.fi +358-40-5118478 http://www.claimid.com/jarilaru PIIA NYKKI, SANNA JRVEL Faculty of Education, University of Oulu, Snellmania, Oulu, P.O. Box 2000, 90014 University of Oulu, Finland Piia.naykki@oulu.fi, sanna.jarvela@oulu.fi Abstract: In this single-case study, small groups of learners were supported by use of multiple social software tools and face-to-face activities in the context of higher education. The aim of the study was to explore how designed learning activities contribute to students learning outcomes by studying probabilistic dependencies between the variables. Explorative Bayesian classification analysis revealed that the best predictors of good learning outcomes were wikirelated activities. According to the Bayesian dependency model, students who were active in conceptualizing issues by taking photos were also active blog reflectors and collaborative knowledge builders in their group. In general, the results indicated that interaction between individual and collective actions likely increased individual knowledge acquisition during the course. Keywords: Case study, Cloud-based social software, Explorative analysis, Higher education, Small-group collaboration
ACCEPTED MANUSCRIPT
1. Introduction Technology is one of the most significant mechanisms currently transforming the learning process. Over the course of history, a range of artefacts has been produced (e.g., invention of the chart) that has modified the way in which people learn in various situated practices (Pea, 1993). In particular, representational tools such as calculators and mind maps have dramatically changed our daily practices in many spheres of life (Slj, 2003). New technologies provide opportunities for creating learning environments that extend the possibilities of old technologies (e.g., books, blackboards, television, radio) and offer new prospects for multiple social interactions (Bransford, Brown, & Cocking, 1999). In recent years, a plethora of digital and networking tools has been established on the Internet. These digital applicationswhich enable interaction, collaboration and sharing among usersare frequently referred to as Web 2.0 (Birdsall, 2007) or social software tools (Kesim & Agaoglu, 2007). These applications are further assumed to be a step change in the evolution of Internet technology in higher education (Wheeler, 2009), which has evolved from being primarily used to distribute course materials, communicate and evaluate to being used to enhance educational processes that support collaborative learning and knowledge building (Collins & Halverson, 2010; Cress & Kimmerle, 2008; Schroeder, Minocha, & Schneider, 2010). Much has been written on the benefits of blogs (Halic, Lee, Paulus, & Spence, 2010;
ACCEPTED MANUSCRIPT
Hemmi, Bayne, & Land, 2009; Wheeler, 2009; Xie, Ke, & Sharma, 2008),wikis (Cress & Kimmerle, 2008; Hemmi et al., 2009; Wheeler, 2009) and social networking sites (Arnold & Paulus, 2010) in education. However, very little formal research focusing on the integration of multiple social software tools in higher education pedagogy has been published as of yet (Uzunboylu, Bicen, & Cavus, 2011; Wheeler, 2009). Crook (2008) and Meyer (2010) have argued a need for more empirical research on the educational use of Web 2.0, its adoption and its impact on higher education. In this single-case study, small groups of learners were supported by multiple social software tools and face-to-face activities in the context of higher education. The purpose of this study was to explore how designed learning activities contribute to students learning outcomes by studying the probabilistic dependencies between the variables. 2. Theoretical background 2.1. Social software to support individual reflection One activity that can promote the use of blogs in education is selfreflective practise (Sharma & Fiedler, 2007; Xie et al., 2008). Self-reflecting is a central concept in metacognitive learning in which students are encouraged to construct explanations, pose questions and provide further information to each other (Cohen & Scardamalia, 1998). While constructing explanations, the students become aware of their thought processes, gaps in knowledge and lack of understanding (Webb, 1989). Through contributing their ideas and making their thought processes visible, the students are able to reflect on their
ACCEPTED MANUSCRIPT
cognitive processes and discuss with others what they do or do not know and understand. Previous research (Xie et al., 2008) has shown that reflection is effortful action that requires external support in order to engage students for extended periods of time. For example, Xie et al. (2008) have introduced various strategies for encouraging reflection, and they have concluded that blogwriting activities, journaling and peer feedback are all appropriate reflection strategies. Weblogs are popular journaling tools that offer students a means of externalising their reasoning and reflecting on their experiences (Xie et al., 2008). Hence, Weblogs can be used as learning logs that capture the cumulative history of a learning project in action and record personally meaningful material that can foster and facilitate reflective practices such as conversations with oneself and others (Halic et al., 2010; Hemmi et al., 2009; Sharma & Fiedler, 2007; Xie et al., 2008). The main idea of blogging is similar to that of network discussions: The students make their thinking visible and externalize their thinking by periodically posting journal entries to their personal or collaborative blogs, allowing other learners to comment on their learning blogs (Xie et al., 2008). Second, in addition to self-reflective blog writing, peer feedback can provide a different perspective and help students to assimilate and accommodate their thinking. Blogs can facilitate reflective thinking, because people can easily access different points of view by looking at peers blogs or
ACCEPTED MANUSCRIPT
comments (Xie et al., 2008). Furthermore, Really Simple Syndication (RSS) offers novel ways to increase access to different points of view by enabling various contributions to be aggregated, even though they may have originated from diverse sources (e.g., blogs, file-sharing tools, and wikis) (Crook, 2008; Lee, Miller, & Newnham, 2008). 2.2. Social software to support collaborative learning The potential of collaborative learning groups has been strongly supported by the literature, which emphasizes students possibilities for constructing knowledge and experiencing shared understanding through these groups (Dillenbourg, Baker, Blaye, & Malley, 1996; Dillenbourg, 1999). Social software applications (e.g., wikis) provide new opportunities for collaborative learning and knowledge building (Cress & Kimmerle, 2008; Dohn, 2009). Moreover, they present significant challenges to the views of knowledge (Cress & Kimmerle, 2008; Dohn, 2009), learning (Crook, 2008; Ravenscroft, 2009) and goals of the procedures implicit in Web 2.0 practises on the one hand (Collins & Halverson, 2010; Crook, 2008; Dohn, 2009) and the educational system on the other (Collins & Halverson, 2010; Dohn, 2009). Dohn (2009) has stressed that Web 2.0 and/or educational practises must be reshaped to fit each other, given that they originate in different contexts. From the perspective of collaboration within Web 2.0 tools, who contributes is less important than the fact that contributions are made and that they stand a chance of being revised by adding, deleting or changing their components until the outcome corresponds to group direction and consensus (Dohn, 2009).
ACCEPTED MANUSCRIPT
Alternatively, Cress and Kimmerle (2008) see an imminent connection between collaborative knowledge building in wikis and learning; from their perspective, one persons individual knowledge can serve as a resource for the learning of others. In their seminal paper on knowledge building with wikis, they describe how people make use of each others knowledge through collaborative knowledge building with artefacts. When interacting with a wiki, individuals can learn as a result of either externalization or internalization. This learning can take place by assimilation (extending knowledge by simply adding new information) or by accommodation (modifying and creating new knowledge). In this study, the pedagogical ideas behind the design are grounded in collaborative learning, and special effort has been placed on enhancing and supporting collaborative learning as a cognitive and social activity (Teasley, 1997). The students learning tasks, including social and individual activities, were supported by designing learning assignments that consisted of recurrent individual and collective phases in which students used Web 2.0 tools in concert to perform the designed tasks. In sum, all these activities to be undertaken with social software tools were also aligned in such way that Web 2.0 characteristics (Dohn, 2009) were taken into account. For example, Web-mediated resources were largely utilised; all created content was open, and wiki pages had distributed authorship; different materials were reproduced and transformed from multiple
ACCEPTED MANUSCRIPT
individual or collaborative learning spaces; and open-endedness and lack of finality were actively promoted to all participating students. 3. Aims of the study In this single-case study, small groups of learners were supported using multiple social software tools and face-to-face activities in the context of higher education. The aim of the study was to explore how designed learning activities contribute to students learning outcomes by studying the probabilistic dependencies between the variables. The research questions are as follows: 1) How much did students learn during the course? 2) Which social software and face-to-face variables were the best predictors for identifying differences between high- and low-performing groups of students? 3) What was the impact of social software and face-to-face sessions on individual students learning gain? 4. Methods This study followed the principles of the case study method. A case study is defined as an empirical study that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between the phenomenon and the context are not evident (Yin, 2003). In practise, the research design of the current study employed a singlecase study with embedded multiple units of analysis. As multiple social software tools and face-to-face activities were used to support learning in a higher education course, the behaviour of students within phases of the
ACCEPTED MANUSCRIPT
learning design and students learning outcomes were considered as the embedded units. These units were analysed using quantitative techniques as the primary approach. In order to return to larger units of analysis, Bayesian methods (Jensen, 2001) were used to classify and model the complex dependencies between the different variables. 4.1. Participants and the research setting The research participants were 21 undergraduate students in a five-year teacher education programme in the Faculty of Education at the University of Finland. All of the students were enrolled in a required course titled Future Scenarios and Technologies in Learning during the spring semester of 2009. The 21 participants included 16 females (76%) and 5 males (24%). The prevalence of females reflects the gender ratio of education majors at the university. 4.1.1. The task The participants worked in groups of four to five students for 12 weeks. Groups were required to complete a wiki project by the end of the semester. In order to complete the wiki project, students needed to participate in recurrent solo and collective phases mediated by the use of social software tools and face-to-face meetings in their respective sessions (see Figure 1). On the first day of the course, in a campus computer lab, the instructor gave all participating students pre-configured accounts to social software
ACCEPTED MANUSCRIPT
services and mobile devices needed for photo-taking activities (see Section 4.1.2). After ensuring that the students in their respective groups understood the instructions provided, no further support was provided during the tasks. In other words, the assignments required the students not only to learn and apply content knowledge, but also to generate their own learning objectives and to determine what information to include in their final contribution in their group wiki to be presented to the class. --- Insert Figure 1 about here --The pedagogical design of this course was as follows: A. Ground [Lecture] (weeks 1-3 and 6-8): Each of six one-week working periods started with a lecture in which students were grounded in main theoretical concepts. The specific themes were in the following order: 1. Learning infrastructure, 2. Learning communities, 3. Metacognition, 4. Selfregulated learning, 5. Learning design, and 6. Social Web as a learning environment. B. Reflect [Discussion] (weeks 1-3 and 6-8): The purpose of this collaborative phase was to reflect on the lecture topic in groups and to formulate a problem to be solved based on the group members shared interests during the following solo learning phases. Groups were advised to set their own learning objectives based on the topic and to write down these objectives in their personal blogs for further reflection.
ACCEPTED MANUSCRIPT
C. Conceptualize [Photo-taking] (weeks 1-3 and 6-8): In this solo phase, individual students were required to conceptualize their group members shared interests. In order to do so, they were required to identify and capture situated pictorial metaphors describing their shared interests. In practise, their tasks were to explore their everyday working and living environments and take photos with a camera phone. D. Reflect and elaborate [Blogging] (weeks 1-3 and 6-8): The task of this phase was to further reflect and elaborate on photos in the students personal blogs. First, they were required to analyse collected visual representations in order to discard ideas that were not relevant to their groups shared learning objectives. Second, they were required to write blog entries about chosen photos in which they further elaborated associations between photos, group-level objectives and students everyday situated practises. E. Review and evaluate [Discussion] (weeks 4 and 9): The first task of this collaborative face-to-face activity was to review group members Weblogs from the previous three-week period. The second activity was to evaluate the usefulness of blog entries in the context of their shared learning objectives and to discard irrelevant ideas. The outcome of this phase was used as material for co-construction of knowledge in the groups wikis. F. Co-construct knowledge [Wiki work] (weeks 4-12): The task in this collaborative assignment was focused on integrating each groups chosen blog entries and visual representations into a cohesive and comprehensive product of all course topics. In other words, the given goal was to formulate
ACCEPTED MANUSCRIPT
what they had learnt in their own words and produce it as uniform material that could be put to authentic use. G. Monitor peer students contributions [Monitor] (whole course): This was not an assignment per se, but it enabled students to obtain different perspectives by seeing what others were doing with social software tools, and it helped students to assimilate and accommodate their thinking. In practise, monitoring activities were done by using cloud-based syndication tools (RSS). 4.1.2. Tools The idea of making use of each others knowledge was operationalized in a socio-technical design. It consisted of recurrent individual and collective phases in which students used multiple Web 2.0 tools and mobile phones in concert to perform designed tasks (Figure 2). First, all students received a personal mobile multimedia computer, which was integrated with features including a 3.2 megapixel digital camera, 3G connectivity and an Internet browser. The mobile device was the main tool for the students in Phase C, who were required to identify and capture situated pictorial metaphors describing their groups shared interests. The device was equipped with a ShoZu cloud-based file-sharing tool, which was used as a bridge to connect mobile phones to the Flickr cloud-based file-sharing service for photos. ShoZu offered functions to add tags, titles and descriptions before putting photos on the Flickr photostream. In addition, the phones Web browser was configured to show students accounts on the
ACCEPTED MANUSCRIPT
Google Reader Mobile cloud-based RSS aggregator. This service was used to show all of the course-related content on the mobile phones at the students disposal (Figure 2). Second, an individual Wordpress.com account was created for each student. This blogging service was used as a personal learning diary for the students in which they individually reflected further on their ideas by writing journal entries regarding the respective pictures/videos sent to blogs via the Flickr file-sharing service (Phase C). The students blogs were used as a storage facility for their groups shared working problems (Phase B) and as an anchor resource in the review and evaluate phase (Phase E). In addition, the blogging service was the platform for course-level activities, a place for course-related announcements. The cloud-based Wikispaces wiki service was also used for two purposes: First, it offered collaboration tools for the groups to use (i.e., empty wiki page and discussion tool) in order to support their collaborative knowledge coconstruction (Phase F). Second, it was used at the course level for distributing resources (i.e., course curricula, lecture slides, hyperlinks and how-to guides) and displaying syndicated content from Flickr (student accounts) and WordPress (course blog, student blogs). In addition, the FeedBlendr and FeedBurner RSS services were used to merge individual, group and class-level feeds from multiple Flickr, WordPress and Wikispaces accounts. In practise, these merged feeds were included as RSS widgets in a sidebar of the respective blog or wiki. These tools enabled the
ACCEPTED MANUSCRIPT
students to combine social software tools, and they may be seen as additional collaborative tools that facilitated relationships between different task phases, the students, the content they produced and the tools used in this study (See Lee et al., 2008). --- Insert Figure 2 about here --4.2. Data collection The data was composed of video recordings, social software usage activity and pre- and post-tests of students conceptual understanding. Respective data variables are stored in parentheses embedded into the descriptions below (see also Appendix I). 4.2.1. Conceptual knowledge test To assess their conceptual understanding, the students completed identical paper-and-pencil pre- and post-tests with a pre-test/post-test quasiexperimental design. Specifically, the conceptual-knowledge measure consisted of six constructed-response questions that were developed based on the key concepts of the course. Students were asked to write definitions of the lecture themes, meaning that each theme was also connected to the learning design described in Section 4.1.1. and was thus used for measuring the students learning outcomes (gain) in a particular week of the course. 4.2.2 Video data Video recordings captured each groups six collaborative reflection sessions (B.discussion) and two collaborative reviewing and evaluation sessions (E.discussion) (42 hours of video data). The duration of those sessions
ACCEPTED MANUSCRIPT
was determined by each group, and the average duration of one session was 44 minutes (where the duration ranged from 13 minutes to 86 minutes). 4.2.3 Social software activity data Social software usage activity data was collected at the student level through multiple sources. First, the total number of Flickr photos per weekly topic and the average number of photos for all topics (C.photo) were calculated. Second, the total number of words in each blog entry and the number of blog entries were measured for each weekly topic. Then, the average values of these were calculated for all topics (D.blog.posts; D.blog.words/post) to be used in the Bayesian multivariate analysis. Third, activity measures of the students wiki usage were calculated by using adds and deletes as coding categories for cumulative history data. A measure of student cumulative involvement in the wiki was given by the sum Activity(u) = add(u) + delete(u), called the edit activity of author u, providing the total number of words (F.wiki.wc.activity) or edits (F.wiki.edits.activity) that u touched by adding or deleting them. This value was used to calculate students active use of their respective group wikis and their interactions in the wiki discussion forum and embedded comments in the wiki (F.wiki.edits.comments; F.wiki.wc.comments). A further characterization of how an author u contributed to the group wiki was given by the difference Net added (u) = add(u) delete(u), called the net number of words added or edits performed, providing the total number of words or edits by which u increased
ACCEPTED MANUSCRIPT
the length of the text (F.wiki.wc.net) or the number of edits (F.wiki.edits.net) when words or edits that u deleted were deducted. This value was used to calculate the amount of new content students contributed to the wiki. Finally, the total number of read RSS items was measured by using statistics collected automatically by Google Reader (G.rss.monitor). 4.3 Data analysis Data was analysed using a quantitative paired samples t-test for the conceptual knowledge tests, qualitative on-task analysis for video recordings and multivariate Bayesian methods for the dependencies between social software usage, face-to-face activities and learning gain. 4.3.1. Quantitative analysis of conceptual knowledge tests In the first stage of analysis, a conceptual knowledge test was analysed in order to answer the first research question: How much did students learn during the course? Three independent researchers (including the first and second authors of this paper) developed the criteria and marked the learning tests (points 0-3). The criteria were as follows: 0 points represented low understanding (the student had no understanding of the concept). One point represented some level of understanding (the student had some understanding (i.e., knew what the concept was connected to) but no detailed knowledge of it). Two points represented a basic level of understanding (the student understood what the concept was connected to and knew some details about the concept). Finally, 3 points represented the highest level of understanding (the student had a deep
ACCEPTED MANUSCRIPT
understanding of the concept and knew very specific details about the concept). The tests were analysed by marking points from 0 to 3 for individual answers. This was done by three researchers who first independently marked the tests and then compared the results and negotiated possible differences. According to the test results, all of the students understanding of the main concepts increased during the course. However, there were differences between their levels of understanding of the different concepts. To analyse the learning outcomes through the pre-test/post-test scores, a paired samples t-test was conducted, and a normalized learning gain was calculated (Hake, 1998). Next, the average normalised gain score was used to identify high-performing and low-performing students for further explorative Bayesian analysis. Note that contrasting the activity and artefacts of high performers to those of low performers is intuitively appealing (Jonassen, Tessmer, & Hannum, 1999) and has been shown to reveal important characteristics and aspects that are not uncovered using other approaches (Wyman & Randel, 1998). 4.3.2. Qualitative analysis of videotaped face-to-face sessions In the second stage, video data transcripts were analysed in order to clarify individual students activity levels in collaborative face-to-face assignments. Results of this analysis were used as an activity measure of faceto-face activities for descriptive analysis of learning phases and explorative Bayesian analysis (research questions 2 and 3).
ACCEPTED MANUSCRIPT
This analysis was adapted from the method that focuses on the duration of on-task and off-task episodes (for further details of the method, see Jrvel, Veermans, & Leinonen, 2008). In this analysis, the focus was placed on the number of task-related utterances, which were used as a measure of on-task activities, while off-task activities, such as discussions about their evening plans, were coded in an independent off-task category. 4.3.3. Descriptive analysis of social software and face-to-face activity variables In the third phase, a descriptive analysis was carried out for all the variables in the course design. First, the average values of an individual students face-to-face and social software activities were calculated for Bayesian analysis (research questions 2 and 3). Second, the mean, standard deviation and max-min values for all students (both high- and low-performing students) were calculated in order to assist in the interpretation of the results of Bayesian classification modelling and to provide an overview of the students activities during the course (See Appendix). 4.3.4. Bayesian multivariate analysis of the impact of social software and faceto-face sessions on learning outcome In the fourth phase, Bayesian analysis (Jensen, 2001) was conducted to study the probabilistic dependencies between the variables (research questions 2 and 3) described in Section 4.2. In practise, the analysis was conducted with the Web-based online data analysis tool B-Course1, which
1 http://b-course.cs.helsinki.fi/obc/
ACCEPTED MANUSCRIPT
allowed users to analyse their data using two different techniques: Bayesian dependency and classification modelling. In general, Bayesian methods have many benefits for explorative analysis, as summarized in Congdon (2003). For this study, the most relevant benefits were as follows: 1) The theoretical minimum for the sample is zero, 2) Different kinds of multivariate variables and distributions are accepted, and 3) It gives statistically robust tools to visualize and categorize complex dependencies between variables. In short, Bayesian methods enabled us to conduct statistical analyses of learning phases in our learning design. The first stage of Bayesian analysis involved conducting classification modelling (Silander & Tirri, 1999) in order to answer the second research question: Which social software and face-to-face variables were the best predictors for determining differences between high- and low-performing groups of students? In the classification process, the automatic search looked for the best set of variables to predict the class variable for each data item. This procedure is akin to the stepwise selection procedure in traditional linear discriminant analysis (Huberty, 1994). The second stage of Bayesian analysis involved building a Bayesian network (Jensen, 2001) in order to answer the third research question: What was the impact of social software and face-to-face sessions on individual students normalized learning gain? Such a Bayesian network was the visualised result of Bayesian dependency modelling, in which the most probable statistical dependency structure between variables was calculated.
ACCEPTED MANUSCRIPT
components (See Figure 3 and Table 3): 1) collected data as ellipses, 2) dependencies visualised as lines between nodes and 3) strength of each dependency as a ratio value in the table (see Table 3) and as a colour in the network. The darker the line, the stronger the statistical dependency between the two variables and the more important (higher ratio value) the dependency. A variable is considered independent of all other variables if there is no line attached to it. 5. Results First, results of the paired samples t-test will be presented to show how much students learned during the course. Second, the best predictors for pointing out differences between high- and low-performing groups will be explored using Bayesian classification analysis. Third, the results of Bayesian dependency modelling showing probability dependencies between the social software, face-to-face sessions and individual students normalized learning gain shall be presented. 5.1 How much did students learn during the course?
A paired samples t-test was conducted to compare pre-test and post-test means. Results showed that students gained higher scores in the post-test (M=7.95) than in the pre-test (M=3.95), t(21)=8.33, p<.000. The effect size (Cohens d) was 1.69. --- Insert Table 1 about here ---
ACCEPTED MANUSCRIPT
Table 1 presents the mean values for pre-test and post-test raw scores and pre-post normalized gain scores. Using the average normalized gain score (M=0.29; SD=0.16), high-performing and low-performing students were identified for explorative Bayesian classification analysis. 5.2. Which social software and face-to-face variables were the best predictors for determining differences between high- and low-performing groups of students? The second analysis explored which variables measuring social software usage and face-to-face activities were the best predictors for pointing out differences between high- and low-performing students. The model for classifying data contained items according to the class variable level of the normalized learning gain (low performers and high performers) with 12 variables of learning activities (descriptive values are shown in Appendix I, and items are described in Section 4.2). The estimated classification accuracy for the model was 81.82%. Table 2 lists the variables ordered by their estimated classification in the model. The strongest variablesthat is, those that best discriminate the independent variablesare listed first. The percentage values attached to each variable indicate the predicted decrease in the classification performance if the variable were to be dropped from the model. The table shows that all variables in the model are equally important; that is, if we were to remove any of the variables from the model, it would weaken the performance by 90.91%. --- Insert Table 2 about here ---
ACCEPTED MANUSCRIPT
Results from the classification analysis showed that the best predictors of higher learning gains were wiki-related activities. First, the mean number of wiki edits (F.wiki.edits.activity; M=68.64; SD=77.90) was two times higher among high performers than low performers (M=34.55; SD=21.16). Second, the high performers were 1.5 times more involved in the wiki editing activities (M=3427.73; SD=3810.10) than the low performers (M=2151.10; SD=2074.12) when the number of words (F.wiki.wc.activity) that they touched by adding or deleting was taken into account. Third, high-performing students increased the length of the text (F.wiki.wc.net) in their groups wikis about 1.4 times more often on average (M=1173.91; SD=444.70) than low-performing students (M=856.45; SD=507.49). In short, the descriptive analysis above shows that high performers were more active in organizing wiki content in a new way and in adding new information. The latter of these contribution categories is an example of assimilation, a process in which information coming from the wiki is perceived and modified in a way that makes it fit into the individuals knowledge. The former category is an example of an activity in which students do not simply assimilate new information into existing knowledge but actually change knowledge in order to better understand the wiki and its information (Cress & Kimmerle, 2008). 5.3. What was the impact of social software and face-to-face sessions on individual students normalized learning gain?
ACCEPTED MANUSCRIPT
The next stage of the analysis involved building a Bayesian network out of the 12 items measuring students learning activities during the course (descriptive values are shown in Appendix I, and items are described in Section 4.2). The rationale for this procedure was to examine dependencies between variables by both their visual representation and the probability ratio of each dependency in order to answer the third research question. A Bayesian search algorithm evaluated the dataset in order to find the model with the highest probability. During the extensive search, 174,987 models were evaluated. Figure 3 shows a visualization of the network, which contains two components: 1) collected data as ellipses and 2) dependencies visualised as lines between nodes. As mentioned, the darker the line, the stronger the statistical dependency between the two variables and the more important the dependency. Table 3 shows the strength of each dependency as ratio values in the probability table. In practise, if one removes the arc from the model with the high probability ratio, it decreases the probability of the model by the same amount. However, in many dependencies in the model, removing the arc between nodes would not change the probability of the final model (listed at the bottom of the probability table). --- Insert Figure 3 about here ----- Insert Table 3 about here --The Bayesian dependency model shows 7 strong (probability ratio >1,000,000) and 25 weaker relationships between variables. However, based
ACCEPTED MANUSCRIPT
on the analysis, only one strong dependency between activities and learning gain was found: the connection between assimilative wiki editing activities (F.wiki.wc.activity) and learning gain (gain), which triangulates with the results in the Bayesian classification model presented above. Furthermore, there was one weak dependency, the one between monitoring other students work via syndication services (G.rss.monitor) and learning gain (gain). Additionally, there were two other connections between other variables (B.discussion, C.photo) and normalized learning gain (gain) included in the visual network model, but their probabilities were so low that they were dropped from the dependency table automatically. It is worth noting that the wiki activities described above were strongly related to commenting on wiki content. When the Bayesian model is further explored, it reveals that the average number of blog posts (D.blog.posts) is the central variable in the model, as it has strong statistical relationships to both assimilative (F.wiki.wc.net; F.wiki.edits.net) and accommodative wiki activities (F.wiki.wc.activity; F.wiki.edits.activity). In practice, it can be said that students who were actively reflecting and elaborating were also active in inserting and modifying information in the wikis. This variable (D.blog.posts) also has a central role in the chain of strong relationships, including all virtual activities in the study design (see Figure 1.): C. Conceptualize, (C.photos), D. Reflect and elaborate (D.blog.posts), F. Co-construct knowledge (F.wiki.wc.activity), and learning gain (Gain). This result demonstrates the successful use of Web 2.0 characteristics in this study, an example of a series of activities in which intermediate learning
ACCEPTED MANUSCRIPT
products were reproduced and transformed. Furthermore, it shows how higher education course students can make use of each others knowledge through collaborative knowledge building (Cress & Kimmerle, 2008). There were also several weaker dependencies in the Bayesian model. First, results showed that active following of RSS feeds was slightly related to an increased number of situated visual representations (C.photos), an increased number of wiki editing activities (F.wiki.*) and learning gain (gain). However, no connection was found between usage of RSS feeds and blogging. Second, both collaborative face-to-face phases (B.discuss, D.discuss) were slightly related to social software usage (D.blog.*; F.wiki.*) except the phase in which students had to take photos. 6. Discussion In our case, we found that using social software tools together to perform multiple tasks likely increased individual knowledge acquisition during the course. Multivariate Bayesian classification analysis revealed that the best predictors of good learning outcomes were wiki-related activities. In addition, according to the Bayesian dependency model, students who monitored their peers work via syndication services and who were active by adding, modifying or deleting text in their groups wiki obtained higher scores. The model also shows that many other learning activities were indirectly related to learning outcome. First, learning scores from pre-test to post-test were statistically significant with high learning effect, indicating a substantial gain in conceptual
ACCEPTED MANUSCRIPT
knowledge test scores from pre-test to post-test. This finding provides support for the learning design used in this study and for the use of multiple cloudbased social software tools in a higher education context, and it was further used to contrast high performers and low performers in the following explorative Bayesian analysis. Second, results from the Bayesian classification analysis revealed differences between high performers and low performers and showed that the best predictors of higher learning gain were wiki-related activities. Descriptive analysis of chosen predictor variables showed that high performers were more active in organizing wiki content in a new way (mean number of wiki edits was two times higher and mean word length of edited content was two times higher when compared to low performers) and in adding new information (mean length of inserted words was 1.4 times higher than that of low-performers). The latter of these contribution categories is an example of assimilation, a process in which information coming from the wiki is perceived and modified in a way that makes it fit into an individuals knowledge. The former category is an example of an activity in which students do not simply assimilate new information into existing knowledge but actually change knowledge in order to better understand the wiki and its information (Cress & Kimmerle, 2008).
ACCEPTED MANUSCRIPT
learning gain, which triangulates with results in Bayesian classification modelling. Furthermore, there was one weak dependency, between monitoring other students work via syndication services and learning outcome. There were two other connections between other variables and learning gain included in the network model, but their probabilities were so low that removing them would not change the probability of the final model, and therefore, those were dropped automatically from the final model during the analysis. It is also worth noting that the wiki activities described above were strongly related to commenting on wiki content. When the Bayesian model is further explored, it reveals that the average number of blog posts per student is the central variable in the model, as it has strong statistical relationships to both assimilative and accommodative wiki activities. In practise, it can be said that students who were actively reflecting and elaborating on visual representations in their own blogs were also active in inserting and modifying knowledge in the wikis. This can be considered an example of learning that is both reflective and collaborative at the juxtaposition of community and personal spaces (Wheeler, 2009). This blog post variable also has a central role in the chain of strong relationships, including almost all social software-related tasks in this study: average number of photos taken and shared by each student, average number of blog posts, total sum of wiki activity, and learning gain. This chain of activities demonstrates the successful use of Web 2.0 characteristics in this study, an example of a series of activities in which intermediate learning
ACCEPTED MANUSCRIPT
products were reproduced and transformed by performing structured collaborative assignments using Web 2.0 tools. It also shows how higher education course students can make use of each others knowledge through collaborative knowledge building (Cress & Kimmerle, 2008). The remaining variables were weaker than those presented above. First, the results showed that monitoring who does what (implicit peer feedback for individual reflection) using syndication tools (RSS) was slightly related to an increased number of situated visual representations (photos), number of wiki editing activities and learning gain. However, the model did not show connections between blog and syndication variables. Therefore, it can be argued that different perspectives on the form of syndicated content did not contribute to reflective blog-writing activities. Instead, the results showed that active monitoring of the activities of others using different social software tools increased students number of wiki activities. Generally, these results further reinforced the findings of Jermann and Dillenbourg (2008), who determined that the tools can provide information to foster group members reflections of their contributions: what to do and who does what. Second, the results revealed that the explicit peer feedback that students received by participating in collaborative face-to-face sessions (sense-making session and meaningmaking session) slightly increased social software usage activities. 7. Conclusion It can be concluded that the carefully crafted pedagogical activities and Web 2.0 tools used together to perform designed tasks likely increased
ACCEPTED MANUSCRIPT
students individual knowledge acquisition during the course. This is in accordance with Meyers (2010) claim regarding how assignments should be structured and orchestrated to encourage learning to occur. It also reinforces findings of Halic et al. that a technological tool works better when its coupled with compatible pedagogical conceptions, and yet interaction is insufficient to achieve cognitive engagement. Some type of facilitation in online environments may be necessary (2010, p. 211). The findings of our case study, together with the described sociotechnical design, illustrate practical implications for designing the use of multiple social software tools to support collaborative learning in higher education. Therefore, by providing an explicit socio-technical example, this study can contribute to pedagogical practices when educators are considering how they should use cloud-based social software as a learning platform (Schroeder et al., 2010; Wheeler, 2009). First, the findings from this study contribute to the emerging body of studies surrounding the empirical research regarding the educational use of Web 2.0 and its adoption and impact (Crook, 2008). Second, this article is also a timely and rare contribution to the emerging discussions on how to design and integrate the use of multiple Web 2.0 tools in higher education contexts in a pedagogically meaningful way instead of using legacy virtual learning environments (Hemmi et al., 2009; Schroeder et al., 2010; Uzunboylu et al., 2011; Wheeler, 2009). This case study was limited by the single-case design and the lack of other student groups completing the same tasks with the same socio-technical
ACCEPTED MANUSCRIPT
design. The rationale for the single-case design is that it is a revelatory case (Yin, 2003). In practise, this study is a rare contribution to the empirical analysis of integrating face-to-face situations and social software in higher education. In addition, the course in which the data collection was conducted was the first implementation of the described socio-technical design at the university. Furthermore, this study used embedded multiple units of analysis in order to qualitatively collect and analyse complex dependencies between different learning phases and students learning outcome, which raises concerns of a small sample size within subunits (Yin, 2003). To overcome the problems raised by the relatively small sample size, data was analysed using Bayesian methods, which do not have theoretical minimums for sample sizes and offer other benefits for explorative data analysis (Congdon, 2003; Jensen, 2001). It also has been argued that research designs in authentic contexts inevitably provide principles that can be localised for others to apply to new settings and to produce explanations of innovative practises (Fishman, Marx, Blumenfeld, Krajcik, & Soloway, 2004). Therefore, research investigations conducted in authentic contexts are still needed as a first step to understand these new opportunities in terms of learning interaction and collaboration that social software can provide.
ACCEPTED MANUSCRIPT
Acknowledgements This research was supported by the Doctoral Programme for Multidisciplinary Research on Learning Environments, Finland, and a grant from the Finnish Cultural Foundation.
ACCEPTED MANUSCRIPT
References Arnold, N., & Paulus, T. (2010). Using a social networking site for experiential learning: Appropriating, lurking, modeling and community building. The Internet and Higher Education, 13(4), 188-196. doi:10.1016/j.iheduc.2010.04.002 Birdsall, W. F. (2007). Web 2.0 as a social movement. Webology, 4(2). Retrieved from http://www.webology.ir/2007/v4n2/a40.html Bransford, J., Brown, A. L., & Cocking, R. R. (1999). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press. Retrieved from http://www.nap.edu/catalog.php?record_id=9853 Cohen, A., & Scardamalia, M. (1998). Discourse about ideas: Monitoring and regulation in face-to-face and computer-mediated environments. Interactive Learning Environments, 6(1-2), 93-113. doi:10.1076/ilee.6.1.93.3610 Collins, A., & Halverson, R. (2010). The second educational revolution: Rethinking education in the age of technology. Journal of Computer Assisted Learning, 26(1), 18-27. doi:10.1111/j.1365-2729.2009.00339.x Congdon, P. (2003). Applied Bayesian modelling. Chichester: Wiley. Cress, U., & Kimmerle, J. (2008). A systemic and cognitive view on collaborative knowledge building with wikis. International Journal of ComputerSupported Collaborative Learning, 3(2), 105-122. doi:10.1007/s11412007-9035-z
ACCEPTED MANUSCRIPT
Crook, C. (2008). Web 2.0 technologies for learning: The current landscape opportunities, challenges and tensions (British Educational Communications and Technology Agency (BECTA) Report: Web 2.0 technologies for learning at Key Stages 3 and 4). Retrieved from http://dera.ioe.ac.uk/1474/ Dillenbourg, P. (1999). What do you mean by collaborative learning? In P. Dillenbourg (Ed.), Collaborative-learning: Cognitive and computational approaches (pp. 1-19). Oxford: Elsevier. Dillenbourg, P., Baker, M., Blaye, A., & Malley, C. O. (1996). The evolution of research on collaborative learning. In E. Spada & P. Reiman (Eds.), Learning in humans and machines: Towards an interdisciplinary learning science (pp. 189-211). Oxford: Elsevier. Dohn, N. (2009). Web 2.0: Inherent tensions and evident challenges for education. International Journal of Computer-Supported Collaborative Learning, 4(3), 343-363. doi:10.1007/s11412-009-9066-8 Fishman, B., Marx, R., Blumenfeld, P., Krajcik, J., & Soloway, E. (2004). Creating a framework for research on systemic technology innovations. Journal of the Learning Sciences, 13(1), 43-76. Retrieved from http://www.jstor.org/stable/1466932 Hake, R. (1998). Interactive-engagement versus traditional methods: A sixthousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66(1), 64-74. doi:10.1119/1.18809
ACCEPTED MANUSCRIPT
Halic, O., Lee, D., Paulus, T., & Spence, M. (2010). To blog or not to blog: Student perceptions of blog effectiveness for learning in a college-level course. The Internet and Higher Education, 13(4), 206-213. doi:10.1016/j.iheduc.2010.04.001 Hemmi, A., Bayne, S., & Land, R. (2009). The appropriation and repurposing of social technologies in higher education. Journal of Computer Assisted Learning, 25(1), 19-30. doi:10.1111/j.1365-2729.2008.00306.x Huberty, C. J. (1994). Applied discriminant analysis. New York: Wiley. Jrvel, S., Veermans, M., & Leinonen, P. (2008). Investigating student engagement in computer-supported inquiry: A process-oriented analysis. Social Psychology in Education, 11(3), 299-322. doi:10.1007/s11218-0079047-6 Jensen, F. V. (2001). Bayesian networks and decision graphs. New York: Springer. Jermann, P., & Dillenbourg, P. (2008). Group mirrors to support interaction regulation in collaborative problem solving. Computers & Education, 51(1), 279-296. doi:10.1016/j.compedu.2007.05.012 Jonassen, D. H., Tessmer, M., & Hannum, W. H. (1999). Task analysis methods for instructional design. Mahwah, NJ: L. Erlbaum Associates. Kesim, E., & Agaoglu, E. (2007). A paradigm shift in distance education: Web 2.0 and social software. Turkish Online Journal of Distance Education, 8(3), 66-75. Retrieved from http://tojde.anadolu.edu.tr/tojde27/
ACCEPTED MANUSCRIPT
Lee, M. J. W., Miller, C., & Newnham, L. (2008). RSS and content syndication in higher education: Subscribing to a new model of teaching and learning. Educational Media International, 45(4), 311-322. doi:10.1080/09523980802573255 Meyer, K. A. (2010). Web 2.0 research: Introduction to the special issue. The Internet and Higher Education, 13(4), 177-178. doi:10.1016/j.iheduc.2010.07.004 Myllymki, P., Silander, T., Tirri, H., & Uronen, P. (2002). B-course: A web-based tool for Bayesian and causal data analysis. International Journal on Artificial Intelligence Tools, 11(3), 369-387. doi:10.1142/S0218213002000940 Pea, R. D. (1993). Practises of distributed intelligence and designs for education. In G. Salomon (Ed.), Distributed cognitions: Psychological and educational considerations (pp. 47-87). New York, NY: Cambridge University Press. Ravenscroft, A. (2009). Social software, web 2.0 and learning: Status and implications of an evolving paradigm. Journal of Computer Assisted Learning, 25(1), 1-5. doi:10.1111/j.1365-2729.2008.00308.x Schroeder, A., Minocha, S., & Schneider, C. (2010). The strengths, weaknesses, opportunities and threats of using social software in higher and further education teaching and learning. Journal of Computer Assisted Learning, 26(3), 159-174. doi:10.1111/j.1365-2729.2010.00347.x
ACCEPTED MANUSCRIPT
Sharma, P., & Fiedler, S. (2007). Supporting self-organized learning with personal web publishing technologies and practices. Journal of Computing in Higher Education, 18(2), 3-24. doi:10.1007/BF03033411 Silander, T., & Tirri, H. (1999). Bayesian classification. In P. Ruohotie, H. Tirri, P. Nokelainen, & T. Silander (Eds.), Modern modeling of professional growth (pp. 61-84). Hmeenlinna: RCVE. Slj, R. (2003). Representational tools and the transformation of learning. In B. Wasson, U. Hoppe, & S. Ludvigsen (Eds.), Designing for change in networked learning environments (pp. 1-2). Dordrecht, The Netherlands: Kluwer Academic Publishers. Teasley, S. D. (1997). Talking about reasoning: How important is the peer in peer collaborations? In L. B. Resnick, R. Saljo, C. Pontecorvo, & B. Burge (Eds.), Discourse, tools, and reasoning: Situated cognition and technologically supported environments (pp. 361-384). Heidelberg, Germany: Springer-Verlag. Uzunboylu, H., Bicen, H., & Cavus, N. (2011). The efficient virtual learning environment: A case study of web 2.0 tools and windows live spaces. Computers & Education, 56(3), 720-726. doi:10.1016/j.compedu.2010.10.014 Webb, M. N. (1989). Peer interaction and learning in small groups. International Journal of Educational Research, 13, 21-40. doi:10.1016/08830355(89)90014-1
ACCEPTED MANUSCRIPT
Wheeler, S. (2009). Learning space mashups: Combining web 2.0 tools to create collaborative and reflective learning spaces. Future Internet, 1(1), 3-13. doi:10.3390/fi1010003 Wyman, B. G., & Randel, J. M. (1998). The relation of knowledge organization to performance of a complex cognitive task. Applied Cognitive Psychology, 12(3), 251-264. doi:10.1002/(SICI)1099-0720(199806)12:3<251::AIDACP510>3.0.CO;2-F Xie, Y., Ke, F., & Sharma, P. (2008). The effect of peer feedback for blogging on college students reflective learning processes. The Internet and Higher Education, 11(1), 18-25. doi:10.1016/j.iheduc.2007.11.001 Yin, R. K. (2003). Case study research: Design and methods (3rd ed.). Thousand Oaks, CA: Sage Publications.
ACCEPTED MANUSCRIPT
Table 1. Pre-test and post-test raw scores and normalized gain scores
Pretest score M SD 3.95 4.27 3.64 2.66 2.87 2.54 Posttest score M SD 7.95 10.00 5.91 2.92 1.95 2.21 Normalized gain M SD 0.29 0.42 0.16 0.16 0.08 0.08
Condition All students High performers Low performers
Note. Normalized learning gain was calculated by using Hake's (1998) approach. Next, the average normalized gain scores were used to identify high-performing and low-performing students for following Bayesian classification analysis.
Table 2. Importance ranking of the social software usage and learning activity variables by the level of normalized gain score Class variable: The level of normalized gain score low-performers high-performers < 0.29 > 0.29 M SD M SD 2074. 3810. 2151.09 12 3427.73 10 507.4 444.7 855.45 9 1173.91 0
Dro pa Predictor variablesb % F.wiki.wc.activit 90.9 y 1 F.wiki.wc. 90.9 net 1 F.wiki.edits.activ 90.9 ity 1 34.55 21.16 68.64 77.90 Note. In the classification modelling process (Silander & Tirri, 1999), the automatic search looked for the best set of variables to predict the class variable for each data item. a. Decrease in predictive classification if item is dropped from the classification model. b. Classification accuracy is 81.82%.
ACCEPTED MANUSCRIPT
Table 3 Dependen cy D.blog.posts -> F.wiki.wc.activity D.blog.posts -> F.wiki.wc.net D.blog.posts -> F.wiki.edits.activity F.wiki.edits.activity -> F.wiki.edits.comments D.blog.posts -> F.wiki.edits.net F.wiki.wc.activity -> Gain Gain -> F.wiki.wc.comments C.photos -> D.blog.posts G.rss.monitor -> F.wiki.wc.activity G.rss.monitor -> F.wiki.wc.net G.rss.monitor -> F.wiki.edits.activity G.rss.monitor -> F.wiki.wc.comments G.rss.monitor -> F.wiki.edits.net D.blog.words/post -> E.discussion G.rss.monitor -> C.photos E.discussion -> F.wiki.wc.activity E.discussion -> F.wiki.wc.net B.discussion -> F.wiki.wc.activity B.discussion -> F.wiki.wc.net E.discussion -> F.wiki.edits.net B.discussion -> F.wiki.edits.net E.discussion -> F.wiki.edits.activity B.discussion -> F.wiki.edits.activity B.discussion -> F.wiki.wc.comments B.discussion -> D.blog.posts C.photos -> F.wiki.wc.comments G.rss.monitor -> Gain G.rss.monitor -> F.wiki.edits.comments G.rss.monitor -> E.discussion D.blog.words/post -> C.photos G.rss.monitor -> B.discussion Probability ratio 1:1.000.000.000 1:1.000.00 0
1:2254 1:975 1:975 1:931 1:880 1:798 1:797 1:72 1:44 1:44 1:44 1:44 1:44 1:44 1:44 1:44 1:44 1:31 1:26 1:17 1:14 1:4.91 1:3.62 1:2.69
ACCEPTED MANUSCRIPT
Note. The probability ratio describes the strength of statistical dependency between the two variables and the importance of the dependency for the model.
ACCEPTED MANUSCRIPT
Appendix Table 1. Descriptive statistics of students activities during the course Descriptive statistics for face-to-face, social software activity and learning gain variables All students (n=21) High-performers (n=10) Low-performers (n=11) Mean Stdev Max Min Mean Stdev Max Min Mean Stdev Max Min
Unit B. Reflect [discussion] B.discussio n C. Conceptualize [phototaking] C.photos D. Reflect and elaborate [blogging] D.blog.post s utteranc es photos
74.27 3.86
28.17 1.25
118 6
23 2
78.18 3.73
23.48 1.10
107 5
49 2
70.36 4.00
32.89 1.41
118 6
23 2
posts 3.99 words/po D.blog.words/post st 88.09 E. Review and evaluate [discussion] utteranc 219.8 E.discussion es 6 F. Co-construct knowledge [wiki-work] F.wiki.edits.activit y edits 51.59 F.wiki.edits. net edits 16.86 2789. F.wiki.wc.activity words 41 F.wiki.wc.ne words 1014.
1.25 37.76
6 153
1.8 9
4.05 101.27
1.03 40.11
5.3 153
1.8 30
3.93 74.91
1.48 31.67
6 128
1.8 9
80.44
390
74
202.64
69.47
327
81
237.09
90.06
390
74
58.37 14.71
271
4 2 320 122
68.64 19.91
77.90 17.47
271
5 3 355 353
34.55 13.82
21.16 11.36
72
4 2 320 122
59 1283 3064.02 0 493.33 2067
59 1283 3427.73 3810.10 0 1173.91 444.70 1854
42 665 2151.09 2074.12 4 855.45 507.49 206
ACCEPTED MANUSCRIPT
t F.wiki.edits.comm ents edits F.wiki.wc.commen ts words G. Monitor peer students contributions [monitor] G.rss.monit read or items Normalized learning gain Gain
68 14.09 277.0 8 120.0 9 9.72 235.46 34 841 2 0 15.82 252.46 11.76 220.18 34 701 2 0 12.36 301.70 7.31 258.10
7 26 841 2 0
199.83
701
76.09
124.81
428
164.09
253.03
701
pre-post 0.29 0.16 0.60 0.0 0.42 0.08 0.6 0.3 0.16 0.08 0.27 0 gain 0 1 Note. Mean, standard deviation and max-min values for all students (both high- and low-performing students) were calculated in order help interpret the results of Bayesian classification modelling and to provide an overview of the students activities during the course.
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Figure 3. Graphical visualization of Ba yesian network (Jensen, 2001) contains three components: 1) collected data as ellipses; 2) dependencies visualised as lines between nodes, and 3) strength of each dependency as color in the network. The dark er the line, the stronger is the statistical dependency between the two v ariables, and the more important (higher r atio value) the dependency is for the model. R emoving the dependency between B.discussion and Gain; C.photos and Gain; D .blog.words/post and F.wiki.wc.net; D.blog.words/post and F.wiki.wc.activity; D.blog.words/post and F.wiki.edits.net; D.blog.words/post and F.wiki.edits.activity; and E.discussion and F.wiki.edits.comments would not change the probabilit y of the final model.

Supporting Small-Group Learning Using Multiple Web 2.0 Tools: A Case Study in The Higher Education Context

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Supporting Small-Group Learning Using Multiple Web 2.0 Tools: A Case Study in The Higher Education Context

Uploaded by

Copyright:

Available Formats

ACCEPTED MANUSCRIPT

Condition All students High performers Low performers

59 1283 3064.02 0 493.33 2067

59 1283 3427.73 3810.10 0 1173.91 444.70 1854

42 665 2151.09 2074.12 4 855.45 507.49 206

You might also like