Research OverviewAn e-Portfolio Design Theory Supporting Ownership, Interaction, and Ease of Use Nathan Garrett, Claremont Graduate Universityhttp://conversation.cgu.edu/garrettn/June 6, 2008

Abstract

The goal of this dissertation project is to create and evaluate the effectiveness of an ePortfoliosystem supporting student ownership, peer learning, and ease of use. Such a system ishypothesized to increase users’ intrinsic motivation, improve student learning, and yield reliableand valid data for assessment. It deals with increasing demands for external accountability bydemonstrating ways to make learning social, engaging, and demonstrable.

Research Question

The central concern of this project is the development of electronic portfolio design theory. Howdoes a portfolio change as it moves from pieces of paper to just another record in a centralizeddatabase? How do system designers ensure their software supports learning while still generatingassessment data? Do current systems support the proper use cases?My research goal is to advance portfolio design theory by evaluating the impact of ownership,ease of use, and peer learning in predicting user satisfaction. Building upon previous work usingsocial software in education, I have developed a number of design features to support theseconstructs. My research approach is to create a software artifact, deploy it in an educationalsetting, and use survey and usage data to evaluate my design features and the overall designmodel.

Reason for Research

As universities react to the Spellings Commission’s call for increased comparability and externalaccountability, portfolios have begun to look like the “golden tool” for assessment. Portfolios provide a rich dataset that can show critical thinking, growth over time, and deep learning(Barrett, 2005). However, as portfolios are increasingly adopted for assessment purposes, their original pedagogical purpose is often subverted (Barrett, 2004). Instead of using portfolios for learning, and their contents for assessment, portfolio initiatives and software are often solelyfocused upon generating data for accreditation bodies.As a result, most ePortfolio systems tend to be large, centralized, and standardized. The need tominimize demands on faculty time has pushed software like OSP and others into rigid checklistand “fill in the blank” designs. Software like LiveText and Iwebfolio encourage administrators tolock down student portfolios to ensure consistency. In Iwebfolio, these controls include limitingthe ability to change page names, add new pages, or choose a non-standard template. Portfoliosare private by default, and students must go though a tedious process for each person they wouldlike to give access.1

Meanwhile, researchers are trying to figure out ways to make portfolios more “sticky” (Jafari, A.,McGee, P., & Carmean, C. 2006b) so that students will be able to see them as the learningopportunity they represent. Professors want to use portfolio software to help students reflect,share, and develop meta-cognitive abilities.Studies of students have shown that they want education software that helps them to connectwith each other, express their individuality, and is easy to use (Jafari, A., Mcgee, P., & Carmean,C, 2006a). Frequently, faculty and staff use free social software on the Internet to do all thesethings, finding it easier to use and more fulfilling than portfolio software. As one faculty member says:“I engage in many e-portfolio-like practices. Those involving dedicated e- portfolio tools have been far less satisfactory than those involving social softwaretools such as blogs, wikis, or social networking sites.” (Becta, 2007, p.24)

Literature Review

Almost all current ePortfolio software packages are driven by a single dominant design model.This paradigm defines a portfolio as a database of individual artifacts organized into a number of different views and submitted to specific audiences. This model is well suited for controllingaccess, providing multiple views of data, and providing assessment data. However, it makes portfolios complex to construct, and the permissions settings discourage sharing and peer learning. Unfortunately, this paradigm is so common that almost all major programs use it. As aresult, Cohn and Hibbitts (2004) identify the “ossification of the current prefabricated, one-size-fits-most portfolio” (p.2).My epistemology for this project is primarily based upon constructivism. In particular, much of this project has been inspired by Schön’s concepts of ‘learning to be’ and reflection (1982, 1986).Schön’s concept of reflection built upon the work of John Dewey, who proposed reflection as amental version of the scientific process (Dewey, 1933). Dewey defined reflection as a set of mental thought experiments. To reflect, a learner must become open to the idea of uncertainty, or not knowing something. They then evaluate possible approaches to structure the problem, andevaluate potential outcomes based upon evidence and rational thought. Learning portfolios arean excellent way to induce reflection (Moon, 2001). The connection between reflection andlearning are discussed in depth in Moon (2001), showing how reflection creates the rightconditions for learning.A portfolio is a fundamentally collaborative work. Zubizarreta (2006) argues that portfoliosrequire three domains of activity: documentation, reflection, and collaboration. Collaborationshould include faculty members (classroom teachers or an advisor) and other students. As hesays, “… reflection is facilitated best by not leaving students individually to their own devices inthinking about their learning but by utilizing the advantages of collaboration and mentoring inmaking learning community property.”Unfortunately, most portfolio software systems do not provide robust features for collaboration.A survey conducted in the United Kingdom (Becta, 2007) found that only a third of the surveyed2

students in the case studies agreed that their system “shows me what my friends are learning”(p.17).Adoption of ePortfolios can be difficult. Goldsmith (2007) found that usefulness and ease of usewere the most important predictors of student adoption. Surveys of students have found that theywant simple education software that helps them connect with each other and express their individuality (Jafari, A., Mcgee, P., & Carmean, C, 2006a). Students frequently request systemsmore like the social software they use outside of school (Jafari et al., 2006a). As one studentsays, “social software has been around for a while now and it’s a lot more user-friendly” (Becta,2007, p.16).Social software supports self-governed, problem-based and collaborative activities (Dalsgaard2006). Because social software is an emerging area, its definition is still fluid. Unsurprisingly,some of the best (or at least most faithful) definitions have been created through social softwaretools. The social software page on Wikipedia says that “[s]ocial software enables people torendezvous, connect or collaborate through computer-mediated communication” (Wikipedia2007). Social software includes a host of different technologies, including blogs (such asBlogger or Xanga), wikis (Wikipedia), social bookmarking (del.icio.us), podcasting (iTunes) and picture sharing (Flickr).Previous educational research on social software has focused on defining student characteristics,social software identifying properties (e.g. web 1.0 versus web 2.0), and appropriate pedagogicaluses, and has often resulted in recommendations for high-level strategic implementations(McAllister et al. 2007). Unfortunately, many academic researchers focus on identifying anddescribing emerging practices, rather than rigorously identifying their effectiveness and impact(McAllister et al. 2007).The United Kingdom’s lead agency for information and communication technology in educationhas found great potential to make connections between e-portfolio processes, such as storing,reflecting and publishing, and learners’ use of emerging social software tools used outside formaleducation (Becta 2007). In addition, McAllister et al. found that social software can provide bothformative and summative assessment (2007).

Theoretical Basis

This research project uses a design science approach. Design science builds upon HerbertSimon’s concept of the science of the artificial. Rather than attempting to theorize and justify,the cycle common to natural science research, I attempt instead to build and evaluate an ITartifact. My artifact will then act as an alternative model (or mindset) for the creation of portfolio software to support higher education. Design science research precisely includes thiskind of theorizing (March and Smith, 1995).Following Walls, Widmeyere and El Sawy (1992, 2004), I establish important requirements for a portfolio system and then connect those requirements to specific features. Not explicitlymentioned are the standard website design requirements, such as automated backups and a fast page response time (including level 1 and level 2 requirements from Zhang. 2007). These are allcritical dimensions, but ultimately are common to all online systems. Because a web portfolio3