Brandman University

Engaging Students with Assessment through Digital Portfolios

WASC Conference April 12, 2013

Dr. Carla Piper, Assessment Coordinator and Instructional Designer Dr. Kathy Theuer, Associate Dean and Director of Accreditation
1

Brandman University Assessment

Portfolio Binders

Digital Cameras

Brandman University Assessment

Personal Portfolio Journey
School-wide Title One Multiple Measures of Assessment Hyperstudio, Filemaker Pro, Access, and VHS Video Portfolios for 4th/5th Grade Language Arts Title One Students University of the Pacific Electronic Portfolio Dissertation 1998 - Hyperstudio Student Portfolios for Elementary Literacy Courses RICA Student Portfolios CSTPs Dreamweaver Archived on Floppy Disks and CDs Chapman University Student Portfolios CSTPs and TPEs Netscape Websites on Floppy Disks PowerPoint Portfolios Portfolio Blogs, Wikis, Epsilen, TeachPort Google Sites Portfolios Blackboard Portfolios Clipboard Early Childhood Portfolio - Developed Web-based Portfolio for Early Reading First Grants (NAEYC/California Competencies) - http://eport.soundpiper.com/clipboard/ TeachPort Developed Web-based Portfolio for TPEs, California Technology Standards for Level 1 & 2, and ISTE NETS - http://eport.soundpiper.com/teachport/ Brandman University Portfolios in LiveText - Program Portfolios and Faculty Report 3

Brandman University Assessment

Themes of Education Reform: Goals 2000

Teacher accountability to professional certification STANDARDS Authentic ASSESSMENT through portfolios demonstrating achievement of teaching and student learning objectives Develop skills with TECHNOLOGY by creating portfolio electronically Nation at Risk Report, 1983
As we move to more standards-based teacher performance assessment, we need new tools to record and organize evidence of successful teaching, for both practicing professionals and student teachers. Barrett (2000)

Brandman University Assessment

Dr. Helen Barrett: University of Alaska, Anchorage
Portfolio Development Electronic Portfolio Development Multimedia Development

Purpose & Audience Collect Interject Select Reflect Direct Inspect Perfect Connect Respect

1. Defining the Portfolio

2. The Working Portfolio 3. The Reflective Portfolio 4. The Connected Portfolio 5. The Presentation Portfolio

Decide Assess Design Plan Develop

Implement Evaluate Present Publish

Website - http://electronicportfolios.org/ Publications - http://electronicportfolios.org/portfolios.html Categories of ePortfolio Tools - http://electronicportfolios.org/categories.html

Brandman University Assessment

Two Conflicting Paradigms
Positivist Portfolios
The purpose of the portfolio is to assess learning outcomes and those outcomes are, generally, defined externally. Positivism assumes that meaning is constant across users, contexts, and purposes The portfolio is a receptacle for examples of student work used to infer what and how much learning has occurred.

Constructivist Portfolios
The portfolio is a learning environment in which the learner constructs meaning. It assumes that meaning varies across individuals, over time, and with purpose. The portfolio presents process, a record of the processes associated with learning itself; a summation of individual portfolios would be too complex for normative description.
6

REFLECT INITIATIVE WHITE PAPER -http://electronicportfolios.org/reflect/whitepaper.pdf

Brandman University Assessment

Balance Between Two Paradigms
How can we meet the needs of the students to support deep learning and also give the institution the information they need for assessment and reporting purposes? A learner-centered electronic portfolio using the learners authentic voice An institution-centered database to collect faculty-generated assessment data based on tasks and rubrics
How do we create an Institution-Centered Assessment and Accountability System without losing the power of the portfolio as a student-centered tool for lifelong learning and professional development? (Barrett, 2005)
REFLECT INITIATIVE WHITE PAPER -http://electronicportfolios.org/reflect/whitepaper.pdf
7

Brandman University Assessment

Dr. Helen Barrett: Balancing Accountability with Learning

Brandman University Assessment

Two Platforms
Digital portfolios have come of age as a viable means of assessment. Assessment Management System

Paired with our Learning Management System

Brandman University Assessment

Outcomes Assessment + Student Choice
University Assessment Purposes Met
Competency with learning outcomes and standards
Organized collection of student-generated work products or artifacts.

Personal reflections require that students think critically about the standards
Students choose and defend their artifacts as demonstrations of their own proficiency.

Expectations are clearly defined with a well-designed rubric

Linked to each web-based portfolio framework. Provides students with specific criteria and clearly defines the level of performance required.
10

Brandman University Assessment

Outcomes Assessment + Student Choice
Student Choice is Valued and Creativity is Encouraged
Students have a vested interest in demonstrating their own competencies Show stronger connections with standards and assessment through the LiveText portfolio process. Students express their competencies beyond the use of the written word Share rich multimedia performance-based artifacts. Students are able to experience professional and academic growth over time. A living, ever-changing record of student progress. Students recognize and celebrate their strengths Reflect on instructor comments Review the criteria for areas where they need improvement.

11

Brandman University Assessment

LiveText Portfolio Documents
LiveText portfolios live in students personal LiveText account as documents.
Can be copied over and over multiple versions Revised and edited for different purposes Resubmitted multiple times for different courses/programs Linked to other portfolios and projects Shared with others in the Brandman system Shared outside Brandman system through a visitors pass Store artifacts in File Manager
12

Brandman University Assessment

Copy Portfolio Documents

13

Brandman University Assessment

Signature Assignments Professional Dispositions Fieldwork Formative/Summative Assessments TPA Assessment (Teacher Performance Assessments) Course Portfolios Program Portfolios Rubric Assessments Projects/Assignments Forms and Surveys Standards Alignment

14

Brandman University Assessment

Highlight Cells in Rubric to Generate Data Used in Accreditation Reports and Program Documents

15

Brandman University Assessment

Types of Portfolios
Course Portfolios Demonstration of Mastery Portfolios Growth Over Time Employment

16

Brandman University Assessment

Course Portfolios
Technology Portfolio (ISTE NETS for Teachers) (EDUU 451/551) MAE in Instructional Technology Portfolio (ISTE NETS for Digital Age Coaches) (EDUU 564) Early Childhood Development Capstone Portfolio (NAEYC) (EDUU 465) Ed Admin Fieldwork Portfolio (EDAU 691) Ed Admin Tier II (EDAU 720) Autism (EDUU 677) Sped Level II- (EDUU 688) PPSC- (CSPU 618) PPSP- (CSPU 641)

17

Brandman University Assessment

Learner Control vs. Organizational Control
Assumption: Greater learner control and degree of student choice lead to more intrinsic student motivation and engagement in the portfolio process.

Motivation and Student Engagement = Student Buy-in

Student Email Responses:
Wow, it's nice to know my hard work mattered! I don't mind if you use my name and/or portfolio I was proud of that myself. Wow! Thank you for your email. I am honored to get this kind of compliment. It would be an honor to have you use screenshots of my work. Feel free to use my portfolio for your presentation. As I have just completed my application for the doctorate program beginning next year, it would be awesome to have my work out there Thank you so much for the opportunity. First I want to thank you for the compliment of you actually remembering me. I took your course at my first term at Brandman over a year ago. It is kind of you to remember me at all with the hundreds of students you have had between then and now. I will be glad if you present something I learned from Brandman. Wow, I'm just in shock and so very honored that you would want to use my work.
18

Brandman University Assessment

Technology Leaders ISTE NETS for Digital Age Coaches EDUU 564 Purpose: Demonstrate competence based on Program Learning Outcomes tied to standards Portfolios include: Information about the Candidate Reflections Responses analytical prompts Artifacts from coursework and fieldwork

19

Brandman University Assessment

Purpose: Demonstrate competence based on Program Learning Outcomes tied to standards Portfolios include: Information about the Candidate Reflections Responses analytical prompts Artifacts from coursework and fieldwork

Technology Leaders ISTE NETS for Digital Age Coaches EDUU 564

20

Brandman University Assessment

Purpose: Demonstrate competence based on Program Learning Outcomes tied to standards Portfolios include: Information about the Candidate Reflections Responses analytical prompts Artifacts from coursework and fieldwork

Early Childhood Development: NAEYC Professional Standards Capstone Portfolio EDUU 465
21

Brandman University Assessment

Purpose: Demonstrate competence based on Program Learning Outcomes tied to standards Portfolios include: Information about the Candidate Reflections Responses analytical prompts Artifacts from coursework and fieldwork

Level One Technology ISTE NETS for Teachers EDUU 451

22

Brandman University Assessment

Purpose: Demonstrate competence based on Program Learning Outcomes tied to standards Portfolios include: Information about the Candidate
Level One Technology ISTE NETS for Teachers EDUU 451

23

Brandman University Assessment

Demonstration of Mastery Portfolios Professional Teaching Portfolios California Teachers of English Learners Portfolio MA in Educational Leadership and Administration MA in Special Education Master of Arts in Education - Ed Leadership Master of Arts in Education -Teaching and Learning

24

Brandman University Assessment

Demonstration of Mastery Portfolios Purpose: Demonstrate competence based on Program Learning Outcomes tied to standards Portfolios include: Information about the Candidate Reflections Responses Analytical Prompts Artifacts from Coursework and Fieldwork
25

Brandman University Assessment

Demonstration of Mastery Portfolio Process Candidate completes online application form and is placed into the appropriate portfolio site Candidate builds portfolio in LiveText Technical Coordinator matches student with assessors Portfolio is assessed by multiple reviewers Technical Coordinator lets candidates know if they have passed Student records are updated accordingly
26

Brandman University Assessment

27

Brandman University Assessment

Portfolio Templates

28

29

Brandman University Assessment

Portfolio Template

30

Brandman University Assessment

Assigning Assessors

31

Brandman University Assessment

32

Brandman University Assessment

Successes
LiveText has allowed us to: Establish consistent expectations of instructors across multiple campuses Utilize multiple assessors for demonstration of mastery portfolios and teacher performance assessments Conduct blind reviews for portfolios and teacher performance assessments

33

Brandman University Assessment

Challenges
LiveText Training for Students and Instructors Communication with Adjuncts Tracking LiveText Applications, Registration, and Usage Multiple Assessors and Consistency Inter-rater Reliability Connections between Artifacts and Reflections Connections to Standards
Brandman Center for Instructional Innovation CII
http://www.brandman.edu/cii/Pages/default.aspx MISSION: The mission of the Center for Instructional Innovation is to promote and enhance teaching and learning effectiveness. VISION: (work in progress) To serve authentic learning by combining faculty expertise with collaborative, meaningful, empowering, on-going professional development. 34

Brandman University Assessment

Tutorials: Instructor Index

https://sites.google.com/site/livetexttutorials/instructor-index Dr. Carla Piper piper@brandman.edu
35

Brandman University Assessment

Brandman University http://www.brandman.edu LiveText: http://www.livetext.com Dr. Carla Piper piper@brandman.edu Dr. Kathy Theuer theuer@brandman.edu
earning Outcomes Piper Digital Portfolio Site: https://sites.google.com/site/digiportfolio/ LiveText Tutorials: https://sites.google.com/site/livetexttutorials/ Tutorials: Instructor Index: https://sites.google.com/site/livetexttutorials/instructor-index Tutorials: Electronic Portfolios https://sites.google.com/site/livetexttutorials/generic-portfolio-tutorial
36