You are on page 1of 634

10th International Conference of the Learning Sciences

The Future of Learning

Proceedings
Volume 2 - Short Papers, Symposia, & Abstracts
July 2 nd - 6 th, 2012
July 2-6, 2012 Sydney, Australia www.isls.org/icls2012
SPONSORED BY: HOSTED BY:

Centre for Research on Computer Supported Learning & Cognition (CoCo)

ICLS 2012 Conference Proceedings Volume 2 – Short Papers, Symposia, and Abstracts
10th International Conference of the Learning Sciences The University of Sydney, Sydney, NSW, AUSTRALIA July 2nd – 6th, 2012, Sydney

THE FUTURE OF LEARNING

Editors: Jan van Aalst, Kate Thompson, Michael J. Jacobson, and Peter Reimann

Title:

Cite as:

Editors:

ISBN: 978-0-578-10704-2

© 2012, INTERNATIONAL SOCIETY OF THE LEARNING SCIENCES [ISLS] Copyright 2012 International Society of the Learning Sciences, Inc. Rights reserved

van Aalst, J., Thompson, K., Jacobson, M. J., & Reimann, P. (Eds.) (2012). The Future of Learning: Proceedings of the 10th International Conference of the Learning Sciences (ICLS 2012) – Volume 2, Short Papers, Symposia, and Abstracts. International Society of the Learning Sciences: Sydney, NSW, AUSTRALIA

The Future of Learning: Proceedings of the 10th International Conference of the Learning Sciences (ICLS 2012) – Volume 2, Short Papers, Symposia, and Abstracts

Jan van Aalst, Kate Thompson, Michael J. Jacobson, and Peter Reimann

Published by: International Society of the Learning Sciences (ISLS) Proceedings printed by: LuLu http://www.lulu.com Proceedings distributed by: LuLu and Amazon Book Cover Design: Dorian Peters Book Cover Photo: Christine Bree

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the International Society of the Learning Sciences. The International Society of the Learning Sciences is not responsible for the use which might be made of the information contained in this book. http://www.isls.org

OUR SPONSORS

ICLS2012 would like to thank our sponsors:
• • •

NSW Trade & Investment The Centre for Research on Computer Supported Learning and Cognition (CoCo) Smart Services Collaborative Research Centre

The Early Career workshop and Doctoral Consortium would particularly like to thank the following for their support:
• • • National Science Foundation The International Society of the Learning Sciences Asia-Pacific Society of Computers in Education

We would also like to thank the following for their generosity in donating their time, skills and information to the conference.
• Inquirium

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

ORGANISERS AND COMMITTEES CONFERENCE CHAIRS
• Michael J Jacobson | The University of • Wouter Van Joolingen | University of

Twente, the Netherlands
• Anindito Aditomo | The University of

Sydney, Australia • Peter Reimann | The University of Sydney, Australia CONFERENCE ADVISORY BOARD
• Shaaron • • • • • • • • • • •

Sydney, Australia EARLY CAREER WORKSHOP
• Tom Moher | University of Illinois, United

States
• Anne Newstead | The University of Sydney,

• •

Ainsworth | University of Nottingham, UK Michael Baker | Telecom ParisTech, Paris, France Katerine Bielaczyc | Nanyang Technological University, Singapore Paul Chandler | Wollongong University, Australia Susan Goldman | University of Illinois, United States Kai Hakkarainnen | University of Helsinki, Finland Yasmin Kafai | University of Pennsylvania, United States Paul Kirschner | Open University of the Netherlands, the Netherlands Marcia Linn | University of California, United States Ric Lowe | Curtin University, Australia Naomi Miyake | University of Tokyo, Japan Stella Vosniadou | University of Athens (Greece) and University of Adelaide (Australia) Uri Wilensky | Northwestern University, United States James Pellegrino | University of Illinois, United States

Australia
• Tak-Wai Chan | National Central Taiwan

University, Taiwan PRE-CONFERENCE WORKSHOPS AND SPECIAL SESSIONS
• Lina Markauskaite | The University of

Sydney, Australia
• Dan Suthers | University of Hawaii, United

States
• Carol Chan | The University of Hong Kong,

Hong Kong
• Nikol Rummel | Ruhr-Universitat Bochum,

Germany PAPER REVIEW COORDINATION
• Peter Reimann | The University of Sydney,

Australia
• Peter Freebody | The University of Sydney,

Australia Kyza | Cyprus University of Technology, Cyprus • Ton de Jong | University of Twente, the Netherlands • Nick Kelly | The University of Sydney, Australia
• Eleni

PROCEEDINGS
• Jan van Aalst | The University of Hong

Kong, Hong Kong
• Brian J. Reiser | Northwestern University,

United States
• Cindy Hmelo-Silver | Rutgers University,

United States
• Kate Thompson | The University of Sydney,

LOGISTICS AND ADMINISTRATION SUPPORT • Sadhbh Warren | The University of Sydney, Australia • Sadhana Puntambekar | The International Society of the Learning Sciences • Dana Gnesdilow | The International Society of the Learning Sciences

Australia
• Stella Wen Tian | The University of Hong

Kong, Hong Kong DOCTORAL CONSORTIUM
• Susan Yoon | University of Pennsylvania,

United States

© ISLS

Page vi

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

PAPER REVIEWERS Abrahamson, Dor, University of California, United States Aditomo, Anindito,, The University of Sydney, Australia Ahn, June, University of Maryland, United States Ainsworth, Shaaron, University of Nottingham, United Kingdom Alac, Morana, University of California, United States Alonzo, Alicia, Michigan State University, United States Alterman, Rick, Brandeis University, United States Andriessen, Jerry, Utrecht University, The Netherlands Ares, Nancy, University of Rochester, United States Arnseth, Hans Christian, University of Oslo, Norway Arthur-Kelly, Michael, University of Newcastle, Australia Asterhan, Christa, Hebrew University of Jerusalem, Israel Avramides, Katerina, London Knowledge Lab, United Kingdom Bagley, Elizabeth, University of Illinois, United States Bairral, Marcelo, Federal Rural University of Rio de Janeiro, Brazil Baker, Michael, CNRS - Telecom ParisTech, France Baker, Ryan, Worcester Polytechnic Institute, United Kingdom Barnes, Jacqueline, Indiana University, United States Barth-Cohen, Lauren, University of California, United States Belland, Brian, Utah State University, United States Berland, Leema, University of Texas, United States Bers, Marina, Tufts University, United States Bielaczyc, Katerine, Singapore National Institute of Education, Singapore Booker, Angela, University of California, United States Brett, Clare, University of Toronto, Canada Brian, Reiser, Northwestern University, United States Buckingham, Brandy, Northwestern University, United States Burke, Quinn, University of Pennsylvania, United States Calabrese Barton, Angela, Michigan State University, United States Carmela, Aprea, Swiss Federal Institute for Vocational Education and Training, Switzerland Carpendale, Jeremy, Simon Fraser University, Canada Castro-Alonso, Cris, University of New South Wales, Australia Cesar, Collazos, Universidad del Cauca, Colombia Chan, Carol, The University of Hong Kong, Hong Kong Chang, Yi-Hsing, Southern Taiwan University, Taiwan Charoenying, Timothy, University of California, United States Chase, Catherine, Carnegie Mellon University, United States Chen, Ying-Chih, University of Minnesota, United States Childs, Joshua, University of Pittsburgh, United States Ching, Cynthia Carter, University of California, United States Chinn, Clark, Rutgers University, United States Christophe, Reffay, ENS Cachan, France Chye, Stefanie, Singapore National Institute of Education, Singapore Clegg, Tamara, University of Maryland, United States Cooper, Benny, University of California, United States Correia, Ana-Paula, Iowa State University, United States Crain, Rhiannon, Cornell University, United States Cress, Ulrike, Knowledge Media Research Center, Germany D'Angelo, Cynthia, University of Wisconsin, United States Damsa, Crina, Utrecht University, The Netherlands Davis, Pryce, Northwestern University, United States De Jong, Ton, University of Twente, The Netherlands de Vries, Erica, University of Grenoble, France Dillenbourg, Pierre, Swiss Federal Institute of Technology, Lausanne, Switzerland

© ISLS

Page vii

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

Drago, Kathryn, University of Michigan, United States Dugan, Therese, University of Washington, United States Duncan, Ravit, Rutgers University, United States Durga, Shree, University of Wisconsin, United States Dyke, Gregory, Carnegie Mellon University, United States Earnest, Darrell, University of California, United States Ehret, Christian, Vanderbilt University, United States Eisenberg, Michael, University of Colorado, United States Elen, Jan, K.U.Leuven, Belgium Engelmann, Tanja, Knowledge Media Research Center, Germany Engle, Randi A., University of California, United States Enyedy, Noel, University of California, United States Erkens, Gijsbert, Utrecht University, The Netherlands Erlandson, Benjamin, Arizona State University, United States Ertl, Bernhard, Universität der Bundeswehr, Germany Fatos, Xhafa, Universitat Politecnica de Catalunya, Spain Femke, Kirschner, Utrecht University, The Netherlands Fields, Deborah, University of California, United States Fischer, Frank, University of Munich, Germany Fishman, Barry, University of Michigan, United States Forest, Dominique, Universitat de Brest, France Freebody, Peter, The University of Sydney, Australia Frode, Guriybe, University of Bergen, Germany Furtak, Erin Marie, University of Colorado, United States Gegenfurtner, Andreas, Technical University of Munich, Germany Gerofsky, Susan, University of British Columbia, Canada Gillen, Julia, Lancaster University, United Kingdom Ginns, Paul, The University of Sydney, Australia Goldman, Susan R, University of Illinois, United States

Goldman, Ricki, New York University, United States Gomez, Kimberley, University of Pittsburgh, United States Gomez, Kimberley, University of California, United States Gonzalez, Carlos, Pontificia Universidad Catolica, Chile Gresalfi, Melissa, Indiana University, United States Groza, Gabriela, University of Illinois, United States Gruson, Brigitte, IUFM de Bretagne, France Gutierrez, Jose, University of California, United States Hajime, Shirouzu, Chukyo University, Japan Hakkarainen, Kai, University of Turku, Finland Halverson, Erica, University Of Wisconsin, United States Halverson, Richard, University of Wisconsin, United States Hawi, Nazir, Notre Dame University, Lebanon Herman, Phillip, University of Pittsburgh, United States Herold, David, Hong Kong Polytechnic University, Hong Kong Herrmann, Thomas, University of Bochum, Germany Hershkovitz, Arnon, Worcester Polytechnic Institute, United Kingdom Hesse, Friedrich, Knowledge Media Research Center, Germany Hickey, Daniel, Indiana University, United States Hmelo-Silver, Cindy, Rutgers University, United States Hoadley, Christopher, Penn State University, United States Holbert, Nathan, Northwestern University, United States Hong, Huang-Yao, National Chengchi University, Taiwan Hora, Matthew, University of Wisconsin, United States Horn, Ilana, Vanderbilt University, United States Howard, Sarah, University of Wollongong, Australia Hulshof, Casper, University of Utrecht, The Netherlands Inge, Molenaar, University of Amsterdam, The Netherlands Ingerman, Åke, University of Gothenburg, Sweden Jackson, Kara, McGill University, Canada
Page viii

© ISLS

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

Jacobson, Michael, The University of Sydney, Australia Jamaludin, Azilawati, National Institute of Education, Singapore Jay, Tim, University of Bristol, United Kingdom Jochen, Rick, Saarland University, Germany Joiner, Richard, University of Bath, United Kingdom Jurow, A. Susan, University of Colorado, United States Kafai, Yasmin, University of Pennsylvania, United States Kanselaar, Gellof, Utrecht University, The Netherlands Kapur, Manu, National Institute of Education, Singapore Keifert, Danielle, Northwestern University, United States Kimmerle, Joachim, University of Tuebingen, Germany King Chen, Jennifer, University of California, United States Kirschner, Paul, Open University, The Netherlands Kollöffel, Bas, University of Twente, The Netherlands Kollar, Ingo, University of Munich, Germany Koschmann, Timothy, Southern Illinois University, United States Kraemer, Nicole, University Duisburg-Essen, Germany Krauskopf, Karsten, Knowledge Media Research Center, Germany Kwon, Samuel, Learning Point Associates, United States Kyza, Eleni, Cyprus University of Technology, Cyprus Lai, Jiang, Katholieke University of Leuven, Belgium Lakkala, Minna, University of Helsinki, Finland Langer-Osuna, Jennifer, University of Miami, United States Law, Nancy, University of Hong Kong, Hong Kong Lazonder, Ard, University of Twente, The Netherlands Leary, Heather, University of Colorado, United States Lee, Kyungmee, University of Toronto, Canada Lee, Chien Sing, National Central University, China Lee, Tiffany, University of Washington, United States Lee, Victor, Utah State University, United States

Lehrer, Rich, Vanderbilt University, United States Lei, Jing, Syracuse University, United States Liao, Chang-Yen, National Central University, China Liesbeth, Kester, Open University of the Netherlands, The Netherlands Limon, Margarita, University Autonoma of Madrid, Spain Lin, Hsien-Ta, National Chengchi University, Taiwan Lindgren, Robb, University of Central Florida, United States Lindwall, Oskar, University of Gotheburg, Sweden Liu, Shiyu, University of Minnesota, United States Liu, Ru-De, Beijing Normal University, China Lone, Dirckinck-Holmfeld, Aalborg University, Denmark Lonn, Steven, University of Michigan, United States Looi, Chee Kit, National Institute of Education, Singapore Louca, Loucas T., European University, Cyprus Lowe, Ric, Curtin University, Australia Lozano, Maritza, University of California, United States Luckin, Rose, London Knowledge Lab, United Kingdom Ludvigsen, Sten, University of Oslo, Norway Lund, Kristine, CNRS, France Magnifico, Alecia Marie, University of Illinois United States Magno, Carlo, De La Salle University, Philippines Manches, Andrew, University of Nottingham, United Kingdom Martell, Sandra, National Science Foundation, United States Martin, Crystle, University of Wisconsin, United States McElhaney, Kevin, University of California, United States McGee, Steven, The Learning Partnership, Canada McKenney, Susan, Open University of the Netherlands, The Netherlands Medina, Richard, University of Hawaii, United States Mercier, Emma, Durham University, United Kingdom Miyake, Naomi, the University of Tokyo, Japan Moher, Tom, University of Illinois, United States
Page ix

© ISLS

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

Moore, Joyce, University of Iowa, United States Mor, Yishay, Open University, United Kingdom Morch, Anders, University of Oslo, Norway Moskaliuk, Johannes, Knowledge Media Research Center, Germany Munoz-Baell, Irma M, University of Alicante, Spain Myint Swe, Khine, Bahrain Teachers College, Bahrain Nückles, Matthias, University of Freiburg, Germany Naidoo, Jayaluxmi, University of KwaZuluNatal, South Africa Nicolau, Juan L., University of Alicante, Spain Noroozi, Omid, Wageningen University, The Netherlands O'Malley, Claire, University of Nottingham, United Kingdom O'Neill, Kevin, Simon Fraser University, Canada Oeberst, Aileen, Knowledge Media Research Center, Germany Olimpo, Jeffrey, University of Maryland, United States Opfermann, Maria, University of DuisburgEssen, Germany Orrill, Chandra, University of Massachusetts, United States Oshima, Jun, Shizuoka University, Japan Oztok, Murat, University of Toronto, Canada Paavola, Sami, University of Helsinki, Finland Papademetri-Kachrimani, Chrystalla, European Univesrity Cyprus, Cyprus Parisio, Martin, The University of Sydney, Australia Pea, Roy, Stanford University, United States Pedaste, Margus, University of Tartu, Estonia Penuel, William, University of Colorado, United States Pluta, William, Rutgers, The State University of New Jersey, United States Polman, Joe, University of Missouri, United States Potgieter, Marietjie, University of Pretoria, South Africa Prins, Frans, Utrecht University, The Netherlands Radinsky, Josh, University of Illinois, United States Raes, Annelies, Ghent University, Belgium Ranney, Michael, University of California, United States Reber, Rolf, University of Bergen, Norway Reimann, Peter, The University of Sydney, Australia
© ISLS

Renninger, K. Ann, Swarthmore College, United States Rivet, Ann, Columbia University, United States Rogotneva, Elena, Tomsk Polytechnic University, Russia Rose, Carolyn, Carnegie Mellon University, United States Rummel, Nikol, University of Freiburg, Germany Rusman, Ellen, Open University of the Netherlands, The Netherlands Ryan, Stephanie, University of Illinois, United States Saleh, Asmalina, Indiana University, United States Sandoval, William, University of California, United States Sayre, Eleanor, Kansas State University, United States Schnotz, Wolfgang, University of KoblenzLandau, Germany Schoerning, Emily, University of Iowa, United States Schwarz, Baruch, Hebrew University, Israel Schwendimann, Beat, The University of Sydney, Australia Scott Curwood, Jen, University of Wisconsin, United States Seifert, Colleen, Univ. of Michigan, United States Sensevy, Gérard, University of Western Brittany, France Shapiro, Amy, University of Massachusetts, United States Shelton, Brett, Utah State University, United States Singer, Susan, Carleton College, United States Slof, Bert, Utrecht University, The Netherlands Smith, Debbie, Clemson University, United States Solomou, Maria, Indiana University, United States Spada, Hans, University of Freiburg, Germany Specht, Marcus, Open University, The Netherlands Stager, Sarah, Pennsylvania State University, United States Stevens, Reed, Northwestern University, United States Stieff, Mike, University of Illinois, United States Strijbos, Jan-Willem, Ludwig-MaximiliansUniversity München, Germany Sudol-DeLyser, Leigh Ann, Carnegie Mellon University, United States

Page x

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

Suthers, Daniel, University of Hawaii, United States Tan, Seng-Chee, National Institute of Education, Singapore Taylor, Martin, University of Texas, United States Tchounikine, Pierre, University of Grenoble, France Tee, Meng Yew, University of Malaya, Malaysia Thayer, Alexander, University of Washington, United States Thompson, Kate, The University of Sydney, Australia Tok, ükran, Pamukkale University, Turkey Trausan-Matu, Stefan, University "Politehnica" of Bucharest, Romania Trninic, Dragan, University of California, United States Tung, I-Pei (Vicky), McGill University, Canada Turns, Jennifer, University of Washington, United States Tzialli, Dora, European University Cyprus, Cyprus van Aalst, Jan, The University of Hong Kong, Hong Kong van Amelsvoort, Marije, Tilburg University, The Netherlands van Es, Beth, University of California, United States van Oers, Bert, University Amsterdam, The Netherlands van Oostendorp, Herre, Utrecht University, The Netherlands Vandewaetere, Mieke, K.U. Leuven, Belgium Vanover, Charles, University of South Florida, United States Vatrapu, Ravi, Copenhagen Business School, Denmark Venkataswamy, Arjun, Community Links High School, United States Virnes, Marjo, University of Eastern Finland, Finland Volker, Wulf, University of Siegen, Germany Walker, Richard, The University of Sydney, Australia Walkington, Candace, University of Wisconsin, United States

Wang, Tsungjuang, National Taipei University of Technology, Taiwan Wardrip, Peter, Northwestern University, United States Warren, Scott, Indiana University, United States Wecker, Christof, University of Munich, Germany Wee, Juan Dee, National Institute of Education, Singapore Weinberger, Armin, Saarland University, Germany Weinstock, Michael, Ben-Gurion University of the Negev, Israel Wessel, Daniel, Knowledge Media Research Center, Germany Wessner, Martin, Fraunhofer IESE, Germany White, Tobin, University of California, United States Wichmann, Astrid, Ruhr University Bochum, Germany Williams, Robert, Lawrence University, United States Wodzicki, Katrin, Knowledge Media Research Center, Germany Wong, Lung-Hsiang, National Institute of Education, Singapore Wouters, Pieter, Utrecht University, The Netherlands Wu, Hsin-Kai, National Taiwan Normal University, Taiwan Yael, Kali, University of Haifa, Israel Yeo, Ai Choo Jennifer, National Institute of Education, Singapore Yilmaz, Kaya, Marmara University, Turkey Young, Michael, University of Connecticut, United States Zagal, Jose, DePaul University, United States Zahn, Carmen, Knowledge Media Research Center, Germany Zannetou, Marilyn, EUC European University of Cyprus, Cyprus Zhang, Jianwei, University at Albany, United States Zingaro, Daniel, University of Toronto, Canada Zywica, Jolene, University of Pittsburgh, United States

© ISLS

Page xi

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

ABOUT ISLS

ISLS is a professional society dedicated to the interdisciplinary empirical investigation of learning as it exists in real-world settings and how learning may be facilitated both with and without technology. ISLS sponsors two professional conferences, held in alternate years. The International Conference of the Learning Sciences (ICLS), first held in 1991 and held biannually since 1996, covers the entire field of the learning sciences. http://www.isls.org/

ABOUT "COCO" AT THE UNIVERSITY OF SYDNEY

ICLS 2012 is hosted by the Centre for Research on Computer Supported Learning and Cognition (CoCo) at the University of Sydney. CoCo's mission is to contribute to theory and research in the field of the learning sciences in order to discover how innovative learning technologies and pedagogical approaches can enhance formal and informal learning. CoCo is a University of Sydney Research Centre operating within the Faculty of Education and Social Work. http://sydney.edu.au/edsw/coco

© ISLS

Page 12

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

PREFACE

Michael J. Jacobson, Peter Reimann, and Jan van Aalst The international and interdisciplinary field of the learning sciences brings together researchers from the fields of cognitive science, educational research, psychology, computer science, artificial intelligence, anthropology, neuroscience, and other fields to study learning in a wide variety of formal and informal contexts. This field emerged in the late 1980s and early 1990s, with the first International Conference of the Learning Sciences (ICLS) being held in 1991 at Northwestern University in Evanston, Illinois, USA. Subsequent meetings of ICLS were held again in Evanston, USA (1996), and then in Atlanta, USA (1998), Ann Arbor, USA (2000), Seattle, USA (2004), Santa Monica, USA (2004), and Bloomington, USA (2006). The first ICLS to be held outside of North America was in Utrecht, the Netherlands (2008), and then back to the USA in Chicago (2010). The ICLS 2012 in Sydney is the first hosting of the conference in the Asia-Pacific region. Papers for this conference were submitted in November 2011, and then went through a process of peer review. Full papers and symposia submissions received three anonymous reviews with a member of the Program Committee summarizing the reviews and making a recommendation. Short paper poster submissions went through the same process, except with two anonymous reviews. See Table 1 for a summary of the conference paper statistics. Table 1. Paper submissions, acceptance, and rejection rates.
Full papers 264 Accepted as Full Papers: 65 (25%) Accepted as Short Paper: 54 (20%) Accepted as Poster: 42 (16%) Rejected: 103 (39%) Short Papers 79 Accepted: 15 (19%) Accepted as poster: 11 (14%) Rejected: 53 (67%) Posters 76 Accepted: 37 (49%) Rejected: 39 (51%) Symposia 27 Accepted: 18 (67%) Rejected: 9 (33%) Total Submissions: 446

The final papers included in the ICLS 2012 Proceedings are for 18 symposia, 60 full papers, 61 short papers, and 62 poster papers, as well as abstracts for the four keynote talks and three special sessions, including the invited Presidential Session on The Future of Learning, and nine workshops. The themes reflected in the papers and presentations at ICLS 2012 cover a wide range of issues and research areas. Some papers deal with long standing theoretical issues, such as conceptual change and knowledge transfer, whereas other papers report on new learning research in conventional subject areas such as science, mathematics, and literacy. Research is also reported on newer knowledge areas such as complex systems, as well as more recent
© ISLS Page xiii

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

perspectives on pedagogy and learning such as knowledge building, inquiry, and productive failure. A number of learning sciences research projects internationally are also exploring ways in which innovative environments for learning may be designed with new technologies such virtual and game environments, modeling and visualization systems, robotics, collaboration technologies, mobile and hand held devices, and educational data mining and learning analytics. Emerging areas in learning sciences research that perhaps may lead to newer perspectives and directions in our understanding of learning include creatively, identity, and embodied cognition. The field continues to engage the broad issues of contributing to and impacting policy and practice more generally. Overall, the research presented in the proceedings of ICSL 2012 contributes numerous perspectives on the conference theme of the future of learning, both as trajectories of research that have been maturing over a number of years and bold new perspectives that promise to shape new trajectories for the future. Making this conference possible, we thank the hard work and the countless hours put in by our international Conference Advisory Board and the various conference subcommittees. We would like to thank all those who assisted with the review process. The Conference is also fortunate to have a number of financial sponsors whose support contributes to conference events: NSW Trade & Investment, Smart Services Collaborative Research Centre, United States National Science Foundation, The International Society of the Learning Sciences, Asia-Pacific Society of Computers in Education, Inquirium, LLC, and The Centre for Research on Computer Supported Learning and Cognition. Any conference is but a snapshot in the dynamic process of articulating and vetting scientifically principled ideas and approaches. That the International Conference of the Learning Sciences has now had its 10th meeting and has entered its third decade are exciting milestones that this proceedings helps to document. We close by reflecting that just as a great movie is about the journey of discovery and development that the characters experience, so might our research field be a journey of discovery and development to more deeply understand how people learn now and as the future unfolds, knowing learning itself as a core essence of life and our humanity.

© ISLS

Page xiv

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

CONTENTS - VOLUME 2

KEYNOTES
Envisioning the Next Generation Classroom and the Next Generation of Learning Technologies
Janet L. Kolodner

1

Productive Failure
Manu Kapur

2 3 4

Lifelong Learning as a Driver for Designing Pervasive Technologies
Judy Kay

Classroom Orchestration: Interweaving Digital and Physical Workflows
Pierre Dillenbourg

INVITED SPECIAL SESSIONS
Invited Presidential Session: The Future of Learning and the Learning Sciences
Susan R. Goldman, Louis M. Gomez, Naomi Miyake, Nichole D. Pinkard, Nikol Rummel

5

Invited Panel Sponsored by the ISLS Membership Committee: Linking Learning Sciences Communities across the World
Nancy Law, Baohui Zhang, Konstantin К. Kolin, Paulo Blikstein, Miguel Nussbaum, Michael Jacobson

7

SYMPOSIA
Hybrid Spaces for Science Education
Ole Smørdal, Jim Slotta, Ingeborg Krange, Tom Moher, Francesco Novellis, Alessandro Gnoli, Brenda Lopez Silva, Michelle Lui, Alfredo Jornet, Cecilie F. Jahreie

9

Building Upon What Is Already There: The Role of Prior Knowledge, Background Information, and Scaffolding in Inquiry Learning
Christof Wecker, Ard Lazonder, Jennifer Chiu, Cheryl Madeira, Jim Slotta, Yvonne Mulder, Ton de Jong, Alexander Rachel, Hartmut Wiesner, Peter Reimann

17

Assessing Interests in the Service of Supporting Personalized Learning Through Networked 25 Resources
Brigid J. Barron, Robert B. W. Ely, Mary D. Ainley, K. Ann Renninger

Supporting Teachers in Capturing and Analyzing Learning Data in the Technology-Rich Classroom
Peter Reimann, Friedrich Hesse, Gabriele Cerniak, Rose Luckin, Ravi Vatrapu, Susan Bull, Matthew Johnson, Wolfgang Halb, Wilfrid Utz, Michal Kossowski

33

Gathering Evidence of Scientific Argumentation Practices: From Pre-Kindergarten to High 41 School
Tiffany Lee, Hiroki Oura, Giovanna Scalone, Kari Shutt, John Bransford, Andrew Shouse, Katie Van Horne, Nancy Vye, Maureen Munn, Randy Knuth

Engaging Middle School-Aged Students in Classroom Science and Mathematics: Implications for Design and Research
Kimberley Pressick-Kilborn, Melissa Gresalfi, K. Ann Renninger, Jessica Bachrach, Nicole Shechtman, Britte Cheng, Patrik Lundh, Gucci Trinidad, Richard Walker

49

Robot Facilitation as Dynamic Support for Collaborative Learning
Naomi Miyake, Sandra Okita, Carolyn Rose

57

Embedded Phenomena for Knowledge Communities: Supporting complex practices and interactions within a community of inquiry in the elementary science classroom Rebecca Cober, Cresencia Fong, Alessandro Gnoli, Brenda López Silva, Michelle Lui, Cheryl

64

© ISLS

Page xv

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

Madeira, Colin McCann, Tom Moher, Jim Slotta, Mike Tissenbaum

Critical Aspects in Learning with Technologies
Ching Sing Chai, Huang Yao Hong, Hwee Ling Koh, Naomi Miyake, Chwee Beng Lee, Choon Lang Quek, Peter Reimann

72

Interactive Surfaces and Spaces: A Learning Sciences Agenda
Michael A. Evans, Jochen Rick, Michael Horn, Chia Shen, Emma Mercier, James McNaughton, Steve Higgins, Mike Tissenbaum, Michelle Lui, James D. Slotta

78

Instructional Approaches to Promote Conceptual Change
Stella Vosniadou, Naomi Miyake, Marcia Linn, Stellan Ohlsson, David D. Cosejo, Michael J. Jacobson, Doug Clark

86

Everyday Interactions and Activities: Field Studies of Early Learning Across Settings
Reed Stevens, Lauren Penney, Danielle Keifert, Pryce Davis, Siri Mehus, RichLehrer

91 99

You're It! Body, Action, and Object in STEM Learning
Dor Abrahamson, Carmen Petrick ,David DeLiema, Mina C. Johnson–Glenberg, David Birchfield, Tatyana Koziupa, Caroline Savio-Ramos, Julie Cruse, Robb Lindgren, Cameron Fadjo, John B. Black, Michael Eisenberg

The Future of Assessment: Measuring Science Reasoning and Inquiry Skills Using Simulations and Immersive Environments
Jodi Davenport, Edys Quellmalz, Jody Clarke-Midura, Chris Dede, Janice Gobert, Kenneth Koedinger, Marty McCall, Michael Timms

110

Scripting Science Inquiry Learning in CSCL Classrooms
Annelies Raes, Tammy Schellens, Bram De Wever, Ingo Kollar, Christof Wecker, Frank Fischer, Mike Tissenbaum, Jim Slotta, Vanessa L. Peters, Nancy Songer

118

Learning as Identity Formation: Implications for Design, Research, and Practice
Rick Jochen, Ben DeVane, Tamara Clegg, Vanessa Peters, Nancy Songer, Susan R Goldman, Cindy Hmelo-Silver

126

Building Bridges between Learning Analytics, Educational Data Mining and Core Learning 134 Sciences Perspectives
Paulo Blikstein, Marcelo Worsley, Bruce Sherin, Ryan Baker, Matthew Berland, Taylor Martin, Ido Roll, Vincent Aleven, Ken Koedinger, Arnon Hershkovitz

The Use of Game Design, Social Learning Networks, and Everyday Expertise to Engage Youth with Contemporary Science
Philip Bell, Leah Bricker, Katie Van Horne, Theresa Horstman, Nichole Pinkard

142

SHORT PAPERS
Developing Primary Students' Argumentation Skills in Inquiry-Based Mathematics Classrooms
Jill Fielding-Wells, Katie Makar

149

Language Formality, Learning Environments, and Student Achievment
Emily Schoerning, Brian Hand

154

Memetic Processes as Conceptual Framework for Idea Improvement in Knowledge Building 157
Karsten Krauskopf, Johanna Bertram, Ya Ping (Amy) Hsiao, Stefan Huber, Katherine Panciera, Nicole Sträfling, Astrid Wichmann, Jan van Aalst

Unraveling Idea Development in Discourse Trajectories
Iassen Halatchliyski, Aileen Oeberst, Martina Bientzle, Franziska Bokhorst, Jan van Aalst

162

Investigating Emergent Dynamics to Understand Interactions in Small Professional Development Groups
Susan Yoon

167

What a Long Strange Trip It's Been: A Comparison of Authors, Abstracts, and References in 172 the 1991 and 2010 ICLS Proceedings
© ISLS Page xvi

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

Victor Lee, Lei Ye, Mimi Recker

Scaffolding Collaborative Sensemaking during Critique of Explanations in TechnologyEnhanced Science Curriculum
Elissa Sato, Vanessa Svihla

177

Collaborative Reading Comprehension with Communication Robots as Learning Partner
Jun Oshima, Ritsuko Oshima, Naomi Miyake

182

Multi-Touch Technology to Support Multiple Levels of Collaborative Learning in the Classroom
Emma Mercier, Steve Higgins, Elizabeth Burd, Andrew Joyce-Gibbons

187

The SunBay Digital Mathematics Project: An Infrastructural and Capacity-Based Approach 192 to Improving Mathematics Teaching and Learning at Scale
Charles Vanover, George Roy, Zafer Unal, Vivian Fueyo, Phil Vahey

Measuring Sophistication of Epistemic Beliefs Using Rasch Analysis
Johan van Strien, Monique Bijker, Saskia Brand-Gruwel, Henny P. A. Boshuizen

197 202

Establishing a Mathematical Practice in a Middle School Classroom
Marta Kobiela, Rich Lehrer

Redesigning Classroom Learning Spaces: When Technology Meets Pedagogy and When They Clash
Elizabeth Charles, Nathaniel Lasry, Chris Whittaker

207

Bridging Design and Practice: Towards a Model-based Collaborative Inquiry Science Learning Environment
Daner Sun, Chee-kit Looi, Baohui Zhang

212

Identity and Digital Media Production in the College Classroom
Erica Halverson, Michelle Bass

217

Mathematics Learning in a Racial Context: Unpacking Students' Reasoning about "Asians are Good at Math"
Niral Shah

222

Dimensions of Scaffolding in Technology-Mediated Discovery Learning Environments
Mark J. W. Lee, Barney Dalgarno

227

How a Power Game Shapes Expressing Opinions in a Chat and in an Argument Graph during a Debate: A Case Study
Gaelle Molinari, Kristine Lund

232

Using the Learner-Generated Drawing Strategy: How Much Instructional Support Is Useful? 237
Annett Schmeck, Luisa Amelie Friedrich, Maria Opfermann, Detlev Leutner

Unpacking traces of collaboration from multimodal data of collaborative concept mapping at 241 a tabletop
Roberto Martinez Maldonado, Judy Kay, Kalina Yacef, Beat Schwendimann

Motivation in the Learning Sciences: Connecting to Practice by Design
Phillip Herman, Peter Wardrip, Jolene Zywica

246

Supporting Student Learning using Conversational Agents in a Teachable Agent Environment
James Segedy, John Kinnebrew, Gautam Biswas

251

Epistemic Mentoring in Virtual and Face-to-Face Environments
Elizabeth Bagley, David Williamson Shaffer

256

Two Models of Authenticity: Signature Pedagogy, Problem Based Learning, and Cultural Context

261

© ISLS

Page xvii

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

Moshe Krakowski

New Evidence on Productive Failure - Building on Students' Prior Knowledge is Key!
Katharina Westermann, Nikol Rummel

266

Explanandum and visibility condition change children's gesture profiles during explanation: 271 implications for learning?
Audrey Mazur-Palandre, Kristine Lund

Bidirectional artifact analysis: A method for analyzing creative processes
Alecia Marie Magnifico, Erica Rosenfeld Halverson

276

Learning from the Folly of Others: Learning to Self-Correct by Monitoring the Reasoning of 281 Projective Pedagogical Agents
Sandra Okita, Azadeh Jamalian

Understanding Influence in Collaborative Group Work: The Importance of Artifacts
Lesley Dookie, Indigo Esmonde

286

Theoretical Issues: Indicators of Decentralized and Centralized Causality as a Gauge for Students Understanding of Complex Systems
Lauren Barth-Cohen

291

Engaging in Design: Reflections of Young Paper Engineers
Rob Rouse, Richard Lehrer

296 301 306

Using Critical Reading Tools to Facilitate the Learning of Argumentation Skills
Jingyan Lu, Nancy Law

Changing Explanations about Sand Dune Movement
Lauren Barth-Cohen

High School Students' Epistemic Engagement in Producing Documentaries about Public Science Concerns
David J. DeLiema, Jarod N. Kawasaki, William A. Sandoval

311

Improving Middle School Students' Understanding of Core Science Ideas Using Coherent Curriculum
Joseph Krajcik, Sung-Youn Choi, Namsoo Shin, LeeAnn M. Sutherland

316

Physical Activity Data Use by Technoathletes: Examples of Collection, Inscription, and Identification
Victor Lee, Joel Drake

321

Locating the Development of Interest: Tools for Studying the Mutual Constitution of Persons and Cultural Practices in Places
William Penuel, John H. Falk, Lynn D. Dierking, Ben Kirshner, Julie Haun-Frank, Adam J. York

326

What to Look for and What to Do: Novice Teachers' Abilities for Noticing and Responding 331 to Their Students' In-Class Inquiry
Loucas Louca, Thea Skoulia, Dora Tzialli

Developing a Technology-Enhanced Scientific Inquiry Curriculum in a Primary School: Outcomes of the School-Based Support
Yau-yuen Yeung, Zhihong Wan, Winnie Wing-mui So

336

Towards Teaching Analytics: Repertory Grids for Formative Assessment
Ravi Vatrapu, Peter Reimann, Abid Hussain

341

The Teacher's Balance between Structure and Flexibility in the Technology-Enhanced Collaborative Inquiry Setting
Marjut Viilo, Pirita Seitamaa-Hakkarainen, Kai Hakkarainen

346

© ISLS

Page xviii

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

Scaffolding and Assessing Knowledge Building among Chinese Tertiary Students Using E351 portfolios
Chunlin Lei, Carol K.K. Chan

Contribution of Motivational Orientations to Student Outcomes in a Discovery-Based Program of Game Design Learning
Rebecca Reynolds, Ming Ming Chiu

356

The Knowledge connections Analzer
Jan van Aalst, Carol Chan, Stella W. Tian, Christopher Teplovs, Yuan Y. Chan, Wing-San Wan

361

Effects of Argumentation Scaffolds and Problem Representation on Students' Solutions and 366 Argumentation Quality in Physics
Carina M. Rebello, Eleanor Sayre, N. Sanjay Rebello

Inevitable Breakdowns in Putting Argumentation into Practice
Baruch B. Schwarz, Nitza Shahar

371

Multiple Solutions and Their Diverse Justifications to the Service of Learning in Early Geometrical Problem Solving
Naomi Prusak, Rina Hershkowitz, Baruch Schwarz

376

ChemVLab+: Evaluating a Virtual Lab Tutor for High School Chemistry
Jodi Davenport, Anna Rafferty, Michael Timms, David Yaron, Michael Karabinos

381

Legitimate Peripheral Participation in Academic Communities of Practice – How Newcomers' Learning is Supported in Student Councils
Julia Eberle, Karsten Stegmann, Frank Fischer

386

Characterizing Collaboration with Metapragmatics: Using PreK Virtual Manipulatives on a 391 Multi-Touch Tabletop
Michael Evans, Kamala Russell, Michaela Hnizda, David McNeill

Development and Validation of a Scale to Place Students along a Learning Progression
Namsoo Shin , Shawn Y. Stevens

396 401 406 411

Learners as Informants of Educational Game Design
Beaumie Kim, Lynde Tan, Mi Song Kim

The Role of Gesture in Solving Spatial Problems in STEM
Matthew Lira, Mike Stieff, Stephanie Scopelitis

Audience Effects: A Bidirectional Artifact Analysis of Adolescents' Creative Writing
Alecia Marie Magnifico

The Use of Text and Process Mining Techniques to Study the Impact of Feedback on Students' Writing Processes
Rafael Calvo, Anindito Aditomo, Vilaythong Southavilay, Kalina Yacef

416

Variation in Fifth Grade Students' Propensities for Managing Uncertainty during Collaborative Engineering Projects
Michelle Jordan

424

Breeding Birds to Learn about Artificial Selection: Two Birds with One Stone?
Aditi Wagh, Uri Wilensky

426

Tracing Ideas and Participation in an Asynchronous Online Discussion across Individual and 431 Group Levels over Time
Alyssa Friend Wise, Ying-Ting Hsiao, Farshid Marbouti, Yuting Zhao

Scaffolding a Knowledge Community for High School Physics
Mike Tissenbaum, Jim Slotta

436 441

Examining System Dynamics Models Together: Using Variation Theory to Identify

© ISLS

Page xix

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

Learning Opportunities in Online Collaboration
Anindito Aditomo, Kate Thompson, Peter Reimann

Effects of Computer-Supported Collaboration Scripts on Domain-Specific and DomainGeneral Learning Outcomes: A Meta-Analysis
Freydis Vogel, Ingo Kollar, Frank Fischer

446

POSTERS
Computer Input Capabilities that Stimulate Diagramming and Improved Inferential Reasoning in Low-performing Students
Sharon Oviatt, Kumi Hodge, Andrea Miller

451

Alternative Spaces for Engagement: Performance and conversation in the Connected Classroom
Rachel Perry, Matthew Kearney

453

The Comparison of The Reinvestment of Collaborative Asynchronous Discourse Observed 455 by Two Main Actors of Pre-Service Teacher Education
Stephane Allaire

Effectiveness of Combining Worked Examples And Deliberate Practice for High School Geometry
Mariya Pachman, John Sweller, Slava Kalyuga

457

Can Technology-Based Gaze Replays of Experts Model Diagnostic Performance of Novices? A Test in Medical Education
Marko Seppänen, Andreas Gegenfurtner

459

Facilitating Teachers' Integrated Technological Pedagogical Content Knowledge
Hsin-Yi Chang

461

Use of A CSCW Platform by Trainers and Trainees. Trace Analysis: Multimodal Analysis Vs Data Mining approach
Jean Simon, Henri Ralambondrainy

463

Social Network Analysis for Knowledge Building: Establishment of Indicators for Collective Knowledge Advancement
Jun Oshima, Yoshiaki Matsuzawa, Ritsuko Oshima, Carol Chan, Jan van Aalst

465

Advancing Understanding Using Nonaka's Model of Knowledge Creation and ProblemBased Learning
Meng Yew Tee, Shuh Shing Lee

467

The Idea Manager: A Tool to Scaffold Students in Documenting, Sorting, And Distinguishing Ideas During Science Inquiry
Camillia Matuk, Kevin McElhaney, Jennifer King Chen, David Miller, Jonathan Lim-Breitbart, Marcia Linn

469

Teacher Education Students' Research Training And E-Research: Current Perspectives And 471 Potential for Development
Carlos González

Promoting Teacher Candidates' Awareness for Teaching Dilemmas: A Field Experiment
Elisabeth Wegner, Iris Kaufmann, Matthias Nückles

473

The Candy Factory Game: An Educational iPad Game for Middle School AlgebraReadiness
Michael Evans, Anderson Norton, Kirby Deater-Deckard, Mido Chang

475

Evaluating Claims in Popular Science Media: Nature of Science Versus Dynamic Epistemological Knowledge
Pryce Davis, Rosemary S. Russ

477

© ISLS

Page xx

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

Adolescent Profiles of Knowledge And Epistemic Beliefs in the Context of Reading Multiple Texts
Leila Ferguson, Ivar Bråten, Helge I. Strømsø, Øistein Anmarkrud

479

Improving Americans' Modest Global Warming Knowledge in the Light of RTMD (Reinforced Theistic Manifest Destiny) Theory
Michael Ranney, Dav Clark, Daniel Reinholz, Sarah Cohen

481

Improving Students' Scientific Reasoning Skills via Virtual Experiments and Worked Examples
Shiyu Liu, Keisha Varma

483

Synthesizing Quantitative Predictors for Interaction in an Asynchronous Online Course
Daniel Zingaro, Murat Oztok, Jim Hewitt

485

Engines of Representation: Processing Raw Student Data into Useable Student Information 487
Suzanne Rhodes, Halverson Richard

The Role of Technologies in Facilitating Collaborative Engagement
Suparna Sinha, Karlyn Adams, Toni Kempler- Rogat, Cindy Hmelo-Silver

489

Integrating Insights from Critical Race and Queer Theories with Cultural-Historical Learning Theory
Indigo Esmonde, Miwa Takeuchi, Lesley Dookie

491

Iterative Technology-based Design with Deaf/Hard of Hearing Populations: Working with Teachers to Build a Better Educational Game
Brett Shelton, Mary Ann Parlin, Jon Scoresby, Vonda Jump, Claudia Pagliaro

493

Enhancing Students' Understanding of What Ideas Are for Knowledge Building
Chieh-Hsin Chiu, Huang-Yao Hong

495

Teacher-Education Students' Beliefs in Science Teaching in a Knowledge Building Environment
Chih-Hsuan Chang, Huang-Yao Hong

497

Investigating Teacher Change in Creating Classroom Knowledge Building Communities
Katerine Bielaczyc, Sunhee Paik, John Ow

409 501

Transformative Experiences and Biological Evolution: Facilitating Deep Engagement
Benjamin Heddy, Gale Sinatra

Argument-based Inquiry and Students with Disabilities: Improving Critical Thinking Skills 503 and Science Understanding
Jonte Taylor, William Therrien, Brian Hand

Fostering Teachers' Use of Talk Moves to Promote Productive Participation in Scientific Practices
William Penuel, Yves Beauvineau, Angela Haydel DeBarger, Savitha Moorthy

505

Inquiry in the Kindergarten Science: Helping Kindergarten Teachers to Implement Inquiry507 Based Teaching
T. L. Loucas, Dora Tzialli, Constantinos Constantinou

Metacognizing Across Self And Socio Dialectics
Azilawati Jamaludin, David Hung

509

Combining Knowledge And Interaction Perspectives to Decipher Learning During A Clinical Interview
Shulamit Kapon

511

Asking for Too Much Too Early? Promoting Mechanistic Reasoning in Early Childhood Science And Mathematics Education

513

© ISLS

Page xxi

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

Loucas Louca, Chrystalla Papademetri-Kachrimani

SMILE - Smartphones in A University Learning Environment: A Classroom Response System
Celia Kaendler, Linus Feiten, Katrin Weber, Michael Wiedmann, Manuel Buehrer, Sebastian Sester, Bernd Becker

515

Stability and Change in Achievement Goals and Transfer
Erkka Laine, Andreas Gegenfurtner

517

Commercial Development of e-Learning Materials for Science and Mathematics Subjects in 519 Hong Kong: Preliminary Evaluation
Yau-yuen Yeung, Irene Chung-man Lam

Relationships between Representational Characteristics, Students' Education Levels, and Beliefs of Models
Silvia Wen-Yu Lee, Hsin-Yi Chang, Hsin-Kai Wu

521

Matchballs - A Casual Game for Learning and Domain Ontology Enrichment
Sabrina Ziebarth, Nils Malzahn, H. Ulrich Hoppe

523

Becoming a Writer: Examining Preschoolers' Interactions, Modes of Participation, and Use 525 of Resources at the Writing Center
Amy Gillespie, Deborah Rowe

Fifth and Seventh Graders' Patterns of Understanding About Cells and Heredity in a Technology-Enhanced Curriculum
Dante Cisterna, Michelle Williams, Joi Merritt

527

Ontological Stances and Systems Thinking in the Favela and Asfalto
Izabel Duarte Olson

529

Challenges of Teaching through Web-Based Inquiry: A Longitudinal Case Study of A Veteran High School Teacher
Eleni Kyza, Iolie Nicolaidou

531

Creating and Sustaining Online Communities of Practice for Science Teachers' Professional 533 Development: Overcoming the Barriers
Yiannis Georgiou, Eleni A. Kyza, Andri I. Ioannou

Reasoning About Mechanism: Children's Explanations of Pop-ups
Mayumi Shinohara, Rob Rouse

535 537

Resources for Reasoning: Use of Video and Text in a CSCL Environment
Cindy Hmelo-Silver, Carolyn Maher, Marjory Palius, Robert Sigley

Weaving Together Parts to Achieve A Whole: Gestural Activity for the Coordination of Information in the Teaching and Learning of Chemistry
Stephanie Scopelitis, Mike Stieff

539

Participatory Learning and Assessment in e-Learning Contexts
Daniel Hickey, Andrea Strackeljahn, Rebecca Itow

541

Irreducible Complexity: How Do Causal Bayes Nets Theories of Human Causal Inference Inform the Design of a Virtual Ecosystem?
M. Shane Tutwiler, Tina Grotzer

543

From Tacit Knowing to Explicit Explanation: Mining Student Designs for Evidence of Systems Thinking
Melissa Gresalfi, Leon Gordon, Sinem Siyahhan

545

The Authority of Ideas: How Students Become Influential in Linguistically Heterogeneous Small Group Discussions
Jennifer Langer-Osuna

547

© ISLS

Page xxii

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

The Power of Improvement Networks to Transform Educational Inquiry: A Preliminary Exploration
Jonathan Dolle, Peter Wardrip, Jennifer Russell, Louis Gomez, Anthony Bryk

549

Understanding Teachers' Cultural Models about Technology
Jen Scott Curwood

551

Design-Based Implementation Research of Spreadable Educational Practices within the Participatory Learning and Assessment Network (PLAnet)
Rebecca Itow, Daniel Hickey

553

Picture-Based Science Attitudes Assessment
Emily Johnson, Amy Bolling, Robb Lindgren

555

Connecting Visitors to Exhibits through Design: Exploring United States census data with CoCensus
Jessica Roberts, Leilah Lyons, Joshua Radinsky, Francesco Cafaro

557

Methods of analysis for identifying patterns of problem solving processes in a computersupported collaborative environment
Shannon Kennedy-Clark, Kate Thompson

559

An Eye For Detail: Techniques For Using Eye Tracker Data to Explore Learning in Computer-Mediated Environments
Marcelo Worsley, Paulo Blikstein

561

Finding the Common Thread: Learners' Intuitive Knowledge of General Patterns that Apply 563 Across Domains
Hillary Swanson

A Case Study of P2PU: New Models for Open and Peer-Focused Learning
Monica Resendes, Stian Haklev

565

UltraLite Collaboration: A Low-Cost Toolkit to Promote Collaborative Learning in the Classroom
Claire Rosenbaum, Paulo Blikstein, Patti Schank, Ken Rafanan, Jeremy Roschelle

567

Learning to Graph: A Comparison Study of Using Probe or Draw Tools in a Web-Based Learning Environment
Libby Gerard, Amber Zertuche, Marcia Linn

569

Bifocal Biology: Combining Physical and Virtual Labs to Support Inquiry in Biological Systems
Tamar Fuhrmann, Daniel Greene, Shima Salehi, Paulo Blikstein

571

Design-Based Research in Practice: A Technology-Based Classroom Experiment that Explores How Students Use Virtual Manipulatives to Order Groups of Fractions
Maria Mendiburo, Gautam Biswas, Ted Hasselbring

573

Learning by Collaborative Design and Evaluation
Shannon Kennedy-Clark, Vilma Galstaun, Martin Parisio, Kate Anderson

575

WORKSHOP ABSTRACTS
Bifocal Modeling: Combining Virtual and Physical Experiments in Real Time Using Lowcost Sensors and Open-source Computer Modeling
Paulo Blikstein, Tamar Fuhrmann, Daniel Greene, Shima Salehi, Claire Rosenbaum, Marcelo Worsley

577

Classroom Orchestration: Moving Beyond Current Understanding of the Field
Yannis Dimitriadis, Pierre Dillenbourg, Miguel Nussbaum, Chee-Kit Looi, Jeremy Roschelle

579

Re-thinking Roles in Future Learning: Teachers as Curriculum Media Designers and Students as Teacher Collaborators

581

© ISLS

Page xxiii

ICLS2012

Volume 2: Short Papers, Symposia, and Abstracts

Eric Hamilton, Gina Chaves, Wendy Chaves, Israel Ramirez

Teachers as Designers of Technology-Enhanced Learning Materials
Yael Kali, Susan McKenney

582 584

Inducing Learners’ Creativeness and Encouraging Learning Across Disciplines
Moseli Mafa, Palesa Khotso, Mamotena Jankie, Makomosela Qhobela

Developing a Competitive Educational Research Proposal for NSF’s Division of Research on Learning
Sandra Toro Martell, Janet L. Kolodner

586

Digital Ecosystems for Collaborative Learning: Embedding Personal and Collaborative Devices to Support Classrooms of the Future
Roberto Martinez, James Slotta, Pierre Dillenbourg, Andrew Clayphan, Mike Tissenbaum, Beat Schwendimann and Chirstopher Ackad

588

Tightening Research-Practice Connections: Applying Insights and Strategies during Design 590 Charrettes
Susan McKenney, Kimberley Gomez, Brian Reiser

Analyzing Collaborative Learning at Multiple Levels
Gerry Stahl, Keith Sawyer, Heisawn Jeong, Dan Suthers

592 593

AUTHOR INDEX

© ISLS

Page xxiv

ICLS2012

Volume 2: Keynotes

Envisioning the Next Generation Classroom and the Next Generation of Learning Technologies
Keynote 1 by Janet L. Kolodner Regents Professor, Georgia Institute of Technology, United States jkolodne@nsf.gov

Abstract
Back in 1990, when I founded The Journal of the Learning Sciences, I expressed the hope, in my opening editorial, that what we learned about learning as learning scientists would make a real difference in the world. That hasn’t happened as fast as I was looking forward to. On the other hand, we know a lot more about learning and promoting learning some two decades later, and in the past few years, we’ve seen a proliferation of new technologies that may make it easier to transition what we have been learning about learning into real use. In this talk, I will present some of the promise I see in what our community has been doing over the past two decades, some opportunities I see for increasing and transforming the opportunities people have for learning, and some of the challenges to making real-world contributions through our research. Some of those challenges have to do with systems in place in the world, and some have to do with how we organize ourselves as a community of researchers. The vision I will present is drawn from my personal experiences as Editor in Chief of JLS for 18 years, as a pedagogy designer and curriculum creator (who co-authored a wonderful science curriculum that is not selling as well as we’d like), as a developer of learning technologies (who could never manage to get any of them used), and most recently, as a program officer at the US National Science Foundation. Note, however, that I won’t give away any NSF secrets, and my talk will not in any way represent plans for support of learning sciences and learning technologies at NSF.

© ISLS

1

ICLS2012

Volume 2: Keynotes

Productive Failure
Keynote 2 by Manu Kapur Associate Professor of Curriculum, Teaching & Learning, National Institute of Education, Singapore manu.kapur@nie.edu.sg

Abstract
I will describe my program of research on productive failure (PF), which is now into its ninth year. PF is a learning design that leverages students’ formal as well as intuitive ideas about concepts they have yet to learn to design learning. PF provides opportunities for students to design solutions to complex, novel problems even though they may fail to produce canonical solutions in the process. This failure however can potentially be the locus of deep learning as it affords students the opportunity to collaboratively generate, elaborate, critique, and refine their representations and solution strategies—a process that is germane for learning. I will share empirical evidence from a series of experimental and quasi-experimental studies with 7th-11th graders in Singapore and Indian mathematics classrooms, and discuss implications for theory and practice.

© ISLS

2

ICLS2012

Volume 2: Keynotes

Lifelong Learning as a Driver for Designing Pervasive Technologies
Keynote 3 by Professor Judy Kay University of Sydney, Australia Judy.Kay@sydney.edu.au

Abstract
This talk presents a vision for lifelong learning as a driver for designing pervasive technologies. It does this via case studies for challenging long-term learning goals associated with health and wellness, collaboration to learn and learning to collaborate. One strand of that vision involves the new learning interfaces across each learner's personal digital ecosystem of devices, ranging from mobiles, to desktops and embedded interactive surfaces on walls and tables. A second strand concerns the huge amounts of data that these devices can, and do, capture about learners. This takes diverse forms, including personal information, learning data and digital footprints. There is a huge and growing amount of this data. It lives across the personal digital ecosystem, on personal devices and in the cloud. The talk illustrates the design of technologies to give this data to the learner at three levels. One concerns learner control over the capture and use of their data. Another involves data mining to transform it into new insights for the learner, their teachers and facilitators, and for education researchers. The final one is the design of interfaces such as in Open Learner Models to scaffold the learner's metacognitive activities of self-monitoring, self-reflection and planning. This keynote is kindly sponsored by NSW Trade & Investment.

© ISLS

3

ICLS2012

Volume 2: Keynotes

Classroom Orchestration: Interweaving Digital and Physical Workflows
Keynote 4 by Professor Pierre Dillenbourg EPFL Switzerland pierre.dillenbourg@epfl.ch

Abstract
Why should students login in an educational tool? The problems that occur at login time may take 2 or 3 minutes, which wastes 5% of the lesson time. Do the functionalities enabled by the login compensate this loss by 5% additional learning gains? This anecdote illustrates the practical flavor of 'classroom orchestration'. I will argue that learning sciences should not only investigate how people learn but also how teachers manage multiple constraints such as time, discipline, physical space, etc... what Nussbaum called the 'logistics' of schools. Some principles emerged for reducing orchestration load: flexibility, control, visibility, physicality and minimalism. They constitute what I refer to as 'modest computing'. But, how to turn this set of anecdotes, examples or principles into the beginning of a theory? One step forward consists in modeling a classroom as two semi-mirrored workflows, a digital and a physical, connected by a certain number of contact nodes. For instance, our experience is that paper-based interfaces are effective orchestration tools, probably because they implement these dual workflows. These findings are close to Hutchins' analysis of the relevance of paper cards in the information flows of an aircraft cockpit. The complexity we face is that, in the context of a classroom, the pilot is alone and he has to serve food to all passengers.

© ISLS

4

ICLS2012

Volume 2: Special Sessions

Invited Presidential Session: The Future of Learning and the Learning Sciences
Chair: Susan R. Goldman, University of Illinois at Chicago, sgoldman@uic.edu Louis M. Gomez, UCLA, Carnegie Foundation for the Advancement of Teaching, lmgomez@ucla.edu Naomi Miyake, School of education, University of Tokyo, Professor, Consortium of Renovating Education of the Future, Deputy Director, nmiyake@p.u-tokyo.ac.jp Nichole D. Pinkard, College of Computing and Digital Media, DePaul University, nicholepinkard@gmail.com Nikol Rummel, Institute of Educational Research, Ruhr-Universität Bochum, nikol.rummel@rub.de

Introduction
In keeping with the ICLS 2012 Conference Theme the Future of Learning and the Learning Sciences, this invited session features four representatives of the Learning Sciences community who will present their visions of the future. The panelists span several generations, countries, and areas of interest. Participants are Louis Gomez, Naomi Miyaki, Nichole Pinkard, and Nikol Rummel. Following their presentations, the panelists and audience will engage in a dialogue about the future. Brief summaries of each presentation are provided. Susan R. Goldman, President of ISLS, will chair the session.

Improvement? The next Wave in the Learning Sciences
Louis M. Gomez Professor of Education, UCLA Senior Fellow, Carnegie Foundation for the Advancement of Teaching lmgomez@ucla.edu Abstract: From the beginning, we, in the learning sciences, have sought to steer a course of scholarship that would guide our efforts so that they would amount to something important in the theater of practice. We have used tools, like design, engineering and implementation and novel social arrangements, like communities of practice to build our base of expertise and our knowledge claims that we matter, to practice. The intentional focus on artifacts and collaborative arrangements has led to notable successes. Yet the problems of practice still loom large. We need to identify ways to work that will accelerate our progress on the enduring problems that vex our ability to provide learning environments that work that work for large populations for diverse learners. I will suggest that Educational Improvement Science might be one way to accelerate our progress. In our conception, Educational Improvement Science draws directly on the ideas of quality improvement pioneered by W. Edwards Demming and Walter A. Shewhart, among others. Their ideas have revolutionized fields from manufacturing to healthcare. We also draw on Douglas Englebart's insights about energizing, within diverse collaboratives, to learn together. Only time will tell if these ideas will gain traction in the next wave of learning sciences. Their promise merits our consideration.

Renovating education of the future: Three scenarios
Naomi Miyake School of education, University of Tokyo, Professor Consortium of Renovating Education of the Future, Deputy Director nmiyake@p.u-tokyo.ac.jp Abstract: I discuss three potential solutions to the issue of sharing with educational practitioners and students the new knowledge being generated by the learning sciences community. The solutions reflect things that are doable now as well as things that will be doable in the foreseeable future. • Restructure basic understanding of how collaborations increase understanding of all participants. • Engage teachers and researchers in collaborative analysis of their collaborative learning processes. This makes more transparent to all the nature of learning and understanding. • Create a dependable and sustainable cloud to enrich classrooms with professional wisdom. Remotely operated robots provide us with new tools for all of the above. They can be mixed among the students in collaborative classes, to let us experiment on what kinds of relation-making foster the learning abilities among the kids. They could work to solicit and record utterances and behaviors of the learners. They would also form a reasonable interface for the people with professional wisdom to answer questions coming from teachers and learners, guiding them toward real-world problems.

© ISLS

5

ICLS2012

Volume 2: Special Sessions

Connecting Learning Through Learning Pathways
Nichole D. Pinkard, PhD Associate Professor College of Computing and Digital Media DePaul University nicholepinkard@gmail.com Abstract: A young person's learning journey will be guided by role-driven learning pathways that link learning across formal and informal spaces. Youth will be required to demonstrate, through artifacts, how learning experiences across home, school, after school and community context contribute to leveling up towards mastery of a set of core standards. Classroom teachers will be charged with managing youths' overall learning pathways ensuring youth are moving forward on their learning journey through participation in the recommend (via pattern analysis) set of learning experiences. Youth will also interact virtually and face-to-face with mentors and peers in variable-size learning groups based upon content, context, and learning dispositions as opposed to the current structure of 30 youth to a class.

Exploring new spaces without reinventing the wheel
Nikol Rummel, Institute of Educational Research Ruhr-Universität Bochum nikol.rummel@rub.de Abstract: When working on a textbook chapter on model learning in 2004, I read many of Bandura’s publications. Also, I got to read a number of articles on mirror neurons. The authors of those articles in their discussion took up ideas that Bandura had been writing about decades before (of course without having any neurophysiological data to support his argumentation). I was astonished to find that none of the papers on mirror neurons that I read, cited Bandura. I am using this example to illustrate a point I want to make in my contribution to this panel: I believe that exciting new “spaces” to explore lay ahead of us as researchers in the learning sciences. New technologies (Web 2.0) afford new forms of learning and enable new kinds of interaction among learners; new methods (e.g. eye tracking) allow us to answer new types of questions; new research foci (e.g. questions concerning emotional and motivational aspects of learning) broaden the scope of our findings. However, in the light of the above example, I would like to caution us to avoid wearing blinders or – in other words – to avoid reinventing the wheel when exploring those new “spaces.” We must make sure to carefully take into account prior work as well as work from other disciplines if it is (or as it becomes) relevant to our research.

© ISLS

6

ICLS2012

Volume 2: Special Sessions

Invited Panel Sponsored by the ISLS Membership Committee: Linking Learning Sciences Communities across the World
Chair: Nancy Law, University of Hong Kong, Hong Kong, nlaw@hku.hk Baohui Zhang, Nanjing University, China, baohui.zhang@nju.edu.cn Konstantin К. Kolin, Institute of Informatics Problems, Russian Academy of Sciences, Russia, kolinkk@mail.ru Paulo Blikstein, Stanford University, USA, paulob@stanford.edu Miguel Nussbaum, Pontificia Universidad Católica de Chile, Chile, mn@ing.puc.cl Michael Jacobson, The University of Sydney, Australia, michael.jacobson@sydney.edu.au

Introduction
One of the goals of ISLS is to expand its membership across a broader geographical and cultural representation. In 2007, members from North American and Western Europe accounts for 89% of all ISLS membership (362). This has grown over the years to 603 in 2011. Another satisfying aspect of this increase in membership is its expansion to regions that were under-represented in our membership. Members residing outside of North America and Western Europe grew from 55 in 2007 to 184 in 2011, an increase of 235%! While this is an encouraging trend, the increase is mainly due to rise in membership from Asia, while the other regions are still seriously under-represented. It is our view that the field of Learning Sciences can benefit from more interactions of our membership with researchers and practitioners in allied disciplines who may not know about or identify themselves as learning scientists. The goal of organizing this symposium is to provide an opportunity for ISLS members to learn more about Learning Sciences related research in different countries around the world, particularly from countries outside of North American and Western Europe, and to explore ways in which ISLS can contribute to enriching the field of Learning Sciences through expanding the international representation of its membership.

Learning Sciences Oriented Research Communities in China
Baohui Zhang Nanjing University, China baohui.zhang@nju.edu.cn Abstract: The panelist will present the preliminary findings from a study conducted in Mainland China by him and his colleagues to identify the key Learning Sciences related organizations, communities, and research agendas being pursued in China. In addition to conducting a comprehensive literature review of learning sciences related publications, empirical data have also been collected using email surveys, phone interviews and face-to-face semi-structured interviews. The study finds that while there are a number of research communities that conduct research on learning, such as the communities of science educators and educational psychology researchers, there is an active academic community that also refers to its field of study as “learning sciences”. This community started around 1986 and has its own research conferences and publication venues. This community does engage in academic exchange with international scholars, but has evolved separately and independently from the mainstream ISLS community. This presentation will highlight the research foci, as well as dominant theoretical and methodological orientations adopted by members of this learning sciences community in Mainland China. Possibilities and implications for building links between this Chinese learning sciences community and ISLS will also be discussed.

Russian Concept of Advanced Education

Konstantin К. Kolin Institute of Informatics Problems, Russian Academy of Sciences kolinkk@mail.ru Abstract: The idea of Advanced Education was formally launched in Russia in 1996 in the 2nd International Congress of UNESCO's "Education and Informatics". The essence of this concept is to reconstruct the content and methodology of the educational process at all levels of the education system so that it is able to prepare the populace for life in the Global Information Society. This is of strategic importance in improving the nation's intellectual potential as the basic human capacity required for successful social adaptation for the 21st century is different from traditional education. One main direction in the formation of Advanced Education in Russia is to develop direct links between universities and institutes of academic science. For this purpose, in Russian universities, specialized scientific and educational centers (SEC) are established to serve as agents for the introduction of new knowledge into the education system. While this is potentially a very effective form of

© ISLS

7

ICLS2012

Volume 2: Special Sessions

integration of academic research and education, funding is a major obstacle. The second obstacle is that the State Educational Standards reflect traditional approaches to education and does not meet the primary objectives of the concept of Advanced Education. Connections between the Russian concept of Advanced Education and the Learning Science will be discussed.

Brazilian education at crossroads
Paulo Blikstein Stanford University, School of Education and (by courtesy) Computer Science Department paulob@stanford.edu Abstract: Brazilian education is at a crossroads. The country is experiencing speedy economic development and education is seen as the only hope to make it sustainable. Large government projects in the last 15 years managed to bring almost all children to school. The challenge now is how to improve quality in a deeply unequal system. Educational research in Brazil has been historically connected to continental philosophy and critical theory, especially due to the work of Paulo Freire. Although highly influential in academic circles and in teacher preparation programs, this approach has been confronted in recent years by researchers and policymakers as too theoretical and ideological. The public debate that has ensued has focused on methodological and philosophical controversies between economists and educators, but has paid little attention to the development of a multidisciplinary area equivalent to the Learning Sciences (LS). The development of LS, in the United States and elsewhere, has allowed for a “third way” which has made possible fast development of technological platforms and new theoretical advances, outside of the traditional realm of education and econometric research. In this presentation, I will discuss how Latin America, and especially Brazil, could be the next frontier in the LS revolution.

Making Progress
Miguel Nussbaum Pontificia Universidad Católica de Chile mn@ing.puc.cl Abstract: Chile is a country where the schools of Education are still in its infancy. A recent national test of graduates of the last year promotion of teachers showed that 42% failed in mastering the contents of what is expected of 8th grade students and only 8% showed an excellent command of the contents. The government, being aware of this fact, sponsored four years ago two centres in the two main Chilean universities to foster research in education. However the research is mainly focused on policy and its scientific productivity is scarce.

The Learning Sciences and Technology in Australia and New Zealand
Michael Jacobson The University of Sydney michael.jacobson@sydney.edu.au Abstract: Researchers at a number of Australian and New Zealand universities have long had a strong national and international profile in a number of fields aligned with the learning sciences, such as artificial intelligence and education (AI & Ed) research, information and communication technologies (ICT), e-Learning, science education, and so on. There has also been a recent “Digital Education Revolution,” which is a nationally funded initiative to provide a laptop to all 9th grade students in Australia, in conjunction with teacher professional development to support innovative teaching and learning with technology. In my remarks, I’ll discuss ways in which we have been promoting learning sciences related research and degree programs at the Centre for Research on Computer Supported Learning and Cognition that have also involved building broader networks of collaborations with other researchers and academics in our region in affiliated areas.

© ISLS

8

ICLS2012

Volume 2: Symposia

Hybrid spaces for science learning: New demands and opportunities for research
1

Ole Smørdal1, Jim Slotta1,2, Tom Moher3, Michelle Lui2, Alfredo Jornet1 The University of Oslo; 2The University of Toronto; 3University of Illinois at Chicago

Abstract: “Hybrid spaces for science learning” refers to the merging of real and virtual worlds to produce new environments and visualizations where physical and digital objects co-exist and interact in real time. Learning science within hybrid spaces can be a fun, engaging, and reflective experience. Further, hybrid spaces are inherently social, facilitating dialogue and social exchange, as well as the construction of knowledge, paralleling the nature of contemporary science. This symposium brings together several research programs that address learning “across contexts,” that span classroom activities, museum visits, and engaging, embodied experience of science phenomena. We include an international set of presenters from Canada, USA and Norway, each engaged in design and empirical investigations of designs that blends conceptual learning with the development of inquiry skills and epistemological knowledge. Each paper presents the research context, method of design and evaluation, research progress, and science learning outcomes.

Introduction to Symposium
The various disciplines of science are rapidly evolving, in part because of the infusion of new technologies and media practices, in a trend sometimes referred to as “science 2.0” (Nisbet & Mooney, 2007; Bell et al., 2007). Cutting edge technologies are increasingly used to represent data in new forms, and by means of new analytic tools, such as 3D visualizations, simulations. Social and aggregative media (e.g., Facebook, YouTube, or wikis) have also entered into the scientific realm, creating new opportunities for collaboration and exchange across disciplines and other contexts. Indeed, scientists exploring the cosmos, the oceans, and the human genome have seized upon the advantages of large, shared datasets and open lab books. While the Internet and other advanced technologies have revolutionized many facets of civilization – including science - the typical science classroom is still dominated by traditional instruction: sequences of curricular units, presented didactically, that rely on previously learned concepts, formalized testing, and teacher-student hierarchies (Krajcik et al., 2008). Our symposium represents a collection of learning science projects that are investigating how new technologies and pedagogical approaches can expand the horizons of science education beyond the space and time restrictions of the classroom, leading to powerful new ways of learning and instruction. Hybrid spaces refer to the merging of real and virtual worlds to produce new environments for science learning. Students manipulating a physical device in a museum setting can produce changes within a digital space that, in turn, influence subsequent curricular activities. Students’ social interactions online can result in changes to their physical classroom environment, or to an aggregate representation – gathered from many students and classrooms through activities in an informal setting. Observations or other data collected using mobile or ubiquitous environments can provide a source of “user contributed content” for curriculum. Students could create large shared repositories (i.e., similar to flickr) of science-related content, including social and semantic metadata that become a resource for a wide community of learners. Social networks can be used to support special interest groups or to coordinate complex pedagogical designs. Tangible and physical computing elements (e.g., multi-touch surfaces, Arduino-controlled objects, or the Microsoft Kinect) can be integrated, allowing direct or embodied manipulation of ideas. We define hybrid spaces along physical, semantic and social variables that can be used as indices for the design of new learning experiences that cut across conventional contexts. We explore three aspects of such spaces: 1) The use of mixed reality for learning, where physical and digital objects (e.g., visualizations and simulations of science phenomena) co-exist and interact in real time. 2) Opportunities gained from digital representations of knowledge and dialogue about science that correspond spatially with student activity in physical environments, such as classrooms and science centers. 3) Understanding how the integration of informal and playful learning experiences in spaces outside the classroom (e.g. field trips to museums and science centers) can foster deeper conceptual and epistemological learning for students in the K-12 classroom. Design has long been recognized in the learning sciences as a crucial dimension of intellectual activity, and many papers have been written about design-oriented methodologies (Brown, 1992; Hoadley. 2002). Recently, there has been an important inquiry into the nature of our design processes, which are said to capture as much of our field’s learning about learning as any scientific outcomes from empirical studies. At the 2011 meeting of Computer Supported Collaborative Learning (CSCL) in Hong Kong, a new book series was

© ISLS

9

ICLS2012

Volume 2: Symposia

announced (Chris Hoadley and Naomi Miyake, editors) that will encourage the “unpacking” and deep discussion of our designs and our design process. This perspective is particularly relevant to research of hybrid environments, which entail dramatically new forms of learning. Several years can be required just to conceptualize our environments and the kind of learning that we wish to engender, before we can conduct careful studies of learning outcomes. By way of analogy, it is challenging to develop a musical score for an instrument that has not yet been designed. Thus, we find ourselves developing the instrument first, as well as the kinds of music – the tonalities, melodies, and harmonies – that that may be suited to it. There is clearly a cyclical set of dependencies in advancing such complex pedagogical, technological and social forms of learning, in which the media are deeply part of the message. Thus, while the papers all refer to learning environments that have been developed and tested to some extent, they are all still in active design mode, and the authors have been asked to focus on their design processes and elements. We have gathered three distinct research groups – one from Canada, one from USA, and one from Norway (with two presentations from the latter) – who have done empirical research into notions of spaces as an affordance and facilitation for interaction with science phenomena. These projects have designed and investigated learning trajectories in such spaces, paying attention to activities that span virtual and physical spaces. Our discussant, Wolff-Michael Roth, who is not affiliated with any of these research programs, will synthesize the ideas and advances in this work and lead a discussion amongst participants.

Paper 1: Science Hub: A digital medium for supporting collective science inquiry in hybrid spaces
1

Ole Smørdal1, Jim Slotta2, and Ingeborg Krange1. InterMedia, The University of Oslo; 2The University of Toronto, correspondence: ole.smordal@intermedia.uio.no

Across the science disciplines, the normal practices and expectations are changing in stride with the emergence of new technologies, media practices, and methodological affordances (NSF Cyberlearning Report, 2008). Bell et al. (2007) argue that the new “data-intensive” nature of 21st century science has moved scientific practices away from individual or even small groups of scientists working with individual databases and computational simulations, toward the use of cutting edge technologies to represent and analyze data in new forms, including powerful visualizations, GIS and satellite mapping, and new experimental tools and methods in every discipline. Social and aggregative media have entered the scientific realm, creating new opportunities for collaboration and exchange across disciplines and contexts, leading some to coin the term, “science 2.0” (e,g., Nisbet & Mooney, 2007). The exciting transitions occurring in science have important implications for science education. Our research will investigate a curriculum that responds to the changing complexion of energy both as a science phenomena and a global socio political challenge in the 21st century in such a way that actually capitalize on students’ digital skills, making science more engaging and at the same time preparing students more deeply for careers in science related disciplines. We address variations and transitions between instructional contexts (Dierking et al. 2003; Rennie, 2007). In particular we address how learning in classrooms and informal settings (e.g., museums and science centers) can be mutually supportive. This paper reports on our design of a digital medium called The Science Hub (SciHub) to support student activities across formal and inform contexts, using rich, interactive media in museums, Web pages at home, and their own smart phones out on the street. The digital infrastructure is important for interconnecting these various forms of interaction, for adding a semantic layer of accessible metadata (including social networking information and patterns of use), and for enabling the design of powerful new forms of aggregated representation, real-time feedback, and pedagogical scripting. This paper will focus on two aspects of the SciHub. First, we present our own design process, which has embodied Science 2.0, occurring across various physical and virtual contexts. We discuss our goals, constraints, guiding principles and technology systems. Second, we present a complex sequence of curriculum activities designed in conjunction with the Norwegian Museum of Science, Technology and Medicine, that engages students at home, in school and at the museum in a variety of forms of learning, including embodied activities, collective inquiry, the creation of shared multi modal resources, and the use of rich visual representations and simulations. The SciHub is being developed to support complex patterns of inquiry that must respond to the unique contexts (both formal and informal) and configurations (individual/small group/whole class interactions) in which learning takes place (Lemke, 2000). This includes learning activities that are physically and temporally distributed, that can and should be described on multiple levels, and that engage users in emergent (i.e., not predictable, a priori) forms of learning and knowledge advancement. The description and coordination of these factors is often referred to as a “script”, which includes the timing and sequencing of activities, planned

© ISLS

10

ICLS2012

Volume 2: Symposia

moments for student reflection, and roles for students and teachers (Dillenbourg & Jermann, 2007, Hakkinen & Makitalo-Siegl, 2007). We regard scripts as a means for specifying and making available structures, procedures, collaboration patterns, roles, materiality and resources of 21st century scientific practices, and making them relevant and engaging for learners. By embedding such scripts within a technology-enhanced environment, we can scaffold their various elements, capture the products of interactions, and provide timely feedback or input to students and teachers. Such feedback can provide participants with a visual sense of the current state of the activity, including aggregated representations from small groups, all students in a class, or everyone who has ever interacted with the script. They can provide teachers with insight into students’ understandings of curriculum topics, and support classroom orchestration through evidence-based decisions (Dillenbourg, Jarvela & Fischer, 2009). The design language of SciHub references concepts from theatrical performances, such as role, actor, script, stage, and performance. In SciHub, students contribute to aggregated products that change over time, with the products themselves serving as resources in the activity. Students cooperatively produce artifacts and experiences, challenging our notion of assessment, and adding important new dimensions of collaboration, reflection, revision and engagement. Designing research to evaluate the impacts of such activities - in terms of student learning as well as a more overarching aspect of progress for the community - has been an interesting challenge. We must consider new forms of electronic discourse that are related to building upon peers’ ideas, connecting artifacts, revising documents, and interacting with physical and tangible media. Emerging notions of social and embodied learning have inspired new ways of measuring progress, capturing participation and evaluating the impact of our innovations. Our first enactments of SciHub supported contextualization of simulations of science phenomena. Students’ inquiry can be structured, or more open-ended, including a wide range of digital interactions such as logs, notes, video recordings, that are available as a resource for subsequent learning activities. Scaffolds and prompts are implemented using a system intelligence, where agents operate on content in databases, state information, proximity awareness and other parameters. SciHub activities serve to connect formal and informal contexts, with a focus on experience-based learning that takes place in the museum, and how this can be related to conceptual learning in the classroom. SciHub supports a wide range of student inputs, and can infer from sensors in a spatial environment (i.e. who is present, types of activity, location, body gestures, gaze, tangibles, mobile devices, etc.).

Paper 2: Embodied science practices in hybrid spaces
Tom Moher, Francesco Novellis, Brenda Lopez Silva, and Alessandro Gnoli University of Illinois at Chicago, correspondence: moher@uic.edu In the Embedded Phenomena framework (Moher, 2006; 2008), simulated spatial phenomena are mapped onto the physical space of self-contained classrooms as subjects of collaborative inquiry. In prior designs employing this framework for inquiry in seismology (RoomQuake), astronomy (HelioRoom), and ecology (WallCology), access to and interaction with the simulated phenomena have been restricted to stationary “portals” (computers) strategically situated within classrooms. While these designs have leveraged embodiment in their utilization of learners’ locations for perspective-taking, distributed collaboration, and enactment of analogical solutions to mathematics problems (Antle, 2009), the digital and natural worlds have remained largely parallel realities rather than hybrid spaces. AquaRoom (Novellis & Moher, 2011) extends the Embedded Phenomena paradigm by introducing hybrid artifacts that richly problematize physical aspects of science practices (Reiser, 2004). AquaRoom is based on the conceit that the classroom is a small town beneath which runs a network of coherent aquifers. Children are challenged to recommend a location for a new chemical plant to be constructed within the town, with the goal of minimizing the impact of potential effluents on underground water supplies. To inform their decision, they were organized into groups to conduct a hydro-geological survey. Using the standard practice of injecting dye tracers into potential aquifer sites, they collect water samples in adjacent areas and use a photospectrometer to analyze the sample for presence of the dye. These analyses allow them to construct a map of the topography, direction and rate of flow of the aquifers under their “town.”

Figure 1. Portable “drilling unit.”

Figure 2. Dye and sample tubes.

Figure 3. “Spectro-photometer.”

© ISLS

11

ICLS2012

Volume 2: Symposia

AquaRoom blends a collection of information-carrying tangible artifacts and analogical (but information free) “props” within the learning environment. Dye injection and sample retrieval are modeled through the use of a “portable drilling unit” (Fig. 1) that includes a non-functional Ethernet cable taped to a suction cup which, when applied to the classroom floor, provides a simulated conduit for liquid flow to and from underground aquifers. Dyes are stored, and samples collected, in test tubes to which are attached special electronic caps (iButtons®), each of which contains a unique identifier (Fig. 2). Students connect the test tubes to iButton readers attached to the drilling units to effect injection and retrieval, carrying samples (in opaque collection test tubes) to a simulated photo-spectrometer (also connected to an iButton reader) to look for traces of the injected dyes (Fig. 3). Students mark up a large, shared paper grid on the wall to reflect the results of their investigations. The design choices in AquaRoom impose physical demands analogous to those required in actual hydrogeological practice, including the selection and recording of injection and retrieval sites, physical travel to those sites, management of field instruments, collection and management of samples, use of laboratory instruments for analysis of samples, and inscription of findings (Duschl et al., 2007). In these ways, AquaRoom represents a more “seamless” model (Ishii & Ullmer, 1997) of a hybrid space than prior Embedded Phenomena, with physical actions in the real world yielding scientifically appropriate and meaningful effects in the virtual environment. In two enactments of AquaRoom in fourth grade classrooms, we have focused on children’s responses to the hybrid artifact designs, their ability to utilize the environment’s affordances to conduct their investigation, their relative sense of presence within the physical and virtual worlds, and their development of understandings of effective spatial strategies for injection and sampling in the context of aquifer mapping. In our first pilot (Novellis & Moher, 2011), fidelity of representation became a prominent thread of discussion, particularly around the issue of whether correct placement of the suction cup on the floor was necessary. (It wasn’t: students self-reported their locations using a coordinate system based on the room’s ceiling tiles.) While opinion split on the necessity of the artifact, there was a strong consensus regarding its utility, with students citing embodied reinforcement of coordinate positions, opportunities for participation within investigative groups, and play value associated with a willing suspension of disbelief (Coleridge, 1817) in support of its use. In a recent enactment, researchers “shadowed” the working groups to track students’ practices surrounding the use of the artifacts. Students accurately placed and reported their injection/sampling locations in all but three of 176 observed cases; surprisingly, artifact fidelity did not become a subject of classroom discourse, though about a quarter of the class manifested skepticism about suction cups during post-unit interviews. Note that the students’ accuracy in self-reporting their locations suggest the viability of a low-cost alternative to indoor location tracking instrumentation, at least for coarsely grained coordinate systems such as those used here. The handling of samples was similarly consistent, with only five of 217 collected samples failing to be subjected to “spectroanalysis.” Across both enactments, students showed a strong ability to transcribe analysis results to the shared map of the “town,” accurately determining the aquifer topographies. Flow direction and rate (measured using stopwatches) were more challenging, but differences of opinion created opportunities for discussion of methods for resolving observational discrepancies and the practice of repeated measurements. In post-unit interviews, students almost universally were able to articulate an effective (concentric circle) inject-and-sample strategy. Analysis of student responses to the question “In your own words, please describe what we’ve been doing in the AquaRoom unit” showed an overwhelming predominance of “domain-oriented” (e.g., “town,” “drill,” “chemical plant”) vs. “implementation-based” (e.g., “computer,” “suction cup,” “classroom”) terms.

Paper 3: Designing Immersive Environments for Collective Inquiry
Michelle Lui, James D. Slotta Ontario Institute for Studies in Education, University of Toronto, Canada, correspondence: michelle.lui@utoronto.ca This research explores how physical space, when augmented with a system of networked tablets, interactive whiteboards, and a web-based learning environment, can support face-to-face interactions as well as asynchronous knowledge co-construction. We present EvoRoom as a case study of an immersive simulation for collective inquiry. The notion of immersive simulations is not a new one, but the use of such media for conceptual learning, including the design of interactions amongst students, peers, teachers and materials, is still in nascent stages. We discuss our research objectives and our design process, as well as the outcomes from a study evaluating student perceptions and behavior within the immersive, physical simulation. Theoretical Perspective: Recently, educators have advocated the approach of forming knowledge communities in classrooms – where members are given the responsibility to generate ideas and build on each others’ ideas within an emergent, collective knowledge base (Bereiter & Scardamalia, 2003). This approach is well suited for today’s science students, who must now develop collaboration and communication skills in
© ISLS 12

ICLS2012

Volume 2: Symposia

addition to learning scientific content and procedures. In our own prior work, we have applied “Web 2.0” collaboration technologies in support of knowledge communities, connecting students with peers to co-develop a shared knowledge base (Peters & Slotta, 2010). We have also advanced the concept of a “smart classroom,” where the physical environment (e.g., walls, furniture, etc.) is deeply infused with a range of digital tools and media, scaffolding students in complex pedagogical designs, through different roles and responsibilities (Lui, Tissenbaum & Slotta, 2011; Slotta, 2010). In response to prior research in virtual worlds, such as River City (Dede, 2009) and Second Life, we now seek a layer of immersivity to transform our smart classroom into a mixed-reality immersive environment. While such virtual worlds may cultivate students’ sense of ownership and play, they are fundamentally limited in terms of bodily-kinesthetic interactions, as users act through avatars via screen-based interactions (Birchfield et al., 2008). We posit that hybrid environments hold the potential to transform typical learning activities into more meaningful, whole-body experiences, promoting collaboration and a sense of knowledge community. Recent research from the Learning Sciences and Human-Computer Interaction (HCI) have advanced the notion of embodied learning, where the human mind and body are viewed as inseparable (Garbarini & Adenzato, 2004). New technologies have led to a surge of interest in augmented physical environments for hands-on and participatory learning. Examples of projects that incorporate ideas of immersion and embodiment include the Cave Automated Visualization Environment (Cruz-Neira, Sandin, & DeFanti, 1993), SMALLab (Birchfield et al., 2008) and Ambient Wood (Rogers & Price, 2004). Inspired by these prior efforts, we seek to create powerful experiences for groups of co-located students to share and develop meaning through embodied interactions with one another and their surroundings, working a knowledge community. We added functionality for immersive simulations to our smart classroom infrastructure, where the room itself is converted into a rich simulation, and conceptual content is embedded using ubiquitous technology that supports students as they engage in carefully designed learning activities. Our research focuses on how such media and activities can support new forms of learning and instruction. Method: In January 2011, we began a co-design partnership with a high-school biology teacher to create our immersive simulation. We considered important design elements and outlined our overall strategy. Our first design decision was the selection of an accessible topic that would allow students to gain enhanced conceptual depth and where their embodied experience would provide distinct opportunities for perception, reflection, or integration of their scientific understandings. We required a topic that was sufficiently challenging and broad enough in scope to engage students for several sessions, including introductory (i.e., non-immersive) activities. We arrived at the topics of biodiversity and evolution, which are notoriously challenging topics for teachers and students alike, and have many characteristics that would support our development of robust, engaging materials and interactions.

Figure 1. The EvoRoom immersive environment.

Figure 2. The smart classroom layout

As the context for students’ scientific inquiry within our immersive environment, we selected the rainforest ecosystem of Borneo and Sumatra. The varied species of the rainforests offer a rich and complex ecosystem from which biodiversity may be studied, and the disappearance of the land bridges that connected Borneo and Sumatra approximately 10,000 years ago offer an interesting phenomenon through which to study evolution. This theme allowed us to develop rich, interactive media, showing various species of flora and fauna, and their interdependencies, which students can directly observe in the context of the simulation (see Figure 1). In order to support such an experience, the room is set up with multiple versions of the simulation. On each of the two long walls of the room, three projection displays are connected to form a 5-meter wide projected surface. These two wide, immersive displays on the facing walls provide a sense of enclosure. On the front wall of the room, serving to bridge these two wide displays, a single projected display provides higher-level symbolic and social information, as a resource for inquiry (see Figure 2). For all student inquiry, we developed specialized applications for tablet and laptop computers, which provided a field guide, collected student observations, and coordinated log-ins and group membership within the room. We also created specialized applications for an interactive white board to support the teacher’s orchestration of activities within the room.

© ISLS

13

ICLS2012

Volume 2: Symposia

To support a sense of identity, each student is given a role of “ecology expert” for a certain species. They are asked to work as “field researchers” in specialized teams (e.g., primate ecologist team), with clearly delineated tasks and scientific agenda. In the biodiversity unit, students work collectively to create a food web of the ecosystem – each bringing to the activity their expert knowledge of their species. To promote connections to the rainforest environment, students create “scientific scenarios” in an introductory activity, predicting how changing one factor in the ecosystem could impact the long-term biodiversity of the rainforest. When students enter the room to find the rainforest simulated with their choice of perturbations, such that the students effectively build their own immersive environment. This helps students build identity with the materials and take ownership of their own learning. In the evolution unit, which occurred several weeks after the biodiversity unit, students returned to the same smart classroom environment, retaining their specialty research teams, to conduct a “field visit” and survey. The teacher coordinated the activity, advancing the simulation through several historical time periods. One of the large walls was programmed to depict prehistoric Sumatra, and the other to depict Borneo in the same time period (e.g., 2 million years ago). Originally, since the two regions were connected by a land bridge, their species are identical. When the teacher advances the simulation to a more recent time period, students observe that the species have diverged! The teacher guides student observations, highlighting historical and climatological records, and drawing attention to important features of each environment,. Findings and Discussion: In our first classroom trial, we examined student behavior and perceptions of the immersive environment using questionnaires given pre- and post-activity as well as video analysis of the activity. Students were highly engaged by the use of the tablet computer supports as well as by the immersivity of the smart classroom. Students made special note of the interactive white board application that collected and shared ideas in real-time, which allowed them to gain powerful insights about evolution. We refined the activities, and are presently enacting the biodiversity and evolution environments as part of a fully integrated biology curriculum. We have added pedagogical supports as well as visualizations to represent the community knowledge base for the duration of the curriculum. The goal is for materials to become more visible (i.e., visually rich) yet less intrusive, responding intelligently to student inputs, as well as capturing the collective wisdom of the classroom community as a resource for all participants in subsequent inquiry activities. We will report on student learning gains, interaction patterns (ie, between peers, and with elements of the environment) and next steps for research.

Paper 4: Epistemic artefacts for inquiry in hybrid spaces
Alfredo Jornet, Cecilie F. Jahreie & Ingeborg Krange. InterMedia, The University of Oslo, correspondence: alfredog@intermedia.uio.no In this paper, we report on our research on the relation between physical experience and conceptual understanding, as mediated by tangible artefacts and visual representations. We explore student’s meaning making as they solve disciplinary problems of their science curriculum in an inquiry-based learning environment that involves experimenting with both tangible phenomena and digital models. Drawing on perspectives on visual representations as social practice (Roschelle & Clancey, 1992; Roth & McGinn, 1998), and on socio-cultural approaches to learning and cognition (Vygotsky, 1962; Wertsch, 1998), we describe semiotic mediation as a useful framework for informing how actual (phenomenological, social, material) experience relates to the development of disciplinary knowledge. The aim of the study is two-fold. On the one hand, we aim to investigate how tangible objects and visual representations become meaningful resources in students’ problem solving along an inquiry-based sequence of disciplinary activities. On the other hand, we aim to inform further development of a technology-enhanced learning environment that integrates tangible and digital experiences. Emerging mobile and ubiquitous computing technologies have opened new possibilities for designing inquiry-based learning environments in which students can engage in embodied experimentation with simulated or augmented phenomena (Moher, 2006). In these environments, both tangible and digital objects are combined as to constitute hybrid spaces where the material and the symbolic become explicitly linked. In these spaces, both digital and tangible artefacts may become objects of, as well as resources for, inquiry. Thus, the boundaries between what can be conceived as a mere artefact and what can be conceived as a representation become blurred. In this sense, scholars have pointed out the need for theoretical accounts of the relations between physicality and representations in hybrid learning spaces (Hornecker & Buur, 2006; Price, 2008). We adopt the term epistemic artefacts (Scardamalia & Bereiter, 2006) in order to refer to any object, whether tangible or digital, that may serve for the advancement of discussion and coordination in a problem solving activity. While there has been much research on visual representations as deployed in traditional desktop computers, such as multiple linked representations (Ainsworth, 2006) or interactive simulations and models (van der Meij & de Jong, 2006), few studies have investigated the role of visual representations in linking physical experiences with conceptual knowledge, or the role of physical artefacts and experience in understanding visual

© ISLS

14

ICLS2012

Volume 2: Symposia

representations. In contrast to traditional views of visual representations as carrying meanings that are to be decoded by mental information processing, a growing body of literature points to the role of visual representations as resources for coordinating social conduct (Roschelle & Clancey, 1992; Roth & McGinn, 1998). From these perspectives, meanings are established through social, situated, material interaction, in semiotic processes by which a sign comes to refer to an object for the agents participating within given activities. This semiotic process is grounded in social interaction and is anchored to material experience. In order to investigate how both tangible and digital objects become epistemic artefacts through social interactions, we draw on video recordings gathered from a pilot study of a multidisciplinary project that aims to develop technology-enhanced solutions for linking experiences at the school and the museum (Jahreie & Krange, 2011; Jornet & Jahreie, 2011). Using Interaction Analysis (Jordan & Henderson, 1995), we follow a group of students and their teacher as they move through a set of activities that go from direct experimentation with physical artifacts to activities involving interaction with digital models. Along the trajectory, we have observed how the meanings of the different objects and representations emerge and evolve as students together make sense of the situations they confront. The students co-construct shared empirical grounds by physically and verbally pointing and particularizing salient features of their material contexts. The students and their teacher make these categories the topic of their discussion and bring them relevant to different discourses, moving from an everyday language to more disciplinary talk. The emotional and motivational components, together with the institutional aspects of the contexts and tasks, seem to influence the kind of discourses the students adopt. The material environment, including both digital and tangible objects, as well as their own talk and bodily gestures and positions, become a complex set of semiotic resources in constant development, and which are in reciprocal relation to the participants’ positions and dispositions in regard to the activities at hand. However, although both tangible objects and representational inscriptions acquire their functions and meanings through similar grounding processes, they appear to have different semiotic potentials as resources for inquiry. Implications of our findings for the design of hybrid inquiry-based learning environments are discussed.

References
Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16(3), 183-198. doi: 10.1016/j.learninstruc.2006.03.001 Antle, A.N. (2009). Embodied child computer interaction: why embodiment matters. Interactions 16, 2 (Mar./Apr. 2009). ACM, New York, NY, 27-30. Bell, G., Gray, J. & Szalay, A. (2007) Petascale Computational Systems, arXiv:cs/0701165v1 [cs.DB]. Bereiter, C., & Scardamalia, M. (2003). Learning to work creatively with knowledge. In E. De Corte, L. Verschaffel, N. Entwistle, & J. van Merriënboer (Eds.), Powerful learning environments: Unraveling basic components and dimensions (pp. 55-68). Oxford, UK: Elsevier Science. Birchfield, D., Thornburg, H., Megowan-Romanowicz, M. C., Hatton, S., Mechtley, B., Dolgov, I., & Burleson, W. (2008). Embodiment, Multimodality, and Composition: Convergent Themes across HCI and Education for MR Learning Environments. Advances in Human-Computer Interaction, 2008, 1–20. Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2), 141-178. Coleridge, S.T. (1817) Biographia literaria. Cruz-Neira, C., Sandin, D., & DeFanti, T. A. (1993). Surround-screen projection-based virtual reality: the design and implementation of the CAVE. In SIGGRAPH '93: Proceedings of the 20th annual conference on Computer graphics and interactive techniques, (pp. 135-142). ACM Press. Dede, C. (2009). Immersive interfaces for engagement and learning. Science, 323, 66–69. Dierking, L.D., Falk, J.H., Rennie, L., Anderson, D., & Ellenbogen, K. (2003). Policy statement of the "Informal Science Education" Ad Hoc Committee. J. of Research in Science Teaching, 40(2), 108-111. Dillenbourg, P., & Jermann, P. (2007). Designing Integrative Scripts. In F. Fischer, I. Kollar, H. Mandl & J. r. M. Haake (Eds.) Scripting Computer-Supported Collaborative Learning (Vol. 6, pp. 275-301): Springer. Dillenbourg, P., Järvelä, S., & Fischer, F. (2009). The Evolution of Research on Computer-Supported Collaborative Learning. In N. Balacheff, S. Ludvigsen, T. Jong, A. Lazonder & S. Barnes (Eds.) Technology-Enhanced Learning. (pp. 3-19): Springer Netherlands. Duschl, R.A., Schweingruber, H.A., and Shouse, A.W. (2007). Taking science to school: Learning and teaching science in grades K-8. National Academy Press. Garbarini, F., & Adenzato, M. (2004). At the root of embodied cognition: Cognitive science meets neurophysiology. Brain and Cognition, 56(1), 100–106. Häkkinen, P., & Mäkitalo-Siegl, K. (2007). Educational Perspectives on Scripting CSCL. In F. Fischer, I. Kollar, H. Mandl & J. M. Haake (Eds.) Scripting Computer-Supported Collaborative Learning. (Vol. 6, pp. 263271): Springer US. Hoadley, C. (2002). Creating context: Design-based research in creating and understanding cscl. In Stahl, G., editor, Computer support for collaborative learning 2002, (pp. 453-462), Mahwah, NJ: Erlbaum.

© ISLS

15

ICLS2012

Volume 2: Symposia

Hornecker, E., & Buur, J. (2006). Getting a grip on tangible interaction: a framework on physical space and social interaction. Paper presented at the SIGCHI conference on Human Factors in computing systems, Montreal, Québec, Canada. Ishii, H. & Ullmer, B. (1997). Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms. Proceeding ACM Conference on Human Factors in Computing Systems (Atlanta, GA, March 22-27, 1997). CHI ‘97. ACM Press, 234-241. Jahreie, C. F., & Krange, I. (2011). Learning in Science Education Across School and Science Museums – Design and Development Work in a Multiprofessional Group Nordic Journal of Digital Literacy. Jordan, B., & Henderson, A. (1995). Interaction analysis: Foundations and practice. The Journal of the Learning Sciences, 4(1), 39–103. Jornet, A., & Jahreie, C. (2011). Designing for immersive learning environments across schools and science museums. Multi-professional conceptualisations of space. Paper presented at the ReLIVE11: Researching learning in immersive virtual environments, Milton Keynes (UK). Krajcik, J., Slotta, J., McNeill, K. L. & Reiser, B. (2008). Designing learning environments to support students constructing coherent understandings. In Kali, Y., Linn, M. C., & Roseman, J. E. (Eds.) Designing coherent science education. (pp.39-64). NY, NY: Teacher College Press. Lemke, J. L. (2000). Across the Scales of Time: Artifacts, Activities, and Meanings in Ecosocial Systems. Mind, Culture, and Activity, 7(4), 273-290. Lui, M., Tissenbaum, M., & Slotta, J. D. (2011). Scripting collaborative learning in smart classrooms: Towards building knowledge communities. Proceedings of the 9th International Conference on ComputerSupported Collaborative Learning (CSCL) – Volume 1, (pp. 430-437). International Society of the Learning Sciences (ISLS). Moher, T. (2006). Embedded Phenomena: Supporting Science Learning with Classroom-sized Distributed Simulations. Proceedings ACM Conference on Human Factors in Computing Systems (Montreal, Canada, April 22-27, 2006). CHI 2006. ACM Press, 691-700. Moher, T. (2008). Learning and participation in a persistent whole-classroom seismology simulation. In Proceedings ICLS, Vol. 2 (Utrech, Netherlands, June 23 - 28, 2008). ICLS 2008. ISLS, 82-90. Nisbet, M. C. & Mooney C. (2007) Framing Science. Science: 316 (5821). Novellis, F. & Moher, T. (2011). AquaRoom: Designing Tangibles and Props to Engage Young Learners in a Full Body Learning Experience. Proceedings 10th international Conference on Interaction Design and Children (Ann Arbor, MI, June 20 - 23, 2011). IDC '11. ACM Press, 90-98. NSF Task Force on Cyberlearning. (2008). Fostering learning in the networked world: The cyberlearning opportunity and challenge, (NSF 08-204). Arlington, VA: National Science Foundation. Peters, V. L., & Slotta, J. D. (2010). Scaffolding knowledge communities in the classroom: New opportunities in the Web 2.0 era. In M. J. Jacobson & P. Reimann (Eds.), Designs for learning environments of the future: International perspectives from the learning sciences (pp. 205-232). Secaucus, NJ: Springer. Price, S. (2008). A representation approach to conceptualizing tangible learning environments. Paper presented at the 2nd International Conference on Tangible and Embedded Interaction, Bonn, Germany. Reiser, B. J. (2004). Scaffolding complex learning: The mechanisms of structuring and problematizing student work. Journal of the Learning Sciences, 13, 273–304. Rennie, L. J. (2007) Learning science outside of school. In Abell S. K. & Lederman N. G. (eds.) Handbook of research on science education. Lawrence Erlbaum Associates Ltd. Rogers, Y., & Price, S. (2004). Extending and augmenting scientific enquiry through pervasive learning environments. Children, Youth and Environments, 14(2), 67–83. Roschelle, J., & Clancey, W. J. (1992). Learning as Social and Neural. Educational Psychologist, 27(4), 435453. doi: 10.1207/s15326985ep2704_3 Roth, W.-M., & McGinn, M. K. (1998). Inscriptions: Toward a Theory of Representing as Social Practice. Review of Educational Research, 68(1), 35-59. Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and technology. In R. K. Sawyer (Ed.), Cambridge Handbook of the Learning Sciences. New York: Cambridge University Press. Slotta, J. D. (2010). Evolving the classrooms of the future: The interplay of pedagogy, technology and community. In K. Mäkitalo-Siegl, F. Kaplan, J. Zottmann, & F. Fischer (Eds.), Classroom of the Future: Orchestrating collaborative spaces (pp. 215-242). Rotterdam: Sense. van der Meij, J., & de Jong, T. (2006). Supporting students' learning with multiple representations in a dynamic simulation-based learning environment. Learning and Instruction, 16(3), 199-212. doi: 10.1016/j.learninstruc.2006.03.007 Vygotsky, L. S. (1962). Thought and language. Cambridge, Mass.: MIT Press. Wang, F., & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research and Development, 53(4), 5-23. Wertsch, J. V. (1998). Mind as action. New York: Oxford University Press.

© ISLS

16

ICLS2012

Volume 2: Symposia

Building Upon What Is Already There: The Role of Prior Knowledge, Background Information, and Scaffolding in Inquiry Learning
Christof Wecker, University of Munich, Department of Psychology, Leopoldstr. 13, D-80802 München, Germany, christof.wecker@psy.lmu.de Ard W. Lazonder, University of Twente, Department of Instructional Technology, P.O. Box 217, 7500 AE Enschede, The Netherlands, a.w.lazonder@utwente.nl Jennifer L. Chiu, University of Virginia, 313 Bavaro Hall, Curry School of Education, Charlottesville, VA 22903, jlchiu@virginia.edu Cheryl Madeira, James D. Slotta, University of Toronto, 252 Bloor St., West., Toronto, Ontario, Canada, M5R2X2 Email: cheryl.madeira@utoronto.ca, jslotta@oise.utoronto.ca Yvonne G. Mulder, Ton de Jong, University of Twente, Department of Instructional Technology, P.O. Box 217, 7500 AE Enschede, The Netherlands Email: y.g.mulder@utwente.nl, a.j.m.dejong@utwente.nl Alexander Rachel, Hartmut Wiesner, University of Munich, Department of Physics, Theresienstr. 37, D-80333 München, Germany Email: alexander.rachel@physik.uni-muenchen.de, hartmut.wiesner@physik.uni-muenchen.de Eva Heran-Dörr, University of Bamberg, Marcushaus, Markusplatz 3, D-96047 Bamberg, Germany, eva.herandoerr@uni-bamberg.de Frank Fischer, University of Munich, Department of Psychology, Leopoldstr. 13, D-80802 München, Germany, frank.fischer@psy.lmu.de Discussant: Peter Reimann, University of Sydney, Sydney NSW 2006, Australia, peter.reimann@sydney.edu.au Abstract: Prior knowledge is one of the most important factors for learning. During the iterative cycles of inquiry learning, learners’ prior domain knowledge is modified, refined, and further developed, provided that learners act upon self-assessments of their understanding and that they can actually think of appropriate hypotheses. Furthermore, knowledge about inquiry strategies influences the quality of the learners’ inquiry activities, and the lack thereof requires compensatory support. This symposium brings together recent work about the role of prior knowledge for inquiry learning and ways to compensate for the lack of it. The four papers focus on the role of learners’ self-assessment of their current understanding for their subsequent inquiry activities, on the gradual refinement of their knowledge on the basis of reflection, and on prior presentation of theoretical background information and concurrent presentation of inquiry strategies as ways to compensate for lack of prior theoretical knowledge and strategy knowledge, respectively. One of the most powerful factors that influence learning is what learners already know (Dochy, Segers & Buehl, 1999). This rather general finding applies to inquiry learning as well (see, e. g., Gijlers & de Jong, 2005). This symposium brings together recent work about the ways prior knowledge influences inquiry learning and ways to compensate for the lack of it. Inquiry learning has been characterized as an iterative process in which knowledge is modified, refined, and thereby further developed (e.g., White & Frederiksen, 1998). Students generate knowledge by asking questions and stating hypotheses, conducting experiments, and drawing conclusions, and each of these cycles is informed by the knowledge the learners have acquired during the previous ones. In this line of reasoning, student learning in subsequent cycles can benefit from accurate metacognitive judgments about the appropriateness of their current knowledge. In fact, interventions that target learners’ self-assessment of their understanding can foster knowledge about the domain content (White & Frederiksen, 1998). Within the framework of Scientific Discovery as Dual Search (SDDS, Klahr & Dunbar, 1988; van Joolingen & de Jong, 1997), learners’ knowledge can be characterized by attributes of hypotheses within the socalled hypothesis space. This space contains all possible hypotheses about a given class of phenomena. A learner’s knowledge is constituted by the information for each hypothesis he or she can think of whether it is considered worthwhile testing, has or has not been tested so far, and if it has been tested, whether it has been rejected or retained for further consideration (cf. van Joolingen & de Jong, 1997; Gijlers & de Jong, 2005). The set of hypotheses a learner can think of, i.e. the so-called learner hypothesis space, is determined by the learners’ knowledge of variables and relations (van Joolingen & de Jong, 1997) and constitutes an important limiting factor during iteratively progressing inquiry learning because learners cannot discover the right hypothesis if it is not within their search space. If learners’ hypothesis spaces are too limited, remedial intervention such as providing background information is required.

© ISLS

17

ICLS2012

Volume 2: Symposia

Another kind of knowledge that is relevant during the cycles of inquiry learning is knowledge about inquiry strategies. While generating hypotheses, designing and conducting experiments, making observations and recording data, and drawing conclusions are activities that operate upon the content of the domain, they are part of inquiry strategies. Accordingly, learners with better knowledge about inquiry strategies are more likely to be successful during their inquiry activities and therefore should acquire more domain knowledge. Conversely, learners lacking sufficient levels of knowledge about inquiry strategies need to be scaffolded appropriately in order to be able to conduct fruitful inquiry activities and learn successfully about the domain. The papers in this symposium cover this array of aspects of the role of prior knowledge during inquiry learning. Jennifer Chiu investigated the effects of learners’ self-assessments of their understanding during a WISE unit about chemical reactions. Cheryl Madeira and Jim Slotta followed the recommendation to apply the iterative character of inquiry learning to teacher training (White & Frederiksen, 1998) and investigated the iterative refinement of teachers’ pedagogical content knowledge under the conditions of practical enactment and peer exchange. Alexander Rachel, Christof Wecker, Eva Heran-Dörr, Hartmut Wiesner and Frank Fischer investigated ways to compensate for elementary school students’ limited hypothesis spaces that do not contain assumptions about theoretical entities related to magnetism. Yvonne Mulder, Ard Lazonder and Ton de Jong focused on scaffolding that may compensate for lacking knowledge about inquiry strategies. They studied the effects of heuristic worked examples demonstrating how to gradually develop an equation that specifies a relationship among a set of variables. The series of the four paper presentations will be complemented by the presentation of a discussant. This role has been taken over by Peter Reimann who is an eminent scholar in the learning sciences and has conducted and published research about inquiry learning himself. He will identify the major achievements and unresolved issues of the four papers. This will provide the basis for an open discussion with the audience.

Paper 1: Student Self-Assessment of Knowledge Integration in a TechnologyEnhanced Chemistry Lesson
Jennifer L. Chiu Supporting self-assessment can help students learn across many domains (e. g., Scardamalia & Bereiter, 1991). Self-assessment skills such as questioning or judging one’s own understanding and making decisions based on those assessments are especially important for inquiry science learning in technology-enhanced environments (Quintana, Zhang & Krajcik, 2005; White & Frederiksen, 1998). Although students benefit from accurate selfassessment, research demonstrates that students have difficulty evaluating their own performance (Dunning, Heath & Suls, 2004). Many factors contribute to learners overestimating or underestimating their understanding, such as the nature of the assessment task, subject-matter knowledge, the learning environment, and motivation. Students may be able to assess their understanding of facts or simple recall items accurately, but overestimate their understanding on open-ended or explanation items (Zoller, Fastow, Lubezky & Tsaparlis, 1999). Even if students can accurately assess themselves, they may or may not go back to improve their understanding. This study explored how high school chemistry students assessed their own understanding in inquiry settings using criteria based on the knowledge integration (KI) framework. The KI perspective calls for conscientious, intentional learning and focuses on the connections among scientific ideas (Linn & Eylon, 2006). Engaging in knowledge integration encourages students to elicit prior knowledge, add new, normative ideas to these existing frameworks, develop criteria to examine new ideas and links that they have formed to their prior knowledge, and evaluate their understanding to sort and distinguish more productive and relevant ideas from less productive ideas. Traditional instruction tends to focus on eliciting and adding ideas, often leaving out critical processes of supporting students to examine, evaluate and refine their understanding (Linn & Eylon, 2011). The KI perspective can particularly help chemistry learners because many students struggle to make connections among concepts at molecular, observable, and symbolic levels. For example, students view equations of chemical reactions, such as CH4 + 2O2 → CO2 + 2H2O as problems to solve instead of a process of breaking and forming bonds among atoms (Krajcik, 1991). Prior research demonstrates that curricula designed with a knowledge integration approach benefit chemistry learners (Linn et al., 2006). This study used the chemical reactions curriculum unit within the Web-based Inquiry Science Environment (WISE), a computerbased environment that scaffolds inquiry science projects (Slotta & Linn, 2009). WISE projects provide various tools to support knowledge integration and scientific inquiry, such as visualizations, concept maps, idea managers, embedded assessments and online discussions. In addition to scaffolding inquiry, WISE provides detailed log reports of how students progress through projects, including where and when they click on particular steps or interact with certain tools, and if and how they revise answers to embedded assessment steps. The WISE chemical reactions unit guides students through a one-week investigation of how chemical reactions relates to climate change using dynamic molecular visualizations. Prior research with this unit demonstrated that students often overestimate their understanding when using dynamic molecular visualizations, and prompting

© ISLS

18

ICLS2012

Volume 2: Symposia

explanations helped students identify gaps in their knowledge (Chiu & Linn, 2008). Building upon this research, this study explored: (1) if students could accurately assess their explanations using explicit knowledge integration criteria; (2) if evaluating explanations encouraged students to revisit and/or revise their explanations, and (3) if students’ ability to assess their explanation had any impact on overall learning of the unit.

Method
High school chemistry students (n = 93) from the same teacher completed chemical reactions as part of normal class instruction. After students’ first interactions with visualizations, they explained what they learned. The next step provided a rubric for students to evaluate their explanation based on the scientific ideas and connections within the explanation. Students rated their explanation and provided a justification for their evaluation. Students’ KI ratings were compared to researcher ratings of the same explanations. Log data were analyzed to determine if students revisited or revised their explanations. Pretest and posttest assessments that contained both conceptual and self-assessment items were used to determine overall impact on self-assessment ability and learning.

Results
Compared to researcher ratings, almost half (48%) of the students accurately assessed their explanations. Students also overestimated (32%) and underestimated (20%) their scores. When asked to justify their selfevaluations, around 30% of students explicitly used KI terms of scientific ideas and/or connections (i.e. “We should get a three because we only made one scientific connection in our explanation and our explanation was not very complex.”) Most students made general statements about their understanding, such as, “i [sic] chose two because in some of the questions i only gave half an answer because some parts i didnt get that much, but i explained what i did know.” Log data results reveal that only 42% of students went back to either their explanation or the associated visualization step immediately after the self-assessment step. If students went back to their explanation, they were likely to revise their explanation for a higher KI score (χ2(1,93) = 14.4, p < 0.01). Student scores significantly increased from pretest to posttest, replicating earlier results that the Chemical Reactions unit helps students make connections among representations in chemistry (t(92) = 15.08, p < 0.01, ES g = 1.08). Regression analysis with pretest score, accuracy of explanation self-evaluation, and revisits to explanations as explanatory variables and posttest score as the dependent variable indicated that neither accuracy of the selfassessment nor revisiting the explanation significantly impacted posttest score. Controlling for pretest ability, explanation evaluation accuracy, and revisiting, students’ average self-ratings residuals tended to decrease from pretest to posttest, indicating less inflated self-ratings on the posttest (R2 = .2, F(4,88) = 5.50, p < .001; β = .44, t = 4.55, p < .001).

Conclusion
Results show that many students could use explicit knowledge integration criteria to assess their open-ended embedded explanations and encouraging students to go back can help them refine their explanations. Results suggest that encouraging students to assess their understanding helped students become more critical of their understanding from pretest to posttest. Since many students did not give KI-based justifications of their scores or go back to revise their explanation after giving themselves a less-than perfect score, students’ selfassessments may need to be accompanied by external feedback or support to encourage students to act upon their judgments.

Paper 2: Iterative Design Enhances Inquiry-Based Teaching
Cheryl Madeira, James D. Slotta Teacher professional development can be realistically said to translate directly into more effective student learning (Davis & Varma, 2008; Gerard et al., 2011; Krajcik & Blumenfeld, 2006). Teachers need to understand their topic domain, make effective choices of instructional strategy, and assess student learning in ways that inform their pedagogical approach. Substantial literature has addressed teacher knowledge and professional development (e.g., Borko et al., 1997), including the different kinds of knowledge held by teachers, such as general instructional strategies for a specific content domain called Pedagogical Content Knowledge (PCK). This type of teacher knowledge depends on experience and builds on prior knowledge. Teacher learning can be very difficult to measure, as it is often implicit and embedded within the teachers’ practice (Davis & Varma, 2008). How can a professional development model help capture and promote teacher learning? This paper presents data from a three-year design-based study that examined teacher learning in relationship to design and enactment of an inquiry-based science lesson. We investigated the impact of two interventions – reflection and peer-exchange – across three stages: (1) lesson planning, (2) enactment, and (3) revision of lesson, with the three stages repeated two or three times, iteratively, by teachers. We describe how

© ISLS

19

ICLS2012

Volume 2: Symposia

teachers developed understandings within the context of these activities, contributing to our understanding of situated collaborative learning and teacher professional development.

Method
This study used a design-based methodology to investigate the development of pedagogical content knowledge of nine science teachers (N = 9) in relation to their instructional practices (e.g., lesson design, preparations, classroom interactions, assessment and feedback), and student understanding. These teachers, who volunteered to participate in this study, entered with a range of experience (between 3 and 30 years) and disciplinary expertise (i.e., physics, biology, chemistry, or general science). The teachers came from 5 different schools located in a large urban city in North America and had a wide variety of technology experience and access. For confidentiality, all participants were given pseudonyms. Data sources include teacher surveys, interview questions, lesson plans, reflections (captured in a wiki), videotaped classroom enactments, field notes, student artifacts and responses, peer exchanges (on wiki, and in group meetings). This paper reports on 3 iterations of the study, which gradually introduced the conditions of the intervention. Iteration 1 of the study included four teachers (n = 4) who worked individually with the researchermentor to co-design, enact and reflect the inquiry-based science lesson. Iteration 2 added five more science teachers and improved the reflective prompts by connecting them directly to lesson planning and enactment, while adding community elements (face-to-face and online). In iteration 3, we continued to refine the reflections and community exchange, connecting teacher activities of lesson planning and enactment to topics of student prior knowledge, project-based learning and technology implementation. In order to analyze the various data from Wiki contributions, interviews and field notes, two coding schemas were developed – one for lesson planning and one for enactment – that included elements to measure teacher knowledge, following Grossman’s (1990) taxonomy: pedagogical content knowledge (PCK), content knowledge (CK), pedagogical knowledge (PK), technology pedagogical content knowledge (TPCK), contextual knowledge (CXK). Each code was ranked for quality with a value from 1 to 3, where 1 represented fragmented evidence of that knowledge, and 3 represented a highly coherent instance of the knowledge, and 2 was anything in between. In addition to these codes, we employed the characteristics of project-based learning as measures of inquiry-based lessons (e.g., student-driven questions, collaborative activities). Thus, if the lesson incorporated more inquiry-based approaches, the lesson plan itself scored higher. These coding schemes allowed us to capture improvements in teachers’ plans and enactments between iterations of our study, reflecting the added benefits of our improved intervention. They also allowed us to correlate those improvements to specific features of teacher knowledge and inquiry-based teaching. Only the lesson planning data will be presented in this paper, in relation to teachers’ prior knowledge. The elements in the coding schema reflected teachers’ understanding of student learning (i.e., PCK), including how teachers would respond to student ideas in their lesson revisions and subsequent enactments. The coding schema for Lesson Planning had high intercoder reliability (Kappa = .80).

Results
We hypothesized that when teachers revise lessons based on their reflections about the enactment, they would improve the quality of those lesson plans. This proved true for five of the six participants who completed two iterations, with the exception of Bill (teacher), whose lesson plan included less student collaboration and interactions in Iteration 2 than it did in Iteration 1 (based on results from analysis of lesson planning coding schema). The correlations provide evidence of a link between the quality of teachers’ reflections and the quality of their designed lessons. Teachers who were able to reflect in detail and link their lesson objectives to student learning were able to improve their lessons in the following iteration. Teachers who used the tools and followed the rules of reflection consistently showed improvements in their overall lesson planning score from the first to the second iteration.

Conclusion
Presumably, these improvements were due to teachers’ reflections about the strengths and liabilities of their lesson in the previous enactment. The teachers were able to ‘see’ what worked and didn’t work with the lesson plan, reflect on this in a concrete way, and then improve on their design and approach for the next iteration. Thus, scaffolded reflection throughout the course of these two planning and enactment cycles appears to play a productive role in helping teachers’ inquiry into their own practice, resulting in improved lesson designs and improved teacher knowledge of lesson planning.

© ISLS

20

ICLS2012

Volume 2: Symposia

Paper 3: Inquiry Learning with Elementary School Children: Prior or Subsequent Presentation of Theoretical Background Information?
Alexander Rachel, Christof Wecker, Eva Heran-Dörr, Hartmut Wiesner, Frank Fischer The idea behind inquiry learning is that learners discover knowledge about scientific phenomena independently (de Jong, 2006). As most learners struggle with the activities required for this purpose, it has been suggested to provide scaffolding for them (de Jong & van Joolingen, 1998). A more far-reaching goal of science education, however, is to acquire knowledge on the theory level that goes beyond observable phenomena. For example, in phenomena related to magnetism, theoretically assumed molecular magnets cannot be observed directly. A problem during inquiry learning might be that theories that assume such unobservable entities cannot easily be discovered because the learners’ hypothesis spaces (Klahr & Dunbar, 1988; van Joolingen & de Jong, 1997) will hardly contain assumptions about these entities. In a prior study we found that secondary school students can acquire more knowledge on the theory level if they receive a presentation of theoretical background information prior to investigating the phenomena themselves, whereas subsequent presentation of theoretical background information after an inquiry phase had no lasting effect (Rachel et al., 2010). The current study focused on the question whether primary school children can likewise acquire knowledge on the theory level and knowledge on the level of phenomena during inquiry activities if supported accordingly. In particular, we investigated the short- and longer-term effects of prior presentation of theoretical background information, subsequent presentation of theoretical background information and specific scaffolding during inquiry learning activities on knowledge on the level of phenomena and knowledge on the theory level.

Method
Three to four intact 4th grade classes from German primary schools were randomly assigned to each condition of a 2x2x2-factorial design with scaffolding (unspecific/specific), prior presentation of theoretical background information (without/with) and subsequent presentation of theoretical background information (without/with) as independent variables. The 612 participants from 31 classes were on average M = 9.25 (SD = 0.52) years old; 305 of them were girls, 307 were boys. The students first completed a 20 minute pretest. Then they worked on an inquiry unit about magnetism for 120 minutes in which they conducted hands-on experiments in dyads at up to eleven learning stations. Finally they completed a 25-minute posttest. Two months later they completed a 25 minute delayed posttest in their classrooms. Dyads in the conditions with unspecific scaffolding received general prompts to engage in the three inquiry activities of predicting, describing and explaining. Dyads in the condition with specific scaffolding received content- and task-specific prompts for these inquiry activities. In the conditions with prior presentation of theoretical background information, initially a 30-minute introduction to the theoretical background of magnetism was provided by a teacher. No such introduction was given in the conditions without prior presentation of theoretical background information. In the conditions with subsequent presentation of theoretical background information a 30-minute teacher-led wrap-up phase about the same topics as in prior presentation of theoretical background information took place at the end of the learning phase, while there was no such phase in the conditions without subsequent presentation of theoretical background information. Identical knowledge tests were used in the immediate and delayed posttests. They consisted of ten groups of tasks with several subtasks with multiple-choice, true-false and an open answering format that required the learners to insert labels into line drawings. The subtasks were coded as correct or incorrect on separate coding variables. The scale for knowledge on the level of phenomena comprised 15 coding variables and had sufficient reliability (immediate posttest Cronbach‘s α = .60; delayed posttest Cronbach‘s α = .61). The scale for knowledge on the theory level comprised 15 coding variables too and had satisfactory reliability (immediate posttest Cronbach‘s α = .81; delayed posttest Cronbach‘s α = .82). A subset of tasks from the parts of the test that captured knowledge on the level of phenomena was used for the pretest (9 coding variables, Cronbach‘s α = .61).

Results
The main results were the following: With respect to knowledge on the level of phenomena, in the immediate posttest no significant main or interaction effects of the independent variables were found. In the delayed posttest a small main effect in favour of the conditions with prior presentation of theoretical background information was detected, F(1; 22.13) = 5.51; p = .03; partial η2 = .02. With respect to knowledge on the theory level, in the immediate posttest a significant main effect in favour of the conditions with unspecific scaffolding, F(1; 22.12) = 8.29; p = .01; partial η2 = .04, and an

© ISLS

21

ICLS2012

Volume 2: Symposia

interaction effect between prior presentation of theoretical background information and subsequent presentation of theoretical background information, F(1; 22.12) = 4.46; p < .05; partial η2 = .02, were found. In particular, in the presence of prior presentation of theoretical background information or subsequent presentation of theoretical background information, learners acquired more knowledge on the theory level than without both of them. In the delayed posttest, however, this interaction effect disappeared. Instead, there was a significant main effect in favour of the conditions with prior presentation of theoretical background information, F(1; 22.10) = 7.55; p = .01; partial η2 = .03.

Discussion
The results indicate a superiority of unspecific scaffolding during inquiry activities with respect to knowledge on the theory level. With general prompts for inquiry, the learners have to set sub-goals for inquiry themselves. As a consequence, the function of each current activity for the overall goal of investigating the theoretical assumptions might be more transparent to the learners than with highly specific questions. Furthermore, prior presentation of theoretical background information appeared beneficial for knowledge on the level of phenomena and knowledge on the theory level. While immediately after the learning phase a summary in the context of subsequent presentation of theoretical background information might be at least a functional equivalent to prior presentation of theoretical background information (as evidenced by the interaction of prior and subsequent presentation of theoretical background information), in the long run prior presentation of theoretical background information seems to be superior. This effect might be explained by the opportunity provided by prior presentation of theoretical background information to apply the theory to be learned during inquiry activities, thereby yielding higher levels of knowledge on the theory level. In sum, this study demonstrates that primary school children can learn about challenging topics involving scientific theories that cannot readily be discovered, provided that they are supported appropriately.

Paper 4: Using Worked Examples to Scaffold Students’ Understanding of the Inquiry Learning Process
Yvonne G. Mulder, Ard W. Lazonder, Ton de Jong Technology-enhanced inquiry learning environments enable students to develop a deep understanding of science content and processes. Computer simulations have long been incorporated in these environments, and are increasingly being supplemented with opportunities for students to build computer models of the phenomena they are investigating via the simulation. Even though inquiry and modeling are potentially powerful ways of learning, students often lack the skills to take full advantage of these activities (e.g., de Jong & van Joolingen, 1998). A recent study showed that domain novices are quite capable of identifying relevant variables, but experience considerable difficulties in specifying the relations between these variables. Instead of gradually working toward a full-fledged scientific equation to specify a relationship, novices tried to induce and model these equations from scratch (Mulder, Lazonder, & de Jong, 2010). Novice learners’ tendency to ‘jump the gun’ points to a lack of knowledge about inquiry strategies that can in principle be controlled by organizing the inquiry learning process in successive phases of increasing complexity (i.e., model progression; White & Frederiksen, 1990). Model progression indeed enhances inquiry and modeling performance (e.g., Mulder, Lazonder, & de Jong, 2011; Swaak, van Joolingen, & de Jong, 1998)—but not to a sufficient degree. It thus seems that students need additional support in order to better understand what the activities in each model progression phase entail, and how they should be performed. The present study examined the effectiveness of heuristic worked examples to deliver this support. Heuristic worked examples were proposed by Hilbert and colleagues (Hilbert & Renkl, 2009; Hilbert, Renkl, Kessler, & Reiss, 2008) as means to extend the application of worked examples from well-structured problemsolving tasks to more ill-structured, and hence more complex tasks. Unlike ‘traditional’ worked examples that provide students with a single algorithm to solve one particular type of problem, heuristic worked examples outline a series of problem solving strategies and demonstrate their usage in or across a range of related tasks. Heuristic worked examples have been successfully applied in, for instance, concept mapping and second language learning tasks (Renkl, Hilbert, & Schworm, 2009), and are expected to be beneficial to inquiry and modeling tasks as well. The present study sought to validate this expectation by comparing the learning and performance of high-school students who worked in an inquiry learning environment with modeling facilities. All students were supported by model progression, but only students in the worked example condition received additional heuristic worked examples that exemplified the activities students should perform within each of three model progression phases. Students in this condition were accordingly expected to exhibit more proficient inquiry and modeling behavior and, as a result, perform better and learn more than students from the control condition who were not supported by heuristic worked examples.

© ISLS

22

ICLS2012

Volume 2: Symposia

Method
Eighty-two high-school students (aged 15-17) were randomly assigned to either the worked examples condition (n = 46) or the control condition (n = 36). Students in both conditions had to investigate a charging capacitor and create a computer model of its behavior. This task was divided in three model progression phases that asked students to first identify and sketch the model with its variables and relations, then specify all relations in qualitative terms (e.g., if resistance increases, then current decreases), and finally transform these qualitative specifications into physics equations (e.g., I = V / R). Students in the worked examples condition could consult two heuristic worked examples for each phase. These examples came in the form of annotated videos that showed the inquiry respectively modeling activities of an anonymous person on a comparable task in a different domain. Students in the control condition did not receive these worked examples. The study took place during four 50-min science lessons. In lesson 1, students were introduced to the learning environment and completed the knowledge pretest which addressed the meaning of key domain concepts (i.e., voltage source, resistance, capacitor, and capacitance) and the physics equations that govern the behavior of a charging capacitor. Lesson 2 and 3 were devoted to the inquiry and modeling task, and lesson 4 was used to administer the knowledge posttest. The posttest contained the same items as the pretest, although phrased in modeling terms to maximize resemblance with the learning task, plus six additional items about all qualitative relations in the simulation's underlying model.

Results
Main findings indicate that students in both conditions had comparable and low pretest scores, F(1, 80) = 0.03, p = .866, needed quite the same amount of time on task, F(1, 80) = 0.70, p = .404, but spent this time differently. As instructed by the worked examples, students in this condition did more experiments with the simulation, F(1, 80) = 12.57, p = .001, and took more time to analyze and interpret the outcomes, F(1, 80) = 9.37, p = .003. Control students, by contrast, largely ignored the simulation and spent most of their time creating and testing their model, F(1, 80) = 57.00, p < .001. This proved rather ineffective, as students in the worked example condition performed significantly better, F(2, 79) = 7.65, p = .001. That is, their models contained both correct variables, F(1, 80) = 15.38, p < .001, and relations, F(1, 80) = 9.45, p = .003. Despite this performance difference, posttest scores were comparable across conditions, F(1, 75) = 0.10, p = .759, suggesting that worked example students performed better, but did not learn more.

Conclusion
These results confirm two of the three expectations. As predicted, heuristic worked examples caused students to perform the inquiry and modeling activities as intended, and enhanced the quality of their models. The expected difference in posttest scores failed to show, which suggests that heuristic worked examples have an immediate effect on students’ learning activities and performance, but little impact on the knowledge students (should) attain from these activities. This suggests that scaffolding inquiry learning strategies is insufficient to acquire domain knowledge; additional content explanations might be needed for students to develop (a deep) understanding of the topic of inquiry.

References
Borko, H., Mayfield, V., Marion, S., Flexer, R., & Cumbo, K. (1997). Teachers’ developing ideas and practices about mathematics performance assessment: Successes, stumbling blocks, and implications for professional development. Teaching & Teacher Education, 13, 259–278. Chiu, J., & Linn, M. C. (2008). Self-Assessment and Self-Explanation for Learning Chemistry Using Dynamic Molecular Visualizations. In International Perspectives in the Learning Sciences: Cre8ting a Learning World. Proceedings of the 8th International Conference of the Learning Sciences (Vol. 3, pp. 16-17). Utrecht, The Netherlands: International Society of the Learning Sciences, Inc. Davis, E. A., & Varma, K. (2008). Supporting teachers in productive adaptation. In Y. Kali, M. C. Linn, & J. E. Roseman (Eds.), Designing coherent science education: Implications for curriculum, instruction and policy (pp. 94–122). New York, NY: Teachers College. de Jong, T. (2006). Technological advances in inquiry learning. Science, 312, 532 f. de Jong, T. & van Joolingen, W. R. (1998). Scientific discovery learning with computer simulations of conceptual domains. Review of Educational Research, 68(2), 179-201. Dochy, F., Segers, M. & Buehl, M. M. (1999), The Relation between Assessment Practices and Outcomes of Studies: The Case of Research on Prior Knowledge. Review of Educational Research, 69(2), 145-186. Dunning, D., Heath, C., & Suls, J. M. (2004). Flawed self-assessment. Psychological Science in the Public Interest, 5(3), 69.

© ISLS

23

ICLS2012

Volume 2: Symposia

Gerard, L.F., Varma, K., Corliss, S.B. & Linn, M.C. (2011). Professional Development for Technology Enhanced Inquiry Science. Review of Educational Research, 81(3), 408-448. Gijlers, H. & de Jong, T. (2005). The Relation between Prior Knowledge and Students’ Collaborative Discovery Learning Processes. Journal of Research on Science Teaching, 42(3), 264-282. Grossman, P. L. (1990). The making of a teacher: Teacher knowledge and teacher education. New York, NY: Teachers College Press. Hilbert, T. S., & Renkl, A. (2009). Learning how to use a computer-based concept-mapping tool: Selfexplaining examples helps. Computers in Human Behavior, 25, 267-274. Hilbert, T. S., Renkl, A., Kessler, S., & Reiss, K. (2008). Learning to prove in geometry: Learning from heuristic examples and how it can be supported. Learning and Instruction, 18, 54-65. Klahr, D. & Dunbar, K. (1988). Dual space search during scientific reasoning. Cognitive Science, 12(1), 1-48. Krajcik, J. (1991). Developing students' understandings of chemical concepts. In S. Glynn, R. Yeany, & B. Britton (Eds.), The psychology of learning science (pp. 117-147). Hillsdale, NJ: Erlbaum. Krajcik, J. S., & Blumenfeld, P. C. (2006). Project-based learning. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 317–333). New York, NY: Cambridge University Press. Linn, M. C., & Eylon, B.-S. (2006). Science education. In P. A. Alexander & P. H. Winne (Eds.) Handbook of Educational Psychology, 2nd edition. Mahwah, NJ: Erlbaum. Linn, M. C., & Eylon, B.-S. (2011). Science Learning and Instruction: Taking Advantage of Technology to Promote Knowledge Integration. New York: Routledge. Linn, M. C., Lee, H.-S., Tinker, R., Husic, F., & Chiu, J. L. (2006). Teaching and assessing knowledge integration in science. Science, 313, 1049-1050. Mulder, Y. G., Lazonder, A. W., & de Jong, T. (2010). Finding out how they find it out: An empirical analysis of inquiry learners’ need for support. International Journal of Science Education, 32, 2033-2053. Mulder, Y. G., Lazonder, A. W., & de Jong, T. (2011). Comparing two types of model progression in an inquiry learning environment with modelling facilities. Learning and Instruction, 21, 614-624. Quintana, C., Zhang, M., & Krajcik, J. (2005). A framework for supporting metacognitive aspects of online inquiry through software-based scaffolding. Educational Psychologist, 40(4), 235-2244. Rachel, A., Wecker, C., Heran-Dörr, E., Waltner, C., Wiesner, H. & Fischer, F. (2010, April/May). A place and a time for expository instruction during inquiry learning? Its role for the understanding of scientific theories that are hard to discover. Paper presented at the Annual Meeting of the American Educational Research Association (AERA) "Understanding Complex Ecologies in a Changing World", Denver, 30 April – 4 May, 2010. Renkl, A., Hilbert, T., & Schworm, S. (2009). Example-based learning in heuristic domains: A cognitive load theory account. Educational Psychology Review, 21, 67-78. Scardamalia, M., & Bereiter, C. (1991). Higher levels of agency for children in knowledge building: A challenge for the design of new knowledge media. The Journal of Learning Sciences, 1, 37-68. Swaak, J., van Joolingen, W. R., & de Jong, T. (1998). Supporting simulation-based learning; the effects of model progression and assignments on definitional and intuitive knowledge. Learning and Instruction, 8, 235-252. van Joolingen, W. R. & de Jong, T. (1997). An extended dual search space model of scientific discovery learning. Instructional Science, 25, 307-346. White, B. Y., & Frederiksen, J. R. (1990). Causal model progressions as a foundation for intelligent learning environments. Artificial Intelligence, 42(1), 99-157. White, B., & Frederiksen, J. (1998). Inquiry, modeling and metacognition: Making science accessible to all students. Cognition and Instruction, 16(1), 3-118. Zoller, U., Fastow, M., Lubezky, A., & Tsaparlis, G. (1999). Students’ self-assessment in chemistry examinations requiring higher- and lower-order cognitive skills. Journal of Chemical Education, 76(1). 112-113.

© ISLS

24

ICLS2012

Volume 2: Symposia

Assessing Interests in the Service of Supporting Personalized Learning Through Networked Resources
Brigid Barron, Caitlin K. Martin, Stanford University School of Education, 485 Lasuen Mall, Stanford, CA 94305-3096 Email: barronbj@stanford.edu, ckm@stanford.edu Robert B. W. Ely, Mary Ainley, Jon Pearce, Disna Wijayawickrama, Josie Chan, Psychological Sciences, University of Melbourne, Victoria 3010 Australia Email: r.ely@student.unimelb.edu.au, maryda@unimelb.edu.au, jonmp@unimelb.edu.au, d.wijayawickrama@student.unimelb.edu.au, josiecchan@gmail.com K. Ann Renninger (Organizer), Mark Chin, Dennis Fan, Department of Educational Studies, Swarthmore College, 500 College Ave., Swarthmore, PA 19081 Email: krennin1@swarthmore.edu, mchin1@swarthmore.edu, dennisfan9@gmail.com Abstract: Papers by four researchers working to assess interest online in order to support learning provide the basis for discussion in this symposium. Session participants will use current work to address four questions: (a) What are the research questions being addressed and what is the context of the research? (b) How are interest and learning conceptualized and measured in this study, especially earlier phases of interest and learning? (c) What adjustments were made to the measures due to context? (d) What are the implications of these data for practice and the design of learning resources? Following these presentations, discussion with the audience will focus on how interests should be assessed for the advancement of theory, pedagogy, and the design of networked resources.

Overview of Symposium Panel
Interest has been repeatedly demonstrated to affect the attention, goals, and learning strategies of learn. Even if they initially have low levels of self-efficacy, lack learning goals, and/or have difficulty taking initiative or selfregulating, their interest can be supported to develop (see Renninger & Hidi, 2011, for a review). How interest can be supported to develop is linked both to triggers for interest and to the forms of interaction that are available, recognized, and engaged (see Ainley, 2012; Barron, Martin, Takeuchi, & Fithian, 2009). The present symposium brings together a group of researchers working to assess interest in a variety of online educational contexts in order to allow discussion of their methods and decision making, and consideration of the implications of their findings for supporting personalized learning. Session participants will use current work to address four session questions: 1. 2. 3. 4. What are the research questions being addressed and what is the context of the research? How are interest and learning conceptualized and measured in this study, especially earlier phases of interest and learning? What adjustments were made to the measures due to context? What are the implications of these data for practice and the design of learning resources?

Following the presentations, the panelists will engage the audience in thinking together about how and why interests should be assessed in online contexts in order to support learning.

Presentation Summaries
Developing Interest in Science: The Role of Prior Experience and Engaging Classroom Experiences Brigid Barron and Caitlin K. Martin Problem
A recent longitudinal study documented a statistically significant association between eighth graders’ projected careers and their completion of a related college degree (Tai, Liu, Maltese, & Fan, 2006). Students who indicated that their ideal jobs were science related were three times more likely to complete a degree in science than students who identified non-science related jobs as ideal. In fact, interest was a stronger predictor of the completion of a science major than achievement test scores in math, a subject that can be a barrier to those interested in natural science. This research suggests that if we want to better understand the origins of interest in science as a career, we focus attention on the resources and experiences that learners encounter that trigger and sustain interest during the preschool, elementary, and middle school years, and that we track experiences both in and out of school (Barron, 2006). In particular, we need to understand how networked affinity groups, learning

© ISLS

25

ICLS2012

Volume 2: Symposia

resources, and online communities can become spark and sustain interest (Ito, et al, 2009). The current paper reports on a comparative study that had two primary goals. The first was to contribute theoretically to our basic understanding of the emergence of interest in science. The second was to provide design recommendations about how networked resources might nurture the development of interest in science as a career, as well as science more generally as a topic relevant to all citizens.

Methods
We focus on a middle school curriculum unit that has strong potential to generate interest, and we carry out quantitative and qualitative research to understand how prior experiences and different aspects of the curriculum interact to generate and sustain patterns The specific research questions we address include: (1) How do students with more and less expressed interest in science experience different aspects of an engaging citizen science curriculum?; (2) How do pre-course science experiences, hobbies, and social learning networks, and use of networked learning resources nurture interest in scientific content? Vital Signs is a citizen science networked system located in the state of Maine, linked statewide to schools and accessible not only to the focal participants (teachers and students in seventh and eighth grade classrooms), but to anyone who wants to learn and contribute. Vital Signs has high potential to generate excitement, interest, and a desire to learn about the natural world by engaging learners directly in observing, documenting, and sharing information about real world phenomena. Through Vital Signs, learners participate in learning activities designed around scientific issues in their local communities using authentic tools and collaborating with scientists. Interest in science was measured through a set of survey items and data was collected from 217 middle school students taught by one of four teachers. We examined this data in relation to the more situated judgments of interest in the unit. In the qualitative analysis, eight case study learners were selected based on their initial differential interest in a multi-week Citizen Science project. Mid-way through the unit, teachers made judgments of learner interest. Half of the cases represent low Vital Signs interest and half high Vital Signs interest. Because we were focused on the development of interest in relation to an opportunity to become interested, we followed learners out in the field, observed classroom sessions, and collected examples of work. Because we also wanted to understand their broader learning ecology, we interviewed students to assess both personal social learning networks related to science activities, and interest in science.

Results
In our presentation we focus on a subset of the survey items. For example, we asked students to indicate how much they agreed with the sentence, “I find working with science very interesting” on a likert-scale from 1 (disagree strongly) to 5 (agree strongly). Of these students, 21% disagreed with the statement, 22% were neutral, and 52% agreed. Open-ended questions that asked students to indicate whether they could become a scientist and explain why revealed the majority of students believed they could not become a scientist and also gave explanations that included low levels of domain interest (e.g., “it is boring”), judgments about the nature of the work (e.g., “ it would be stressful”), and low ratings of competence or other personal characteristics (e.g., “I’m not smart enough”; “I have no patience”). This disparity in beliefs about science and science learning was also found between individual student ratings of their science interest and teacher ratings of a child’s interest in the Vital Signs project. Our case study data helps to provide an account of these patterns and we will showcase a subset of our cases in our talk. For instance, Laura, an outgoing 13-year-old whose science teacher saw her as highly engaged with Vital Signs but who indicated that she did was not interested in working with science. More detailed observations and interviews shared that Laura had always been extremely involved with sports but also had an affinity towards science. She enjoyed experimenting and told of mixing household products together to see what would happen when she was in elementary school. Though she used Facebook and played some online games, she was not active online with science-related content until Vital Signs. Her team worked efficiently to get things done and Laura and a teammate even set up a mini-experiment within the observation. They captured a bug they knew ate the plant they were looking for and put both it and a leaf sample from their plant in a container to see what happened. She recounted seeing the natural world around her differently due to her species exploration and documentation and was very proud of the positive ID she made and the science expert confirmation she received online. Josh on the other hand, a shy and quiet seventh grader, was rated as not so engaged in Vital Signs by his teacher, but rated himself very engaged in working with science. Interviews with Josh shared stories of many activities done with his entire family, including dirt-biking in the woods around a trailer park, playing first-person-shooter video games, and using Internet resources to pursue science-related topics. Josh’s parents became interested in a dwarf star after Josh shared the story he had seen on a you tube video and the entire family talked about new developments, checked for more information online, and rented related videos from Netflix. The family was also using Google Earth together to locate relatives’ houses around the United States. Despite these exciting family-wide science and technology activities, Josh did not connect with the Vital Signs project. Josh’s team did not manage to successfully submit their submission to Vital Signs and he felt that his team members did not listen to him, though he expressed interest in learning more about the

© ISLS

26

ICLS2012

Volume 2: Symposia

things that frustrated him, including developing a research question and uploading and organizing digital photos between a camera and the computer.

Discussion
Our detailed case accounts have implications for the design of both classroom experiences and networked resources. We conceptualize topical or activity-based interest as a dynamic psychological state that in its early phases is subject to rapid shifts in relation to a learners’ affective experience of the activity and in relation to identity fit (Hidi & Renninger, 2006). In addition to these metrics of individual interest, we assess the child’s environment for its potential to stimulate interest. Our research is intended to inform the design of networked resources. Our approach builds on methods developed within learning sciences research programs designed to provide rich case study portraits that can guide theory development and design. We plan to develop partnerships with the case study teachers that will allow us to engage them in imagining and prototyping new features and resources that will amplify the potential of cyberlearning for citizen scientist work. We are particularly focused on identifying opportunities to design resources that will allow for personalized pathways for teachers and students based on their specific domain or activity interests.

Establishing the Interests of Young People, a New Exploratory Approach: The My Interest Now for Engagement (MINE) Project Robert B. W. Ely, Mary Ainley, Jon Pearce Problem
A central concern for educators involves identifying what young people are really interested in, and at what level are they interested? Is it possible to trigger and/or establish both the content and level of student’s interests using a process of online exploration? In contrast to traditional approaches to identifying interest content and level—using ethnographies, observations, questionnaires and/or interviews, or even a good chat with a student—we have developed a new methodological approach to measure and trigger interest: the My Interest Now for Engagement (MINE) tool, using iFISH software (Pearce, 2008). The measurement of triggered interest “in the moment” is possible as the MINE tool possesses the ability to trigger and measure interest as part of the one process, and requires the student to respond with minimum reflection. Identifying the content and phase of interest in students can be critical in the teaching and learning processes of all students, but more particularly for those students who are disengaged from the learning process in school. The Four-Phase Model of Interest Development (Hidi & Renninger, 2006) provides an experiential, cognitive, and affective framework for investigating the development of interest as a dynamic psychological state. Interest is triggered, grows, stabilizes, or recedes depending on an individual’s experience, cognition, and affect.

Method
MINE uses an interactive and playful environment to facilitate students’ reporting and commenting on their interests. Students indicate the experiential, cognitive, and affective dimensions of their involvement in a wide range of interest content generated by, and specific to, the context of their student community. A wide range of factors including age appropriate content, language, culture, sub-culture and gender has a relationship with the potential for interest in content. Students identifying interest content appropriate to their own context was perceived as integral to the MINE method. The interactive nature of the tool allows students to explore and select from a pool of 60 potential interests aggregated from an original 160 potential interests generated by six participants aged 14 to 16. Interactive sliders that represent five dimensions of experience, indoor-outdoor, creative-practical, technological-natural, solitary-social, and serious- fun, facilitate exploration. Manipulation of these sliders animates and re-orders graphical representations of the 60 potential interests in an online and real-time environment. Each student is able to select from three to eleven different interests. In contrast to previous approaches, this process allows for triggering of new interests as well as reporting existing interests. Analysis of the reliability and validity of the process of the MINE tool is derived from data generated by 136 first-year university students and 60 secondary students. Analysis of content is derived from 260 secondary students, aged 12 to 15, enrolled in two low-socioeconomic-status (low SES) secondary schools.

Results
The findings of the project were obtained in a three-stage process. Firstly, 136 first-year university students established values for the settings of five interactive sliders for 60 potential interests. Secondly, evidence for reliability and validity of the MINE process was established using 60 secondary students in a test/re-test pilot study. There was correlation between experiential, cognitive, and affective dimensions of interests selected by the same students on two occasions, two months apart. Finally, once development of the MINE tool was completed, data gathered using MINE established patterns of interest content aggregated into broader categories as described in Figure 1. Results describe a complex pattern of the 1599 interests for the 260 students in these two schools.

© ISLS

27

ICLS2012

Volume 2: Symposia

Figure 1. Interest content for 260 students. The students as a cohort most frequently reported being the most interested in electronics (10%) and money (8%), and were very interested in sport (5%), clothes (5%), game consoles (5%) and shoes (3%). Students had no reported interest in “being an individual” or “global warming.” Quantitative data relating to the experiential and cognitive and affective dimensions of interest were supplemented by qualitative data gathered from students’ comments. Four profiles of interest dynamic were identified by cluster analysis of the experiential and cognitive variables: duration (how long?), frequency (how often?), effort (how much?) and flow (sense of time passing?). The first profile is characterized by relatively low duration and frequency and moderate effort and flow, represents a beginning interest. The second profile is defined by high duration and frequency, low effort, and very low flow and represents maintained interest. The third interest profile is characterized by very high duration, frequency, effort, and flow and represents active interest. The fourth interest profile showed very high duration, frequency, and flow but negligible effort, indicating effortless interest. The affective dynamics associated with these four interest profiles are also examined. The exploratory nature of the MINE tool allows student to select objects, activities, or ideas that interest them with duration of “just now,” and there were 53 instances of such selections. These 53 triggered interests possess a lower frequency but greater effort and flow than beginning interests. Further analysis of qualitative data suggests that students whose interest was triggered by the MINE process itself may have had previous experience with the selected interest content, emphasizing the dynamic nature of interest development.

Discussion
Almost by definition, educators wish to engage students in the process of learning. The MINE findings provide data that will be useful for the development of content for learning resources intended to provide engaging curriculum. Establishing interest content, and identifying four interest profiles—beginning interest, maintained interest, active interest, and effortless interest—describes both what these 260 students are interested in, individually and collectively, and at what level they are interested. This understanding can inform what curriculum is presented to them, and how these students may be engaged with that curriculum. Subsequent research may use the unique exploratory nature of the MINE process to examine content students might be interested in, described by Hidi and Renninger (2006) as triggered situational interest.

The Tension Between Autonomy, Choice, and Structure when Task Interest is Low Mary Ainley, Disna Wijayawickrama, Josie Chan Problem
It is clear from a range of research findings that interest and self-regulated learning skills function together to produce high-quality learning and performance. Students who are interested in the task, or who have welldeveloped individual interest in the task domain, readily accept new learning challenges and revel in being able to choose their own topic. However, this is not the case for students who are not interested in the task domain or who do not have their interest triggered when the new task is presented. One interpretation of these general findings is that the functional relation between interest, self-regulation, and task characteristics may take different forms according to students’ level of initial task interest. Online tasks designed to encourage problem solving, reasoning, and argumentation skills by providing opportunities for choice and decision making may be

© ISLS

28

ICLS2012

Volume 2: Symposia

advantageous for students who are highly motivated because their interest has been activated, and with strong motivation, their learning skills are energized. At the other end of the spectrum where students respond to the task with low interest, those same task characteristics providing autonomy and choice may mean that students never quite get into the task. The skills they have are not engaged, with resulting low levels of achievement. Just as students with triggered situational interest may require support to sustain that interest, providing support in the form of prompts and scaffolds may serve to increase interest in students who initially have low task interest, thereby assisting them to engage with the task. Hence, it is our expectation that providing more structure for students who have low initial task interest will function to increase interest as the task progresses. In addition, task engagement will increase as manifested in the quality of note taking, and higher task engagement will be associated with higher quality answers.

Methods
We compare how students with high task interest and students with low task interest respond to two versions of the same online, open-ended problem task, solving a murder mystery. One version of the task gave students autonomy to direct their own path through the task. Students chose when and how to access a set of information resources to generate a reasoned answer to the problem. In the second version, access to the information resources followed a fixed sequence and each information resource came with prompts or scaffold questions to guide how students approached the information. Two groups of 8th-grade students from the same secondary school participated in different years. The first version (autonomy version) was completed by 80 students, 55 male and 25 female, and the second version (scaffolding version) by 65 students, 34 males and 31 females. Interest was conceptualized as an immediate psychological state and measured as a rating on a 5-point Likert scale. However, while for some students this state can be akin to a triggered situational interest, for other students who bring to the task some level of interest in the task domain, this immediate psychological state may represent deeper knowledge and valuing of the domain. To record the trajectory of on-task interest, the online interest rating scale appeared three times; after the task had been explained but just before students started work on the problem, mid-way through the problem, and immediately after students submitted their answer. Note-taking quality was assessed as (i) the number of items of important information recorded, and, (ii) the level of organization of the items of information recorded using the SOLO taxonomy (Biggs & Collis, 1982). Answer quality was assessed using an index derived from the theme development scoring in the Wechsler Individual Achievement Test (WAIT-II, 2002) and the Solo Taxonomy.

Results
When high- and low-initial-task-interest groups were compared across the autonomy and scaffolding versions of the task, there was no significant difference in the interest trajectory for the high interest groups. Initial interest was high and this was maintained across the task. However, there were significant differences in the trajectories for the low interest groups. For the autonomy version low interest remained low across the task, while for the scaffolding version interest increased across the task. Note-taking quality also varied between the groups. Of particular significance is the finding that quality of note taking for the low interest group with the scaffolding version showed significantly higher-quality notes both on number of important items of information and integration of information. In addition, this higher-quality note taking was significantly related to increased interest. Further analyses will model relations between interest, note taking, and answer quality.

Discussion
In this investigation, interest has been conceptualized as an immediate psychological state that may represent an immediate response to the new task—a triggered situational interest. It may also draw on more developed interests that have been activated by the new task. However, in previous research we have shown that within the immediate situation of an online open-ended problem task, the initial level of interest triggered is predictive of the trajectory across the whole task. The scaffolding version of the task was an attempt to provide support for students who initially have low task interest and who generally show little task engagement if left to make their own way. The implication is that learning conditions supporting self-regulation and task engagement may vary according to students’ interest. These findings are consistent with the relation between interest development and teacher support as described in the Four-Phase Model (Hidi & Renninger, 2006). In addition, they indicate that the provision of support for student learning cannot be reduced to “one size fits all.”

© ISLS

29

ICLS2012

Volume 2: Symposia

Interest and Productive Disposition: Pre-service Teachers and Virtual Fieldwork in Mathematics K. Ann Renninger, Mark Chin, and Dennis Fan Problem
Session questions will be addressed using findings from a multi-method, use-informed study of pre-service teachers’ (PTs) work with the Math Forum’s Virtual Fieldwork Sequence (VFS)—online modules designed to enable PTs to develop their capacities to work with challenging mathematics and develop their abilities to mentor elementary-aged students. Evidence from research on interest (e.g. Hidi & Renninger, 2006), self-concept of ability (e.g., Denissen, Zarrett, & Eccles, 2007), anxiety (e.g, Ma & Xu, 2004), and level of prior mathematics (e.g., Carlson, Oehrtman, & Engelke, 2010) suggests that each of these variables points to distinct ways in which PTs are likely to engage with mathematics. There are also indications that these variables are potentially coordinated. In the present study, we first determined that for the PTs, these variables formed two clusters which we termed higher and lower productive disposition, following on the National Research Council’s (2001) discussion of productive disposition as a critical but little understood strand of mathematical proficiency. We then examined the impact of the level of PTs’ productive disposition on (a) the PTs’ developing abilities to engage in mathematical thinking and mathematical communication, and (b) their perceptions of mathematics and the learning environment. Lastly, we considered the implications of these findings for design.

Methods
Target participants included a total of 88 (13 M, 75 F) PTs who participated in one of four different class implementations (add-on, supplement, integrated supplement, and class focus). A matched control group for each implementation included a total of 49 (4 M, 45 F) PTs. Three sources of data inform the present study: (a) pre-and post-module online surveys that included tasks parallel to the virtual fieldwork, (b) artifacts from PTs’ work with module tasks, and (c) instructor interviews. The pre- and post-module online descriptive surveys were used to gather data regarding motivation and learner characteristics expected to describe their productive disposition. Artifacts from the PTs’ work with the modules were used to assess their mathematical thinking and mathematical communication. Finally, interviews of the PTs instructors were used to inform understanding of data from PTs’ responses to the survey and their work with the modules.

Findings
While PTs varied individually, two clusters of PTs were clearly identified: those with lower productive disposition had low interest in mathematics, low self-concept of ability in mathematics, high anxiety about mathematics, and no calculus background, whereas those with higher productive disposition had high interest in mathematics, high self-concept of ability in mathematics, low anxiety about mathematics, and calculus background. In terms of engagement and learning, PTs with lower productive disposition engaged the content of the modules and learned from them, while those with higher productive disposition did not, or at least not in the same way. The PTs with lower productive disposition made positive gains with respect to their ability to communicate about mathematics as well as their ability to address mathematical content during mentoring. Their matched controls did not. The work of the PTs with higher productive disposition, on the other hand, declined on the same indicators on which the PTs’ with lower productive disposition improved. Their matched controls did not evidence the same declines. It appears that the present design of the VFS may primarily benefit the PTs with lower productive disposition and may not meet the needs of the cluster of PTs with higher productive disposition. Given that the performance of the PTs with higher productive disposition declines, it seems that they could benefit from different supports to engage with mathematics. For example, while PTs with lower productive disposition appear to be learning to be fluent and to communicate about mathematics, this is different from learning to provide elementary students with either effective strategies for working with mathematics or supportive mathematical feedback. They are not yet able to accomplish goals such as these. However, developing strategies for working with mathematics and learning to provide supportive mathematical feedback could be set as expectations for PTs with higher productive disposition. PTs with higher productive disposition were fluent with mathematics when they started work with the VFS and appear to need support to continue to develop their abilities to work with mathematics in their mentoring.

© ISLS

30

ICLS2012

Volume 2: Symposia

Discussion
Interest is here defined developmentally as both a psychological state and a predisposition to return to engagement with particular disciplinary content (e.g., mathematics)(Hidi & Renninger, 2006). Phases in the development of interest have been identified as ranging from an initial triggered situational interest that may last only a few moments to a well-developed individual interest that is relatively long lasting. In the present study, interest was assessed using survey items addressing the PTs’ feelings, knowledge, and value for mathematics; however, inconsistency in response patterns on Likert ratings and PTs’ descriptions of their preferences and patterns of engaging mathematics revealed that what “mathematics” meant to PTs varied, requiring that their responses be cross-validated. As such, PTs who did not like to work with challenging problems or talk about mathematics were not considered to have a developed interest in mathematics despite their own rating of themselves as high on “liking.” We think that the PTs in this study, regardless of whether they have lower or higher interest, are likely to be in earlier phases of interest development. As a result, we consider the distinction we have made between levels of productive disposition as lower and higher to be relative to the population, potentially requiring qualification of how the findings from this study are understood. In terms of implications, findings from the present study do suggest that learners in both phases of interest that were identified, and with differing levels of productive disposition, need to be supported to learn, and that their strengths and needs as learners differ. In the case of the VFS, it appears that at least two different tracks could be built into the VFS modules allowing learners to opt for the type of problems and/or the types of challenges they take on in providing mentoring to others. Providing instructors with information about differences in the ways in which their students are positioned by their productive disposition to work with online modules is also warranted.

References
Ainley, M. (2012). Students’ interest and engagement in classroom acitivities. In S.L. Christenson, A.L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 283-302). New York: Springer. Azevedo, F. W. (2006). Personal excursions: Investigating the dynamics of student engagement. International Journal of Computers for Mathematical Learning, 11, 57–98. Barron, B. (2006). Interest and self-sustained learning as catalysts of development: A learning ecologies perspective. Human Development, 49, 193–224. Barron, B., Kennedy-Martin, C., Takeuchi, L., & Fithian, R. (2009). Parents as learning partners in the development of technological fluency. International Journal of Learning and Media, 1(2), 55–77. Biggs, J. B., & Collis, K. (1982). Evaluating the quality of learning: The SOLO taxonomy. New York: Academic Press. Carlson, M., Oehrtman, M., & Engelke, N. (2010). The precalculus concept assessment: A tool for assessing students’ reasoning abilities and understanding. Cognition and Instruction, 28(2), 113–145. Denissen, J. H., Zarrett, N. R., & Eccles, J. S. (2007). “I like to do it, I’m able, and I know I am”: Longitudinal couplings between domain-specific achievement, self-concept, and interest. Child Development, 78, 430–447. Hidi, S., & Renninger, K. A. (2006). The four-phase model of interest development. Educational Psychologist, 41(2), 111–127. Ito, M., Sonja B., Matteo B., boyd, D., Cody, R., Herr-Stephenson, B., Horst, H.A., et al. 2010. Hanging out, messing around, and geeking out: Kids living and learning with new media. Cambridge, MA: MIT Press. Ma, X. & Xu, J. (2004). The causal ordering of mathematics anxiety and mathematics achievement: A longitudinal panel analysis. Journal of Adolescence, 27(2), 165–179. Maltese, A. V., & Tai, R. H. (2011). Pipeline persistence: Examining the association of educational experiences with earned degrees in STEM among US students. Science Education, 95(5), 877–907. National Research Council (2001). Adding it up: Helping children learn mathematics. J. Kilpatrick, J. Swafford, and B. Findell (Eds.), Mathematics Learning Study Committee, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press. Pearce, J. M. (2008). A system to encourage playful exploration in a reflective environment. Paper presented at the EdMedia 2008 Conference. Renninger, K. A. & Hidi, S. (2011). Revisiting the conceptualization, measurement, and generation of interest. Educational Psychologist, 46(3), 168-184. The Psychological Corporation. (2002). Wechsler Individual Achievement Test (2nd ed.). San Antonio, TX. Tai, R. H., Liu, C. Q., Maltese, A. V., & Fan, X. (2006). Planning early for careers in science. Science, 312(5777), 1143–1144.

© ISLS

31

ICLS2012

Volume 2: Symposia

Acknowledgments
The research entitled “Developing Interest in Science: The Role of Prior Experience and Engaging Classroom Experiences” (Brigid Barron and Caitlin K. Martin), was supported by a grant from the National Science Foundation (REC-1124568). The research entitled “Interest and Productive Disposition: Pre-service Teachers and Virtual Fieldwork in Mathematics” (K. Ann Renninger, Mark Chin, and Dennis Fan) was supported by a grant from the National Science Foundation (DUE-0717732)

© ISLS

32

ICLS2012

Volume 2: Symposia

Supporting teachers in capturing and analyzing learning data in the technology-rich classroom
Peter Reimann, MTO, Tübingen, Germany; p.reimann@mto.de (Organizer); Friedrich Hesse, Knowledge Media Research Centre, Tübingen, Germany F.hesse@iwm-kmrc.de (Organizer); Peter Freebody, University of Sydney, Australia; peter.freebody@sydney.edu.au (Discussant) Gabriele Cierniak, Birgit Imhof, {g.cierniak, b.imhof}@iwm-kmrc.de; Barbara Wasson, Cecilie Hansen. University of Bergen & InterMedia, Uni Helse, Uni Research, Bergen, Norway; Cecilie Hansen {Barbara.Wasson, Cecilie.Hansen}@uni.no; Wilfrid Utz, BOC, Vienna, Austria, wilfrid.utz@boc-eu.com; Wolfgang Halb, Joanneum Research, Graz, Austria, wolfgang.halb@joanneum.at; Ravi Vatrapu, Computational Social Science Laboratory (CSSL), Dept. of IT Management; Copenhagen Business School, vatrapu@cbs.dk; Susan Bull, Matthew Johnson, Electronic, Electrical & Computer Engineering, University of Birmingham, s.bull@bham.ac.uk, m.d.johnson.1@bham.ac.uk; Rosemary Luckin, Brock Craft, Katerina Avramides, The London Knowledge Lab, Institute of Education, London. r.luckin@ioe.ac.uk.; Michal Kossowski, BOC Information Technologies Consulting, Warsaw, michal.kossowski@boc-pl.com Abstract: While the technology-rich classroom makes it comparatively easy to gather, store and access data on students’ activities, turning those into information on learning that can inform pedagogical decision-making is still hard to achieve. In this symposium, it is argued that teachers are an important if not the most important source of knowledge about the necessary diagnostic assessment methods, and that therefore teachers should be supported in describing, sharing and deploying these methods. We also argue that teachers should play a more central role in analyzing learning data for the purpose of creating knowledge about how specific pedagogical and technical innovations play out in the context of specific classroom situations, and describe a new approach to teacher-led inquiry into students’ learning data. Introduction Schools are slowly yet inevitably entering the information age. But while the level of technology infusion is increasing, and with it the capacity to distribute information for learning and gather information about learning quickly and efficiently, we are still far away from the vision of the school as a “high performance, personalized learning community” (Hamilton & Jago, 2010). The barrier to that is increasingly not the absence of information, but the absence of the right information, at the right time, in the right format. The classroom may be increasingly data-rich, but is still comparatively information-poor. One reason is that a good part of the data made available to teachers and students have limitations to inform pedagogical decision making: Large-scale assessment data are usually not linked to classroom practices and outcomes, and are not available close enough in time to learning and teaching activities (Crawford, Schlager, Penuel, & Toyama, 2008). Another reason is that classroom technologies that are closer to the performance level focus on activity tracking rather than knowledge tracking. For instance, classroom response systems, also known as "clickers", are an effective, easyto-use way for teachers to generate instant data about students’ thinking. However, classroom response systems do not in any way guide the teacher in formulating the 'right' questions, such as questions that can help to identify residual misconceptions in students. A third group of reasons for why learning data are often not brought to bear on classroom-level decision-making has to do with teacher capacity. Some studies report that teachers as harboring views of the nature and the role of evidence that are not conducive to data-oriented decision-making (Parr & Timperley, 2008). And even when teachers are appreciative of learning data, they often do not feel sufficiently qualified and/or not provided with sufficient time to be working with detailed learning data. This symposium brings together a number of researchers who are addressing the challenges of the datarich classroom in an international research cooperation funded by the European Commission, the NEXT-TELL project (www.next-tell.eu). NEXT-TELL seeks to address the emergent needs of teachers and schools through the participatory development of a set of methods and tools which support teachers’ engagement with advanced learning technologies, with a particular focus on the use of student data to promote innovation and change as a form of teacher professional development aligned to schools’ strategic planning goals. The project’s philosophy is that teachers need not only be seen as the users of classroom technologies, and the recipients of information, but also as the innovators of technology-supported teaching and assessment practices, and as the creators of knowledge about students’ learning. The project’s methodological approach is design-based research (Barab & Squire, 2004; Wang & Hannafin, 2005). In accordance with design-based research, we think that multiply

© ISLS

33

ICLS2012

Volume 2: Symposia

cycles are needed to successfully develop methods and tools together with teachers and within the classroom. Cierniak et al. present findings from a baseline study that probed into participating teachers’ conceptions of formative assessment in the classroom, and their conceptions of teacher-led research. As participating teachers come from a number of European countries, the findings reflect not only individual differences, but also differences that can be attributed to variations in educational policies and teacher training across nations. Using the base-line study and findings from the wider research literature as basis, the next three contributions describe a first set of interventions, which will be further refined by means of intervention studies with teachers in schools. Reimann, Utz and Halb describe an intervention that aims at enabling teachers to describe ICTintegrated formative classroom assessment methods, to share these with other teachers, and to deploy these rapidly in their classrooms. The main idea of this intervention is to make the process of formative assessment an open one, ‘open’ the sense that it can be ‘inspected’ by stakeholders, as well as open in the sense that all stakeholders—teachers, students, parents—can actively contribute to how it is conducted. Vatrapu, Bull and Johnson extend this idea to the manner learning data get analysed by suggesting a new form for distributing analytical work on such data. This contribution provides also a link from tactical, classroom-level decision making (What do I do next with my students?) to strategic, school-level decision making: What kind of innovations and reforms work for our students? The Teacher Inquiry into Student Learning (TISL) strand of work in the NEXT-TELL project is concerned with fostering in-service teachers’ professional development by providing new methods and tools for designing and sharing inquiry projects into students’ learning. It also aims to increase a school’s capacity for data-driven decision-making by means of leadership development, including ICT support for the strategic planning of teachers’ professional development. Luckin et al. describe the first steps into the development of the TISL methodology and the TISL software model.

Teachers’ Views of the Role of Formative Assessment and Teacher-led Research. Gabriele Cierniak, Birgit Imhof, Friedrich Hesse, KRMC, Tuebingen, Schleichstraße 6, Germany, {g.cierniak, b.imhof, f.hesse}@iwm-kmrc.de; Barbara Wasson, Cecilie Hansen. University of Bergen & InterMedia, Uni Helse, Uni Research, Bergen, Norway; Cecilie Hansen {Barbara.Wasson, Cecilie.Hansen}@uni.no. The NEXT-TELL project aims on the one hand to enhance adaptive teaching practices in the technology-rich classroom by co-designing and co-developing ICT that support evidence-based formative assessment. On the other hand the project aims to support teachers’ professional development with regard to educational ICT-use and formative E-assessment by co-designing and co-developing a method of teachers’ inquiry into student learning (TISL). Hence, the project wants to support teachers as key stakeholders of educational innovation with regard to formative E-assessment (Reimann, Utz, Halb, this symposium) and teacher research (Luckin, Craft, Avramides, & Kossowski, this symposium). In order to work with teachers interested in the project according to their current teaching practices, we investigated their current views on and practices in formative E-assessment as well as teacher research in a baseline study. We first developed two semi-structured interviews. Interview 1 was designed to find out more about teachers’ current practices concerning (1) lesson planning, (2) classroom teaching, (3) monitoring, (4) assessment, (5) providing feedback, (6) homework, (7) communication with students, and (8) communication with parents. At the beginning of each topic teachers were asked to tell how they currently practice the topic (e.g., how they assess their students). If not already included in teachers’ reports, we asked whether they use any kind of ICT in practice and whether they wished to be supported in any activities by ICT. Interview 2 was designed to find out more about the (country-) specific school systems the teachers are in concerning (1) assessment, (2) teacher research, and (3) professional development. Here, teachers were asked about their understanding of the three topics and their interrelatedness and to describe how they see the topics are handled within their educational systems. We conducted the interviews with 34 teachers teaching in secondary classes in five EU-countries (Austria (8 teachers), Denmark (3 teachers), England (7 teachers, 4 pre-service teachers: in England the interview questions were embedded in general discussions in workshops), Germany (6 teachers), Norway (6 teachers). The teachers’ teaching experiences varied from 1 to 38 years. The analysis of the interview data analyses showed that despite many similarities across the teachers, there are crucial variations with regard to several dimensions in current practices of lesson planning, assessment, and providing feedback. Concerning lesson planning, teachers differed with regard to the dimensions of how collaboratively they plan, how much they consider student assessment, and how as well as how much ICT they use for planning. Concerning assessment, teachers differed with regard to how formative their assessment practices are and how much ICT they use in assessing their students. Concerning the provision of feedback, teachers also differed with regard to whether they use ICT or not and which type of feedback they provide. Here, the teachers’ descriptions of their feedback practices also reflected their knowledge concerning formative

© ISLS

34

ICLS2012

Volume 2: Symposia

assessment (or assessment for learning: Black, Harrison, Lee, Marshal, & Wiliam, 2003). Whereas some teachers were interested in enhancing their possibilities to enhance feed-forward, others did not mention any emphasize on feedback. Moreover, teachers differed with regard to their explicit knowledge of formative assessment (e.g., Black & Williams, 1998) and teacher research in general (Bannan-Ritland, 2008). Whereas some of the teachers are from systems that promote formative assessment initiatives and teacher research approaches (e.g., teachers in England), others have not yet heard about such initiatives (e.g., teachers in Germany). Thus, a few teachers were irritated and even showed misconceptions when they were confronted with the term ‘teacher research’. In general, the more elaborated teachers’ conceptions on formative assessment and teacher research were, the more interrelated they saw the different levels of teaching practices concerning formative assessment with professional development and with teacher research, but also with school development as whole. Although our results are not representative in a statistical sense, they are nevertheless in line with what is known about national and school-level differences (e.g., Law, Pelgrum, & Plomp, 2006) and hence, suggest different teacher profiles (see Figure 1) which result from the level of understanding of the components (classroom practices/pedagogy, teacher research, and school strategy) as well as their interrelatedness: 1. Particularised-incomplete: Teachers with this profile are not yet familiar with all components to be supported in NEXT-TELL. They also see connections between the different components of formative (E)-assessment as classroom practices, teacher research as professional development, and school development as rather weak or loose. 2. Particularised: Teachers recognize the importance of all components (even if not practiced so far) but do not have a fully integrated view of their interrelations 3. Integrated: Teachers have a differentiated understanding of the components and are aware of their interrelationships.

Figure 1: Visualized examples of teacher profiles. In order to integrate teachers with different profiles concerning formative assessment and teacher research into the project appropriately, we will chose different interventions as next steps in our process of co-development (Penuel, Roschelle, & Shechtman, 2007).

Open Models of ICT-embedded Formative Classroom Assessment. Peter Reimann, MTO, Tübingen, Germany (p.reimann@mto.de); Wilfrid Utz, BOC, Vienna, Austria (wilfrid.utz@boc-eu.com); Wolfgang Halb, Joanneum Research, Graz, Austria (wolfgang.halb@joanneum.at). As mentioned in the introduction, an important step for making classroom data useful for teachers’ and students’ ‘tactical’ decision making (What to do next?) is to express the information on the level of knowledge and skill development, processes of learning, motivation and engagement. Teachers usually get this information from direct observations, and from formative assessments, such as quizzes or problem solving exercises. In the technology-rich classroom, a third source of information are the recordings of learning activities as they unfold in digital media, such as software applications (e.g, MS Excel) , learning management systems (e.g, Moodle), and increasingly on “Cloud” tools and services (such as Google Docs). While these digitally enacted learning activities are easily recorded (e.g., as log files), they usually need further processing in order to be interpretable as information on learning and knowledge. So far, methods to do this automatically have been confined to socalled Intelligent Tutoring Systems, e.g., Cognitive Tutors (Koedinger & Corbett, 2006) and personalized learning systems (Heller, Steiner, Hockemeyer, & Albert, 2006). This approach, however, needs a very detailed analysis of the knowledge/skill structures to be learned, and a very fine-grained learning application in order to trace the students’ actions on a level that is appropriate for the diagnostic algorithms. As a consequence, such

© ISLS

35

ICLS2012

Volume 2: Symposia

systems have only been developed so far for a few curriculum areas, and the diagnostic machinery has been coupled with specifically designed learning software. Teachers that don’t teach in domains covered, and/or don’t want to use the specific application with their students are out of luck. In addition, even if more teachers would have access to these applications (some of which are web-based by now), it is still an open question how many would actually make use of them for formative assessment purposes, given that the assessment method is opaque. Teachers would face the dilemma that for providing learning-relevant information to their students they would need to do so based on an assessment method they cannot explain, and which is hidden in programming code. We therefore decided to build on a “glass-box” approach to technology-enhanced formative assessment; both the diagnostic data transformation procedures as well as the resulting learner model should in principle be open. And beyond that: not only should the assessment be ‘open’, it should also be the teacher who, in principle, can develop and modify technology enhanced classroom assessment methods. This takes also into account that diagnostic information on students’ learning can come from many sources: we consider in particular teachers, students, (in the role of self and peer assessors), parents, and software applications. They all can produce diagnostically relevant information that are candidates for visualisation in the Open Learner Model (OLM). An important design consideration in NEXT-TELL is that we believe that all assessment methods, independent of who employs them (teacher, student, parent, software) should adhere to certain quality criteria, in particular concerning their validity and reliability. For establishing validity, we build on the Evidencecentered assessment Design methodology (Mislevy & Riscontente, 2006), and for establishing reliability of assessments in NEXT-TELL the users need to document their assessment process, thus creating provenance data (Groth et al., 2006). In short, we require that humans as well as computational assessment services describe how they, starting from observations on what learners do in the course of their learning activities (performance) and from the artifacts produced in the course of learning activities, come to conclusions about learners’ knowledge and skills.

Figure 2: One of the views of the NEXT-TELL authoring tool, integrating learning activities with formative assessments. On the left, a “Notebook” is shown that renders the attributes that are available with each modeling object, and makes them available for editing. To achieve this, the creators of assessment methods, including teachers, need to be provided with a ‘language’ to express their assessment ideas. If, as in our case, the assessment is to be of the formative kind, it needs to be integrated with the teaching/learning process. Therefore, we equip teachers with an authoring tool for designing learning activity sequences and for relating these to expected knowledge changes (learning progressions) as well assessment methods (see Figure 2). We treat any assessment process as an instantiation of an assessment model, and any learning activity sequence as an instance of a learning sequence model. Technically, we use a meta-modeling shell and the Open Models approach (Karagiannis, Grossmann, & Hoefferer, 2008) for modeling formative assessment processes as well as learning activity sequences). The models, then, are not only descriptions, but can also serve as the basis for rapid deployment in an ICT environment. Currently, we provide adaptors to Moodle and to Google Apps.

Towards Visual Analytics for Teachers: An Open Learner Model Approach

© ISLS

36

ICLS2012

Volume 2: Symposia

Ravi Vatrapu, Computational Social Science Laboratory (CSSL), Dept. of IT Management, Copenhagen Business School, vatrapu@cbs.dk; Susan Bull, Matthew Johnson, Electronic, Electrical and Computer Engineering, University of Birmingham, s.bull@bham.ac.uk, m.d.johnson.1@bham.ac.uk The NEXT-TELL project aims to provide computational and methodological support for teachers in real-time and in-situ classroom settings. Towards this end, we integrate emerging developments in visual analytics and the established methodological approach of design-based research (DBR) in the learning science into “Teaching Analytics” and propose a model of teaching analytics, termed “triadic model of teaching analytics (TMTA)” (Vatrapu, et al., 2011), discussed next. We adapt and extend the dyadic model of pair analytics in visual analytics to a Triadic Model of Teaching Analytics (TMTA) as shown in Figure 3:

Figure 3: Triadic Model of Teaching Analytics (TMTA) At its core, our model sees collaborative knowledge building between teachers, analysts and researchers. We think of the relationships between the TE, VAE and DBRE as a dynamic socio-technical system. The design considerations are about creating feedback loops between the three individuals, such that each one drives the other two to higher levels of performance on the positive side (with the cost of anxiety in the negative case). That is, feedback from the teacher inspires the VAE to create new, better visualizations and for the researcher to better understand the ongoing teaching and learning processes while feedback from the VAE – perhaps in the form of visualization artifacts – allows the teachers to better understand what is going on in the classroom from a learning activity design perspective and the research to hypothesize, test and predict student learning trajectories and performance outcomes. All in all, these feedback loops should culminate in the teacher providing timely, meaningful actionable, customized and personalized feedback to students. The key point here is that each member of the triumvirate of TE, VAE, and DBRE can gain from the other two, not that each partner's role is to highlight deficiencies of the other two. Therefore, TMTA involves a close collaboration between the TE, VAE, and the DBRE. It includes teaching practitioners in the design process and invites them to contribute significantly to the innovation of the visual analytics tools. This allows these learning analytics tools to address pedagogical issues as they arise and evolve in real classrooms. In the next section, we outline an approach to TMTA based on open learner models (OLM). An obvious starting point for developing the TMTA approach is to base it around the existing work in Artificial Intelligence in Education, on open learner models. A learner model holds information (usually) about an individual learner, and the model is automatically and dynamically updated during the user's interaction with a computer-based/online educational environment. The learner model typically includes data about the learner's knowledge state, which may include specific difficulties and misconceptions; and it can also have data on other aspects of the learning process (e.g. representation, content, teaching style preferences; motivational, social, affective attributes). The learner model is then used by the educational environment to adapt its teaching to the specific needs of the individual learner (the environment 'understands' the user's understanding). An "open learner model" is a learner model that can also be externalised to the user (Bull & Kay, 2007). This externalised (open) learner model may be simple or complex in format using, for example: text, skill meters, concept maps, hierarchical structures, animations (Bull et al., 2010). Normally the user who accesses the learner model is the learner. Common purposes of externalising the learner model to learners are to promote metacognitive activity such as awareness-raising, reflection, selfassessment and planning (Bull & Kay, 2008). Some learner models have, however, also been made available to teachers (Bull & McKay, 2004; Eyssautier-Bavay, Jean-Daubias, & Pernin, 2009; Zapata-Rivera, Hansen, Shute, Underwood, & Bauer, 2007). Teacher access to the learner models of their students can help them to better understand learners' needs as individuals and as a group, and can therefore enable teachers to adapt their teaching. Of particular interest in NEXT-TELL is the possibility of open learner models to support the routine but dynamic decision-making that teachers need to perform in the classroom. While the above describes the typical situation of open learner models, it is easy to envisage this being extended for use in TMTA. A range of visualisations or externalisations of the learner model have been
© ISLS 37

ICLS2012

Volume 2: Symposia

explored (e.g. Bull et al., 2010), and these are currently being further extended to support the synthesis of work between teaching experts, visual analytics experts and design-based research experts, as required for the proposed TMTA approach within the NEXT-TELL project.

Teachers’ Inquiry into Student Learning: Supporting Design-based Research towards Formative Assessment in the Technology-rich Classroom. Rosemary Luckin, Brock Craft, Katerina Avramides, The London Knowledge Lab, Institute of Education, London. r.luckin@ioe.ac.uk.; Michal Kossowski, BOC Information Technologies Consulting, Warsaw, michal.kossowski@boc-pl.com A recent report on the impact and use of ICT in schools across 27 EU countries confirms that schools’ effective engagement with ICTs is patchy and that there exist considerable differences in levels of e-maturity within and between countries (Empirica, 2006). At the same time, the existence of these ICT resources is raising expectations for increased and improved communication about students’ learning (Becta, 2008, 2009). Teachers and schools face a continual struggle in understanding how to effectively embed these new tools into their everyday pedagogic practices, particularly in areas where ICTs are not perceived to be core elements (Clark et al., 2009; Freebody et al., 2008). In response to these dissonances around technology use in education, some design researchers suggest that a more holistic and systemic approach to understanding the learning context is needed (Fishman, 2004; Luckin, 2010; McKennay et al., 2008; Ravenscroft, 2009). The development of the TISL methodology and software tool has therefore adopted a participatory approach through which teachers directly inform the design process and outputs. Our approach is supported by two theoretical frameworks (1) Teacher Design Research (TDR) (Bannan-Ritland, 2008); and (2) The Ecology of Resources model and design framework (Luckin, 2010). The former is used to generate a dialogue around a conceptualisation of teachers’ as innovators, whilst the latter is used to expand the TDR model to take into account the complexity of technology-rich learning contexts and the emergent properties of interactions between teachers, learners and available resources. In practice, the literature on teacher inquiry, teacher design research, and teacher-led research has shown that, in order to pursue this kind of research, teachers often require guidance and support. For TISL, such guidance and support must be developed in collaboration with teachers. To facilitate this need, TISL, as with related components within NEXT-TELL, is being developed in two phases: (1) researcher-led; and (2) teacher-led. The TISL approach outlined in this paper is being developed as part of the researcher-led phase of the project. It provides a preliminary framework for identifying relevant tools and approaches on the use of available data for evaluating students’ learning (Table 1) in alignment with teachers’ professional development. TISL aims to enable teachers to conduct sustainable and relevant inquiry into students’ learning and related school-based practices (e.g. teaching, assessment, etc.). Testing and evaluating the initial TISL method. Table 1: 10 steps to systematising teacher inquiry with TISL TISL Method establishing a trigger choosing a lens (researcher or teacherled) planning for and collecting evidence analysing practices enacting and adapting an action/innovation TISL Tools and Data Handling identifying tools (ALTs) and potential data sources planning for data capture and data sharing collaborative data analysis and interpretation evaluating data and reflecting on inquiry process data-driven decision-making for innovating practice

1. 2. 3. 4. 5.

1. 2. 3. 4. 5.

The modelling procedure for the TISL inquiry planner is drawn from the initial 5-Step TISL method (see Table 1 above). It identifies the systematic sequence of the 5 steps to be taken by the user (e.g. teacher participants) when planning an inquiry process (into students’ learning). The TISL method is designed to produce results that can be used by other teachers to inform teaching, learning and formative assessment. . The structure and dependencies for the TISL inquiry process planner are shown in Figure 4 below.

© ISLS

38

ICLS2012

Volume 2: Symposia

Figure 4: Time line view of the TISL Inquiry Process Planner. An example of these steps in action drawn from empirical work with teachers would be: Step 1 Establishing a trigger: how can video data be used to support teacher inquiry into students’ learning? Step 2 Choosing a lens: constrain to 1 minute video segments at the beginning-middle-end of a learning activity to focus inquiry. Step 3 Plan and Collect Evidence: seek appropriate segments of video from system repository, or plan collection of fresh data. Step 4 Analyse Practices: explore of learning revealed in the video clips for formative assessment and identify framing patterns for future practice and analysis. Step 5 Adapt and Enact: implement and test framing patterns in practice. References Bannan-Ritland (2008). Teacher Design Research. An emerging paradigm for teachers' professional development. In A. E. Kelly, R. A.Lesh & J. Y. Baek (Eds.), Handbook of design research methods in education (pp. 246-262). New York: Routledge. Becta (2008), Harnessing Technology Review 2008, 2009: The role of technology and its impact on education. Coventry: Becta. Black, P., Harrison, C., Lee, C., Marshal, B., & Wiliam, D. (2003). Assessment for learning. New York: Open University Press. Black, P. & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5 (1), 7-74. Bull, S., & Kay, J. (2007). Student Models that Invite the Learner In: The SMILI Open Learner Modelling Framework. International Journal of Artificial Intelligence in Education 17(2), 89-120. Bull, S., & Kay, J. (2008). Metacognition and Open Learner Models. In I. Roll & V. Aleven (Eds.), Proceedings of Workshop on Metacognition and Self-Regulated Learning in Educational Technologies, International Conference on Intelligent Tutoring Systems (pp. 7-20). Bull, S., & McKay, M. (2004). An Open Learner Model for Children and Teachers: Inspecting Knowledge Level of Individuals and Peers. In J. C. Lester, R. M. Vicari & F. Paraguacu (Eds.), Intelligent Tutoring Systems (pp. 646-655). Berlin Heidelberg: Springer. Bull, S., Gakhal, I., Grundy, D., Johnson, M., Mabbott, A., & Xu, J. (2010). Preferences in Multiple View Open Learner Models. In M. Wolpers, P. A. Kirschner, M. Scheffel, S. Lindstaedt & V. Dimitrova (Eds.), EC-TEL 2010 (pp. 476-481). Berlin Heidelberg: Springer. Clark, W., Logan, K., Luckin, R., Mee, A. and Oliver, M. (2009) 'Beyond Web 2.0: mapping the technology landscapes of young learners', Journal of Computer Assisted Learning 25(1), 56-69. Crawford, V. M., Schlager, M. S., Penuel, W. R., & Toyama, Y. (2008). Supporting the art of teaching in a datarich, high-performance learning environment. In E. B. Mandinach & M. Honey (Eds.), Data-driven school improvement (pp. 109-129). New York: Teachers College Press. Empirica (2006), Benchmarking Access and Use of ICT in European Schools 2006. Bonn: Empirica. Eyssautier-Bavay, C., Jean-Daubias, S., & Pernin, J.-P. (2009). A Model of Learners Profiles Management Process. In V. Dimitrova, R. Mizoguchi, B. D. Boulay & A. Graesser (Eds.), AIED09 (pp. 265-272). Amsterdam: IOS Press. Fishman, B. et al. (2004) Creating a Framework for Research on Systemic Technology Innovations, Journal of the Learning Sciences, 13(1), 43-76 Freebody, P., Reimann, P. and Tiu, A. (2008) Alignment of perceptions about the uses of ICT in Australian and New Zealand Schools. University of Sydney.

© ISLS

39

ICLS2012

Volume 2: Symposia

Groth, P., Jiang, S., Miles, S., Munroe, S., Tan, V., Tsasakou, S., & Moreau, L. (2006). An architecture for provenance systems Retrieved from http://eprints.ecs.soton.ac.uk/12023/ Hamilton, E., & Jago, M. (2010). Toward a theory of personalized learning communities. In M. J. Jacobson & P. Reimann (Eds.), Designs for learning environments of the future (pp. 263-282). New York: Springer. Heller, J., Steiner, C., Hockemeyer, C., & Albert, D. (2006). Competence-based knowledge structures for personalised learning. International Journal on E-Learning, 5(1), 75-88. Karagiannis, D., Grossmann, W., & Hoefferer, P. (2008). Open Model Initiative. A feasibility study. Vienna: University of Vienna. Koedinger, K. R., & Corbett, A. (2006). Cognitive tutors. In R. K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (pp. 61-77). New York: Cambride University Press. Law, N., Pelgrum, W. J., & Plomp, T. (2008). Pedagogy and ICT Use in Schools around the World: Findings from the IEA SITES 2006 Study. Berlin: Springer. Luckin, R. (2010) Re-designing Learning Contexts: Technology-Rich (Learner-Centred) Ecologies, London: Routledge. McKenney, S. et al. (2006) Design Research from a Curriculum Perspective in Educational Design Research (Ed. van den Akker, J. et al.), London: Routledge Mislevy, R. J., & Riscontente, M. M. (2006). Evidence-centered assessment design. In S. M. Downing & T. M. Haladyna (Eds.), Handbook of test design (pp. 61-90). Mahwah, NJ: Lawrence Erlbaum. Parr, J. M., & Timperley, H. S. (2008). Teachers, schools and using evidence: Considerations of preparedness. Assessment in Education: Principles, Policy & Practice, 15(1), 57-71. Penuel, W. R., Roschelle, J., Shechtman, N. (2007). Designing formative assessment software with teachers: An analysis of the co -design process. Research and Practice in Technology Enhanced Learning, 2(1), 5174. Ravenscroft, A. (2009) ‘Social software, Web 2.0 and learning: status and implications of an evolving paradigm’, Journal of Computer Assisted Learning, 25(1), 1-5 Vatrapu, R., Teplovs, C., Fujita, N., & Bull, S. (2011). Towards Visual Analytics for Teachers' Dynamic Diagnostic Pedagogical Decision-Making. Paper presented at the 1st International Conference on Learning Analytics & Knowledge (LAK 2011), Banff, Canada. Wang, F., & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research and Development, 53(4), 5-23. Zapata-Rivera, D., Hansen, E., Shute, V. J., Underwood, J. S., & Bauer, M. (2007). Evidence-Based Approach to Interacting with Open Student Models. International Journal of Artificial Intelligence in Education, 17(3), 273-303.

© ISLS

40

ICLS2012

Volume 2: Symposia

Gathering Evidence of Scientific Argumentation Practices: From Pre-Kindergarten to High School
Tiffany R. Lee, Hiroki Oura, Giovanna Scalone, Kari Shutt, John Bransford, Andrew Shouse, Katie Van Horne, Nancy Vye, University of Washington, LIFE Plaza, Box 354941, Seattle, WA, USA 98195-4941 Email: tlee13@uw.edu, oura@uw.edu, gscalone@uw.edu, shuttk@uw.edu, bransj@uw.edu, awshouse@uw.edu, katievh@uw.edu, nancyvye@uw.edu Maureen Munn, University of Washington, Genome Sciences, Box 355065, Seattle, WA, USA, 98195, mmunn@uw.edu Randy Knuth, Knuth Research, Inc., 717 East Blackhawk Drive, Spokane, WA, USA 99208, randy@knuthresearch.com Philip Bell (discussant), University of Washington, LIFE Plaza, Box 354941, Seattle, WA, USA 98195-4941, pbell@uw.edu Abstract: The purpose of this symposium is to explore students’ scientific argumentation practices in relation to the recently released Framework for K-12 Science Education (NRC, 2011) in the United States. We will present data from four research efforts on prekindergarten, upper elementary school, and high school students’ science learning in classroom settings, followed by a reflective presentation by a discussant. Given that our work spans multiple grade levels of formal school instruction, we will discuss our observations across these groups of students while mapping to the goals and suggested progression of argumentation practices described in the NRC framework.

Symposium Overview
The purpose of this symposium is to explore students’ scientific argumentation practices in relation to the recently released Framework for K-12 Science Education (NRC, 2011) in the United States. We will present data from four research efforts on pre-kindergarten, upper elementary school, and high school students’ science learning in classroom settings, followed by a reflective presentation by a discussant. Given that our work spans multiple grade levels of formal school instruction, we will discuss our observations across these groups of students while mapping to the goals and suggested progression of argumentation practices described in the NRC framework. Argumentation is recognized as a necessary and central component of science, and an understanding of scientific argumentation is essential for all people – for scientists to conduct their work and for citizens to productively engage with everyday topics such as the environment or personal health. The NRC framework (2011) specifically highlights the importance of scientific practices in addition to crosscutting concepts and disciplinary core ideas, in quality science education. Indeed, this aligns with trends in science education and the emphasis on moving away from teaching science as purely factual knowledge and instead incorporating scientific discourse, explanations, and argumentation (NRC, 2007, 2009; AAAS, 1989, 1993). Not only does the overall emphasis on scientific discourse better align with actual disciplinary practices of professional scientists (e.g., Bell, 2004; Edelson & Reiser, 2006; Newton, Driver, & Osborne, 1999), but opportunities to engage in argumentation in the science classroom have been shown to increase students’ conceptual understanding and reasoning skills (e.g., Mercer, Dawes, Wegerif & Sams, 2004). Despite the general consensus on the importance of understanding how to argue scientifically, it is still often missing or ineffectively utilized in typical science instruction across grade levels. The research presentations in this symposium highlight four curricular design efforts to support students’ capacities to engage in scientific reasoning and argumentation while also integrating their personal experiences, interests, and relevant topics in their community to increase their knowledge of and participation in science. Our goals for the symposium are to examine how students engage in scientific argumentation within these units and add to the discussion of the progression of scientific argumentation (Berland & McNeill, 2010; NRC, 2011) practices across pre-kindergarten through high school science education. The first paper examines the developing abilities of pre-kindergarten children, ages 4-5 years, to engage in scientific argumentation in classroom science discussions. Although these young children do not systematically use evidence and construct complete arguments every time, these data show that they are capable of using data to support their claims. Often the students draw upon their everyday experiences and knowledge of the natural world, but they also reference shared classroom experiences or artifacts as they make their claims. The next two papers focus on fifth grade students’ participation in re-designed science curricular units. One paper looks at how 5th Grade students in a science classroom engage in collaborative debate on a controversial issue on Triclosan. The other paper compares argumentation across three curricular contexts. One

© ISLS

41

ICLS2012

Volume 2: Symposia

unit is a traditional, teacher-directed unit. The other two units, of our own design, feature student agency and student-directed investigations. Argumentation varies widely across contexts with differing amounts of agency. The last paper examines high school students’ argumentative reasoning during hypothesis development and testing with a real dataset in a refined science curriculum designed for engaging students in an authentic practice of genomic and epidemiological research. The authors analyzed student conversations during their investigations. They found that argumentative reasoning emerges during both hypothesis development and statistical analysis. Students did frequently articulate logical relationships among claim, evidence, and reasoning, and they worked to develop alternative explanations when they encountered results contradictory to their original hypothesis. However, they also found such reasoning is limited during causal inference. The authors developed a hypothesis that both students and teachers may have developed a “singular” view of scientific method for causal reasoning as a reflection of the existing science curricula, and it might interfere when they learn other scientific methods. We intentionally present data from across pre-kindergarten through high school curricular enactments to advance conversations about the development of and expectations for school aged youths’ argumentation practices. Our discussant, Philip Bell, will provide reflective commentary on the presentations.

Paper 1: Supporting Pre-Kindergarten Students’ Emergent Scientific Argumentation Practices (Tiffany R. Lee)
Views on science education have undergone significant changes in the twentieth century. Science has traditionally been taught as a well-defined process using the scientific method (hypothesis, experimentation, control of variables, etc) with theories, concepts and vocabulary to memorize. In the last 50 years, science education reforms have worked to impart more accurate accounts of scientific ideas, practices, and reasoning skills to schoolchildren. In particular, the recently released NRC Framework for K-12 Science Education (2011) has explicitly called for increased attention to students’ understanding of scientific practices. In this paper, we identify instances of pre-kindergarten students engaging in scientific argumentation during classroom science instruction and provide evidence of the progression of argumentation practices. Osborne (2010) suggests that the nature student participation in reasoning and argumentation is complex, dependent upon their existing domain-specific knowledge and their reasoning capabilities at a given age. For very young children, science instruction is often limited to direct observations and hands-on activities, with the assumption that they are not yet capable of abstract thinking and reasoning. Much of the existing early science curricula and teaching approaches are based upon an overextension of Piaget’s conjecture that young children are concrete, simplistic thinkers, resulting in unnecessary and oftentimes inappropriate constraints on young children’s learning (Metz, 1995). The research effort described in this paper presupposes that young children are capable science learners and should be provided with opportunities to support their developing science knowledge and skills (cf. Brown, Campione, Metz, & Ash, 1997; Eshach & Fried, 2005) and builds upon recent research that demonstrates young children’s developing knowledge and experiences with science (Lee, 2010) and their abilities to engage in extended discourse and argumentation (e.g., Lee, 2011; Corsaro, 2003; Gallas, 1995). The overarching goals of the curricular design are to leverage young children’s existing knowledge and interests about the natural world and engage them in the processes of science. We draw upon theory about how people learn, particularly in relation to engaging learners’ interests and preconceptions (NRC, 2000); Vygotsky’s (1978) notion that knowledge is co-constructed by people and that children’s learning is optimally supported by identifying and leveraging zones of proximal development; and applications of Brown and Campione’s (1994) work on creating classroom communities of learners. This paper focuses specifically on pre-kindergarten students’ engagement in scientific argumentation and their use of everyday experiences and knowledge to support their claims. We will show the emergent argumentation practices of these young students and discuss how the classroom teachers facilitated these opportunities for argumentation.

Methods
The data presented in this analysis are part of a year-long curricular intervention in two half-day prekindergarten classrooms in the Northwest region of the United States. Each of the pre-kindergarten classrooms comprised of 12-15 students (ages 4-5 years) one lead teacher, and one assistant teacher. Researchers worked with the teaching team to develop weekly science lessons that emphasized student-led discourse, argumentation, and connections to their existing knowledge of the natural world. Lessons included exploration of artifacts brought into the classroom (e.g., live animals, plants, fossils), observations of the outdoors, and questions to spark student discussion, and typically lasted between 30 to 60 minutes each. To preserve the authenticity of student-generated topics of study, the lessons were designed throughout the school year and built upon students’ interests and questions. Data sources included 1) video recordings of the weekly science lessons, 2) fieldnotes from classroom observations and teacher interviews, 3) digital photographs of classroom activities, 4) artifacts

© ISLS

42

ICLS2012

Volume 2: Symposia

such as student-made drawings and writing samples, materials used for instruction, etc, 5) teacher interviews to reflect on science instruction throughout the year, and 6) parent surveys about students’ out-of-school activities.

Analysis and Findings
Video data of class discussions were analyzed for examples of scientific argumentation between two or more students. As hypothesized, questions that tapped into children’s common experiences but did not have obvious factual answers allowed for extended discourse and argumentation. One unit at the beginning of the school year focused on living vs. non-living things. We began by asking the students what it means to be alive and having them identify characteristics of living things. Students offered ideas from their existing knowledge of the living world: it moves, it talks, it eats, etc. Teachers then asked questions to challenge students’ ideas. One question, “Are plants alive?” prompted an extended discussion between the students – do plants make noise and do they eat? In this discussion, we observed students making references to their existing knowledge of plants (e.g., knowing that plants cannot make noises or move on their own), and then trying to reconcile these ideas to support the argument that plants are living things. In another lesson, students are presented with two hamsters – one real hamster and one toy hamster that runs on a wheel. The students are asked to decide whether each hamster is alive based on the qualities they have identified for living things. Students easily agree that the real hamster is alive – it moves, breathes, eats, and maybe it can make a noise. They also know that living things can reproduce (often talked about as “making more” or “having babies”), and they guess that the real hamster can do this. The toy hamster is puzzling at first: several students claim that it could be alive because it moves on its own. Upon closer investigation, one student notices that it has wheels instead of feet and guesses that it uses a battery to move. After more discussion, they agree that the toy hamster is not alive. Throughout the data, we see that students are often quick to give an opinion in response to the teacher’s questions. When the teacher follows up by asking for a reason behind that opinion or encourages other students to voice a differing opinion, the students are given opportunities to explain their thinking. In these instances, we see that the pre-kindergarten students often support their claims with evidence from their out-of-school experiences or shared classroom activities.

Discussion
Findings from this work show that even very young children are able to engage in scientific argumentation and the development of these practices needs to be supported in science instruction. In these data, pre-kindergarten students readily engage in extended discourse and argumentation when presented with topics that relate to their early experiences with the natural world and encouraged by the teacher to give opinions and agree or disagree with their peers. Although these students do not systematically use evidence to support their claims or necessarily construct thorough arguments each time, they easily leverage their experiences and knowledge to support their claims. Increased opportunities for young children to engage in argumentation practices will likely improve their abilities to successfully use data to support their claims and allow them to become more proficient in their argumentation practices over time. Further research is being conducted to document the range of young children’s abilities, particularly as they relate to how children engage in scientific discourse and argumentation and how they make connections to their everyday experiences. In addition, we seek to better understand the role of the teacher in creating an environment that supports this kind of discourse and science exploration. As a result of these and other related findings about young children’s reasoning abilities, we argue that the goals and approaches for science education need to be revised to support the development of early scientific skills and prepare children for future success in science.

Paper 2: Achieving Coordination in Collaborative Debate (Giovanna Scalone)
Teaching students to argue and collaboratively debate are central features of inquiry in the natural sciences (Bell, 2004). We draw on Bell’s framing of argumentation and debate in science as the exploration of a theoretical controversy involving the coordination of evidence with theoretical ideas (p. 138). When students engage in collaborative argumentation, they are “arguing to learn”. This implies that students are collaboratively exploring solutions dialogically (Andriessen, 2006). From a Vygotskian perspective, it is through language that we come to know things, and transform knowledge into actions, objects, and other symbol systems. Argument is a form of discourse, which according to Kuhn (1991) requires one to not only think in a metacognitive and metalinguistic framework, but to also place one’s ideas in the planes of the possible, the probable, and the disputable (Pierut-Le Bonniec & Valette, 1991). Elementary school students are rarely asked to participate in collaborative debate and discussion of controversial topics in science education (Levinson & Turner, 2001). As a result, in classroom debate students find difficulties in anticipating objections in their own arguments as well as in the arguments of others (Reznitskaya et al., 2001). This issue also arises because students have difficulties in understanding the

© ISLS

43

ICLS2012

Volume 2: Symposia

evidentiary nature of scientific arguments and the social dimensions of science (Bell, 2004; Kuhn, 1993; Driver et al., 1996). In this paper, we discuss the findings of two elementary school science debates on ‘Should hand sanitizers be banned?’ Perspective taking is essential in science where students come to understand the standpoints of various stakeholders associated with the controversy around triclosan. In addition, we wanted students to move beyond their epistemological naiveté and consider a theory as well as evidence and its bearing on a theory (Kuhn, 1991). Helping students to think about their own thinking will help them learn how to evaluate evidence. Finally, it is possible that helping students weigh different forms of evidence enables them to formulate and reformulate ideas, which, in turn, fosters conceptual change (Koslowski, 1996). In this paper we ask: How do elementary school students in science achieve coordination in debate?

Methods & Data Sources
This study took place in two fifth grade science classrooms at Granite Elementary School in an urban neighborhood. There were five iterations of the Micros & Me curriculum, in which the content was framed around microbiology and health; and students were given choice and agency to document community-relevant health topics; design fair tests; and choose a personally-relevant, related topic and conduct research. We report on the findings from the fifth iteration, specifically on a debate, ‘Hand-sanitizers should be banned’. We helped student’s structure their debates by drawing on Bell’s (2004) framing of argumentation and debate. We think of argument as a product, which includes claims, evidence, and reasoning; and we think of argumentation as the process of assembling these components in order to make one’s case. Students debated on whether hand sanitizers that contain Triclosan (an antibacterial, antifungal agent used in consumer products, such as hand sanitizer, toothpaste, etc.) should be used since it affects the microorganisms in the water. In Fall of 2010, participants included 29 consented fifth graders (all between 10 and 11 years old) and two teachers, Ms. Jones and Mr. A. Ethnographic techniques were incorporated into the design to leverage students’ everyday expertise. The data sources came from participant observation using field notes and digital photography of instruction in the science classrooms, video recording, student illustrations, and students’ notebooks.

Findings
By following Bell’s (2004) framing of argumentation and debate within student discourse as well as verbal and nonverbal (cf. Lemke, 1998) argument, in this paper we endeavor to understand how students coordinate their evidence in classroom debate. Preliminary findings reveal that as students engage in different modes of evidence to support their position in the triclosan debate, some were faced with difficulties separating theory and evidence, producing pseudoevidence (Kuhn, 1991). Other students made good evidence-based arguments, however their arguments where disassociated with the counterarguments.

Discussion
The findings have implications for how we can help students not only think about perspective taking in science, but also scaffold their thinking in ways that can help them critically evaluate arguments for the opposing position and make every argument associated with a counterargument. Overall, this contributes to helping students understand how scientists work by having them engage in critique and evaluation. Consequently, students come to understand that science is a body of knowledge that is rooted in evidence-based argumentation (NRC, 2011).

Paper 3: Student Agency and Argumentation Across Curricular Contexts (Kari Shutt, Nancy Vye, & John Bransford)
An important goal for science education is to provide students with the opportunity to engage in authentic science practices (National Research Council, 1996, 2011). Argumentation is a key component of scientific discourse and plays a central role in the construction of knowledge. It is the means by which the scientific community reaches consensus, and is therefore a crucial scientific practice (NRC, 2007, 2011). In this study, we compare argumentation across three different curricular contexts that differ in terms of their level of student agency. Each unit is taught to 5th grade students over the span of 12 weeks. One unit, the FOSS Environments unit, is a kit-based unit that features discrete, teacher-directed investigations. The other two units, the Isopod Habitat Challenge and My Skokomish River Challenge, were designed by our research team. The cornerstones of the latter units are student agency and student-directed collaborative investigations related to an overarching problem (Vye et al., 1998).

Methods
The study took place in a mid-sized suburban school district. We selected one teacher and class from each curricular context as cases for consideration. Within each class, we selected one small group of students to

© ISLS

44

ICLS2012

Volume 2: Symposia

follow closely. Each of the three units involves a series of investigation cycles. We chose to examine an investigation cycle that took place midway through the units. We focused our attention on the selection of the research question and the experimental design process. The primary data source was naturalistic video. Videos were transcribed for analysis, and episodes were coded using an adaptation of Chin and Osborne’s (2010) argumentation coding scheme.

Findings Case 1: FOSS Environments (Comparison)
In the FOSS-Environments unit (Delta Education (Firm), Lawrence Hall of Science, & University of California Berkeley, 2000), students undertake a series of life science investigations. In most instances, students investigate questions that are determined by the teacher, and students typically follow prescribed procedures to investigate those questions. In this investigation cycle, Ms. Stark gave her students the investigative question at the start of the class session, and asked students to brainstorm ways that they might conduct the investigation. George: Well if it’s like too cold or something it [the isopod] could roll up into a ball or just stay very still. Eileen: I think it rolls up into a ball when it's scared because um George: Or when it just doesn't like the climate. Drew: You can pick yours [isopod] up now [Off Task] Student interaction was limited for the Comparison students, and there was minimal evidence of argumentation, with only one weak rebuttal offered during the conversation.

Case 2: Isopod Habitat Challenge (IHC)
The Isopod Habitat Challenge addressed the same learning objectives as the FOSS Environments unit, but in the redesigned unit, the IHC, small groups of students conducted sustained, original inquiry in which they determine what and how to design a classroom habitat for isopods. Ms. Atwell began this investigation cycle with a 90 minute, student-led discussion about researchable questions. She interjected open-ended questions periodically to move the discussion forward. After this discussion, Ms. Atwell released the students to generate a question in their small groups. The students considered seven possible questions before settling on their choice. Design of the procedure flowed naturally from the conversation about investigative questions, with students designing their experiment as they debated the merits of the possible questions. In total, IHC students spent 59 minutes in small group conversation around design issues. The IHC students offered more new ideas about investigative questions and experimental design than students in the FOSS, and they demonstrated more evidence of argumentation around those ideas. This included an emerging ability to indentify weaknesses in claims. Lisa: Do isopods drink water? John: You can't find that out. We can't find that out. Jessi: All organisms need water. Lisa: Nu-uh. They could eat like- they could eat the- they eat the carrots that has moisture in them as their water. John: Really, you can't tell if they're drinking really. Lisa: You could. Really, you could. You put a certain amount of water in a little cup or something and measure it and then you see if later on there's a little less amount. John: Well it would be so tiny you could barely find it

Case 3: My Skokomish River Challenge (MSRC)
MSRC is an earth science unit based on similar design principles to the IHC. It is challenge-based and features opportunities for sustained inquiry, revision of ideas, and student agency. It also features a number of studentled investigations. To set up the investigative question selection, Ms. Donovan told students what experiments they might do and how they might go about conducting them. When given the opportunity to select their question in their small group, one student simply stated a question from the list. For the procedure, Ms. Donovan gave students a “fill-in-the-blank” procedure sheet. Students inserted their variables on the blank spaces; the remainder of the procedure was provided for them. There was no discussion of the procedure in the small group. Despite the fact that the unit was designed for student agency--where students would generate questions to investigate and design ways to investigate these questions--that agency was not offered to them in this investigation cycle. These activities were highly-constrained and structured by the teacher. Only 14 minutes were provided for small group conversation. This particular group did not choose to interact during that time, and thus, there was no evidence of argumentation.

© ISLS

45

ICLS2012

Volume 2: Symposia

Discussion
Findings from the IHC unit suggest that authentic, “agentive” science inquiry can provide a rich context for engaging students in productive argumentation. However, we saw widely different outcomes in the IHC and the MSRC, despite their common design principles. Curriculum can create an environment that fosters productive argumentation, but particular pedagogical moves are needed to support this practice. Positioning students as knowledgeable and capable of independent inquiry, for example, by asking open-ended questions and encouraging student-to-student interaction and decision-making, appear to foster agency and productive argumentation.

Paper 4: High School Students’ Argumentative Reasoning During Hypothesis Development and Testing In Epidemiological Analysis (Hiroki Oura, Katie Van Horne, Andrew Shouse, and Maureen Munn, Randy Knuth)
Introduction
Developing a scientific argument can be seen as marshaling relevant evidence and justifying an explanation relative to other alternative explanations for particular observations or phenomena (Osborne & Patterson, 2011). As a basic form for instruction, students should develop an argument with a set of claim, evidence, and reasoning, and they also need to hedge their argument by articulating the limitations and possible rebuttals (Toulmin, 1958; McNeill & Krajcik, 2012). The key for meaningful classroom practice is to provide the appropriate contexts in which students may reach multiple reasonable explanations, for instance, by analyzing a real dataset (instructional context), by supporting explanations with relevant evidence (argumentative product), and by defending their explanations and questioning explanations from others (argumentative process) (Berland & McNeill, 2010). The authors have refined an existing curriculum aimed at engaging students in an authentic practice of genomic and epidemiological research (Munn et al., 2010) and developed a set of scaffolds in collaboration with genome scientists in a three-year project. The goal of this design-based research is to engage students in authentic data analysis with a real dataset in which they develop arguments for factors that increase a person’s risk of becoming a regular smoker. Smoking is a multifactorial trait, and various environmental and physiological (or genetic) factors contribute to its initiation and the difficulty in quitting (cessation). The dataset is from a real case control study of nearly 300 adult smokers and non-smokers and includes items from the research survey completed by subjects and limited genotyping on their DNA. Students develop explanatory hypotheses by defining the exposures and non-exposures for selected survey items as candidate factors. Students test their hypotheses by calculating a significance test for association and determining causal inference by applying a set of criteria used by epidemiologists. For each test (limited to only a few), students estimate the odds ratio as the magnitude of the difference in likelihood for the exposure between regular smokers (cases) and those who tried smoking but did not continue (controls), and evaluate the significance with the 95% confidence interval. Once students identify the association by the statistical evidence and their reasoning, they may make causal inference by scrutinizing the degree to which their results meet a set of criteria for causality such as the strength of association, temporal sequence, consistency with other studies, and lack of confounding factor (Bradford Hill, 1965).

Method
In this study, we examined high school students’ argumentative reasoning as they conduct hypothesis development and testing during our first-year implementation. Our data include video- and audio- recordings during data analysis and associated field notes from 13 high school student groups (2 to 4 students per group) in urban or suburban public schools or a local summer program. All teachers in the classrooms had participated in a training workshop in which they learned basic concepts in the curriculum and practiced data analysis with their own investigations. In the classrooms, students conducted their investigations after learning basic concepts related to the model of smoking, case control study, and evaluation with the odds ratio and confidence interval, mostly in a lecture format. We analyzed student conversations during their data analysis and evaluated their argumentative reasoning using the instructional model articulated by McNeill & Krajcik (2012). Due to the variability in the lengths and situations of the recordings among the participating classrooms, we focus on and report noteworthy themes on students’ typical patterns and performance followed by our discussion along with the learning progression and instructional concerns.

Findings
Our analyses indicate that argumentative reasoning can emerge during hypothesis development based on informal evidence such as students’ beliefs, experience, and observations. Students often developed their hypothesis as they identified a survey item related to their overarching hypothesis and justified their hypothesis to the item by drawing on such informal evidence. As an ideal case, for instance, when one student in a group
© ISLS 46

ICLS2012

Volume 2: Symposia

was developing a test to see if having a close relationship with parents who smoke can increase a risk of smoking, another student proposed an alternative hypothesis that people might not smoke because their parents smoke by drawing on her father’s case (they called it “rebelling”). They decided to test this hypothesis and found that the odds ratio was insignificant; they concluded the hypothesis was not supported, though they each appeared to maintain their belief in their own hypothesis after testing. During statistical analysis, students frequently consider and articulate logical relationships among claim, evidence, and reasoning when they evaluate the association from the odds ratio and confidence interval. Most groups appropriately evaluated the significance of the association from the confidence interval by reasoning in a way such as “one is outside the interval, so there is association” (the null is one). Their reasoning seemed to become more active particularly when they encountered results contradictory to their original hypothesis, and students worked to develop an alternative explanation fitting the results. Lastly, we found that argumentative reasoning rarely emerged during discussions of causal inference, and when students did talk about the criteria for causality, they focused on the strength of association. This is likely due to the curriculum structure, as its main focus is placed on how to appropriately interpret the odds ratio and confidence interval rather than asking students to evaluate and construct causal inferences. Yet, these criteria were explicitly shown in a table in the curriculum and teachers had practiced the causal reasoning in the prior training workshop. We identified some cases that both students and teachers seemed to reject the idea that they could make causal inference because the case control study is not “controlled” (according to their explanation of characteristics of a study needed to infer cause). In reality, however, epidemiologists often make causal inferences based on evidence for association from observational studies due to several ethical and practical reasons (Koepsell & Weiss, 2003). We could hypothesize that some teachers and students may hold a “singular” view of scientific method for causal reasoning (Lederman et al., 2002), and it might interfere when they learn other methods, since the randomized and manipulated control experiments are the predominant form of hypothesis testing in today’s K-12 science curricula and standards (Windschitl et al., 2007). Although it is hard to generalize the issue within the present study, this “epistemological” aspect of argumentative reasoning is worth further investigation to the existing literature in our future works.

References
American Association for the Advancement of Science/Project 2061 (1989). Science for all Americans. New York: Oxford University Press. American Association for the Advancement of Science/Project 2061 (1993). Benchmarks for science literacy. New York: Oxford University Press. Andriessen, J. (2006). Arguing to learn. In R.K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 443-460). Cambridge, UK: Cambridge University Press. Bell, P. (2004). Promoting students’ argument construction and collaborative debate in the science classroom. In M. C. Linn, E. A. Davis & P. Bell (Eds.), Internet environments for science education (pp. 115-143). Mahwah, NJ: Erlbaum. Berland, L. K., & McNeill, K. L. (2010). A learning progression for scientific argumentation: Understanding student work and designing supportive instructional contexts. Science Education, 94(5), 765-793. Bradford-Hill, A. (1965). The environment and disease: Association or causation? Proceedings of the Royal Society of Medicine, 58: 295–300. Brown, A. L., & Campione, J. C. (1994). Guided discovery in a community of learners. In K. McGilly (Ed.), Classroom lessons: Integrating cognitive theory and classroom practice (pp. 229–270). Cambridge, MA: MIT Press/Bradford Books. Brown, A., Campione, J., Metz, K. E., & Ash, D. (1997). The development of science learning abilities in children. In A. Burgen & K. Harnquist (Eds.), Growing up with science: Developing early understanding of science. Goteborg, Sweden: Academia Europaea. pp. 7-40. Corsaro, W. A. (2003). We’re friends right? Inside kids’ culture. Washington, DC: Joseph Henry Press. Delta Education (Firm), Lawrence Hall of Science, & University of California Berkeley. (2000). FOSS Environments: Grades 5-6. Nashua, NH: Delta Education. Driver, R., Leach, J., Millar, R. & Scott, P. (1996). Young people’s images of science. Buckinghamshire, England, Open University Press. Edelson, D.C., & Reiser, B.J. (2006). Making authentic practices accessible to learners. In R.K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 335-354). Cambridge, UK: Cambridge University Press. Eshach, H., & Fried, M. N. (2005). Should science be taught in early childhood? Journal of Science Education and Technology, 14(3), 315-336. Gallas, K. (1995). Talking their way into science: Hearing children’s questions and theories, responding with curricula. New York: Teachers College Press.

© ISLS

47

ICLS2012

Volume 2: Symposia

Koepsell, T. D., & Weiss, N. S. (2003). Epidemiologic methods: Studying the occurrence of illness. Oxford: Oxford University Press. Koslowski, B. (1996). Theory and evidence: The development of scientific reasoning. Cambridge, MA: MIT Press. Kuhn, D. (1991). The skills of argument. Cambridge: Cambridge University Press. Kuhn, D. (1993). Connecting scientific and informal reasoning. Merrill-Palmer Quarterly, 39(1), 74-103. Lederman, N. G., Abd-El-Khalick, F., Bell, R. L., & Schwartz, R. S. (2002). Views of Nature of Science questionnaire: Toward valid and meaningful assessment of learners’ conceptions of nature of science. Journal of Research in Science Teaching, 39(6), 497-521. Lee, T. R., Vye, N., & Bransford, J. D. (2010, March). Social influences on the development of young children’s understandings of science. Paper presented at the annual meeting of NARST, Philadelphia, PA. Lee, T. R., Salgado, N. L., Vye, N., & Bransford, J. D. (2011, April). Argumentation in the pre-kindergarten classroom: Young children’s use of everyday experiences and knowledge in scientific discourse. Paper presented at the annual meeting of NARST, Orlando, FL. Lemke, J. L. (1998). Multiplying meaning: Visual and verbal semiotics in scientific text. In J. R. Martin & R. Veel (Eds.), Reading science: Critical and functional perspectives on discourses of science (pp. 87 – 113). New York: Routledge. Levinson, R. & Turner, S. (2001). Valuable lessons: Engaging with the social context of science in schools. London: The Wellcome Trust. McNeill, K. L. & Krajcik, J. (2012). Supporting grade 5-8 students in constructing explanations in science: The claim, evidence and reasoning framework for talk and writing. New York, NY: Pearson Allyn & Bacon. Mercer, N., Dawes, R., Wegerif, R., & Sams, C. (2004) Reasoning as a scientist: Ways of helping children to use language to learn science. British Educational Research Journal, 30(3), 367-385. Metz, K. E. (1995). Reassessment of developmental constraints on children’s science instruction. Review of Educational Research, 65(2), 93-127. Munn, M. Brown, M., Martinez, K., Alan, D., Booth, G., Kelly, C., Mouat-Rich, N., Santucci, S., Thomson, L., and Welch, MM 2010. Investigating the Effects of genes and Environment on Smoking Behavior. Available at: https://gsoutreach.gs.washington.edu/files/investigating_smoking_behavior_jan2011.pdf National Research Council. (1996). The National Science Education Standards. Washington, D.C.: National Academy Press. National Research Council. (2000). How people learn: Brain, mind, experience, and school. Washington, D.C.: National Academy Press. National Research Council. (2007). Taking science to school: Learning and teaching science in grades K-8. Washington, D.C.: National Academies Press. National Research Council. (2009). Learning science in informal environments: People, places, and pursuits. Washington, D.C.: National Academies Press. National Research Council. (2011). A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas. Washington, D.C.: The National Academies Press. Newton, P., Driver, R., & Osborne, J. (1999). The place of argumentation in the pedagogy of school science. International Journal of Science Education, 21(5), 553 – 576. Osborne, J. (2010). Arguing to learn in science: The role of collaborative, critical discourse. Science, 328, 463466. Osborne, J., & Patterson, A. (2011). Scientific argument and explanation: A necessary distinction? Science Education, 95(4), 627-638. Pierut-Le Bonniec, G., & Valette, M. (1991). The development of argumentative discourse. In G. Pierut-Le Bonniec & M. Dolitsky (Eds.), Language bases to discourse basics. Amsterdam: Benjamins. Reznitskaya, A., Anderson, R. C., McNurlen, B., Nguyen-Jahiel, K., Archodidou, A., & Kim, S. (2001). Influence of oral discussion on written argument. Discourse Processes, 32(2-3), 155-175. Toulmin, S. E. (1958). The uses of argument. Cambridge: Cambridge University Press. Vye, N. J., Schwartz, D. L., Bransford, J. D., Barron, B. J., Zech, L. K., & Cognition and Technology Group at Vanderbilt. (1998). SMART environments that support monitoring, reflection, and revision. In D. Hacker, J. Dunlosky & A. Graessar (Eds.), Metacognition in Educational Theory and Practice (pp. 305-346). Mahwah, NJ: Erlbaum. Vygotsky, L. S. (1978) Mind in society: The development of higher psychological processes. (M. Cole, V. JohnSteiner, S. Scribner, & E. Souberman, Eds. and Trans.). Cambridge, MA: Harvard University Press. Windschitl, M., Dvornich, K., Ryken, A. E., Tudor, M., & Koehler, G. (2007). A Comparative Model of Field Investigations: Aligning School Science Inquiry with the Practices of Contemporary Science. School Science and Mathematics, 107, 1, 382-390.

© ISLS

48

ICLS2012

Volume 2: Symposia

Engaging Middle School-Aged Students in Classroom Science and Mathematics: Implications for Design and Research
Kimberley Pressick-Kilborn University of Technology, Sydney, PO Box 222, LINDFIELD NSW, AUSTRALIA 2070 Email: kimberley.pressick-kilborn@uts.edu.au Melissa Gresalfi Indiana University, 1900 E. 10th Street, Eigenmann 532, Bloomington, IN 47406 Email: mgresalf@indiana.edu K. Ann Renninger & Jessica E. Bachrach Department of Educational Studies, Swarthmore College, Swarthmore, PA 19081 Email: krennin1@swarthmore.edu, jbachra1@gmail.com Nicole Shechtman (Organizer / Chair), Britte Cheng, Patrik Lundh, & Gucci Trinidad Center for Technology in Learning, SRI International, 333 Ravenswood Ave., Menlo Park, CA 94025 Email: nicole.shechtman@sri.com, britte.cheng@sri.com, patrik.lundh@sri.com, gucci.trinidad@sri.com Richard Walker (Discussant) University of Sydney, Education Building A35, NSW 2006 Email: richard.walker@sydney.edu.au Abstract: A critical educational challenge and core issue for the ICLS 2012 theme of “the Future of Learning,” is how to keep students, particularly those who attend the most underresourced schools, engaged in science and math as they progress through secondary school. The four papers in this symposium discuss key issues, challenges, and progress around interest and engagement in science and math for students in grades 5 to 7. From a range of theoretical and methodological perspectives, the panelists discuss what interest and engagement are as research constructs; some of the design principles, key features of instructional materials, and contextual factors that trigger and shape interest and engagement; how learner characteristics influence the ways students select or respond to opportunities to engage; the roles that interest and engagement play in learning; and issues of measurement. This 90-minute session will include both formal presentations and moderated discussion among the panelists and audience.

Symposium Overview
A critical educational challenge is how to keep students, particularly those who attend the most under-resourced schools, engaged in science and math as they progress through secondary school. Motivation and engagement are essential to learning at all ages (National Research Council, 2000), yet it is well-documented that students’ interest and enjoyment in both science and math generally decline as they move from primary to secondary school (e.g., Hidi & Harackiewicz, 2000; Krapp, 2006; Speering & Rennie, 1996; Eccles & Wigfield, 1992; National Center for Education Statistics, 2006). The late primary / middle school years are a particularly important and vulnerable transition point in the school trajectory, as concepts become increasingly difficult and abstract (Nathan & Koellner, 2007; Leinhardt, Zaslavsky, & Stein, 1990). Many students begin to lose interest in science and math, fall behind in achievement (Oakes, 1990), and consolidate motivational attitudes (Eccles, Wigfield, & Schiefele, 1998). Declining interest is of special concern for urban inner-city students who face challenges of poverty, cultural differences, and language. In the US, the National Research Council-commissioned book, Engaging Schools: Fostering High School Students’ Motivation to Learn (National Research Council & Institute of Medicine, 2004), reports on a federal charge to review and synthesize the broad spectrum of educational research to establish a set of recommendations for the educational community for how to promote and maintain student engagement and genuine improvements in achievement throughout the high school years. This symposium focuses on the Committee’s first recommendations, that “courses and instructional methods be redesigned in ways that will increase adolescent engagement and learning” (p. 214). This charge is highly relevant to the international ICLS 2012 theme of “the Future of Learning,” in particular to the key prototypic research questions:

© ISLS

49

ICLS2012

Volume 2: Symposia

 

How can the energies and motivations that accompany a learner’s interests be matched with learning resources to enable productive learning pathways? How can teachers productively create teaching and learning environments that support the needs of learners of diverse linguistic, cultural and economic backgrounds?

These questions indicate the need for deep understanding of motivation as a research construct and its implications for design: What are key aspects of instructional materials and context that trigger and shape interest and engagement? How do learner characteristics influence the ways students select or respond to opportunities to engage? The four papers in this symposium provide complementary approaches to addressing these issues in the teaching of science and mathematics, especially for students in the late primary / middle grade years. The two papers by Pressick-Kilborn and Gresalfi describe design principles and design research undertaken with the aim of developing and understanding specific instructional affordances for motivation in science and math classrooms. Renninger & Bachrach report on findings addressing the relation among triggers for interest and learner characteristics in an out-of-school science program for at-risk youth, and Shechtman et al. consider issues of operationalization and measurement in a study of engagement as a multidimensional construct in the urban mathematics classroom. The papers represent a range of theoretical perspectives, rooted in both sociocultural and psychological approaches; a range of methodological approaches, including design research, qualitative case studies, and psychometrics; both formal and informal instructional contexts; technology-based curriculum and curriculum less reliant on technology; and settings in both the US and Australia. Furthermore, within these issues, the papers speak to a number of themes important to the ICLS community: collaborative learning, metacognition and self-regulation, developing flexible understandings that can be used beyond formal schooling and throughout life, and the capabilities of interactive technology and computational models for supporting STEM learning. The 90-minute session format will be as follows. The first hour will be formal presentation; each paper will be presented for 15 minutes. The last half hour will be dedicated to a discussion among the panel and a moderated discussion with the audience. The discussant is Richard Walker, whose research interests center on ways of enhancing the learning, motivation, and academic achievement of students at all levels of education. His work investigates from a sociocultural perspective learning in electronic learning environments designed to support collaborative and cooperative interactions among students, the use of textbooks and other learning resources, after-school homework support, and identity formation. He will pose questions to the panelists about the design implications from their findings regarding the prototypic questions, for example, (a) the match between middle school students’ interests and learning resources; and (b) how teaching and learning environments might productively address the needs of learners of diverse linguistic, cultural and economic backgrounds.

Paper 1: Instructional Design Principles for Engaging Students in Learning Science and Fostering Interest Development
Kimberley Pressick-Kilborn

Problem and Theoretical Rationale
A number of studies have reported that while students have a great deal of enthusiasm for science in the primary years, interest and enjoyment generally decline during the early years of learning science in secondary schools (Hidi & Harackiewicz, 2000; Krapp, 2006; Speering & Rennie, 1996). This finding leads to the question of what it is that makes primary science engaging and interesting. More specifically, what features of primary contexts for science learning contribute to creating, maintaining and developing student engagement and interest? The focus of this paper is the articulation and illustration of guiding instructional principles for supporting the development of students’ interest in learning science. The project was framed from a Vygotskian sociocultural perspective, which emphasises the social origins of learning and motivation (Walker, PressickKilborn, Sainsbury & Arnold, 2004; Walker, 2010). Strengths of adopting a sociocultural perspective lie in the emphasis on studying interest as it develops in authentic learning contexts over time, enabling the relationship between situational and individual aspects of interest to be considered as inclusively separate (Valsiner, 1998) so that a study of the total activity is maintained.

Method
The principles were developed as part of a qualitative, classroom-based project that drew on a design-based research methodology (Brown, 1992; Guthrie & Alao, 1997), aimed at informing and extending both theory and practice. Seven guiding instructional principles were established at the outset of the research, informed by the literature relating firstly to student motivation and interest (for example, Mitchell, 1993; Paris & Turner, 1994;

© ISLS

50

ICLS2012

Volume 2: Symposia

Guthrie & Alao, 1997; Bergin, 1999; Ainley, 2001), and secondly to classrooms as learning communities (Bereiter & Scardamalia, 1993; Brown, 1997; Collins, 1998). Two science learning units were subsequently designed in collaboration with a classroom teacher, guided by these principles, and team-taught in a grade 5 classroom in Sydney, Australia. Detailed researcher field notes were recorded for observations of class activities, student engagement and interest, as students participated in weekly science lessons over 6 months. Students shared self-reports of their interest through charting retrospective interest trajectory graphs and semistructured interviews were conducted with six focus students at different time points during the study.

Results
The key features of the learning units that contributed to creating a community of learners, designed to create and support interest and engagement in science and technology, were identified and considered in relation to (1) physical features of the classroom context (classroom layout, access to materials), (2) features to support social interaction within and beyond the classroom (grouping structures and strategies, access to experts in the wider community) and (3) pedagogical features (opportunities for hands-on involvement, varying degrees of choice, reflection on learning, interactive noticeboard). Individual student interest trajectory ratings were collated to create a class mean trajectory of interest development, which revealed patterns of interest in the classroom learning community. Interest peaked during engagement in hands-on investigations and design and make activities with multiple pathways for task completion and possibilities for negotiation. Interest peaks also were evident for guest speaker presentations and an excursion. Troughs were observed for class discussions, when there was relatively more passive engagement of students who were listening to peers, more limited opportunities for active participation and greater teacher control.

Discussion and Implications
The seven instructional principles articulated at the outset of the research related to the organisation of science learning units in three distinct phases, and emphasised the importance of active student engagement in hands-on tasks with possibilities for choice. The principles also supported the inclusion of both individual and collaborative activities, as well as opportunities for engagement with topics and issues of concern to the broader scientific community, and contact with domain experts and communities of practice beyond the classroom. In addition, the principles guided the teacher to share emotions during the learning process and to lead the learning as a more expert learner, whilst also fostering student reflection on the learning process. My researcher perspective as a participant observer in that classroom led to the refinement, elaboration and extension of the initial guiding design principles. In particular, three additional principles were developed, focusing on play, imagination and the physical layout of the classroom. The focus of this paper on guiding instructional principles has clear implications for classroom practice and more specifically, science curriculum design. The principles are illustrated by drawing on data gathered in the classroom-based research project, with the focus on elaborating the relationships amongst interest, engagement and learning.

Paper 2: Designing to Support Critical Engagement with Content
Melissa Gresalfi

Problem and Theoretical Rationale
In this paper I describe a set of design experiments that were undertaken with the goal of supporting a particular form of engagement that we call critical engagement, which involves making intentional choices about which procedures to leverage in order to support particular claims. The past twenty years have brought technological innovations that have changed the world as we know it. Continuous access to online information has supported a transformation of the relationship between individuals and knowledge; with information so readily accessible, people have been repositioned to move beyond acquisition of facts to consider when to access those facts, interrogate them, and integrate them into activity. For these reasons, knowledgeable participation in mathematics must involve more than proficiency using key procedures. Instead, knowledgeable participation must involve engaging in acts of decision-making, determining which procedures enable the resolution of defensible solutions, how, and why. It is clear that supporting students to engage with mathematics in this way is not simply a matter of learning more mathematics content. Instead, it is crucial that students have opportunities to learn new content in ways that are consistent with how we actually want them to use that information (Boaler, 2000; Bransford & Schwartz, 1999; Cobb, Stephan, McClain, & Gravemeijer, 2001; Engle, 2006; Greeno, 1991; Lave, 1997). When creating new curricula, it is therefore crucial that designers attend not just to the content goals of the unit (the mathematical ideas you want students to learn and understand), but also participatory goals of the unit (how you hope students will engage). This does not mean that becoming a successful engineer, for example, requires

© ISLS

51

ICLS2012

Volume 2: Symposia

learning all mathematics at the elbow of a practicing engineer. Instead, this suggests that the kinds of practices that a practitioner is expected to leverage (such as experimenting, reviewing, collaborating, justifying, testing, and inventing) are the practices engaged in during the learning experiences.

Data Sources and Methods
This paper presents two rounds of a design cycle of a statistics unit, created in the context of an online, interactive videogame called Quest Atlantis. The design experiment was conducted with the same teacher over two years, focusing on two different 5th grade math classes. For both studies, the entirety of the two-week unit was captured with videotapes of whole class interaction, small group discussion, and individual interviews. In addition, students’ submitted work was collected and coded for accuracy and for evidence of critical engagement.

Results and Significance
Examining the design history of the unit, this paper demonstrates how iterative refinements of the unit supported increasing critical engagement with the content, and specifically, the importance of fostering both intentionality and experiences of consequentiality in the designs. Finally, the paper considers the implications of the lessons learned for this targeted intervention for our understanding of how elements of design support students’ mathematical reasoning more generally.

Paper 3: Design Implications from Study of Potential Triggers in an Out-ofSchool Biology Workshop
K. Ann Renninger & Jessica E. Bachrach

Problem and Theoretical Rationale
For educators, an essential question is how to pique, or trigger, the interest of learners. However, little is understood about how triggering really works outside of the experimental setting. A working assumption among motivation researchers is that potential triggers for interest result in engagement even though research findings sometimes suggest otherwise (e.g., Durik & Harackiewicz, 2007). Moreover, the assumption that potential triggers result in engagement presumes that all learners perceive and respond to potential triggers in the same way—a point that has also been challenged (e.g., Bartlett, 1932; E. J. Gibson, 1982; J. J. Gibson, 1950, 1977, 1979; Herrenkohl, Tasker, & B. White, 2011; Norman, 1999). Better understanding of the triggering process is needed to begin to explain when and why potential triggers for interest are effective. In order to consider the relation between potential triggers for interest and learner engagement, as well as the relation between engagement and learner characteristics, findings from two studies of the activity of eight inner-city middle school participants in an out-of-school biology workshop will be described. Study 1 addresses the identification and generalizability of potential triggers for interest across activities. Study 2 is a post hoc analysis that explores the relationship between triggers for interest and learner characteristics.

Participants
Eight youths (3 males, 5 females) participated in the biology workshop. They were African-American or racially mixed and ranged in age from 9-12 years; mean age was 10.5 years. The youth were participants in a rigorous choral training program for inner-city youth and were enrolled in the workshop as a summer enrichment activity. They had had no previous experience with formal instruction in science and they were identified as being in the earliest phase of interest development.

Data Sources and Methods
Informed by grounded theory (Charmaz, 2000; Strauss & Corbin, 1998), potential triggers for interest were tracked using participant observation notes collected as part of a larger project focused on participant learning and motivation in an out-of-school, inquiry-oriented biology workshop for inner-city youth. Multiple additional sources of data (workshop artifacts, participant interviews, caretaker interviews, and educator reports) were available to the researchers as reference material for each of the two studies. Analyses for both studies were undertaken at the level of the activity and/or the group. Each of the studies to be reported targeted participants’ behaviors during ongoing workshop activity (Stake, 2005); other studies of the workshop have focused on individual participants. Categorical directed content analysis (Hickey & Kipping, 1996; Hsieh & Shannon, 2005; Potter & Levine-Donnerstein, 1999) was used for the first part of both Study 1 and Study 2. This approach involves using existing research and theory to inform data reduction, in addition to allowing identification of emergent categories for coding. The categories identified in Part I of each study were then used to code data in Part II of each study.

© ISLS

52

ICLS2012

Volume 2: Symposia

Potential Triggers for interest identified based on a review of the literature and tracked for purposes of analysis included affect, autonomy, challenge, character identification, computers/technology, group work, hands-on activity, instructional conversation, novelty, ownership, and personal relevance.

Findings and Significance
Taken together, findings from both studies confirm that learners do not perceive and respond identically to potential triggers for interest and provide insight into the circumstances affecting when potential triggers will and will not work for learners who are new to science. They indicate that the triggering process is nuanced and is informed both by the features of the activity and the readiness of the learner to pick up on potential triggers. Findings from Study 1 indicate that triggers examined in experimental studies could be identified in the naturally occurring workshop setting. Furthermore, the same triggers could be identified in multiple activities, although they may or may not work for all learners at all times. Moreover, some activities included multiple triggers. As in studies from work on situational interest, the present findings indicate that when triggers work, attention is captured. In this sense, triggering enabled engagement (Fredricks, et al., 2004) and was often dependent on other people in the environment, the design of activities, or, as reported in Study 1, serendipity such as when a participant walked into a spider’s web. In contrast to discussions suggesting that situational interest might be expected to work the same way for all learners (e.g., Hidi & Berndorff, 2001), the present findings suggest that the process of triggering is complex and idiosyncratic, and that knowledge of the learning environment and the learner may be essential for predicting whether potential triggers will work. Findings from Study 2, furthermore, underscore that learner characteristics are likely to affect when and how potential triggers work. While this notion is not new, Study 2 represents an initial attempt to systematically investigate this relation and to do so in a naturally occurring context with all of its complexity. A number of previously unreported relations among potential triggers and learner characteristics surfaced. In particular: (a) Some learner characteristics were found to be more relevant to some triggers than others (e.g., sociability appears most likely to influence whether group work works as a trigger for interest); (b) The success of some potential triggers was found to always be tied to particular learner characteristics (e.g., personal relevance was always affected by awareness); (c) Furthermore, potential triggers were found to be affected by different numbers of learner characteristics (e.g., affect, challenge, and group work were affected by all of the identified learner characteristics).

Paper 4: Unpacking the Black Box of Engagement: Cognitive, Behavioral, and Affective Engagement in Learning Mathematics
Nicole Shechtman, Britte Cheng, Patrik Lundh, & Gucci Trinidad

Problem and Theoretical Rationale
Keeping students engaged in mathematics as they progress through secondary school is a critical national educational challenge for practitioners and researchers. It is also called out in many recent national policy documents (e.g., Engaging Schools [National Research Council& Institutes of Medicine, 2004], National Mathematics Advisory Panel [NMAP, 2008], Adding It Up [Kilpatrick et al., 2001]). However, “engagement in mathematics learning” is still somewhat of a black box—the field is grappling with a gap in our understanding of what engagement in mathematics truly is, how to measure it, what its relationships to learning are, and how we could leverage what we understand to increase engagement for all students. The goal of the Math Engagement Project (MEP) is to examine ways to unpack this black box. MEP is an exploratory research study of middle school mathematics learning and engagement to build theory and instrumentation around engagement in mathematics learning—which we treat as a complex, multidimensional, context-specific construct. We will present findings with respect to a theoretical framework we are developing to capture the dimensions of engagement in math learning as it is embedded in the classroom as an activity system. In a review of the broader literature on school engagement, Fredricks et al. (2004) describe several dimensions that are implicit within “engagement.” We apply this specifically to the domain of mathematics learning, teasing apart three components: (a) behavioral engagement (e.g., being on-task), (b) affective engagement (e.g., enjoying or being interested), and (c) cognitive engagement (e.g., doing the cognitive work). Within the cognitive space, there are also a variety of “ways” of engaging generally (e.g., memorizing facts vs. solving novel problems) and specifically with mathematical content (e.g., critical [see Paper 2] vs. procedural vs. conceptual). We also distinguish engagement at two different time horizons: (a) “dispositional engagement” over a long period of time (e.g., engagement in secondary math), and (b) “proximal engagement” in the moments of instruction (e.g., engagement in today’s math lesson). We use this framework to unpack the elements of engagement and learning of middle school students (ages 12-13) in the classrooms of two teachers teaching a 4-week unit on rate and proportionality. In prior large-

© ISLS

53

ICLS2012

Volume 2: Symposia

scale RCTs, this unit, which integrates curriculum and the interactive, dynamic software SimCalc MathWorlds, was shown to promote gains in conceptual understanding for a wide variety of students (see Roschelle et al., 2010). However, while learning gains overall were robust, there was also substantial variation between students—indicating that not all students engage with the unit to the same extent or in the same ways. In the current study, we use multiple methods to examine individual students’ behavioral, affective, and cognitive engagement during several lessons in the unit. We seek to characterize the range of engagement and examine how it may be shape the mathematics that students learn. Furthermore, we examine how the ways individual students engage may be shaped by (1) the dispositions students bring to the unit, and (2) the affordances for engagement provided by the curriculum and the teachers.

Methods
For the 302 seventh-grade students across the two teachers’ 11 classrooms, sources of data include: (1) pre-unit survey of attitudes and dispositions toward learning and mathematics (e.g., achievement orientation, interest in mathematics, mathematical confidence and anxiety, school engagement); (2) unit pretest and posttest to measure baseline knowledge and learning gains for the unit’s mathematical content; (3) survey administered multiple times throughout the unit to capture students’ self-reported behavioral, affective, and cognitive engagement in the lessons; (4) teachers’ report of students’ behavioral, affective, and cognitive engagement for a subset of 98 students; (5) analyses of written mathematical work for evidence of cognitive/mathematical engagement; and (6) classroom observations focused on the mathematical discourse. Additionally, 11 case study students (7 girls, 4 boys) were interviewed before, during, and after the unit to investigate their engagement and learning in greater depth. This paper will focus on the behavioral, affective, and cognitive engagement of the students over the course of the unit, using the 11 case study students as illustrative examples. We draw from these multiple sources of data to characterize the different ways students engage, how their engagement relates to their learning, and the dispositional and contextual factors that shape their engagement.

Preliminary Results and Discussion
The context of the study was a suburban area in Northern California. The school has less than 10% of students qualifying for the free or reduced lunch program, and the majority of students are Caucasian or Asian/Pacific Islander (Note that we are in the process of recruiting for the next study which will focus on students in highpoverty, high-minority schools). The 11 classrooms across the two teachers represented the remedial and gradelevel pre-algebra tracks (more advanced students had a different teacher). Not surprising in this context, student attendance was high and behavioral disruptions were relatively low. Both teachers taught with high fidelity to the workbook. On average, students reported a relatively positive classroom climate, in terms of both teacher and student support. The mean student pretest and gains were slightly above the average of the RCT treatment group, which represented a wide variety of classrooms across the state of Texas. Within this context, the 302 students represent a range of prior achievement levels (as measured on the pretest) and prior attitudes and dispositions toward learning and mathematics. While most of the students tended to be behaviorally engaged (i.e., on task) much of the time, preliminary findings suggest marked differences in the ways students engage affectively and cognitively. Affectively, while some students’ engagement is marked by interest and enthusiasm, other students’ is marked by boredom, anxiety, or indifference. Cognitively, while some students engage in building their own knowledge as the curriculum was intended through inquiry with the software and asking questions, other students appear quite satisfied to passively absorb the lessons, while others are strongly disaffected and simply trying to get by. Preliminary findings also suggest that these ways of engaging may be mediated by dispositions students had reported at baseline (e.g., how they see themselves as learners in the classroom, achievement orientation as mastery vs. performance, mathematical self-efficacy), the ways in which the teachers scaffold the lesson, and opportunities to interact productively or unproductively with peers. For example, the more passive cognitive engagement was observed in a highly math-anxious student in the classroom of the teacher who scaffolded the lesson in a way that emphasized correct answers. Findings also suggest that these ways of engaging have consequences for the depth of conceptual understanding that students attain. These and other themes will be discussed with respect to the broader literatures on engagement and motivation. We will explore implications for the measurement of engagement and how to find leverage points for developing ways to support engagement in learning mathematics.

References
Ainley, M. (2001). Interest in learning and classroom interactions. In D. Clarke (Ed.), Perspectives on practices and meaning in mathematics and science classrooms (pp. 105-130). Dordrecht: Kluwer Academic Press. Bartlett, F. C. (1932). Remembering: A study in experimental and social psychology. London: Cambridge University Press.

© ISLS

54

ICLS2012

Volume 2: Symposia

Bereiter, C., & Scardamalia, M. (1993). Surpassing ourselves: An inquiry into the nature and implications of expertise. Chicago: Open Court. Bergin, D.A. (1999). Influences on classroom interest. Educational Psychologist, 34(2), 87-98. Boaler, J. (2000). Exploring situated insights into research and learning. Journal for Research in Mathematics Education, 31, 113-119. Bransford, J. D., & Schwartz, D. L. (1999). Rethinking transfer: a simple proposal with multiple implications. Review of Research in Education, 24, 61-100. Brown, A.L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences, 2, 141-178. Brown, A.L. (1997).Transforming schools into communities of thinking and learning about serious matters. American Psychologist, 52(4), 399-413. Charmaz, K. (2000) Grounded theory: Objectivist and constructivist methods. In N. Denzin & Y. Lincoln (Eds.), Handbook of qualitative research (2nd ed., pp. 509-536). London: Sage Publications. Cobb, P., Stephan, M., McClain, K., & Gravemeijer, K. (2001). Participating in classroom mathematical practices. Journal of the Learning Sciences, 10, 113-164. Collins, A. (1998). Learning communities: A commentary on papers by Brown, Ellery and Campione, and by Riel. In J.G. Greeno & S.V. Goldman (Eds.), Thinking practices in mathematics and science learning. Mahwah, NJ: Lawrence Erlbaum Associates. Durik, A., & Harackiewicz, J. M. (2007). Different strokes for different folks: How individual interest moderates the effects of situational factors on task interest. Journal of Educational Psychology, 99(3), 597-610. Eccles, J. S., & Wigfield, A. (1992). The development of achievement-task values: A theoretical analysis. Developmental Review, 12, 265-310. Eccles, J. S., Wigfield, A., & Schiefele, U. (1998). Motivation to succeed. In N. Eisenberg (Ed.), Social, emotional, and personality development handbook of child psychology (Vol. 3, pp. 1017-1096). New York: Wiley. Engle, R. A. (2006). Framing interactions to foster generative learning: A situative explanation of transfer in a community of learners classroom. Journal of the Learning Sciences, 14(4), 451-498. Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59-109. Gibson, E. J. (1982). The concept of affordances in development: The renascence of functionalism. In W. A. Collins (Ed.), The concept of development: The Minnesota symposia on child psychology (Vol. 15, pp. 55-82). Lawrence Erlbaum Associates, Inc. Gibson, J. J. (1950). The perception of the visual world. Boston: Houghton Mifflin. Gibson, J. J. (1977). The theory of affordances. In R. Shaw & J. Bransford (Eds.), Perceiving, acting, and knowing: Toward an ecological perspective (pp. 67-82). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers. Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin Company. Greeno, J. G. (1991). Number sense as situated knowing in a conceptual domain. Journal for Research in Mathematics Education, 22, 170-218. Guthrie, J.T., & Alao, S. (1997). Designing contexts to increase motivations for reading. Educational Psychologist, 32(2), 95-105. Herrenkohl, L. R., Tasker, T., & White, B. (2011). Pedagogical practices to support classroom cultures of scientific inquiry. Cognition and Instruction, 29(1), 1-44. Hickey, G., & Kipping, C. (1996). Issues in research: A multi-stage approach to the coding of data from openended questions. Nurse Researcher, 4, 81-91. Hidi, S. & Harackiewicz, J.M. (2000). Motivating the academically unmotivated: A critical issue for the 21 st century. Review of Educational Research, 70(2), 151-179. Hidi, S., & Berndorff, D. (2001). Interest, reading, and learning: Theoretical and practical considerations. Educational Psychology Review, 13(3), 191–209. Hsieh, H.-F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277-1288. Kilpatrick, J., Swafford, J., & Findell, B. (2001). Adding it up: Helping children learn mathematics. Washington, DC: National Academy Press. Krapp, A. (2006). Structural and dynamic aspects of interest development: Theoretical perspectives from an ontogenetic perspective. Learning and Instruction, 12(4), 383-409. Lave, J. (1997). The culture of acquisition and the practice of understanding. In D. Kirshner & J. A. Whitson (Eds.), Situated cognition: Social, semiotic, and psychological perspectives (pp. 17-35). Mahwah, NJ: Erlbaum. Leinhardt, G., Zaslavsky, O., & Stein, M. (1990). Functions, graphs, and graphing: Tasks, learning, and

© ISLS

55

ICLS2012

Volume 2: Symposia

teaching. Review of Educational Research, 60, 1-64. Mitchell, M. (1993). Situational interest: Its multifaceted structure in the secondary school mathematics classroom. Journal of Educational Psychology, 85(3), 424-436. Nathan, M. J., & Koellner, K. (2007). A framework for understanding and cultivating the transition from arithmetic to algebraic reasoning. Mathematical Thinking and Learning, 9(3), 179-192. National Center for Education Statistics. (2006). The nation's report card: Mathematics 2005 (No. NCES 2001– 571). Washington, DC: U.S. Department of Education. National Mathematics Advisory Panel. (2008). Foundations for success: The final report of the National Mathematics Advisory Panel. Washington DC: United States Department of Education. National Research Council. (2000). How people learn: Brain, mind, experience, and school, Expanded edition. In J. D. Bransford, A. L. Brown, & R. R. Cocking (Ed.), Committee on Developments in the Science of Learning, Committee on Learning Research and Educational Practice, Commission on Behavioral and Social Sciences and Education. Washington D.C.: National Academy Press. National Research Council & Institute of Medicine. (2004). Engaging schools:Fostering high school students’ motivation to learn. Washington, DC: The National Academies Press. Norman, D. A. (1999). Affordance, conventions, and design. Interactions, 6(3), 38-42. Oakes, J. (1990). Opportunities, achievement, and choice: Women and minority students in science and mathematics. Review of Educational Research, 16, 153-222. Paris, S.G., & Turner, J.C. (1994). Situated motivation. In P.R. Pintrich, D.R. Brown & C.E. Weinstein (Eds.), Student motivation, cognition and learning (pp. 213-237). Hillsdale, NJ: Lawrence Erlbaum. Potter, J., & Levine-Donnerstein, D. (1999). Rethinking Validity and Reliability in Content Analysis. Journal of Applied Communication Research, 27, 258-284. Roschelle, J., Shechtman, N., Tatar, D., Hegedus, S., Hopkins, B., Empson, S., Knudsen, J. & Gallagher, L. (2010). Integration of technology, curriculum, and professional development for advancing middle school mathematics: Three large-scale studies. American Educational Research Journal, 47(4), 833878. Speering, W., & Rennie, L. (1996). Students’ perceptions about science: The impact of transition from primary to secondary school, Research in Science Education, 26(3), 283-298. Stake, R. E. (2005). Qualitative case studies. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (3rd ed., pp. 443-466). Thousand Oaks, CA: Sage Publications. Strauss, A., & Corbin, J. (1998). Basics of qualitative research techniques and procedures for developing grounded theory (2nd ed.). London: Sage Publications. Valsiner, J. (1998). Dualisms displaced: From crusades to analytic distinctions. Human Development, 41, 350354. Walker, R.A. (2010). Sociocultural issues in motivation. In P. Peterson, E. Baker & B. McGaw (Eds.), International Encyclopedia of Education (3rd ed., Vol. 6, pp. 712-717). Oxford: Elsevier. Walker, R.A., Pressick-Kilborn, K., Arnold, L. & Sainsbury, E.J. (2004). Investigating motivation in context: Multiple dimensions, domains and assessments. European Psychologist, 9, 245-256.

© ISLS

56

ICLS2012

Volume 2: Symposia

Robot Facilitation as Dynamic Support for Collaborative Learning
Naomi Miyake, The University of Tokyo, Hongo 7-3-1, Bunkyo-ku, Tokyo 113-0033, Japan, nmiyake@p.u-tokyo.ac.jp Sandra Y. Okita, Teachers College, Columbia University, 525 West 120th Street, New York, NY 10027, okita@tc.columbia.edu Abstract: Automated as well as remotely controlled robots have high potential to expand our research and practice in collaborative learning environments. The area of computer supported collaborative learning (CSCL) is one in which advanced technologies have been used effectively to automatically monitor and dynamically structure group discussions for learning. Traditional desktop environments for collaborative learning have included interactions between groups and computer agents with conversational abilities. Robots can take the collaboration out of the computer and into the three dimensional, real world environments of students. This symposium raises the important question of how advanced technologies, such as robotics, may enable us to take the same insights from the area of scripted collaboration and use them to support groups of learners in these environments. Other topic areas include design choices in robotic features that may influence scripted interactions, social behavior, meaningful engagement, and conditions that dynamically support situational factors and generate collaborative learning among students.

Introduction
As collaborative learning has proven to be effective in helping students learn (Chi et. al, 2001; Graesser, Person, & Magliano, 1995; Palincsar & Brown, 1984), it is only natural to ask how sophisticated artifacts can be used as facilitating tools to dynamically support collaborative learning. Conversational agents, avatars, and humanoid robots are highly directable, allowing researchers to capitalize on a dynamic nature that enables humans to share knowledge and ideas. Similar to the use of conversational agent technologies in desktop CSCL environments, having a robot participate in collaborative learning allows researchers to control some part of the collaborative activities, which enables us to better understand the basic mechanisms of collaborative learning. This work has built on theories and findings from a wide range of areas of the Learning Sciences as well as the technical fields of Machine Learning, Language Technologies, and Human-Computer Interaction. Some of the most cutting edge work in this area has been the use of robots to support collaborative learning. The symposium takes a close look at three areas that may contribute to the unique value of robots in assisting collaborative learning and assessment. One area that is critical in the design of meaningful engagement is scripted collaboration. In the past decade, the area of scripted collaboration has produced tremendous gains in terms of insight into how to elevate the occurrence of valuable discussion-based learning behaviors from students in online and offline settings (Dillenbourg, 2002; Kollar, Fischer, & Slotta, 2005). This insight has also led to advanced technologies, such as robotics, that may enable us to take these findings and use them to support groups of learners in the three-dimensional world. More recently, advances in the area of automatic collaborative learning process analysis have enabled the first attempts at dynamic support for collaborative learning, which builds on the insights learned from static forms of scripted collaboration (Rosé, Wang, Cui, Arguello, Stegmann, Weinberger, & Fischer, 2008). The second area takes from the theories and findings of Human-Computer Interaction (HCI) and Human-Robot Interaction (HRI). This area examines how specific features in robots may influence; how interactive scenarios are delivered to humans (e.g., with an angry voice, with gesture), how behavioral scripts based on familiar narratives and routines can be generated from humans (e.g., common children play routines like tea party, play doctor), and how robotic features may influence the quality of interaction (e.g., testing the robot or meaningful engagement). In examining the situational factors that promote collaborative learning, much attention has been given in making machines more responsive and sophisticated. The advancement in sensors and audio-visual tools has helped robots detect human behavior (e.g., facial expressions), while the sophisticated automation and expressive tools have helped robots provide dynamic and expressive responses toward humans (e.g., gesture, tone of voice, attention). Identifying the delicate balance between detection and response has improved the overall quality of human-robot interaction. However, little research has been done to examine how these features when combined with specific scripts and scenarios can assist collaborative learning and meaningful engagement. The final area of interest in this symposium is taking the insights from the first two areas (i.e., scripted collaboration and human-robot interaction) and exploring conditions that dynamically support and generate the optimal level of collaboration among groups of students. Robots and other agent technologies can participate in
© ISLS 57

ICLS2012

Volume 2: Symposia

and support reflection in active ways, for example by providing learners with precise replay of past learning experiences upon request, which has not been as feasible with human support. In addition to these capabilities, however, the use of robots goes beyond that, allowing us to implement basic design principles of group activities in more tangible ways. They offer the unique opportunity to support groups of students in real world activities. Automated as well as remotely controlled robots have high potential to expand our research and practice in collaborative learning. As a learning collaborator, robots can present their prepared explanations to their human partners, comment on others’ ideas, and exchange questions and answers, at least to the extent those exchanges could be either prepared prior to the classes or provided by human operators. As a research partner, robots can run controlled experiments during a collaborative learning process in classroom-like settings. One example of such control is to have a robot in each discussion group and to have it deliver the same information to different groups, so that we can explore the effects of a particular discourse in collaboration. Reflectively it is illuminating that the various effects of this kind of dynamic and interactive support have not been carefully researched in the past. Collaborative learning has been practiced widely in the community of learning scientists, with relative to profound change in the quality of learning achievements (Bransford et al., 1999; Sawyer, 2006). In order to better design such practices within a wider variety of cultural settings, we need stronger scientific evidence for such success, as well as a more precise explanation of the mechanisms of how and why it works (cf., Miyake, 2008). One of our motivational factors for proceeding in this direction is the need for us to better equip ourselves with stronger sets of evidence of the power of collaborative, learner-centered orientation in education, to work effectively with students in the collaborative setting.

Target Audience
Our audience would include a wide range of learning science researchers who seek to evolve research on collaboration. We particularly aim this symposium at practice-oriented, classroom-based research, from which we could learn how to build sustainable, effective communities, to foster impacts on real world learning. We are also keen to invite engineers and robotics researchers seeking focused research fields to foresee what is needed for the creation of symbiotic robots. Collaborating with them would open up a stronger promise for us to reach out to real world classrooms.

Symposium Presentations
The symposium presentations will focus on three topic areas: 1) dynamic collaborative learning support techniques with scripted collaboration, 2) influence of robotic features on interactive scenarios and facilitating engagement, and 3) exploring conditions that dynamically support situational factors that generate the optimal level of collaboration among groups of students. The research in the first presentation covers interactions that are text-based in a two-dimensional chat environment with group interactions, but the architecture and dynamic collaborative learning support techniques are highly applicable to physical robots environment. The research in the second presentation involves a close examination of physical humanoid robots and humans in a one-to-one (robot-to-human) interaction (not in groups), but the findings has helped identify important perceptual cues and responses that need to be recognized and handled respectively that are applicable to collaborative groups settings. The research in the final presentation involves multiple physical humanoid robots spread across multiple student groups and addresses many of the common insights and issues from the first two presentations. To help highlight the three interest areas, we discuss them separately, but all three are at play in each presentation. The symposium consists of two parts. The first part involves presentations of recent work that shows promising results, strengths of, and challenges in robot facilitation. The second part of the session leads into a discussion organized by discussants that engage presenters and audiences to explore practical use of robots and agents/avatars in collaborative learning (e.g., how different situational factors require different dynamic support, and how robots may facilitate in each setting).

Presentation 1 Basilica 3D: Towards Architecture for Operating Robotic Accountable Talk Facilitation Agents
Carolyn Penstein Rosé Language Technologies Institute and Human-Computer Interaction Institute School of Computer Science, Carnegie Mellon University cprose@cs.cmu.edu

© ISLS

58

ICLS2012

Volume 2: Symposia

While theories of discussion-based learning both within the computer supported collaborative learning (CSCL) community as well as in the classroom discourse community are many, there is consensus about what types of group interactions are desirable. For example, the value placed on taking ownership of reasoning, articulation of reasoning, evaluation of reasoning, and building on the reasoning of self and others are almost ubiquitous. Yet it is widely acknowledged that groups do not operate at an ideal level without support. Thus, researchers in the area of scripted collaboration have worked to develop design principles to guide development of support that elicits the kind of group behaviors that are valued within the CSCL community. However, while much of that work was developed and evaluated with static forms of support, there has been a growing interest in making that support dynamic, both in the sense of triggering based on real time analysis of the ongoing collaboration (Rosé et al., 2008; Mu et al., 2011), as well as in the sense of support that is itself interactive (Kumar et al., 2006). Conversational agents have a long history of successful support for individual learning with technology (Rosé et al., 2001; Rosé & Van Lehn, 2005; Kumar et al., 2006). A series of results offer hope that they can be used productively to offer support for collaborative learning, especially in chat environments (Kumar & Rosé, 2011; Kumar et al., 2010; Kumar et al., 2011; Chaudhuri et al., 2009; Ai et al., 2010; Howely, Mayfield, & Rosé, 2011). The Basilica architecture has been developed for the purpose of enabling efficient development of intelligent agents with rich conversational behaviors to participate with groups of students in online chat environments and to play a facilitative role (Kumar & Rosé, 2011). Early work in this area simply imported the same conversational agents used in individual learning with technology into group learning settings. However, more recent work has directly employed techniques from the classroom discourse community to develop conversational agents that can employ to some limited extent the practices of classroom facilitation techniques. In early work on dynamic support for collaborative learning, agent behaviors were triggered through analysis of text-based interactions in the chat environment. Recent work on monitoring group interactions through speech (Gweon et al., 2011) gives hope that this dynamic collaborative learning support technique can be adapted from the two-dimensional text-based chat environment to the three-dimensional, face to face collaboration setting where the facilitation may be conducted by robots controlled through this same architecture. This talk takes a visionary look at how this Basilica work may be extended for this purpose.

Presentation 2: Multimodal Approach to Facilitating Affective Human-Robot Interactions
Sandra Y. Okita Communication, Computing, and Technology in Education Teachers College, Columbia University okita@tc.columbia.edu The advancement in sensors and audio-visual tools has helped robots detect human behavior (e.g., facial expression), while sophisticated automation and expressive tools have helped robots provide dynamic and expressive responses toward humans (e.g., gesture, tone of voice, attention). Identifying the delicate balance between detection and response has improved the overall quality of human-robot interaction. However, little research has been done to examine how these features when combined with specific interactive scenarios can facilitate meaningful and affective engagement. The research shares some preliminary findings on how specific features in robots may influence how interactive scenarios are delivered to humans (e.g., with an angry voice, exaggerated gesture), how familiar scenarios and behavioral scripts can be generated from humans (e.g., hide-and-seek), and how robotic features may influence the quality of interaction (e.g., child testing the robot but not communicating with the robot), affective engagement (e.g., robotic voice scares child) and learning (e.g., peer-like robot vs. teacher-like robot). Having physical humanoid robots in the collaborative learning setting provides researchers with a range of design choices (e.g., tone of voice, attention, gesture, detection and response timing) on how interactive scenarios can be delivered. For example, is it the content of the interaction or the timing of feedback that is more important? If the interactive scenario is accompanied with robotic gestures (Ng-Thow-Hing, Luo, & Okita, 2010; Okita, Ng-Thow-Hing, & Sarvadevabhatla, 2009), will the interpretation be different? Will interaction be more affective if the robot maintains longer eye contact, pay more attention, mimic human gestures, or stand more closely to students? As some features may be common across different platforms (e.g. computer desktop, virtual reality environments, robot) some features are unique to the physical environment (e.g., proxemics). Examining these characteristics can be potentially useful when incorporating the findings into designing interactive and dynamic scenarios for collaborative learning. In the first line of work, we applied familiar play routines when interacting with robots (e.g., turntaking scenario, setting up the dinner table). Often times, the child’s ability to pretend or engage in

© ISLS

59

ICLS2012

Volume 2: Symposia

collaborative play was constrained by what the robot could do in response (e.g., timing and limitation of response). In another line of work we found that when the robot activated a familiar schema (e.g., going to the zoo, story-telling time, The Three Little Pigs), the child sustained a social relationship with the robot (Okita, Ng-Thow-Hing, & Sarvadevabhatla, 2011). Children seemed to start out by drawing on prior knowledge through which they decide the “can and cannot do interactions” with robots (Okita & Schwartz, 2006). The studies helped identify important perceptual cues and responses that needed to be recognized and handled respectively to improve the quality of interaction. Until robots have the intelligence to flexibly respond to a wide range of interactive bids, following a well-known script or scenario seemed quite successful in guiding the interaction. In the third line of work, we examined whether familiar game scenarios (e.g., children's game "Captain May I?") could influence physical distance (e.g., proxemics) between human and robots (Okita, NgThow-Hing, & Sarvadevabhatla, 2012). Current limitations in sensor technology made physical distance a crucial factor to detect and respond to situations (e.g., avoid collision during collaborative tasks), user state (e.g., facial expression, speech recognition), and better choice of expressive communication (e.g., verbal or gesture).

Presentation 3: Robots facilitate “constructive listening” for strengthening individualized learning in collaborative learning situations
Naomi Miyake Consortium for Renovating Education of the Future, School of Education The University of Tokyo nmiyake@p.u-tokyo.ac.jp Remotely operable robots in collaborative classrooms can serve at least two roles. One is to work with children as “a learning friend” who helps members in the class enjoy and stay engaged in dynamic learning, where each individual member promotes her/his own understanding. The second role is to work with researchers as a datacollector and a reflector of our own facilitating behavior so that we can learn both the basic mechanisms and the design principles of productive activities in collaborative classrooms. In this presentation I will focus on the former, to illustrate what kind of new research can be developed by using robots, and then lead the way toward the second role. In order to establish the research context for this study, we have devised a strongly scripted yet dynamically collaborative learning situation based on the Jigsaw method. We call this framework the Knowledge Constructive Jigsaw, where it is emphasized that each individual student is responsible for integrating perspectives given by the learning materials and from other students with their own understanding. The class design involves a shared question to be answered and some relevant learning materials from different perspectives distributed among the different groups first in expert groups, to be later exchanged and integrated to answer the question in the jigsaw groups (Miyake, 2011). In accordance to some basic mechanisms of constructive interaction (e.g., Miyake, 1986), the design naturally requires each student to become a task-doer in the jigsaw group. It also provides each student with a chance, or chances, to be a monitor who infers what the other students say and why they say that, to integrate others’ ideas with their own. Yet the proportion of being the doer could differ from individual to individual. We now have data from nearly thirty classes of different grade levels on different topics. The analyses of these classes of middle schools on different science topics have revealed that there is little correlation between the achievement levels and the proportion of the role exchange. Rather, the monitors, who could spend almost the entire class without “speaking up,” learned a lot from just attending to others’ talks and inwardly working to integrate such inputs into their own understandings. This result suggests the importance of “constructive monitoring,” similar to the notion of “constructive listening” proposed by Greeno and van de Sande (2007), for better analyses and design of the interactive learning processes (Damsa et al., 2010). In a series of follow-up studies of this construct, we used remotely operable robots so that we could control the levels of soliciting constructive monitoring. We used the established learning plans and their associated materials with new groups of children and a robot, to see whether we could recreate similarly successful classes with robots as facilitators, and also to see what kind of conversational cues the monitoring children would use to start taking responsibility of their own learning. The basic analyses of these classes have revealed that children around 10 year olds have a keen sense of distinguishing whether the robot “knows” the answer but does not say it or the robot is “just like the other kid who does not know the answer, but sincerely working to know the answer.” When the robots are accepted as the latter, the children had a better chance to get involved in the constructive conversation, resulted in better learning (Miyake, 2011; Miyake, et al., 2011; Oshima, et al., 2011).

© ISLS

60

ICLS2012

Volume 2: Symposia

Discussion Topics (Tentative) 1. Comparison with alternative technologies. How robots can enhance spontaneous, creative collaborative discourse among learners without interrupting or distracting from the inter-personal interaction (Gerry Stahl)
How can robots create values beyond existing devices for learning and collaboration? Can we use existing measures for collaborative learning to evaluate this type of learning partnership? 2. Pragmatic considerations of deployment to formal and informal learning environments. What needs to be considered when taking robots out of the lab and into real world (e.g., user support).

Discussant 1: Gerry Stahl
College of information Science and Technology Drexel University Gerry.Stahl@drexel.edu

Area of Expertise:
Dr. Stahl is trained in computer science, artificial intelligence, social philosophy, cognitive science, and learning science. Since 2002, he has taught human-computer interaction (HCI), computer-supported collaborative learning (CSCL), computer-supported cooperative work (CSCW) and social informatics (SI) at Drexel University. Dr. Stahl’s research approach includes theory building, system development and empirical studies of software usage. He has developed software systems and prototypes to explore support for collaborative learning, design rationale, perspectives and negotiation. His theory combines various sources from philosophy, education, sociology, communication and anthropology. He has developed a methodology of fine-grained empirical investigation into how groups of people learn to use artifacts like groupware systems in real-world settings such as school classrooms and virtual math teams. Dr. Stahl is a world-class researcher in CSCL, having organized international conferences, founded an international journal, published a volume on Group Cognition in MIT Press and one on Studying Virtual Math Teams in Springer Press and written over 150 professional papers. His website and blog are major resources for the CSCL research community. Retrieved excerpts from Drexel University faculty www site (November 7, 2011).

Discussant 2: Frank Fischer
Munich Center of the Learning Sciences Ludwig-Maximilians-Universität München Frank.fischer@psy.lmu.de

Area of Expertise:
Frank Fischer earned his Doctorate in Psychology in 1997, and his Habilitation (professorial dissertation) in Psychology and Educational Science in 2002, both from the University of Munich. He is a Full Professor of Educational Science and Educational Psychology at the University of Munich. Since 2008 he is serving as the Director of the Department of Psychology at this university. Since 2009, he is the speaker of the Munich Center of the Learning Sciences, an interdisciplinary collaboration of more than 30 research groups focusing on advancing research on learning “From Cortex to Community”. In this context, he is also directing the Doctoral Training Program “Learning Sciences”. He is currently a member of the Board of Directors of the International Society of the Learning Sciences. His own research revolves around learning and instruction, with projects on collaborative learning, problem‐ based learning as well as inquiry and simulation‐ based learning. An overarching question is how technology-enhanced learning environments can advance knowledge and skills of learners in school, higher and further education. With respect to methodology, he has been contributing to the development of use‑ inspired basic research approach in the field of learning and instruction. He has published more than 100 journal articles and book chapters, and co‐ edited 6 books and special issues of scientific journals.

© ISLS

61

ICLS2012

Volume 2: Symposia

Retrieved excerpts from Ludwig-Maximilians-Universität München faculty www site (November 7, 2011).

Implications
In order to better design such practices within a wider variety of cultural settings, we need stronger scientific evidence for such success, and more precise explanation of the mechanisms of how and why things works. The studies in the presentations provide some portrayal on how students interact, engage in conversation, respond to, and work with robots and agents in a collaborative learning setting. This has some practical importance, as we hope to examine whether or not advanced technologies have promising roles for students in collaborative learning. The recent findings in the presentations, and the insight and expertise from the discussants, is sure to generate a strong discussion on the pros and cons, challenges and promises in using such artifacts in formal and informal learning environments. What is most crucial in such an endeavor is have an opportunity to gather, exchange ideas, rack brains, and plan future steps that will lead to successful research and implementation.

References
Ai, H., Kumar, R., Nguyen, D., Nagasunder, A., Rosé, C. P. (2010). Exploring the Effectiveness of Social Capabilities and Goal Alignment in Computer Supported Collaborative Learning, in Proceedings of Intelligent Tutoring Systems. Bransford. J., Brown, A.L. & Cocking R.R. (Eds.). (1999). How People Learn: Brain, Mind, Experience, and School. Washington D.C.: National Academy Press for National Research Council. Chaudhuri, S., Kumar, R., Howley, I., Rosé, C. P. (2009). Engaging Collaborative Learners with Helping Agents. In Proceedings of AI in Education Chi, M.T.H., Silver, S.A., Jeong, H., Yamauchi, T., Hausmann, R.G. (2001). Learning from human tutoring. Cognitive Science, 25, 471-533. Graesser, A.C., Person, N., Magliano, J. (1995). Collaborative dialog patterns in naturalistic one-on-one tutoring. Applied Cognitive Psychologist, 9, 359-387. Damsa, C. I., Kirschner, P. A., Andriessen, J. E. B., Gijsbert, E. & Sins, P. H. M. (2010). Shared Epistemic Agency: An Empirical Study of an Emergent Construct. The Journal of the Learning Sciences, 19, 143186. Dillenbourg, P. (2002). Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL Can we support CSCL (pp. 61-91). Heerlen: Open Universiteit Nederland. Drexel University (2011, November 7). Faculty Details Gerry Stahl, Ph.D. Retrieved November 7, 2011, retrieved excerpts from http://www.ischool.drexel.edu/Home/people/faculty/facultydetails/?facultyid=23 Greeno, J. G., & van de Sande, C., (2007). Perspectival understanding of conceptions and conceptual growth in interaction, Educational Psychologist, 42, 9-23. Gweon, G., Agarwal, P., Udani, M., Raj., B., Rosé, C. P.(2011). The Automatic Assessmnet of Knowledge Integration Processes in Project Teams, in Proceedings of Computer Supported Collaborative Learning Howley, I., Mayfield, E., Rosé, C. P. (2011). Missing Something? Authority in Collaborative Learning, in Proceedings of Computer Supported Collaborative Learning Kollar, I., Fischer, F., & Slotta, J. D. (2005). Internal and external collaboration scripts in web-based science learning at schools. In T. Koschman, D. Suthers, & T.W. Chan (Eds.), Computer Supported Collaborative Learning 2005: The Next 10 Years! (pp. 331-340). Mahwah, NJ: Lawrence Erlbaum Associates. Kumar, R., Rosé, C. P., Aleven, V., Iglesias, A., Robinson, A. (2006). Evaluating the Effectiveness of Tutorial Dialogue Instruction in an Exploratory Learning Context, Proceedings of the Intelligent Tutoring Systems Conference. Kumar, R., Ai, H., Beuth, J. and Rosé, C. P. (2010) Socially-capable Conversational Tutors can be Effective in Collaborative-Learning situations, Intelligent Tutoring Systems, Pittsburgh, PA Kumar, R. and Rosé, C. P. (2011), Architecture for Building Conversational Agents that Support Collaborative Learning, IEEE Transactions on Learning Technologies Kumar, R., Beuth, J., Rosé, C. P. (2011). Conversational Strategies that Support Idea Generation Productivity in Groups, in Proceedings of Computer Supported Collaborative Learning Ludwig-Maximilians-Universität München (2011, November 7). Curriculum Vitae, Frank Fischer, Retrieved November 7, 2011, retrieved excerpts from http://www.psy.uni-muenchen.de/ffp_en/download/lebenslaeuufe/cv_frank_fischer.pdf

© ISLS

62

ICLS2012

Volume 2: Symposia

Miyake, N. (1986) Constructive interaction and the iterative processes of understanding, Cognitive Science, 10, 151-177. Miyake, N., (2011) Fostering conceptual change through collaboration: Its cognitive mechanism, socio-cultural factors, and the promises of technological support, Proceedings of the 9th International Conference on Computer-Supported Collaborative Learning, (CSCL2011), Hong Kong Miyake, N., Oshima, J., & Shirouzu, H. (2011). "Robots as a research partner for promoting young children's collaborative learning." Proceedings of the 6th ACM/IEEE International Conference on Human-Robot Interaction. Lausanne, Switzerland. Mu, J., Stegmann, K., Mayfield, E., Rosé, C. P., Fischer, F. (2011). ACODEA: A Framework for the Development of Classification Schemes for Automatic Classification of Online Discussions, in Proceedings of Computer Supported Collaborative Learning. Ng-Thow-Hing, V., Luo, P., & Okita, S. Y. (2010). Synchronized gesture and speech production for humanoid robots. Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). (pp. 4617-4624). October 18-22, Taipei, Taiwan. Okita, S. Y., Ng-Thow-Hing, V., & Sarvadevabhatla, R. K. (2012). Captain May I? Proxemics Study Examining Factors that Influence Distance between Humanoid Robots, Children, and Adults during Human-Robot Interaction. Proceedings of the 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI), (pp.203-204). March 6-8, Boston, MA. Okita, S. Y., Ng-Thow-Hing, V, & Sarvadevabhatla, R. K. (2011). Multimodal Approach to Affective HumanRobot Interaction Design with Children. ACM Transactions on Interactive Intelligent Systems (TiiS). ACM, 1,1, Article 5, 1-29. Okita, S. Y., Ng-Thow-Hing, V., & Sarvadevabhatla, R. K. (2009). Learning together: ASIMO developing an interactive learning partnership with children. Proceedings of the 18th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) (pp.1125-1130). September 27-October 2, Toyama, Japan. Okita, S. Y., & Schwartz, D. L. (2006). Young children’s understanding of animacy and entertainment robots. International Journal of Humanoid Robotics (IJHR), World Scientific, 3, 393-412. Oshima, J., & Oshima, R., (2011) “Collaborative Reading Comprehension with a Robot as a Learning Partner: Implementation of Robots in the Jigsaw Method,” Proceedings of the 9th International Conference on Computer-Supported Collaborative Learning, (CSCL2011), Hong Kong Palincsar, A.S., Brown, A.L. (1984). Reciprocal teaching of comprehension-fostering and comprehension monitoring activities. Cognition and Instruction, 1, 117-175. Rosé, C. P., Moore, J. D., VanLehn, K., Allbritton, D. (2001). A Comparative Evaluation of Socratic versus Didactic Tutoring, In Proceedings of the Cognitive Sciences Society Rosé C. P., & VanLehn, K. (2005). An Evaluation of a Hybrid Language Understanding Approach for Robust Selection of Tutoring Goals, International Journal of AI in Education 15. Rosé, C. P., Wang, Y.C., Cui, Y., Arguello, J., Stegmann, K., Weinberger, A., Fischer, F., (2008). Analyzing Collaborative Learning Processes Automatically: Exploiting the Advances of Computational Linguistics in Computer-Supported Collaborative Learning, The International Journal of Computer Supported Collaborative Learning, 3, pp237-271. Sawyer, R. K. (Ed.). (2006). The Cambridge handbook of the learning sciences. New York: Cambridge University Press.

© ISLS

63

ICLS2012

Volume 2: Symposia

Embedded Phenomena for Knowledge Communities: Supporting complex practices and interactions within a community of inquiry in the elementary science classroom
Rebecca Cober1, Cresencia Fong1, Alessandro Gnoli2, Brenda López Silva2, Michelle Lui1, Cheryl Madeira1, Colin McCann1, Tom Moher2, Jim Slotta1, and Mike Tissenbaum1 1 Ontario Institute for Studies in Education, University of Toronto 2 Department of Computer Science and Learning Science Research Institute, University of Illinois at Chicago rebecca.cober@utoronto.ca, cresencia.fong@utoronto.ca, agnoli2@uic.edu, brenda.ls@gmail.com, michelle.lui@gmail.com, madeirc@gmail.com, colindmccann@gmail.com, moher@uic.edu, jslotta@gmail.com, miketissenbaum@gmail.com Abstract: The work presented here is a product of a collaborative effort to develop a knowledge community and inquiry curriculum for elementary science, where students engage in extended investigations of simulated scientific phenomena presumed to occupy the physical space of their classrooms. By their immersive nature, these “embedded phenomena” lend themselves to a collective epistemology, and hence to new forms of learning and instruction that depart from the conventional didactic approach. The symposium centers on the design and enactment of a seven-week elementary school ecosystems unit, WallCology, developed in close collaboration with partner teachers and school administrators during summer and fall of 2011. Six posters highlight different facets of our effort, including descriptions of the immersive environment, the instructional narrative, the inquiry support technologies, the role of aggregate representations, discourse processes, and the classroom experiences of the 37 students and two teachers who participated in the unit.

Introduction
The goal of establishing a science classroom where students and teachers work together as a community of learners has been championed by Brown and Campione (1996), Scardmalia and Bereiter (1996; 2006), Bielaczyk & Collins (2006) and many others. In this vision, student’s work together with peers to investigate issues or phenomenon, develop their own theories, build upon one another’s ideas, and make progress toward some commonly held learning goal. While this “knowledge community approach” (Slotta & Najafi, 2010) has received many accolades for its vision of a collective epistemology, it has been quite difficult for researchers to enact. The FCL approach (Brown & Campione, 1996) has never been fully replicated, and the knowledge building model has been recognized by Scardamalia (2006) and others (van Aalst & Chan, 2008) as being quite challenging to enact. What are some of the major obstacles to making a knowledge community approach more tractable for researchers and practitioners alike? First, there must be some object of inquiry. The design or selection of this object of inquiry is of crucial importance, as it must be sufficiently intriguing for students, accessible to their investigations, and broad enough in scope to ensure that all students must be involved to make progress in the inquiry. Second, there must be pedagogical and technological scaffolds to support student inquiry, allowing students to engage in authentic science practices and helping make ideas visible for students, peers and teachers. Such scaffolds, when well designed, help students focus on important scientific aspects of the inquiry objective. Third, a community knowledge base serves to captures the aggregated observations and insights of all participants, and serves as a resource to their subsequent inquiry. Finally, there is an important epistemological element, where students must come to understand the purpose of their inquiry as being quite different from “individual learning,” and instead more concerned with the progress of the whole class in terms of inquiry activities and theoretical ideas. These are extremely important theoretical constructs in the learning sciences that are fundamentally distinct from the dominant body of work that explores individual learning, or even the more conventional models of collaborative learning. These ideas represent a revolutionary departure from conventional instruction, and a step toward inclusive, equitable, engaging designs—if only we can get past the substantial barriers to entry. We must assume that our own understanding about how to conduct such research will grow in parallel with our scientific output, as we observe and participate in the evolution of learning and instruction within our partner classrooms. Perhaps this is why such research remains limited to just a few exemplars, despite its compelling theoretical position. The work presented here is a product of a new collaborative effort to engage in a knowledge community approach in an elementary classroom. One group, led by Tom Moher at the University of Illinois at Chicago, has spent the past six years developing a design framework called Embedded Phenomena (Moher, 2006), which engages learners in extended investigations of simulated scientific phenomena presumed to occupy the space of their classrooms. By their immersive nature, these embedded phenomena lend themselves to a collective epistemology, and hence to new forms of learning and instruction that depart from the conventional
© ISLS 64

ICLS2012

Volume 2: Symposia

didactic approach. The other group, led by Jim Slotta from the University of Toronto, has developed a new pedagogical model called Knowledge Community and Inquiry (KCI) that provides a scaffolding framework for collective inquiry (Slotta, 2007). The project seeks to enrich the representational space of the embedded phenomena framework with the addition of comprehensive inquiry support layer for students’ and teachers’ enactment of complex investigations. The symposium centers on the design and enactment of a seven-week elementary school ecosystems unit, WallCology, developed in close collaboration with partner teachers and school administrators during summer and fall of 2011. Six posters highlight different facets of that process, including descriptions of the immersive environment, the instructional narrative, the inquiry support technologies, and the classroom experiences of the 37 students and two teachers who participated in the unit. The symposium will begin with a fifteen-minute overview to orient participants to the poster suite, followed by an hour of individual discussion with poster presenters. Chris Quintana from the University of Michigan will serve as chair and discussant, offering comments during the final fifteen minutes of the symposium.

WallCology: Representational space and instructional narrative
Brenda López Silva, Alessandro Gnoli, and Tom Moher In WallCology, learners are challenged to maintain the biodiversity of a simulated ecosystem contained within a controlled environment imagined to occupy the walls of their classroom. The ecosystem contains several species of flora and fauna which students observe through persistent (continuously animated) portals (“WallScopes”) attached to the walls of the classroom, allowing them partial access to the controlled habitats. In addition, students are given control over several environmental variables that differentially impact vegetation growth: temperature, humidity, and lighting (Fig. 1). While initially stable, over the course of seven weeks, the ecosystem is perturbed in ways that threaten the viability of one or more species. Students are asked to respond to threat through the manipulation of the environmental controls, based on their understanding of the system food web and population dynamics. Students engage in a progressive series of activities designed to foster learning across a range of ecosystem concepts and science practices. Initially, students document the biodiversity and abundance of the Figure 1. “WallScope” view of a WallCology system through systematic observation of habitats and h bit t species that they see in the WallScopes. In the second phase, students focus on identifying the life cycles of the animal species by observing egg hatchings, transitions from larvae to pupa, etc. These understandings are needed in order for students to estimate species populations, an activity that is repeated periodically throughout the unit in order to monitor population trends that are not readily apparent to the naked eye. (Population estimates are obtained by “counting” species visible within the WallScopes and computing estimates based on the ratio of the area of the classroom wall to that of the display screens). Students observe the feeding patterns of species in order to construct the food web defining the indirect interactions within the ecosystem. An initial perturbation (the “failure” of the lighting system in several of the habitats), and its concomitant impact on population levels, alerts the children to the sensitivity of the system. While this “failure” is quickly resolved, it creates an opportunity to engage the learners in consideration of “what-if” scenarios involving other potential threats to the system. Rather than experimenting on the target ecosystem directly and risking possible catastrophic results, learners are provided with an ecosystem-modeling tool that allows them to rapidly model outcomes associated with variations in population mixtures and environmental conditions. The community knowledge growing out of this phase becomes critical when a second perturbation is introduced: the arrival of an invasive species in the form of a new predator. In the final phase of the unit, students directly intervene in the WallCology ecosystem by modifying environmental conditions to mitigate the impact of the predator and ensure the survival of all of the original species. Student activity within the unit involves a combination of structured observation, question and hypothesis formulation, experimental manipulation and hypothesis testing, and whole class and small group discussion, all within the context of a technology tool suite that allows students and teachers to contribute to, and draw from, an emerging community knowledge base. Students record observations, utilize the modeling tool, and synthesize con-tributions to the knowledge base using tablet computers; teachers orchestrate discussions using a SmartBoard interface that gives them access the knowledge base and scaffolds themed discussions aligned with unit milestones.

© ISLS

65

ICLS2012

Volume 2: Symposia

Smart Classrooms for Knowledge Communities: EPIC Technology Environment
Jim Slotta, Mike Tissenbaum, Michelle Lui, and Matt Zukowski New forms of knowledge media offer a wealth of opportunity for researchers who can take advantage of the varying contexts (i.e., within the classroom, at home, or in field activities) and devices (e.g., laptops, smartphones, interactive tabletops, and large format displays) to design exciting new kinds of instruction where students dynamically generate knowledge, build on peers’ ideas, and investigate questions as a knowledge community. We have advanced the notion of a “smart classroom,” which integrates such technologies for purposes of supporting a spectrum of collaborative inquiry and knowledge construction activities. The work centers around the development of a flexible open source platform called SAIL Smart Space (S3), which offers a framework for integrating devices, materials, and user interactions, as well as a set of core technologies: (1) a portal for student registration and software application management; (2) an intelligent agent framework for data mining and tracking of student interactions in real time; (3) a central database that houses the designed curriculums and the products of student interactions; and (4) a visualization layer that controls how materials are presented to students (Slotta, 2010). We have implemented S3 in several distinct scenarios, including (1) PLACE. Web: a content community for high school physics (Tissenbaum & Slotta, 2011); (2) EvoRoom: an immersive, room-sized simulation of a Sumatran rainforest where students investigate evolution (Lui, Tissenbaum & Slotta, 2011), and (presently) (3) Embedded Phenomena for Inquiry Communities, where tablet computers and shared interactive displays support student investigations of simulations running within their classroom walls, floors and ceilings. Through these developments, we investigate how smart classrooms can scaffold the collection, aggregation, and representation of information across contexts and environments. We also strengthen the underlying S3 “core technologies” so they may be extended to an increasing range of applications. Our poster will detail the S3 implementation for the EPIC designs. We focus on several dimensions, including the role of the core Figure 2. The S3 Software Architecture. technology (e.g., to register students, support groups, store and retrieve data, etc), our use of intelligent agents (i.e., in support of the various interaction designs discussed in other posters within this session), and the specific forms of visualization used for various purposes. Our Wallcology enactment relied on two loosely coupled systems: (1) the Embedded Phenomena server, which managed the bug colonies (and all perturbations, environmental dependencies, etc.) within the classroom walls, and (2) an S3 server that supported student and teacher interactions, including log-ins, data collection and management, tablet and Smartboard applications. S3 supported our development of a peer-to-peer network of tablet-based input devices, where students add their observations of the phenomenon (e.g., counts of species appearing on the wallscopes), as well as more abstract representations (e.g., the relationships between two species of insects). Observational data are aggregated in real time, using Extensible Messaging and Presence Protocol (XMPP), which updates student tablets continuously. Intelligent agents are also implemented using XMPP; agents are implemented as software routines that simply “listen in” to the relevant message stream, and take action whenever certain conditions occur. WallCology employs an Archivist agent, which translates incoming messages from the tablets into database entries, and a Notetaker agent, which passes student notes from tablet to database and to the Smartboard. Several distinct forms of representation were designed, including aggregate tables of student observations, graphical representations (e.g., of species counts, dependencies across locations in the room, etc.), and the phenomenon itself – both in a concrete form (i.e., “1 bug in the wall is 1 bug on the screen”) and a more symbolic form (graphs of bug populations). The representations were populated by active calls to the database that occurred via XMPP in real time, so that all tablets and the Smartboard were updated instantaneously. Our poster will discuss the role of smart classroom technologies in supporting research of complex pedagogical designs, including the scaffolding of interactions, as well as distinct affordances for real time aggregation and execution of pedagogical logic (e.g., dynamic grouping, real time assignment of resources, and content sensitive feedback), using intelligent agents. S3 offered an accessible means of developing the sophisticated array of interdependent applications needed to coordinate our Wallcology designs.

Materials that scaffold collective inquiry: The role of aggregate representations
Rebecca Cober, Colin McCann, Jim Slotta, and Tom Moher

© ISLS

66

ICLS2012

Volume 2: Symposia

In previous enactments of WallCology, students used paper lab books to record observations about the phenomena (Moher et al., 2008). In the current version, our interest was to help students work collectively as a "knowledge community,” with the contributions of individuals or small groups readily accessible to others. Moreover, the results of individual inquiry actions, such as observations or reflections, should ideally combine into aggregate representations (ARs) that make important patterns visible to both teachers and students. To achieve this goal, we developed a new technology layer to support student investigations and to share knowledge with each other. A custom tablet application was developed to support student investigations of Wallcology, effectively replacing the lab books of previous iterations. Most importantly, students had access to ARs, displayed on the tablets and interactive whiteboards, comprised of their own and their peers' observations and insights, compiled in real-time using XMPP (a Twitter-like messaging technology). Our presentation will describe the design and evaluation of these representations and the role they played within the WallCology narrative. Our guiding research question: How do aggregate representations of collective knowledge advance students' and teachers' understanding of the object of scientific inquiry and provide a resource for subsequent inquiry activities? Method. Our designs were informed by the Knowledge Community and Inquiry (KCI) framework, which seeks to integrate elements of community knowledge construction with scaffolded inquiry (Slotta, 2007). We designed a tablet application to facilitate four learning activities within the WallCology unit: recording observations of (1) the organisms and (2) habitats of the ecosystem, and constructing representations of (3) life cycles and (4) food webs (Fig. 3).

Figure 3. Aggregate representations of community knowledge. Text-based data tables for learning activities 1 and 2, and pair-wise matrices depicting tallies of relational statements, for learning activity 4. We begin by describing the targeted pedagogical goal for each AR, in relation to its corresponding learning activity. We then evaluate each AR in the context of the wider instructional design: does it serve to achieve the broader desired pedagogical goals of the unit? Next, we evaluate the enactment of the curriculum, recognizing that it may diverge considerably from the intended design, due to variations in teaching styles and classroom dynamics. Finally, for each AR, we capture and represent the pedagogical patterns that actually occurred in this iteration of WallCology, and compare them against our original design goals. Outcomes. The AR designs supported collective inquiry in three important ways. First, teachers used the ARs to guide whole-class discussions (WCDs). One pattern, called "aggregate observations and discuss" occurred across all four ARs, where teachers displayed relevant strands of student-contributed observations to facilitate discussion, moving the classroom inquiry toward a desired pedagogical goal. The outcome of these WCDs frequently set the stage for further work, revealing important next steps in the investigation of the EP. Second, ARs were used to form a basis for consensus – particularly the relationship tallies in ARs 3 and 4. High tallies were strong indicators of the veracity of relational statements (e.g., that green bugs hatch from dark blue eggs), enabling the community to resolve the life cycle and food web relationships (led by the teacher). Third, the ARs revealed areas of disagreement, moving the “knowledge community” forward in their understanding of the embedded phenomena. Closely contested observations (e.g., two organisms tallied almost equally for the same life cycle stage) were readily apparent in the ARs, providing specific areas for students to focus on in their continued investigations, and for teachers to focus discussions.

© ISLS

67

ICLS2012

Volume 2: Symposia

The Role of Discussion in Orchestrating Inquiry
Cresencia Fong, Rafael Pascual-Leone, and Jim Slotta WallCology engages students in cognitive, social, and technological interactions with their environment orchestrated by the teacher, typically through discussions that served to advance the understanding of the community. How do such “idea-centred” (Scardamalia & Bereiter, 1996) discussions proceed? How should the teacher “orchestrate” (Fischer & Dillenbourg, 2006) and “script” (Kollar, Fischer & Slotta, 2007) such discussions to foster students’ sense of contribution and personal progress? What role does technology play in representing student ideas, and supporting the growth of collective understanding? We investigate those questions by examining discussions that occur within WallCology classrooms, describing: (1) the pedagogical characteristics of discussions that occur throughout the unit, (2) how the various WallCology technologies were employed within those discussions, and (3) how a new discourse tool impacted those pedagogical characteristics. We begin by analyzing the discussions that occurred in each classroom. We then introduce “Common Knowledge” (CK)—a technology environment designed to promote idea growth and exchange while enabling teachers to guide discussions. Method. In WallCology, the teacher’s role as ‘guide on the side’ is pivotal in facilitating meaningful discussions. Such discussions may be short and spontaneous to orient students toward productive paths, or lengthier and strategically placed in the curriculum to promote conceptual learning goals. Using video and observation notes, we analyzed discussions occurring during the first month of the enactment, in terms of their goal or purpose, conversational dynamics, the growth of ideas, and pedagogical outcomes or consequences. We also analyzed the use of technology and paper-based resources. In CK, a handheld computer tablet application enables students to contribute their questions, evidencebased hypotheses, theories, and ideas; and “tag” their contributions from a list of science content and practice keywords (e.g., “Theory”, “Observation”, “Question”). These notes dynamically appear on all tablets and on the classroom’s SMART Board, allowing teachers to manipulate notes during oral discussions, swiftly filtering topic-related notes as the discussion progresses. We integrated CK discussions into the WallCology curriculum, specifying discussion goals and pre-programming relevant keyword tags. Teachers also launched spontaneous CK discussions, as they felt warranted. Outcomes. Four Figure 4. Tablet (left) and SMART Board (right) mediated discussions. WallCology sessions in both grades 5/6 classrooms were analyzed as described above. Data sources included classroom observations, video and audio recordings, teacher-researcher debriefings, paper artifacts of student work, teacher and student interviews, and data logs of CK discussions. The goal was to produce a schema for productive inquiry discourse, and a model by which to evaluate CK discussions. Discussion analysis revealed several distinct discourse patterns, including: • Encouraging new ideas. Teacher displays observation data or CK notes on SMART board; helps students recognize that there might be additional avenues for reflection or observation; • Resolving divergence. Teacher displays observation data or CK notes on SMART board; helps students notice divergence, discuss each position, and consider ways to resolve the divergence; • Introduce topics or processes. Teacher builds upon existing ideas from observation data or CK notes, grouping ideas together to promote insights about a new topic (e.g., organism life cycles); • Motivate approaches. Teacher encourages students to reflect on data or hypotheses, brainstorm limitations to current approaches, and possible new activities (e.g., comparing numbers of species in different habitats). We analyzed the CK discussions that occurred within the first 4 weeks of the Wallcology curriculum, to see how teachers achieved these patterns using the technology environment. Students were engaged, with 4 times the number of in-class CK contributions per minute than oral discussions, and substantial reading and response to peer notes. Idea growth and symmetric knowledge advance were heavily reliant on teacher orchestration of lesson activities and spontaneous teacher scaffolding. Design improvements are concerned with smarter filtering and commenting.

Teacher Orchestration of Complex Inquiry Patterns
Cheryl Madeira, Cresencia Fong, and Richard Messina

© ISLS

68

ICLS2012

Volume 2: Symposia

How do teachers orchestrate complex inquiry-based activities, and how can technology scaffolds be developed that support teachers as they acquire such skills? This paper examines teacher practice within the Wallcology project, following two master teachers as they worked closely with researchers and technologists to co-design, enact, and revise (often in real time) the inquiry-based activity patterns for their grade 5-6 classes. Indeed, from the outset, these teachers (“knowing better”) asserted that it would not be possible to choreograph every twist and turn of the curriculum, but that they would need substantial leeway to interpret the situation and respond accordingly. Such “adaptive practice,” illustrated below, can be viewed as “orchestration” of an inquiry design: After reviewing the day’s lesson plan, the teacher, Brendon (pseudonym), notices that students have not entered details in their Observation tab. He says, “Only a couple of notes on life cycle were entered…maybe today we can observe more.” Then he instructs: “under the Organism tab, you can talk about organism again; and then go into life cycle.” Even though the plan had called for whole class session of Observation of organisms, Brendon realized that by turning kids’ attention forward to “Lifecycle observations,” he could transition into the important step of “Relationships”. (FieldNotes 2011_1017). This except illustrates adaptive teacher practice, where the teacher gages students' prior knowledge (Penuel & Gallagher, 2009), develops collaborative opportunities for students' inquiry processes (Slotta & Linn, 2009), and facilitates students' emerging knowledge (Zhang et al., 2010). In recent years, learning scientists have embraced the term 'orchestration,' defined as “the process of productively coordinating supportive interventions across multiple learning activities occurring at multiple social levels” (Fischer & Dillenbourg, 2006). In orchestrating the flow of activities, materials, and interactions within the classroom, teachers develop pedagogical knowledge. Method. To capture the role of the teacher in our EPIC classrooms, we examined teachers’ orchestration, including their adaptive responses to student ideas, and how they make use of technology in ways both intended (i.e., by designers) and unintended. While the teachers in our study were both masters of inquirybased pedagogy, having used it exclusively for 3 and 5 years respectively, neither had any experience with embedded phenomena nor the specific forms of student inquiry it was designed to support. Thus, while our teachers entered with substantive pedagogical knowledge, there would be many challenging and novel aspects of teaching with the Wallcology materials and activities. Our research seeks to understand how these teachers develop new practices (and implicitly, new pedagogical knowledge), and to inform our understanding of how such orchestration leads to in-service professional development. We observed the teachers throughout the co-design and enactment of the 8-week Wallcology unit that addressed science topics related the diversity of life. Data included classroom observation notes, video, photos and audio recordings, time logs of teacher’s activities, teacher and student interviews, and other relevant school documentation. Ten co-design sessions were held (all audiotaped) where teachers, administrators, researchers and technologists designed the curriculum activities, reviewed drafts of materials and technology tools, and reflected actively about the nature of learning and instruction with Wallcology. We observed all classroom sessions, and debriefed regularly with teachers, including larger scale research meetings where the teachers met with the entire research team to discuss pedagogical issues and inform ongoing design. Working first from observation notes, then from video, we segmented each lesson into an “orchestrational move.” For each segment, we described the purpose, or goal, the role of technologies or representations, and any consequential outcomes. We paid particular attention for “adaptive moves” where teachers responded to characteristics of the classroom, evidence of student knowledge, or intuitions about where more attention might be necessary. Outcomes. It became apparent early in the implementation that the teachers were not at ease unless they were able to adapt and improvise at any moment during the enactment. They felt strongly about their decisions to veer away from certain design requirements. We saw evidence of teachers ignoring the technology when face-to-face communications were more comfortable and effective. In one segment after the next, there were indications that teachers assessed student engagement and understanding, then responding in ways that departed from the lesson plan. Each night, a revised lesson plan was made for the following day. While we had envisioned that the plan would be a reference guide (expecting some variance), Brendon commented that he “had the score in his head” – summoning images of a “jazz performance” rather than “orchestral conducting.” At least for these early trials, our curriculum was specified more as a series of “chord progressions” than a strict melody. Our summary of teacher enactment patterns, including technology use, can inform the design of EPIC curriculum to reinforce those patterns and encourage teachers with less expertise to engage in them.

© ISLS

69

ICLS2012

Volume 2: Symposia

Fidelity of science practice in complex instructional designs
Alessandro Gnoli, Cheryl Madeira, Tom Moher Contemporary research in science pedagogy emphasizes the importance of engaging learners in authentic science practices, not only for the development of specific inquiry skills, but also for the development of agency and understandings about the nature of scientific work (Duschl, Schweingruber, & Shouse, 2007; Zimmerman, 2007). The embedded phenomena (EP) framework affords classroom learners opportunities to engage in a broad range of science practices, particularly the kind of “patient science”—extended inquiry over long time periods— that elementary school students rarely encounter (Fulp, 2002). The underlying simulations are designed to present an extended narrative, gradually revealed through the accumulation of data over time. The clarity of that narrative, as a result, is highly sensitive to students’ consistent and accurate applications of observational and manipulative procedures, making the timely detection and remediation of practice errors critical capabilities in successful enactment. When errors—of observation, of computation, of transcription, of interpretation—go undetected, and erroneous data are added to the historical representation of the investigation, they can obscure the underlying narrative of the unit, undermining days or weeks of classroom effort. In WallCology, students gather and transcribe substantial amounts of “objective” data involving habitats, life cycle patterns, producer/consumer relationships, and environmental conditions. They “count” individual animals within WallScopes (samples) and computer population estimates based on physical measurements of the classroom walls. Errors in these practices can have significant impacts on the resultant representations used by students to reason about the state of the ecosystem. For example, an incorrect characterization of the life cycle of one species can result in incorrect classification of individuals within sampling activities, leading to population estimation errors. Inconsistent procedures for counting individuals (even if properly classified), or computational errors in determining count averages can have similar effects. In WallCology, the cumulative impact of these errors can easily mask Figure 5. Errors in counting mask important population shifts or introduce representational population trends in prior WallCology enactment. artifacts unrepresentative of the actual state of the ecosystem. Method. Moving embedded phenomena from paper to technology-based media creates the opportunity to identify observational, transcription, and computational errors of practice at the point of data entry. While straightforward enough to implement, our goal is not to use discrepancies as the basis for directly notifying students of their errors (e.g., using pop-up dialogs to inform students that the data they just recorded are incorrect). Rather, our longer-term strategy is provide notifications to teachers, who can use the information to more subtly intervene within undermining students’ agency as investigators (e.g., by walking over to a group that’s making significant errors in practice and suggesting that their data seem unusual, perhaps warranting a “second look” by the group). Research surrounding this strategy will focus on the capacity of teachers to manage this information flow, the choice of media to be used for delivery (e.g., tablet notifications, smartphone text messages, Bluetooth earphones), message representations (individual, small group, whole class), and information origination (e.g., “push” vs. “query” strategies). Outcomes. We have developed software agents that continuously compare student data entries with the current state of the simulation, and we used those agents to collect baseline data on the frequency and magnitude of errors during the present WallCology enactment. Preliminary analysis suggests that moving to electronic rather than paper-based media has not impacted the rate of practice errors. As in prior enactments, cumulative errors obscured the representation of population trends, requiring us to “dramatize” the impact of ecosystem perturbations in order to motivate population modeling and interventions. The rate of observed errors suggests that the design of notification mechanisms could be a significant challenge, requiring a careful balance between real-time and off-line approaches, and the adoption of strategies that are sensitive to the severity of errors as well as historical error frequency among individuals and within groups. We will be designing and developing our initial tools and procedures to support practice error notification and remediation for a second iteration of the “new” WallCology in spring 2012.

© ISLS

70

ICLS2012

Volume 2: Symposia

References
Bielaczyc, K., & Collins, A. (2006). Fostering knowledge-creating communities. Collaborative Learning, Reasoning, and Technology, 37-60. Brown, A. L., & Campione, J. C. (1996). Psychological theory and the design of innovative learning environments. Environments, 289-325. Duschl, R., Schweingruber, H. & Shouse, A. (2007). Taking Science to School: Learning and Teaching Science in Grades K-8. National Research Council report. Fischer, F., & Dillenbourg, P. (2006). Challenges of orchestrating computer-supported collaborative learning. Paper presented at the 87th Annual Meeting of the American Educational Research Association (AERA), San Francisco, California, USA. Fishman, B., Marx, R. W., Best, S., & Tal, R. (2003). Linking teacher and student learning to improve professional development in systemic reform. Teaching and Teacher Education, 19, 643–658. Fulp, S. (2002). The Status of Elementary Science Teaching. Horizon Research (One of a series of reports from the NSF-sponsored 2000 National Survey of Science and Mathematics). Retrieved from: http://2000survey.horizon-research.com/reports/elem_science.php Kollar, I., Fischer, F., & Slotta, J. D. (2007). Internal and external scripts in computer-supported collaborative inquiry learning. Learning & Instruction, 17(6), 708-721. Moher, T. (2006). Embedded phenomena: Supporting science learning with classroom-sized distributed simulations. Proceedings ACM Conference on Human Factors in Computing Systems (CHI ‘06), Montreal, Quebec, Canada. 691-700. Moher, T., Uphoff, B., Bhatt, D., López Silva, B., and Malcolm, P. (2008). WallCology: designing interaction affordances for learner engagement in authentic science inquiry. Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems. CHI '08. ACM, New York, NY, 163-172. Penuel, W. R., & Gallagher, L. P. (2009). Preparing teachers to design instruction for deep understanding in middle-school earth science. Journal of the Learning Sciences, 18, 461–508. Prieto, L. P. (2010). The many faces of orchestration: Towards a (more) operative definition, Unpublished Technical Report, University of Valladolid, Spain. Retrieved from http://www.mendeley.com/research/faces-orchestration-towards-more-operative-definition/. Scardamalia, M., & Bereiter, C. (1996). Engaging students in a knowledge society. Educational Leadership, 54(3), 6-10. Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and technology. The Cambridge Handbook of the Learning Sciences.Cambridge University Press, Cambridge, 97-115. Slotta, J. D. (2007). Supporting collaborative inquiry: New architectures, new opportunities. In J. Gobert (Chair), Fostering peer collaboration with technology. Symposium conducted at the biennial Computer Supported Collaborative Learning (CSCL) Conference, New Brunswick, NJ. Slotta, J.D., Tissenbaum, M., & Lui, M. (2011). SAIL smart space: Orchestrating collective inquiry for knowledge communities. 9th International Computer-Supported Collaborative Learning Conference (CSCL), Hong Kong, China. 3, 1081-1088. Slotta, J. D., & Linn, M. C. (2009). WISE science: Web-based inquiry in the classroom Teachers College Press. Slotta J D and Najafi H (2010), Knowledge Communities in the Classroom. In: Penelope Peterson, Eva Baker, Barry McGaw, (Editors), International Encyclopedia of Education, Volume 8, pp. 189-196. Oxford: Elsevier. Tissenbaum, M. (2011). Co-designing collaborative smart classroom curriculum for secondary school science. Proceedings of the Red-Conference: Rethinking Education in the Knowledge Society, Switzerland. van Aalst, J., & Chan, C. K. (2007). Student-directed assessment of knowledge building using electronic portfolios. Journal of the Learning Sciences, 16(2), 175-220. Zhang, J., Scardamalia, M., Lamon, M., Messina, R. & Reeve, R. (2007). Socio-cognitive dynamics of knowledge building in the work of 9 and 10 year olds. Education Tech Research Dev, 55, 117-145. Zimmerman, C. (2007) The development of scientific thinking skills in elementary and middle school. Developmental Review 27(2), 172-223.

Acknowledgments
The authors would like to thank Joel Brown, Julia Murray, Armin Krauss, Francesco Novellis, and Ben Peebles, Gabriel Resch for their invaluable contributions to the design and enactment of the WallCology unit. This material presented here is based upon work supported by the U.S. National Science Foundation under grant IIS1065275 and Canadian Social Sciences and Humanities Research Council under grant 410-2011-0474.

© ISLS

71

ICLS2012

Volume 2: Symposia

Critical Aspects in Learning with Technologies
Ching Sing Chai, Nanyang Technological University, chingsing.chai@nie.edu.sg Choon Lang Quek, Nanyang Technological University, choonlang.quek@nie.edu.sg Chwee Beng Lee, University of Western Sydney, chwee.lee@uws.edu.au Huang-Yao Hong, National Chengchi University, Taiwan, hyhong@nccu.edu.tw Joyce Koh Hwee Ling, Nanyang Technological University, joyce.koh@nie.edu.sg Naomi Miyake, University of Tokyo, nmiyake@p.u-tokyo.ac.jp Peter Reimann, University of Sydney, peter.reimann@sydney.edu.au Abstract: In this symposium, we present a comprehensive overview on the unique position of technologies in the context of learning sciences. According to Jonassen, Howland, Marra and Crismond (2008), learning with technology necessitates the designs of learning and the learning environments in order to engage learners. The overarching purpose of this symposium is to bring forth a comprehensive discussion on the critical aspects of learning with technologies which are fast becoming an integrated part of learning. To accomplish our purpose, the presenters will (a) provide an overview of the context of learning by discussing the importance of technological pedagogical content knowledge and socio-cultural issues, (b) examine issues related to the designs of learning, this includes scaffolding learning and intentional knowing, (c) discuss one of the most important outcomes of learning, which is conceptual change. In our symposium, our presenter will also discuss the roles of technologies in fostering conceptual change.

Introduction
Technologies are pervasively used in education. But are we using it for productive thinking and meaning making? What are the critical aspects to consider when we intend to integrate technologies into meaningful learning? Have we made progress in designing for learning with technology rather than merely from technology? With these questions in mind and relating to the emphasis of the powerful role that technologies play in transforming learning (Sawyer, 2006), we seek to assess the position of technologies in the context of learning sciences. According to Jonassen, Howland, Marra and Crismond (2008), when learning with technologies, it necessarily consists of the designs and the environments that engage learners. Most importantly, technologies should function as intellectual partners where the cognitive responsibility is distributed to the partner that performs it better. The overarching purpose of this symposium is to bring forth a comprehensive discussion on the critical aspects in learning with technologies. To accomplish our purpose, the presenters will (a) provide an overview of the context of learning by discussing the importance of technological pedagogical content knowledge and sociocultural issues. Learning scientists argue that deep learning is most likely to occur in complex social and technological environments (Sawyer, 2006) and this is supported by Koehler and Mishra (2009) while previously Mishra and Koehler (2006) who also argue that good teaching requires knowledge of content, pedagogy, and technology. Understanding the situative perspective (Greeno, 2006) of learning is critical as this helps researchers and educators to understand the complexity nature of learning for better design of learning environments, (b) examine issues related to the designs of learning, this includes intentional knowing. One of the main concerns of learning sciences is to provide powerful learning environments that foster deep learning (Sawyer, 2006). Our symposium will also examine the importance of intentional knowing in relation to the use of technologies for learning, (c) discuss one of the most important outcomes of learning, which is conceptual change. According to Vosniadou (2008), perhaps one of the biggest breakthroughs in the learning sciences was the examination of conceptual change. In our symposium, our presenter will also discuss the multiple roles of technologies in fostering conceptual change. Our discussion begins with Ching Sing Chai, Huang-Yao Hong, Joyce Koh Hwee Leng giving the general perspectives of technological pedagogical content knowledge and design thinking. Next, Naomi Miyake will address socio-cultural issues such as collaborative learning and the contexts of learning. She will discuss urgent needs for researchers pay more attention to these “harder to observe” collaborative processes. Chwee Beng Lee and Choon Lang Quek will discuss the components of intentional knowing and its role. Lastly, Peter Reimann will focus on the pivotal roles of technologies in the context of conceptual change research.

Technological pedagogical content knowledge and design thinking
Ching Sing Chai, Nanyang Technological University

© ISLS

72

ICLS2012

Volume 2: Symposia

Email: chingsing.chai@nie.edu.sg Huang-Yao Hong, National Chengchi University, Taiwan Email: hyhong@nccu.edu.tw Joyce Koh Hwee Ling, Nanyang Technological University Email: joyce.koh@nie.edu.sg The advancement of networked technology and the development of myriad e-learning platforms and social networks have provided ample opportunities where ideas, insights, experiences, and knowledge can be articulated, shared, co-constructed and distributed (Chai & Lim, 2011). These ideas, insights, experiences, and knowledge can be referred to as conceptual artifacts or world 3 objects (Bereiter, 2002; Popper, 1978). Once publicized through networked technologies or traditional media, these cognitive objects become improvable ideas. They are subjected to examination, critique, refinement, elaboration, re-contextualization etc. Bereiter and Scardamalia (2006) argue that ability of an individual to participate and contribute to a community effort in improving the ideas is the key competency that citizens of the Knowledge Age must possess. Many educators have also advocated shifting the focus of education from reproductive transmission to that of creative construction (Collins & Halverson, 2010; Paavola, Lipponen, & Hakkarainen, 2004). In this paper, building on Bereiter and Scardmalia’s (2006) proposal of building students’ “ability to work collaboratively with conceptual artefacts in design mode" (p. 702), we propose that teachers need to develop and create the corresponding technological, pedagogical and content (TPACK) knowledge (Mishra & Koehler, 2006) through continuous design thinking. We also argue that nurturing design epistemology is the key task that educators should focus on, beginning from the teachers and cascading down towards the students, especially when supported by the affordances of collaborative learning platforms. In other words, we argue in this paper that teachers need to become knowledge creators who generate the know-how in using collaborative technologies to help students to co-construct knowledge. This involves teachers’ capacity in providing the epistemic framing (Elby & Hammer, 2010) for classroom learning, scaffolding students’ collaborative sense making processes and building students’ epistemic repertoire before, during and after their learning episodes.

The learning of a monitor: How much and why
Naomi Miyake, University of Tokyo Email: nmiyake@p.u-tokyo.ac.jp One intriguing outcome of socially well-organized collaborative learning is the students who sat silent during the group work still come out with high performance at the end. In this presentation, I will showcase three examples of this “learning of a monitor,” give some theoretical framework to understand how this could happen, and then propose promising use of technology to promote this kind of learning. I will take examples of the learning from “the Hypothesis-Experiment Instruction,” (Itakura, 1997, henceforth HEI), “the Knowledge Constructive Jigsaw,” (Miyake, 2010, henceforth KCJ), and “Manabi-noKyodotai (Comminity for Learning, Japan, led by Manabu Sato, henceforth MK). They differ in their basic curriculum designs, but all support strongly the learners’ discussions either in whole class (HEI), in small groups with member change (KCJ), or four-member groupings (MK). Careful analyses of the students’ learning processes and the associate learning outcomes often reveal that the students who mainly monitored during the class did show substantial learning toward the end of class. Theoretically this could be the case, as suggested by studies on two-person, understanding and problem solving process such as trying to find an answer to how a sewing machine makes its stitches (Miyake, 1986), because in her analyses, the monitor, who took the role of listening and integrate what the other member, the task-doer, tried to explain to her/his own understanding, did progress her/his part of understanding almost independently from the task-doer’s. It was not rare in her other analyses that the monitors played the critical role to bring the pair to find a hidden course to solve the problem, mainly because they were pursuing some different course from the task-doers. If this theoretical framework should work at classrooms where more than two people engage in the groups simultaneously, we should be able to identify cases where those students who mainly stay silent should also learn, as well as those who actively participate in the group work. For the HEI, I will take a case where 21 third graders discussed to predict forthcoming results of experiments, to build a first stage understanding about the air as substance, (i.e., to understand and become able to explain why the air does not share the same space with water), based on Saito’s analyses (Saito & Miyake, 2010). She analyzed the frequency and the order of class utterances of all of the students, and found that the frequency of talk differed greatly, but all of them could predict the result of the last experiment correctly. In addition, all of them could write after the class the reason for their choice in scientifically acceptable forms. The KCJ is a jigsaw class, with a carefully designed “class question” to be answered only by integrating different pieces of information contributed by different expert groups into the

© ISLS

73

ICLS2012

Volume 2: Symposia

jigsaw. For the KCJ, I will report three cases which all showed that those relatively silent students could write what they learned during the class as well as those who had been active during the class. The MK uses group work more flexibly yet the recommended form of grouping is four, so that when a teacher asks the class to solve a problem with expected answer rate of 50%, there is some assurance that often two of the members could explain how and why of the solution. In MK also, there has been observed that the students who had been identified as “not fit” for the subject of the class sometimes gives a critical utterance to lead the whole class to clear understanding. After showcasing these examples, I will develop a model for this type of “learning of monitor” based on my constructive interaction framework. To conclude my presentation, I would like to discuss urgent needs for us to collect and study the learning processes in more detail during both those experimental and regular classes, so that we could pay more attention to these “harder to observe” processes. I will illustrate how technology could help us do this more extensively, with more ease, so that not only researchers like us but teachers could participate in this endeavour to understand how people learn, and how we could support the learning.

The role of intentional knowing in learning with technology
Chwee Beng Lee, University of Western Sydney, School of Education Email: chwee.lee@uws.edu.au Choon Lang Quek, Nanyang Technological University Email: choonlang.quek@nie.edu.sg Berieter and Scadamalia (1989) highlighted that learners are not only active in their construction of meaning, but they can be intentional. They are intentional because they are fulfilling their goals that in turn determine their human behaviour. This would also mean that they are cognitively engaged in the learning process, monitoring and regulating their learning (Sinatra & Pintrinch, 2003). To learn intentionally, students must consciously understand and be able to define their strengths and weaknesses, their learning processes, how they examine the way they execute learning tasks, monitor learning, evaluate learning and whether they innovate in order to learn intentionally. Berieter and Scadamalia (1989) used the term intentional learning to refer to cognitive processes that have learning as a goal rather than an incidental outcome. It is an “achievement, not an automatic consequence of human intelligence.” (pp. 366). As put forth by Scadamalia and Berieter (2002), intentional cognition is more than self-regulated learning; it is the active pursuit of a mental life whereas selfregulated learning is usually a set of study skills and learning-to-learn strategies. They mentioned that there is a need for metaknowledge to foster intentional learning. Such an understanding is very much aligned with Efklides’s (2008) metatheoretical model in which at the personal-awareness level, there are conscientious efforts in monitoring and controlling processes. Bereiter and Scadamalia (1989) defined it as knowledge about knowledge, access to one’s own knowledge, and skills, which take the knowledge as an object to operate on. From a constructivist standpoint, learners are aware of the central role of ideas in the development of knowledge and how ideas are revised through a process of conjecture, argument, and test (Smith, Maclin, Houghton, & Hennessey, 2000). This type of processing is not a mere response to the external circumstances or environment. In a sense, “intentional level processing is not only initiated by the learners, it is under the learner’s conscious control” (Sinatra & Pintrich, 2003, pp. 4). On the other hand, if a learner is not intentional, the learning process is determined largely by the external factors such as prior knowledge, types of tasks, facilitating conditions, etc. In this section of the symposium, we will introduce the term intentional knowing through the lens of metacognition and we argue that it is a critical element to propel intentional learning. We have conducted several empirical studies to re-conceptualize metacogntion and a six-factor model was found to explain intentional knowing. Next, we argue that intentional knowing plays a pivotal role in learning with technology. This stance aligns with what most learning scientists would advocate, and that is the emphasis of technology in fostering deep learning, for instance, helping learners to collaborate meaningfully or to reflect on their understanding (Sawyer, 2006). Using technology to represent knowledge, articulate thinking, manipulate and revise conceptual understanding and to reflect upon knowledge and learning process are effortful and goaloriented activities for the learners. Such activities necessarily require intentional knowing which is the conscious knowing of one’s capacity and learning process, the knowing of how one examines his or her own monitoring, evaluation and execution processes, in addition, how he or she innovates to cope with the demand of tasks. Intentional learners not only cognitively engage in the learning process, but also regulate their learning. For instance, Vosniadou and Kollias (2003) reported that when using Knowledge Forum for science learning, students were able to collaborate effectively and question their own conceptual framework. From the pedagogical perspective, in order to develop learners who have the intentionality to learn, we suggest that educators not only create learning opportunities to cognitively challenge the students, but also embed

© ISLS

74

ICLS2012

Volume 2: Symposia

meaningful tasks and activities in technology-rich environments to them to apply thinking skills and take stock of their own strengths and weakness in the learning process. In this section of our symposium, we discuss the components and importance of intentional knowing in using technology for learning through our recent empirical study (N=2404). In this study that was conducted with high school students, we developed a statistical model to explain the intricate relationships between intentional learning and intention to use technology.

The multiple roles of technology for learning: Lessons learned from conceptual change research
Peter Reimann, University of Sydney, Faculty of Education and Social Work Email: peter.reimann@sydney.edu.au Research on conceptual change is an interesting case for studying various conceptualizations of the role educational technology can play for learning. While the notion of technology as a tool to learn with has probably found its strongest expression in math education, science educators have seen the role of technology more conservative: more to learn from than to learn with. At the same time, research on science education particularly concerned with conceptual change has been strongly influenced by Vygotskian notions of symbolic and social mediation. Scientific concepts and theories are cultural resources provided through (school, university, adult) education. Vygotsky made a crucial distinction between experiences produced by the immediate contact of the individual with the environment, and experiences mediated by symbolic tools, with writing as the major class of symbolic mediators (Kozulin, 2003). With respect to science education, unmediated contact gives rise to empirical learning, leading to the acquisition of spontaneous conceptions (of which some will be misconceptions), whereas the second, mediated form of contact affords theoretical learning, resulting more often in scientific conceptions (Karpov, 2003). In effect, for the case of theoretical learning, this does away with the dualism between process and content: How humans learn is largely dependent on the kind of concepts they master in the course of learning. In a profound sense, human learning is all about “learning to learn”. Process and content are thus two sides of the same coin. While a socio-cultural perspective allows us to overcome some of the problems inherent in the processcontent dualism, it is committed to the mind-environment dualism (with environment comprising the social as well as the natural). Due to the commitment to internalization, problem solving and thinking are achieved in the mind, by operations on a represented environment. In socio-cultural theory, the focus on the symbolic nature of tools goes along with a focus on internalization: Those tools are ‘out there’, and provided to us through education, but in order to use them we have to gain command over them by making them part of our cognitive repertoire. Cognition is seen as rooted in the mind/brain, 'between our ears'. However, for a number of reasons, not the least the rapid growth of mobile devices, the question of what needs to be learned from tools versus what we can accomplish with tools (Salomon, 1990) needs to be revisited. This requires us to make a less strong distinction between mind and environment. Jonassen’s (1996) for instance, identified the role that widely available computational tools, such as spreadsheet software, can play as “mindtools”. Despite its success, and the publication of a second edition (Jonassen, 2000), it is probably fair to say that the central idea has not been fully taken up. What we find today in (science) education is the acceptance of computers as ‘productivity tools’, which are good for creating artifacts such as essays and graphs and models. Learning, however, is still seen as resulting from these activities only in the form of ‘cognitive residues’, as changes in long-term memory. What is true for education in general also holds for the perceived value of ICT for conceptual change: the predominant question asked is: How does technology x contribute to fostering change? And the place we look for the answer is in the learner’s head, typically in a setting where the use of the technology is not allowed. To fully unravel the potential of computers, this view needs to be widened; the vision articulated in Jonassen (1996) needs to be revitalized. This should not be so hard, given that the socio-cultural perspective widely accepted by educators provides us with an appropriate conceptual framework: Learning scientific concepts means learning to make use of cultural tools, of the concepts and methods that science has developed and continuous to develop. If we are willing to entertain the idea that the distinction between psychological and technical instruments (Vygotsky, 1930/1981) is getting increasingly blurred, then the notion of learning as extending to changes in the use of external tools—not confined to changes to long term memory—should be a rather natural one. For instance, the notion of learning by modeling has evolved from a notion of learning from models (Reimann & Thompson, 2009), but to play out the full potential we need to move on to the notion of knowledge creation/problem solving/decision making with modeling. When the formal learning ends, tool use does not necessarily end. For example, students today can continue to carry their Netlogo software (Tisue & Wilenski, 2004) with them – on their laptop, hopefully soon also on their smartphone or tablet device. Indeed, further learning (including communal and societal learning) requires the continuous use and further

© ISLS

75

ICLS2012

Volume 2: Symposia

development of such tools. In order for such a (realistic) view of technology to take hold in schools, what is required are changes in the manner we assess learning. A still largely underconceptualized and underused potential of technology lies in its role for enhancing perception (Young, 2004). The main value of ICT may be that it provides ways to ‘see things differently’ by providing conceptual tools, and to make those tools available persistently and ubiquitously. The best use of ICT maybe augmentation--scaffolding without the fading. Computer tools for augmenting ‘reality’ can enhance perception in addition to thinking and problem solving.

References
Bereiter, C., & Scardamalia, M. (1989). Intentional learning as a goal of instruction, In L. B. Resnick (Ed.), Knowing, learning and instruction: Essays in honor of Robert Glaser (pp.361-392). Hillsdale, NJ: Lawrence Erlbaum Associates. Bereiter, C. (2002). Education and mind in the knowledge age. Mahwah, NJ: Lawrence Erlbaum. Bereiter, C., & Scardamalia, M. (2006) Education for the Knowledge Age. In P. A. Alexander, and P. H. Winne (Eds.), Handbook of Educational Psychology (2nd ed.). (pp. 695-713). Mahwah, NJ : Lawrence Erlbaum. Chai, C. S. & Lim, C. P. (2011). The Internet and Teacher Education: Traversing between the Digitized World and Schools. The Internet and Higher Education, 14, 3-9. Collins, A., & Halverson, R. (2010). The second educational revolution: rethinking education in the age of technology. Journal of Computer Assisted Learning, 26, 18-27. Elby, A., & Hammer, D. (2010). Teachers' personal epistemology and its impact on classroom teaching. In L. D. Bendixen & F. C. Feucht (Eds.), Personal epistemology in the classroom: Theory, research, and implications for practice (pp. 409-434). Cambridge: Cambridge University Press. Efklides, A. (2008). Metacognition: Defining its facets and levels of functioning in relation to self-regulation and co-regulation. European Psychologist, 13(4), 277-287. Greeno, J. (2006). Learning in activity, In K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 79-96). NY: Cambridge University Press Itakura, K. (1997). Kasetsu-Jikken-Jugyo no ABC, Dai 4 han. (The ABC of the Hypothesis-ExperimentInstruction: Invitation to enjoyable classes, Ver.4.) Tokyo: Kasetsu-Sha. [in Japanese] Jonassen, D. (1996). Computers in the classroom: mind tools for critical thinking. New Jersey: Prentice Hall. Jonassen, D. (2000). Computers as mindtools for schools. engaging critical thinking. (2nd ed.): Merrill. Jonassen, D., Howland, J., Marra, R., & Crismond, D. (2008). Meaningful learning with technology. Pearson: Upper Saddle River, NJ. Nussbaum Karpov, Y. (2003). Vygotsky's doctrine of scientific concepts: Its role for contemporary education. In A. Kozulin, B. Gindis, V. S. Ageyev & S. M. Miller (Eds.), Vygotsky's Educational Theory in Cultural Context (pp. 65-82). Cambridge: Cambridge University Press. Koehler, M. J., & Mishra, P. (2009). What is technological pedagogical content knowledge? Contemporary Issues in Technology and Teacher Education, 9(1). Retrieved from www.citejournal.org/vol9/iss1/general/article1.cfm Kozulin, A. (2003). Psychological tools and mediated learning. In A. Kozulin, B. Gindis, V. S. Ageyev & S. M. Miller (Eds.), Vygotsky's Educational Theory in Cultural Context (pp. 15-38). Cambridge: Cambridge University Press. Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for integrating technology in teacher knowledge. Teachers College Record, 108(6), 1017-1054. Miyake, N. (1986) Constructive interaction and the iterative processes of understanding, Cognitive Science, 10(2), 151-177. Miyake, N., Oshima, J., & Shirouzu, H. (2011). "Robots as a research partner for promoting young children's collaborative learning." Proceedings of the 6th ACM/IEEE International Conference on Human-Robot Interaction. Lausanne, Switzerland. Paavola, S., Lipponen, L., & Hakkarainen, K. (2004). Models of Innovative Knowledge Communities and Three Metaphors of Learning. Review of Educational Research, 74(4), 557-577. Popper, K. (1978, April 7). Three worlds. Retrieved 16th March 2010 from http://www.tannerlectures.utah.edu/lectures/documents/popper80.pdfReimann, P., & Thompson, K. (2009). Ecosystem modeling for environmental education: From stocks and flows to behavior and interactions. In P. Blumenschein, D. Hung & D. Jonassen (Eds.), Model-based approaches to learning: Using systems models and simulations to improve understanding and problem solving in complex domains (pp. 111-148). Rotterdam, The Netherlands: Sense Publishers. Salomon, G. (1990). Cognitive effects with and of computer technology. Communication Research, 17(1), 2644. Saito, M., & Miyake, N. (2011) “Socially constructive interaction for fostering conceptual change,” Proceedings

© ISLS

76

ICLS2012

Volume 2: Symposia

of the 9th International Conference on Computer-Supported Collaborative Learning, (CSCL2011), Hong Kong, Sawyer, K. (2006). The Cambridge handbook of the learning sciences. NY: Cambridge University Press Scardamalia, M., & Bereiter, C. (2003). Knowledge Building. In Encyclopaedia of Education. (2nd ed., pp.1370-1373). New York: Macmillan Reference, USA. Sinatra, G. M., & Pintrich, P. R. (2003). The role of intentions in conceptual learning. In G. M. Sinatra & P. R. Pintrich (Eds.), Intentional conceptual change (pp. 1-18). Mahwah, NJ: Lawrence Erlbaum. Smith, C. L., Maclin, D., Houghton, C., & Henessey, M. G. (2000). Sixth-grade students’ epistemologies of science: The impact of school science experiences on epistemological development. Cognition and Instruction, 18(3), 349-422. Tisue, S., & Wilenski, U. (2004, May). NetLogo: A simple environment for modeling complexity. Paper presented at the International Conference on Complex Systems, Boston. Vygotsky, L. S. (1930/1981). The instrumental method in psychology. In J. V. Wertsch (Ed.), The concept of activity in Soviet psychology (pp. 134-143). Armonk, NY: Sharpe. Vosniadou, S., & Kollias, V. (2003). Using collaborative, computer-supported, model building to promote conceptual change in Science. In De Corte, E., Verschaffel, L., Entwistle, N., & Van Merrienboer, J. (Eds.). Powerful learning environments: Unravelling basic components and dimensions (pp.181-196). Amsterdam: Earli. Vosniadou, S. (2008). International handbook of research on conceptual change NY: Routledge. Young, M. (2004). An ecological psychology of instructional design: learning and thinking by perceiving. In D. H. Jonassen (Ed.), Handbook of Research on Educational Communication and Technology (2 ed., pp. 169-177). Mahwah, NJ: Lawrence Erlbaum.

© ISLS

77

ICLS2012

Volume 2: Symposia

Interactive Surfaces and Spaces: A Learning Sciences Agenda
Michael A. Evans, Virginia Tech, 306 War Memorial Hall, Blacksburg VA 24060, mae@vt.edu Jochen Rick, EduTech, Saarland University, Campus C5 4, Saarbrücken, D-66123, j.rick@mx.uni-saarland.de Michael Horn, Northwestern University, 2120 Campus Drive, Evanston IL 60208, michael-horn@northwestern.edu Chia Shen, Harvard University, 33 Oxford St, Cambridge MA 02138, cshen@seas.harvard.edu

Emma Mercier, James McNaughton, Steve Higgins, Elizabeth Burd Durham University School of Education, Leazes Road, Durham, DH1 1TA, UK emma.mercier, j.a.macnaughton, s.e.higgins, liz.burd @durham.ac.uk Mike Tissenbaum, Michelle Lui, and James D. Slotta, University of Toronto, 252 Bloor St. W., Toronto, Canada miketissenbaum@gmail.com, michelle.lui@utonronto.ca, jslotta@gmail.com Roberto Martinez Maldonado and Andrew Clayphan, University of Sydney, roberto@it.usyd.edu.au, andrew.clayphan@sydney.edu.au Abstract: Interactive surfaces and spaces are entering classrooms and other learning settings. This symposium brings together leaders in the field to establish a coherent research agenda for interactive surfaces inside the learning sciences. We demonstrate the broad applicability of these technologies, outline advantages and disadvantages, present relevant analytical frameworks, and suggest themes to guide future research and application.

Supporting Learning with Interactive Surfaces and Spaces
Though the mouse-and-keyboard controlled interface is still the standard in classrooms, museums, and homes, interactive surfaces and spaces are emerging as a prominent alternative in the form of handhelds, tablets, tabletops, and whiteboards, used alone or in provocative configurations. As these devices become more capable and affordable, the case for applying these technologies to support learning becomes increasingly compelling. Now is the time for the learning sciences to get involved—to research the benefits and deficits of these technologies and to develop learning solutions that take advantage of the former while avoiding or compensating for the latter. The goal of this interactive poster session is to exhibit current work in this field.

Gesture, Metapragmatics & Multimodal Analysis Techniques for Surfaces & Spaces
Michael A. Evans Our work examines PreK students (ages 4-5), a group of three boys compared to a pair of girls attempting to solve geometric puzzles on a tabletop (Evans, Feenstra, Ryon, & McNeill, 2011). Results showed significant differences in the way that boys and girls negotiated the task at hand. These differences can be located in the relative frequency of object co-references and task co-references, and also in the function of metapragmatic speech, either to assert one’s presence and correctness or to co-create a coherent structure for the interaction. Accordingly, the boys and girls also differed in the way that they involved the teacher in their interactions. Where our work extends the current literature is by investigating multimodal interaction among peers and triads around a multi-touch tabletop surface. For the reported research we focus on the SMART Table™, a tabletop computer that projects bottom-up images on a horizontal surface that can be directly manipulated by users simultaneously. The incorporation of the SMART Table in our research allows us to experimentally manipulate learning conditions with changes to the software, requiring less reliance on verbal cues and policy. Thus, our research examined the interrelationships among social constraints (free, divided, and single ownership modes) and instructional technologies (physical and virtual manipualtives) using a multi-touch surface. The questions that follow from this arrangement of social constraints and instructional technologies are these: 1) How does discourse (quality and type) differ among the three levels of social constraint? 2) How does discourse (quality and type) differ between physical and virtual manipulatives? 3) Are there distinct patterns of interaction between social constraint and instructional technologies (i.e., physical and virtual manipulatives? Multi-modal techniques that examine talk, gesture, gaze, and activity are not unknown to the LS research community. Works by Strijbos and Stahl (2007) have demonstrated the benefits of using multimodal techniques to examine collaborative learning. Though Cakir et al. (2010) were investigating the use of a digital whiteboard in a virtual mathematics chat room setting, results from this work corroborate our emphasis on focusing on coreferential or joint problem solving moments in the discourse to find traces or evidence of group cognition. Moreover, the techniques adopted by Cakir et al. (2010) justify our adoption of microgenetic ethnographic methods to identify discrete moments of group cognition. Where our work is distinguished from theirs is the emphasis of co-located interaction and collaboration. The virtual chat room space decreases, or entirely removes, indications rendered by gesture. Though our work aligns in emphasizing gaze and how it might establish a dual space, we extend these efforts by including the gestural component that has been found critical in conveying mathematical ideas. Collaboration emerges when participants explicitly describe themselves as collaborating, or implicitly fit themselves into the structures we associate with collaboration as a way of interacting, such as turn-taking

© ISLS

78

ICLS2012

Volume 2: Symposia

structures marked by both time and repeating poetic structures from turn to turn. Within the scheme of references and co-references, metapragmatic speech that functions to make an interaction coherently collaborative is necessarily co-referential (McNeill et. al., 2010). Collaboration operates on a scale larger than a moment; an emergent collaborative character of ‘what we are talking doing now’ depends on chain linkages of collaborative metapragmatics throughout an interaction.

Proportion: A Tablet Application for Supporting Collaborative Learning
Jochen Rick I designed the Proportion iPad application to research the potential of one table to support collaborative learning. For this application, the tablet is positioned vertically on a table in front of two learners, aged 9–10. Learners work together to solve a series of ratio / proportion problems. The interface has two columns—one on the left and one on the right. For each problem, the children must size the left and right columns in proportion to their respective numerical labels. Through using Proportion, they gain competence in proportional reasoning.

Figure 1. Four interfaces to scaffold learners in solving problems. Proportional reasoning is a challenging mathematical domain. One problem is that it is usually taught and tested with mathematical notation through word problems; these provide neither feedback on task progress nor tools to scaffold users. Proportion provides several interfaces to scaffold users (Figure 1). Without any support (a), learners must estimate the ratios. Embodied proportional reasoning, relying on rules-of-thumb (e.g., larger denominator means smaller amount) and estimation (e.g., 9 is about twice as much as 4), are particularly important for learners to relate their everyday experiences to mathematical concepts (Abrahamson & Trimic, 2011). Proportion provides two levels of feedback on task progress. If the ratio of the two columns is close to the correct answer, a small star with a “close” label is shown. If the ratio is within a very small zone, then it is pronounced as correct, a large star with a “correct” label is shown, and the application moves on to the next problem. With a fixed 10-position grid (b), learners have precise places that they can target, thereby using their mathematical understanding of the task to quickly solve problems. One strategy is for users to select the grid line that corresponds to their respective numbers. This works well for simple ratios (e.g., 4 : 9). For the common-factor problem shown in Figure 1b, that strategy does not work. As illustrated, the children tried a novel strategy of positioning the columns based on the last digit of the number. Of course, this did not work and they were able to realize that this was not a viable strategy. With relative lines (c) that expand based on the position of the columns, learners can use counting to help them solve the problem. They can also learn more embodied strategies, such as maximizing the size of the larger column to make it easier to correctly position the smaller column. When the lines are labeled (d), other strategies can be supported. For instance, in the fractionbased problem shown, a useful strategy is to arrange columns so that whole numbers (e.g., 1) are at the same level. Proportion has been through two rounds of user testing to improve the interface and fine-tune the sequence of problems. The research with Proportion aims to shed light on two broad research topics. First, it will investigate how children communicate to collaborate. Previous work on interactive tabletops has demonstrated that children readily use their interactions with the interactive surface to communicate with their partners (Rick, Marshall, & Yuill, 2011). This work aims to tease apart the role of verbal and gestural communication. Second, it will investigate issues of equity of collaboration for tablet-based collaboration. On tabletops, it becomes difficult for users to access all parts of the surface; therefore, users tend to concentrate their interactions in areas closer to their position at the tabletop (Rick et al., 2009). Such separation is not possible for a tablet: Every user has good access to all parts of the interactive surface. Proportion was designed to have an interface split across the users. Children quickly grasp that they should control the column on their side. Do children stay with this convention? What happens when the convention breaks down? How does this affect the equity and effectiveness of the collaboration?
© ISLS 79

ICLS2012

Volume 2: Symposia

Multi-Touch Tabletops in Natural History Museums
Michael S. Horn and Chia Shen In this presentation, we discuss one exhibit that we have designed for this project called Build-a-Tree (or BAT for short). Build-a-Tree is a multi-level puzzle game that asks visitors to construct trees showing the evolutionary relationships of organisms (Figure 1). We represent organisms using black silhouettes superimposed on colored circles that visitors drag around the table. Touching any two circles together joins them into a tree. The reverse action, dragging circles apart, removes an organism from an existing tree. We designed the game levels to be increasingly difficult and to build on one another conceptually. For example, level four asks visitors to construct a tree showing the relationships of spiders, scorpions, and insects. These same relationships appear again as sub-problems in larger puzzles on levels six and seven. We conducted a study of Build-a-Tree at a well-known university natural history museum. In the study, we recruited and video recorded 30 family groups (consisting of at least on parent and one child) using the exhibit. We transcribed visitor conversation up to and including the first six levels of game play.

Figure 1. Screenshot of the Build-a-Tree game (left) and visitors playing the game in the museum (right). Based on these data, we iteratively developed a coding scheme that included three high-level categories: game talk, content talk, and off-topic talk. Among our findings, we saw that visitors spent long periods of time interacting with the exhibit (14 minutes on average; SD = 6) and engaged in minimal off-topic conversation (off-topic utterances occurred 0.16 times per minute on average; SD = 0.3). However, while engagement was high, evidence of visitor reasoning about evolutionary relationships was less satisfactory. Content related talk made up 26.6% of the overall conversation, but statements about relationships were rarely backed up with explanation. We see this study as one point in an ongoing design-based research effort, and we are in the midst of updating our design based on this analysis to increase evolutionary reasoning in visitor conversation. In attempting to explain these results, it was clear that game talk played a significant role in helping visitors coordinate their activity around the tabletop. For example, negotiating turn-taking was a common mechanism for groups consisting of multiple children. This is perhaps not surprising. In ethnographic research of children playing video games in homes, Stevens, Satwicz, and McCarthy (2007) observed a variety of learning arrangements that young people spontaneously adopted to support game play. In effect, when visitors encountered our game in the museum, they already had a repertoire of social practices on hand to support collaborative interaction. Identifying similar ways to cue effective social practices will be critical to support the use of tabletop surfaces in informal learning environments.

Orchestrating Learning in the Multi-touch Classroom: Developing Appropriate Tools
Emma Mercier, James McNaughton, Steve Higgins, Elizabeth Burd Interactive surfaces provide new ways to support collaborative learning with technology, however, new technology in classrooms has rarely lived up to its promise, in part because less attention has been paid to the pedagogical and classroom orchestration issues of introducing new technology (Dillenbourg & Jermann, 2010; Slotta, 2010). The SynergyNet multi-touch classroom (Higgins, et al., 2011) was designed with four networked, multi-touch student tables, teacher tools and an interactive whiteboard (IWB) that allow for the movement of content between the tables and to the IWB, and teachers tools to manage table functions, deliver content and monitor activities. Over two years we have developed tools for the teacher to use, iteratively designing the teacher controls as we learn more about the orchestration needs of a multi-touch classroom.

Stage 1: The Multi-touch Classroom and Orchestration Desk
Initially teacher orchestration tools were on an angled multi-touch table (Fig. 1). These controls were used to select activities for the class, send content to and monitor the student tables, and project content to the IWB.

© ISLS

80

ICLS2012

Volume 2: Symposia

This classroom was used to study a series of collaborative learning activities; two researchers with prior teaching experience taught the classes. For each of the activities, the teacher started the activity by sending the content to the student tables, then walked around the room, intervening to support groups where necessary, and locking the tables for whole-class discussions. For this the teacher projected content from one of the student tables to the IWB; the group whose table was displayed described their process and ideas. The teachers noted that not having access to controls as they moved around the room constrained their ability to take an interesting idea from one group to the whole class to support uptake of student ideas.

Stage 2: Mobile Orchestration Devices
To allow control while the teacher moved around the room, the teacher controls were transferred from the orchestration desk to an iPad (Fig. 2), which the teacher carried and used to make changes to the activities and view information about students’ contributions to tasks. The control features allow the teacher to start, pause or end an activity, project the content of a student table to the IWB, and move content from one table to the next. In addition, the teacher can view the work that the students are doing to all quick intervention if a student appears to be struggling. In one study, a member of the research team taught three classes using the iPad, reporting that the ability to control the activities from anywhere in the room was useful, but that having to hold and interact with the tablet took away from some of the ability to support the students by working directly on the student tables.

Stage 3: Gesture-based Teacher Controls
Currently, we are moving away from using direct-touch devices for orchestration, building on tracking capabilities of depth cameras such as the Microsoft Kinect. The teacher is identified by the Kinect, which is mounted on the teacher orchestration desk. Using a series of pre-defined gestures the teacher can manage the student tables from anywhere in the classroom (Fig 3), including locking and unlocking the students’ tables, taking screen-shots of a particular table and projecting the content from one table onto the IWB. These should provide the teacher with enough control to manage the technology, while future development will explore the use of teacher-specific tools that the teacher can activate on any of the interactive surfaces in the room, building on the recognition capabilities of the Kinect, which would allow the teacher to conduct more complex functions without the need for an additional device. Issues of orchestrating collaborative learning in a technology enhanced classroom are concerned with both the management of the technology and ways in which technology can provide additional information for optimal teacher intervention when groups get stuck or go astray. While our orchestration tools have been designed to support both needs, some of the management issues and technical interaction and control challenges must be solved before a teacher can give all their attention to the learning activities of their students.

Gesture, Scaffolding Collaborative Knowledge Construction in High School Physics with Tablet Computers and interactive White Boards
Mike Tissenbaum and James D. Slotta To effectively integrate collaborative technologies and practices into Science, Technology, Engineering, and Math (STEM) curriculum many learning theorists have advocated a knowledge community approach, where students work together to investigate salient issues, collaboratively develop theories, build ideas, and develop conclusions with technology as a central scaffold to the learning process (Brown & Campione,1996). Handheld tablets are increasingly popular within these technology mediated environments, as they allow for a 1:1 device to student ratio which can improve the organization and distribution of materials, and the coordination, communication, and negotiation of students within real-time activities (Zurita & Nussbaum, 2004). However, handheld screen sizes can inhibit group collaboration, and as such, should be augmented with large-format displays in order to better facilitate small group and whole class interactions (Tissenbaum, Lui, Slotta in press). By using large, interactive surfaces, we can more easily mimic the ecological dexterity of paper, allowing students to rearrange, reassemble, and annotate aggregated products of prior interactions (Everitt et al., 2005). The use of such technologies can allow students to collaboratively interact within an information space, build shared representations, and allow students to “slide” information between their own device and these representations. Another aspect of this combination of technologies (i.e., within a single physical setting) is the ability to dynamically adjust the representations and scripts sent to devices within the room. This capacity to

© ISLS

81

ICLS2012

Volume 2: Symposia

“orchestrate” the pedagogical flow of students, materials and activities can facilitate specific instructional designs, and serve to embody the broader epistemological goals of the research (Tissenbaum, Lui, Slotta, in press). The “room” could conceivably respond to individual students, moving them between groups or activities, or sending relevant materials. To achieve these complex pedagogical moves we require a robust technology infrastructure, including a role for intelligent software “agents” that can perform data mining and help coordinate pedagogical flow (Slotta, 2010). To this end, our group has been developing a flexible, open source “smart classroom” framework called SAIL Smart Space (S3) that supports the integration of technologies, including the aggregation, interpretation and response to student activities. The goal is to develop a technology environment capable to supporting a wide range of collaborative inquiry and knowledge construction scripts including interactive media and multi-touch surfaces. Our current design involves two grade 12 high school physics classes each involved in a two-day smart classroom activity, where students developed their understandings of real-world physics phenomena, in part through classifying physics problems, mapping those problems to the phenomena, and “setting up” problems concerning the phenomena, in ways similar to physics experts (Chi, Feltovich & Glaser, 1981).

Figure 1: In-Class collaborative equation negotiation

Figure 2: Smart Classroom configuration (Step 1)

Figure 3: SMART Board aggregated principle screen

Students first worked at home to sort various physics problems according to a core set of underlying principles (Newton’s laws, conservation of energy, and equations of motion), before working in small groups in a unique form of “equation sorting and negotiation” towards solving assigned problems. Throughout the activity intelligent agents performed student grouping based on past work, and tracked in-class activity, providing student groups feedback about their progress on consensus on their final equation sorts (Figure 1). The teacher used his own touchscreen tablet to “flip” through the aggregated group pages (like pages in a book), gaining insight into groups’ progress. During the smartroom activity students entered the room (Figure 2) in batches of 12, and divided into 4 groups of 3. Each student was given a set of sub-principles from one of three larger themes, and moved from station to station (by a “traffic flow” agent) and: (1) watched a video clip of a popular Hollywood movie that illustrated or violated one or more physics principles, and (2) “flung” (swiped from tablet to wall) any of their assigned principles at the video wall, before moving on to the next station as directed by the agent. As students moved throughout the room, flinging principles, the aggregated collections appeared on the SMART boards at the front (Figure 3). Once completed, agents regrouped students to one of the four stations based on their tags from the previous step and provided a “scaffolding question” to help them approach the video as a real physicist would. Using their tablets students submitted “assumptions” about the scenario within the video (e.g., the weight or speed of a car, if relevant to “solving” the implicit problem) and collectively debated these assumptions, towards forming consensus. In the next “pedagogical step”, students moved to a new station, where they saw the assumption of the previous group and received a small set of problems, assigned by an agent based on the principles attached to the video (from the homework stage). Their task was to pick the problem that most directly helped in understanding the video, and its related equations (attached during the preactivities). The groups were reconfigured for a final step, where they used the work of previous groups to order the sequence of equations towards solving the implicit physics problem by dragging equations to a field on their tablets and negotiating any conflicts (similar to the Day 1 activity). The activity concluded with the teacher engaging the class in a debrief around the final results and using the SMART boards to show the evolution of the class’ constructed knowledge. This intervention highlights how a smart classroom setting coupled with tangibly interactive technologies can achieve a variety of class configurations, interactions with student generated content, and epistemological goals that could not be achieved by traditional paper and pen approaches and physical layouts. Our demo will create a simplified activity that can be performed by session participants within the context of

© ISLS

82

ICLS2012

Volume 2: Symposia

this symposium (i.e., in 15 minutes) and also provide a video of our actual smart room session and accompanying poster.

EvoRoom: An Immersive Simulation for Smart Classrooms
Michelle Lui and James D. Slotta EvoRoom is a room-sized simulation of a rainforest environment designed for a “smart classroom” in an urban high school. Collaborating closely with a biology teacher, we designed immersive learning activities for teaching topics on biodiversity and evolution. These immersive simulation activities engages students in collective inquiry, where students take on the role of “field researchers” and use aggregated student inputs and collective knowledge to solve a problem or to understand a phenomena. Students work in various group configurations (e.g., individually, small groups, and as a collective) to complete tasks delivered to them on their personal tablet computers. The tablets, which contain custom designed application, provide scaffolds, place students in small groups, and give real-time updates and dynamic resources. Student contributions (e.g. written reflections) are aggregated and displayed on interactive whiteboards (IWBs) as they come in, and stored within a collective knowledge base. There are a number of surfaces involved in an immersive simulation, and each affords a different interaction (e.g., student-to-student interactions, student-to-teacher, and student-to-digitalartifacts). This poster examines how students engaged with the various components of EvoRoom. Below, we describe the environment, outline one of the immersive activities and detail the outcomes with respect to the various surfaces involved in the sessions, relating student engagement to the kinds of interactions that these components encourage. Our technology framework, Sail Smart Space (S3) supported the activity by providing a portal that where students login to the experience, an intelligent agent architecture that tracked real-time communications, and a central database that housed curriculum materials and the products of student interactions (Slotta, 2010). The room is set up with two sets of large projected displays (6 meters wide, achieved by “stitching” together 3 projector displays) that students would examine together (Figures 1, 2). The simulation files are networked and controlled with a custom tablet application, allowing the teacher to manage the time spent in each portion of the activity, controlling the pedagogical flow within the room. Two IWBs are located at the front of the room to display aggregated student inputs, which both teacher and students manipulate as a locus of discussions. In this study, 45 students from two sections of our co-design teacher’s grade 11 Biology course spent 75 minutes (i.e., one class period) working on the activity. Students were video recorded and student artifacts (e.g. written notes) were collected for analysis. A post activity questionnaire that queried student perceptions of environment was answered by 39 of 45 students.

Figure 1. Smart classroom setup

Figure 2. Three projectors make up a sidewall of EvoRoom

Students who visited EvoRoom were each given a tablet to work with; on the walls were different versions of the rainforest ecosystem, each showing an outcome from a specific environmental variable (e.g., high rainfall, tsunami…etc.). We designed a tablet-based interface that challenged students to explore the differences between these rainforests and to match them to the different environmental variables. As part of the activity, a “sorting agent” assigned students different organisms to look for. When students scanned QR codes at the different stations, the agents recognized their location and sent further, contextualized instruction to the tablets. For each station, students were prompted to record whether their assigned organisms were present. All responses were aggregated on the IWB at the front of the room, presenting a tally for reference by teachers and students alike. Students worked in groups of two or three to begin eliminating rainforest stations that likely did not result from their assigned variable. During group activities, different tasks were distributed to different students in each group. For example, in a group of three, one person was designated to be the scribe, while the second group member was instructed to look up information from field guide, with the third assigned to look up prediction from their class website (accessed as a link on the tablet). All of the group's decisions and notes were

© ISLS

83

ICLS2012

Volume 2: Symposia

collected on the IWB at the front of the room. The aggregated results were all shown on the IWB, which the teacher used to lead a discussion of how the variables affect the flora and fauna of the rainforest ecosystem. Our analysis of student performance and perceptions in EvoRoom uncovered several findings about student learning in immersive environments. Results of the classroom trial indicate that students are able to effectively allocate their attention between the immersive simulation and the various technologies supporting their tasks (e.g., tablets, laptops). Of the 16 groups that participated in the activity, 8 groups (50%) correctly identified the related rainforest as their first choice, while 4 (25%) groups chose the correct rainforest as their second choice. Video analysis will allow researchers to better understand which patterns of interactions were more conducive to supporting collective inquiry. On a scale of 1 to 10 (with 1 being unsuccessful and 10 being very successful), students rated the use of personal tablets in the smart classroom with an average of 7 (SD = 2.76). The immersivity of EvoRoom was rated highly, at an average of 9 out of 10 (SD = 1.48). One student indicated that the QR scanning feature offered a more “hands-on” experience compared to, for example, selecting an option from a list of item. On the whole, we found that students were excited about the immersivity of EvoRoom and the use of tablet computers for supporting their learning.

Analysing and Supporting Collaborative Learning at the Interactive Tabletop
Roberto Martinez Maldonado and Andrew Clayphan One of the major challenges faced when developing computer-supported collaborative learning tools is the provision of adequate support to the activities according to the particular needs of the group members. In group work, teachers often divide their attention among groups and they typically focus on the groups which they think need more advice and guidance. As a result, they typically only see the final product. The process to build such a solution and individual contributions may be hard to determine. Emerging pervasive shared devices, such as multi-touch interactive tabletops, are very promising, providing an enriched learning environment and a medium to capture groups’ social dynamics to adapt the affordances of the tool to students and their teachers’ needs. We present Collaid (Martinez et al., 2011), a digital framework to support collaboration at the classroom by providing both support to learners and awareness tools for their teachers in a multi-tabletop classroom. Collaid augments the capabilities of the hardware with a set of key features: identification of proximity of learners around a tabletop and authorship of each touch (using a depth sensor situated on top of the surface device), recognition of people speaking and capture of multimodal data about collaboration (video, audio and application logs in a common repository). Any touch or oral participation around the tabletop is automatically traced, with its author, and logged.

Figure 1. Examples of Collaid in use. We show how personal devices can be used to establish a link so to identify the source, provide personalisation and tracking of learners during activities and across activities. We demonstrate how our system uses technology for ways to capture the digital footprints of users without impacting the main task. This provides the opportunity to model groups’ participation and mirror back indicators of collaboration to their teachers so they can best orchestrate the collocated collaborative process. Collaid is presentedin two learning domains: collaborative concept mapping and brainstorming. Concept mapping is a technique that permits learners to visually externalise a representation of their knowledge about a topic. It permits group members to confront different perspectives to solve misunderstandings and build new knowledge. The interface permits learners to merge their individual concept maps in face-to-face sessions at the tabletop. Secondly, we demonstrate brainstorming (Clayphan et al., 2011) on an interactive digital tabletop as a collaborative activity to help groups generate original ideas encouraging egalitarian and cooperative participation. Digital tabletops combine natural face-to-face discussion found in conventional settings with increased flexibility gained from computerised support. Tabletops allow collaborative process to be captured, helping measure learner

© ISLS

84

ICLS2012

Volume 2: Symposia

engagement. We present work on tabletops and brainstorming with scripted collaboration around user control. We identify stages in the process and enable mechanisms to guide learners with the aim of improving support. We demonstrate the system with a recent integration of user tracking, allowing deeper metric collection and understanding of active contribution.

Conclusion
As demonstrated in this symposium, interactive surfaces and spaces offer a new computing paradigm to supplement, complement, or even supplant the desktop user interface. Learning scientists and educational technologists adopting these technologies have begun to carve niches for themselves in education. Over time, more powerful hardware will become available at a more affordable price, software development environments will improve, more applications will be available, and our understanding of how to use them to support learning will increase. Consequently, these interactive environments will play an increasing role in research and in the general support of learning.

References
Abrahamson, D., & Trimic, D. (2011). Toward an embodied-interaction design framework for mathematical concepts. In Proceedings of IDC ’11 (pp. 1–10). New York: ACM Press. Brown, A. & Campione, J. C. (1996). Psychological theory and the design of innovative learning environments. 289325. Cakir, M.P., Zemel, A., & Stahl, G. (2010). The joint organization of interaction within a multimodal CSCL medium. International Journal of Computer-Supported Collaborative Learning,4(2), 115-149. Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5(2), 121-152. Clayphan, A., Collins, A., Ackad, C., Kummerfeld, B. and Kay, J. Firestorm: A brainstorming application for collaborative group work at tabletops, Proc. of Interactive Tabletops and Surfaces ITS'11, ACM (2011), pages 162-171. Dillenbourg, P., & Evans, M. (2011). Interactive tabletops in education. International Journal of ComputerSupported Collaborative Learning, 6(4), to appear. Evans, M.A., Feenstra, E., Ryon, E., & McNeill, D. (2011). A multimodal approach to coding discourse: Collaboration, distributed cognition, and geometric reasoning. International Journal of Computer Supported Collaborative Learning, 6(2), 253-278. Higgins, S. E., Mercier, E. M., Burd, E., & Hatch, A. (2011). Multi-touch tables and the relationship with collaborative classroom pedagogies: a synthetic review. International Journal of Computer-Supported Collaborative Learning, 6(4), 515–538. Martínez R., Collins A., Kay J., Yacef K., Who did what? Who said that?: Collaid: an environment for capturing traces of collaborative learning at the tabletop, Proc. of Interactive Tabletops and Surfaces ITS'11, ACM (2011), pp. 172-181. McNeill, D., Duncan, S., Franklin, A., Goss, J., Kimbara, I., Parrill, F., Welji, H., Chen, L., Harper, M., Quek, F., Rose, T. & Tuttle, R. (2010). “Mind-Merging”. In a Festschrift for Robert Krauss, Ezequiel Morsella (ed.), Expressing oneself / expressing one's self: Communication, language, cognition, and identity. London: Taylor and Francis. Rick, J., Harris, A., Marshall, P., Fleck, R., Yuill, N., & Rogers, Y. (2009). Children designing together on a multi-touch tabletop: An analysis of spatial orientation and user interactions. In Proceedings of IDC ’09 (pp. 106–114). New York: ACM Press. Rick, J., Marshall, P., & Yuill, N. (2011). Beyond one-size-fits-all: How interactive tabletops support collaborative learning. In Proceedings of IDC ’11 (pp. 109–117). New York: ACM Press. Slotta, J. D. (2010). Evolving the classrooms of the future: The interplay of pedagogy, technology and community. In K.M\{a}kitalo-Siegl, F. Kaplan, J. Zottmann & F. Fischer (Eds.), The classroom of the future orchestrating collaborative learning spaces (pp. 215-242). Rotterdam: SensePublisher. Stevens, R., Satwicz, T., & McCarthy, L. (2007). In-game, in-room, and in-world: Reconnecting video game play to the rest of kids’ lives. In K. Salen (Ed.), The Ecology of Games: Connecting Youth, Games, and Learning, pp. 41–66. MIT Press Strijbos, J. W., & Stahl, G. (2007). Methodological issues in developing a multi-dimensional coding procedure for small group chat communication. Learning & Instruction. Special issue on measurement challenges in collaborative learning research, 17(4), 394-404. Tissenbaum, M., Slotta, J. D., & Lui, M. Co-designing collaborative smart classroom curriculum for secondary school science. Jorunal of Universal Computer Science. Zurita, G., & Nussbaum, M. (2004). Computer supported collaborative learning using wirelessly interconnected handheld computers. Computers \& Education, 42(3), 289-314.

© ISLS

85

ICLS2012

Volume 2: Symposia

Instructional Approaches to Promote Conceptual Change
Stellan Ohlsson (stellan@uic.edu) Stella Vosniadou, Department of Philosophy and History of Science, National and Kapodistrian University of Athens, Greece, (svosniad@phs.uoa.gr) Naomi Miyake, School of Computer and Cognitive Sciences, Chukyo University, Japan, (nmiyake@sist.chukyo-u.ac.jp) David D. Cosejo, Department of Psychology, University of Illinois at Chicago, USA, (dcosej1@uic.edu) Marcia Linn (mclinn@berkeley.edu) Doug Clark, Graduate School of Education, University of California at Berkeley & Vanderbilt University, USA, (doug.clark@Vanderbilt.Edu) Michael J. Jacobson , Education and Social Work, University of Sydney, Australia, (michael.jacobson@sydney.edu.au) Keywords: learning, conceptual change, re-categorization, instructional analysis, integration, classroom discussion

Introduction
Learning usually involves the enrichment of prior knowledge and constructive approaches to learning and instruction emphasize the activation of prior knowledge so that the new information can be productively integrated into what is already known. What happens however when the new, to-be-learned, information is highly counter-intuitive and contradicts what is already known? What are the instructional interventions that can be used to help students restructure what they already know and achieve conceptual change? The problem of belief revision in the process of conceptual change is one of the most important problems in learning and instruction research and a problem that unfortunately has not received adequate attention so far. The purpose of the present symposium is to present current theoretical analyses of this problem and to discuss constructivist instructional approaches that attempt to deal with it. Researchers coming from different perspectives will present new empirical evidence as well as new theoretical accounts of the processes involved in conceptual change and the instructional interventions that can facilitate it. A number of different proposals will be presented and their implications for the design of curricula and instruction will be discussed. Attention will be given to the role of implicit learning, to explicit hypothesis testing, to the importance of integrating students’ ideas, to the deliberate use of instructional analogies, and finally to productive use of collaboration and classroom discussion. Even though the problem of conceptual change is fundamental for a theory of learning it has had little presence so far in the conferences of the International Learning Science Society. The aim of this symposium is to bring some of the current discussions and different perspectives on instructional interventions to promote conceptual change to the attention of the learning science audience. All the speakers are key developers of the different approaches and have led their development.

Explicit and Implicit Processing in a Laboratory Model of Conceptual Change
Stellan Ohlsson and David D. Cosejo. Conceptual change is a temporally extended process with infrequent and course-grained expression in observable behavior. Developmental psychologists, educational researchers and historians of science work with temporally scarce data that provide scant information about the processes involved. To study these processes under controlled conditions, we have developed a laboratory model of conceptual change that we call recategorization. In a re-categorization paradigm, participants learn an unfamiliar category, to criterion, at which point the target category is re-defined. The participants are not informed of this but continue the standard categorization cycles of stimulus presentation, categorizations, and feedback until the revised category has been learned to criterion. The question is how, by what processes, the initially acquired category – the “misconception” – is unlearned, and the revised, target category is acquired. We present some preliminary data obtained with this paradigm that supports the view that conceptual change (in this paradigm) is an interaction between explicit and implicit processing. Explicit processing such as deliberate hypothesis testing is sequential and hence strongly impacted by the complexity of the category to be learned, while implicit processing (e.g., automatic updating of association strengths) occurs in parallel and hence is less affected by category

© ISLS

86

ICLS2012

Volume 2: Symposia

complexity. The relation of the laboratory model to naturalistic conceptual change is raised, and the question of how the model can be improved to capture more of naturalistic processes is discussed.

Using instructional analogies to promote the comprehension of counterintuitive text
Stella Vosniadou Learning science often requires children to radically restructure their naïve explanations of physical phenomena. Are children successful in engaging in such restructuring processes and can instructional analogies help in this process? For sometime now we have been investigating primary and secondary school students’ comprehension of counter-intuitive text in science and mathematics with and without the use of verbal analogies and/or dynamic visual models. Our results showed that students fail to understand even the most fundamental ideas in counter-intuitive science and math texts. In addition, the students often generate erroneous inferences which reveal intrusions from prior knowledge and which are similar in many respects to the misconceptions, or synthetic conceptions, revealed in cross-sectional and longitudinal developmental studies. We hypothesized that analogies would have the potential to help in the restructuring process because they draw upon students’ existing knowledge from a different but familiar domain to facilitate the understanding of a new and unfamiliar explanation. We also hypothesized that the presence of a dynamic visual model would facilitate the comprehension of the scientific explanation not only because it draws on familiar past knowledge but also because it can make more explicit the structure and function of the explanatory mechanism. Results confirmed both hypotheses showing significant gains for the analogies and dynamic models groups compared to the controls. They also showed that the interventions were not equally successful for all students but had the greatest effect on the children who were in the transition to understanding the scientific explanation and had rejected some of the presuppositions of their naïve theories that constrained their understanding of the scientific explanations. In general, the students who were able to understand the analogies were also able to use them to restructure their physical explanations.

Synergies in conceptual change perspectives
Marcia Linn & Doug Clark Seeking similarities between varied perspectives on conceptual change helps to clarify the field and offers a foundation for future research (Clark & Linn, in press). We identify synergies between framework (Vosniadou, 2008) and elemental (Clark, 2006; diSessa, 2008; Linn & Eylon, 2011) perspectives. Researchers offer differing perspectives about whether students’ understandings in science are better characterized as a coherent unified scheme of theory-like character or an ecology of quasi-independent elements or ideas. But the disagreement is a matter of degree. To identify synergies we consider a graduated spectrum for the magnitudes of influence that elements may exert on one another. We draw on research exploring knowledge integration to clarify the synergies. An examination of the diverse, creative, and unique ideas that students formulate has led researchers to argue for a constructive process of knowledge generation and change. When students are asked to explain scientific phenomena in an abstract de-contextualized frame, they often respond quite differently from when asked to explain an observed phenomenon. Analyzing the conceptual change processes involves explaining how students make sense of ideas when they occur in a new situation. Students’ methods for grappling with new observations strengthen the view that students engage in a creative process of trying to make sense of their world (Hatano & Inagaki, 2003). Many researchers have focused on the diversity and character of student ideas. These ideas have been referred to as “misconceptions,” “alternative conceptions,” “beliefs,” “intuitive ideas,” and “constructed ideas.” Any theory of conceptual change needs to explain the emergence of these views, as well as their role in conceptual change. A large body of evidence has led researchers to argue that students construct multiple, contradictory, and fragmented ideas that stem from their interactions with the material and social world. Many researchers have shown that the ideas students generate arise from observations, analogies with related events, cultural practices, or colloquial uses of language.

© ISLS

87

ICLS2012

Volume 2: Symposia

The knowledge integration perspective on conceptual change has emerged through a series of empirical studies. It was spurred by evidence of the impact of context on student reasoning (Linn, 1983). It celebrates the ideas students generate and views these ideas as intellectual accomplishments rather than intellectual constraints (Linn & Hsi, 2000). Important evidence for this view comes from a longitudinal study carried out over five years that gives insight into student lifelong learning (Clark, 2006; Clark & Linn, 2003; Linn & Hsi, 2000). This longitudinal study illustrated how students maintain conceptual ecologies involving multiple conceptual elements and ideas at various levels of sophistication, connection, and conflict. These conceptual elements and ideas include cultural and observational information and beliefs spanning both epistemological and ontological aspects of knowing and learning. As with Vosniadou et al.’s (2008) framework theory perspective and diSessa’s knowledge in pieces perspective (1993), the knowledge integration perspective acknowledges that ideas can be introduced (through schooling and other experiences) that result in conflicts, fragmentation, or integration. Learning occurs through a process of restructuring and reorganizing these ideas. These studies reveal underlying (although often unarticulated) ideas that shape students’ thinking, explanations, and predictions. Students use these multiple ideas to interpret the phenomena they encounter in their everyday lives. The particular ideas students consider and connect depend on contextual cues. Some connections arise from experience (e.g., metals feel cold), some connections are situationally specific and less broadly useful (e.g., cooling on the stove is different from heating), some are imported from another domain such as electricity and may or may not be useful or accurate in the new domain (e.g., glass is not a conductor of electricity so it will be a poor thermal conductor), and some have their roots in classroom instruction (e.g., metals have heavier molecules). Some connections that students make are spontaneous and ephemeral, whereas some connections are much more durable and persistent. As students learn, they reorganize, reconnect, and sort through their ideas. Some ideas become much more central and pivotal as a student uses them as focal points around which to integrate other ideas, while other ideas are demoted in priority and centrality. The knowledge integration framework emphasizes creating opportunities so students can productively distinguish among their ideas to achieve conceptual change and coherent understanding. Researchers have shown that the ideas students articulate to make sense of school and everyday situations illustrate students’ capabilities to sort out confusing observations rather than illustrating developmental constraints (e.g., Gilbert & Boulter, 2000; Redish, 2003). For example, students often argue that metal must be a naturally “colder” material because metal feels cold at room temperature. Some researchers see these efforts as evidence for powerful reasoning ability that can be guided by instruction. More specifically, when students make an effort to sort out ideas, even if the view they formulate is not supported by all the empirical data, they are engaging in the sort of reasoning that can lead to understanding. In summary, the knowledge integration framework calls for capitalizing on students’ ability to make sense of scientific phenomena by empowering them to consider new ideas, distinguish among existing and new ideas, and promote the most promising ones. Students generate a broad range of ideas about any scientific phenomenon. These ideas represent multiple types of explanations, vary across contexts, and may not be recognized as applying to the same topic. The knowledge integration framework takes these ideas as building blocks and mobilizes the same processes that generated them to focus the learning trajectory on coherent understanding. Synthesizing framework theory perspectives with the knowledge integration perspective and other elemental perspectives reveals overlaps and consistencies. In particular, the magnitude of influence of certain ideas from framework theory perspectives can mesh well with the focus on the rich interactions within conceptual ecologies highlighted by elemental perspectives. These synergies clarify and strengthen the accounts of conceptual change collected across studies. This integrated perspective also supports the value of instructional sequences, such as the knowledge integration pattern, that scaffold students in refining and consolidating their conceptual ecologies around productive focal ideas. Essentially, highlighting the magnitude of influence of certain ideas in students’ conceptual ecologies clarifies systematicities in student explanations while respecting the rich range of ideas and interactions between ideas. This approach clarifies the nature and process involved in the fragmentation of ideas and emergence of synthetic models. It also sheds light on the difficulties students have when grappling with the abstract formal ideas introduced in science classes.

Designs and analyses of multi-person constructive interaction in real classrooms for adaptive conceptual change

© ISLS

88

ICLS2012

Volume 2: Symposia

Naomi Miyake The University of Tokyo This presentation aims at clarifying structures of successful classroom discussion that support learners to change their folk knowledge into scientific understandings. It has been suggested that sequential, cumulative discussion across classes has strong positive effects to help learners expand their understandings adaptively (e.g., Hatano & Inagaki, 1991). Yet, the details of such classroom discussion have rarely been analyzed fully to explore whether there is some specific structure leading to adaptive conceptual change. In this talk I report the results of our analyses of two types of classroom discussions, the Hypothesis-Experiment Instruction (henceforth HEI; Itakura, 1997) and the Knowledge Constructive Jigsaw classes (henceforth KCJ; Miyake, 2011). To analyze these two types of classroom activities, I adopt the frameworks from previous research on collaborative conceptual change (Roschelle, 1992) and two-person, constructive interaction for the abstraction of levels of understanding (Miyake, 1986). The combined framework requires that there occur interaction, or role exchange, between the task-doer engaged in explaining or externalizing the on-going problem solving at hand and the monitor who watches over such development and tries to integrate her/his own understanding. It also suggests that the progression of the aimed conceptual change could be analyzed in terms of levels of understanding, from more directly related to every-day, mundane understanding to more abstract, scientifically acceptable understanding. Choosing classrooms where the students succeeded in attaining primitive yet scientific understandings, I analyzed their discussion patterns to see whether there occurred the role-exchange and if so in what forms. The relationship between the shifts of the identified patterns and the levels of understandings of the students, exhibited in forms of their conversations, class presentations, and the end-of-theclass written reports. For the HEI, four classes of third graders were analyzed (Saito & Miyake, 2011). The topic was to understand physical identity of objects, or to understand that “two objects like air and water cannot share the identical physical space” to be specific. Overall, our analyses show that the kids’ discussion represents the socially expanded version of two-person, constructive interaction, where not only an individual student but also small groups of students could serve the role of task-doing while the rest of the class monitors. The proportion of the two roles differs from class to class, but more group-based interaction tended to lead to changes of understandings. The class design framework of the KCJ involves a shared question to be answered and some relevant learning materials from different perspectives that are distributed among the different groups first in expert groups, to be later exchanged and integrated to answer the question in the jigsaw groups (Miyake, 2011). The design naturally requires each student become a task-doer in the jigsaw group, yet the proportion of the two roles could differ from individual to individual. We have analyzed data from several classes of different grade levels from elementary to middle school on different topics of science. The analyses of three classes of middle schools on different topics have revealed that there is little correlation between the achievement levels and the proportion of the role exchange. Rather, the monitors, who could spend almost the entire class without “speaking up,” tended to learn a lot from just attending to others’ talks and inwardly working to integrate such inputs to their own understandings. I will report some evidence for this silent yet active learning identifiable from the memos taken during the class by such monitors.

References
Clark, D. B. (2006). Longitudinal conceptual change in students' understanding of thermal equilibrium: An examination of the process of conceptual restructuring. Cognition and Instruction, 24(4), 467-563. Clark, D. B., & Linn, M. C. (In press). The Knowledge Integration Perspective: Connections Across Research and Education. In S. Vosniadou (Ed.), Handbook of Research on Conceptual Change (2nd ed.). New York: Routledge. Clark, D. B., & Linn, M. C. (2003). Scaffolding knowledge integration through curricular depth. Journal of Learning Sciences, 12(4), 451-494. DiSessa, A. (2008). A Bird’s-Eye View of the “Pieces” vs. “Coherence” Controversy (From the “Pieces” Side of the Fence). In S. Vosniadou (Ed.), Handbook of Research on Conceptual Change. New York: Routledge. Gilbert, J. K., & Boulter, C. J. (2000). Developing models in science education. Dordrecht: Kluwer. Hatano, G. & Inagaki, K. (1991). Sharing cognition through collective comprehension activity. In B. Resnick, J. M. Levine, & S. D. Teasley (Eds.). Perspectives on socially shared cognition. 331-348. APA Hatano, G., & Inagaki, K. (2003). When is conceptual change intended?: A cognitive-sociocultural view. In G. M. Sinatra & P. R. Pintrich (Eds.), Intentional conceptual change (pp. 407-427). Mahwah, NJ: Lawrence Erlbaum Associates.

© ISLS

89

ICLS2012

Volume 2: Symposia

Itakura, K. (1997). Kasetsu-Jikken-Jugyo no ABC, Dai 4 han. (The ABC of the Hypothesis-ExperimentInstruction: Invitation to enjoyable classes, Ver.4.) Tokyo: Kasetsu-Sha. [in Japanese] Linn, M. C. (1983). Content, context, and process in adolescent reasoning. Journal of Early Adolescence, 3, 6382. Linn, M. C., & Eylon, B.-S. (2011). Science Learning and Instruction: Taking Advantage of Technology to Promote Knowledge Integration. New York: Routledge. Linn, M. C., & Hsi, S. (2000). Computers, Teachers, Peers: Science Learning Partners. Mahwah, NJ: Lawrence Erlbaum Associates. Miyake, N. (1986) Constructive interaction and the iterative processes of understanding, Cognitive Science, 10(2), 151-177. Miyake, N., (2011) Fostering conceptual change through collaboration: Its cognitive mechanism, socio-cultural factors, and the promises of technological support, Proceedings of the 9th International Conference on Computer-Supported Collaborative Learning, (CSCL2011), Hong Kong Redish, E. F. (2003). Teaching Physics with the Physics Suite. New York: John Wiley and Sons, Inc. Roschelle, J. (1992). Learning by collaboration: convergent conceptual change, The Journal of the Learning Sciences, 2, 235-276. Saito, M., & Miyake, N. (2011) “Socially constructive interaction for fostering conceptual change,” Proceedings of the 9th International Conference on Computer-Supported Collaborative Learning, (CSCL2011), Hong Kong Vosniadou, S. (Ed.). (2008). International Handbook of Research on Conceptual Change. New York: Routledge.

Acknowledgment
The work entitled “Using instructional analogies to promote the comprehension of counter-intuitive text” (Stella Vosniadou) is supported by the project ANALOGY: Human-The Analogy Making Species, financed by the FP6 NEST Program of the European Commission. (STREP Contr. 029088).

© ISLS

90

ICLS2012

Volume 2: Symposia

Everyday Interactions and Activities: Field Studies of Early Learning Across Settings
Reed Stevens, Lauren Penney, Danielle Keifert, Pryce Davis Northwestern University, School of Education and Social Policy 2120 Campus Drive, Evanston IL, 60208 USA Email: reed-stevens@northwestern.edu, lauren.penney@u.northwestern.edu, keifert@u.northwestern.edu, pryce@u.northwestern.edu Siri Mehus, University of Washington, College of Education 1100 NE 45th St., Suite 200, Box 354941, Seattle, WA 98195-3600 USA Email: smehus@u.washington.edu Discussant: Rich Lehrer, Vanderbilt University Abstract: This symposium presents research that uses video-based ethnographic methods to sample the range of ecological events and contexts in young children’s lives and thereby document the what and how of naturally occurring learning and development. All four papers presented focus on the interactional arrangements of social, cultural, and material supports for learning within contexts. The specific focus is on the social functions and social occasioning of practices across settings. The first two papers examine two common interactional arrangements—questioning and imitation—and the qualities of these arrangements across settings. The final two papers examine two common childhood activities in context—building with blocks and watching television—in order to understand how children engage in these activities and how that engagement is mediated by different interactional arrangements across settings. The combined effect of these papers is to begin to map the socio-material arrangements and interactional routines that contribute to young children’s everyday learning, a neglected focus in the learning sciences.

Symposium Overview
The learning sciences have largely neglected the study of young children and have generally ceded learning at this age to more traditional developmentalists and their methods. The dominant methods of developmental research over recent decades have been experimental. Laboratory studies offer assurances of statistical reliability, internal validity, and generalizability. What this dominant tradition lacks as a focus is a commitment to adequately sample the range of ecological events and contexts in young children’s lives and thereby to directly study the what and how of naturally occurring learning and development. For this we need to capture and analyze the daily learning activities and settings of early childhood. These studies involve video-based microethnographic analyses of children’s routine activities at home and/or preschool. The data are drawn from two large-scale field projects investigating children’s interactions and learning in and across the socio-material contexts of their everyday lives. The first three papers are based on video data recorded in young children’s (ages 2-5) preschool classrooms and homes. Activities in seven classrooms at three preschools were recorded weekly for three-five months in each school. In addition, ten focal children (five boys and five girls) were video-recorded in out-of-school contexts. Approximately 500 total hours of interaction were video-recorded. The video records were viewed, logged, and selectively transcribed for further microanalysis. The analysis in the final paper draws from data collected for a study of children’s learning from screen media, particularly television. We simultaneously recorded the video stream from the television and the embodied activities of children and others in the viewing space, synchronizing the two videos to allow analysis of how children’s interactions in the room were affected by and coordinated with the television shows they viewed. Participants included 13 children (ages 1-6 years old) from nine families, as well as their parents, friends and siblings. Approximately 60 hours of video were collected for this study. Two overarching themes unify these studies. First, each study documents the interactional, organizational, and material supports for learning in the naturally occurring activities of young children. Secondly, these studies specify qualities of how children think and learn with others. Two further analytic foci connect these studies—foci distinctively available to field methods; these studies all seek to explore the social functions and social occasioning of children’s practices within and across settings. By social functions, we refer to the work that specific practices (e.g. questioning) do among people (e.g. how questions serve attentiongetting functions for children with adults as well as information-getting functions). By social occasioning, we refer to the precipitating contextual conditions that give rise to a specific practice (e.g. a family-local practice of inquiry about scientific or natural things). Both of these questions are important for the learning sciences in that they address questions of how and under what conditions practices emerge in the flow of activity and where they lead in downstream activity.

© ISLS

91

ICLS2012

Volume 2: Symposia

These studies could be seen to lack unity if viewed from the perspective of the four focal practices: questioning, imitating, block building, and TV viewing. Conventional wisdom would suggest two of these practices are to be understood as much more ‘general’ (i.e. imitating and questioning) and two are more ‘specific’ (e.g. block building and TV viewing). By bringing studies on these four practices together, we mean to problematize this conventional wisdom. We mean for a comparison of these practices to open up new questions about the movement and transformation of practices across contexts and about the movement of people across contexts, into and out of practices. There are arguments of course that imitation is likely to be a general learning practice, because imitating is built into the human genetic structure (Rizzolatti, Fadiga, Fogassi, & Gallese, 2002) and that questioning is a general practice, because questions are encoded into the syntactic and/or prosodic structure of languages. TV viewing and block building would not seem to have these sorts prestructured affordances, but perhaps that only seems the case if we bias our view of prior structure to personinternal structure (e.g. grammar, prosodic production, or mirror neurons) and not extend considerations of prior structuring to the external environment as well. TV viewing makes a strong case for thinking in these terms as well, because screen-based viewing opportunities (on televisions, computers, tablets, and smart phones) are nearing ubiquity, at least in Western culture contexts. The comparative questions we want to ask with the different practices are fundamentally transfer questions, understood differently from traditional mentalist transfer: how do practices circulate and stabilize across contexts and how do people circulate among practices across contexts? Together, these four papers examine learning over several levels of interactional arrangement—from the moment-to-moment analysis of questioning and imitation to the more extended interactional arrangements that surround play and media usage. These papers also explore learning across multiple contexts and among multiple interactional partners, which include parents, siblings, and school peers. The combined goal of these papers is to partially map the socio-material arrangements and interactional routines that contribute to young children’s everyday learning and to open up new questions about learning and participation in practices, within and across contexts.

Questioning among Preschool Children
Lauren Penney & Reed Stevens It is generally accepted that children use questions as tools to learn about the world, and for the past century much research has been done on the topic of children’s questions. Researchers have catalogued the types of questions children ask (e.g. what, where, why, how) and at what age they begin to ask each type of question (Davis, 1932; Meyer & Shane, 1973; Kearsley, 1976; Tyack & Ingram, 1976; Hood & Bloom, 1979). Brown (1968) studied the grammar and structure of the question forms that children produce. Other researchers have analyzed the different ways adults respond to children’s questions (Tizard, Hughes, Carmichael, & Pinkerton, 1983; Van Hekken & Roelofsen, 1981). More recently, there has been investigation into how children’s questions contribute to their causal reasoning (Frazier, Gelman, Wellman, 2009; Chouinard, 2007; Callanan & Oakes, 1992; Hood & Bloom, 1979). This prior research analyzes the spontaneous questions of children, yet focuses specifically on questions that indicate information gathering and knowledge building (i.e. Whquestions: what, where, when, why, how), and analyzes questions out of the context of the larger conversation. Our research extends prior work by more closely examining the role that questions play in the lives of young children. We use conversation analysis (e.g. Goodwin & Heritage, 1990) as well as quantitative analysis to explore all of the spontaneous questions asked by two-year-old to five-year-old children in their everyday environments. In this paper we expand upon existing taxonomies that have been created to categorize the various functions that questions play in the conversations of children. Our expanded taxonomy includes a missing category of questions we call “social” which serve various social functions. We then argue that unlike existing research, children’s questions should be analyzed at a unit level larger than just the question or question-answer pair. Using turn-by-turn analysis we take a detailed look at several sequences we call lines of question-based inquiry (LOQBI) and explore how different the analysis would look if these sequences were analyzed as individual question-answer pairs. We conclude this paper with a discussion about how questions function as a tool that children employ to interact socially with each other. Ultimately this paper makes two new arguments: 1) children’s questions often need to be analyzed in extended sequences (as opposed to broken into question-answer pairs), and 2) these questions are multifunctional and frequently not only seeking information. The question sequences we are calling LOQBI have been noted, but not fully explored by prior research (Chouinard, 2007; Tizard et al., 1983). In this paper we document and analyze inquiry sequences from video-recordings of preschool children across multiple settings. Our findings extend recent work by Chouinard (2007) in that we find children’s questions are not one-offs but are sustained for sense making or information gathering purposes. In Chouinard’s study, questions continued until satisfactory adult answers were provided, while in our study children continue questioning, even after initial answers are provided, to explore a line of inquiry. For example, when Charlie commented that he was supposed to see Santa Claus but didn’t, his teacher

© ISLS

92

ICLS2012

Volume 2: Symposia

replied, “Maybe he’s still working in the North Pole.” Charlie then asked several questions about this topic, starting with, “What is the North Pole?” This LOQBI continued for two minutes and was question-heavy, with Charlie asking 11 questions – all of which had something to do with understanding the Santa Claus story. This conversation covered where Santa Claus lives, what he does, with whom he associates, and how he does his work. Charlie received an adequate response to his initial “What is the North Pole” question, yet he continued asking questions about the Santa Claus story. This three-year old child was asking a stream of questions about a topic he was interested in, and asked different questions as he learned more about the topic. These questions were all related and the topic was the reason for the continued questioning – not inadequate answers. Next we argue that children’s questions perform various functions in a conversation beyond seeking information. Freed’s (1994) taxonomy of question functions in adult didactic conversations, for example, is a continuum from “information sought” (the speaker seeks information from the hearer) to “information conveyed” (the speaker conveys information to the hearer). Freed says questions that fall into the information sought category ask for factual information (e.g. “What time are you going home?”) while information conveyed questions are used to express importance or emotion (e.g. “And you know what’s upsetting?”) (p. 626-629). Research on children’s questions has focused mainly on questions that are about information gathering, yet the research on adult questions indicates this is not the only way questions are used in conversation. Many taxonomies of children’s questions have been created, many of which stem from Piaget’s (1932) categorization of “why” questions (Piaget, 1923; Davis, 1932; Meyer & Shane, 1973), others of which are developed independently (Tyack & Ingram, 1977; Callanan & Oakes, 1992). However, these taxonomies focus primarily on the information gathering and knowledge building questions asked by children. This field study captures and begins to explore the full range of questions asked by young children in their natural environments. One of our findings is that many of the questions children direct towards other children perform a social function, that is, children use questions to invite others to play (“Who’s the conductor?”), as well as to negotiate and arrange their play together (“Wanna watch some dog movies?” or “How much player is this?”). These questions are not asked to simply seek information, but the children also use them as an interactional social tool. Our first argument in this paper is that analyzing these LOQBI as sequences allow us to see the connected nature of this type of questioning – something that is lost when questions are analyzed individually or as question-answer pairs. Our second argument is that children’s questions perform various functions in a conversation beyond seeking information; children use them as they socially interact with each other during pretend play. We hope this paper begins a discussion about the range of questions asked by young children.

Children’s Imitation in its Natural Environments
Siri Mehus, Reed Stevens, & Lauren Penney Imitation is one of the fundamental means by which humans learn from other humans. Our particular propensity for doing as we see and hear others do is quite possibly unique to humans and forms the basis of human culture (e.g. Gergely & Csibra, 2005; Meltzoff et al., 2009; Meltzoff, 1988; Tomasello, 1999). While the importance of imitation for human learning has been well demonstrated in experimental laboratory research, few studies have investigated the ways in which children imitate other children and adults in their natural habitat—e.g. in the homes and childcare classrooms in which they spend their days. Even fewer have employed the ethnographic approach of investigating such practices from the participants’ perspectives. Laboratory studies help us understand how children can learn through imitation—i.e. what the human brain is capable of—but they do not tell us much about how children do learn through imitation—i.e. the role of imitative learning in the course of children’s development in their social worlds. In this study we set out to investigate imitation in terms of how it is occasioned and how it functions in the socio-material contexts of children’s everyday lives. We ask, in other words, when does imitation happen, and when it does, what happens next? Our data consist of video-recorded interactions between young children and their families, teachers and peers in home and preschool environments. Our analysis reveals that imitation in children’s daily lives often occurs in the context of some social routine, game, or shared activity. Within these interactional environments, the purpose of making an appropriate and successful move in the activity motivates the imitative act. Examples of social moves that often involve some form of imitation include reciprocating, one-upping, joining in, aligning, mocking, arguing for equity, and participating in improvisational games. Many of children’s imitative acts in everyday life are partial and transformative. By this we mean that the imitator repeats some part or aspect of another person's initial action, but also adds or changes something in the repetition. We submit that the ways in which children imitate—e.g. whether and how they transform or adapt the repeated act—are shaped by the activities in which these imitative acts are embedded. For instance, we analyze one sequence that takes place in a preschool classroom in which a group of children (3-5 years old) are seated around a table. They engage in a spontaneous improvisational game that involves repeating and upgrading the previous utterances of other children (Transcript 1). We compare this to a sequence that occurs in the home of a three-year old girl (Marie), in which she and her five-year old brother prepare to engage in a

© ISLS

93

ICLS2012

Volume 2: Symposia

battle against their father, employing Star Wars ships they have built out of Legos (Transcript 2). We find that in the context of this activity, direct imitation with minimal transformation is a resource for the child in forming an alliance with her brother, in opposition to their father. Transcript 1: “I’m a tiger” Transcript 2: Forming an Alliance
5 6 7 8 9 Hey, what about this? What if I made a ship and you guys, I let you guys destroy it? Evan: Yeah:::! Marie: Yeah:::! Dad: Would that be fun? All right. … ((approx. 1.5 minutes while Dad builds his ship and repeatedly predicts that he will win the battle, which Marie and Evan contest.)) 49 Dad: I'm having a little trouble keeping my ship together. 50 Do you have any glue? Am I allowed to use glue? 51 Evan: No! 52 Marie: No! 53 Dad: Okay:: The door is opening. ((sound effect)) That'll be my driver. 54 Marie: Well [(I have your driver) ((chanting)) 55 Dad: [Oh. I know what I need. (.) This. The ultimate56 You will be fighting the ultimate iron giant. Ha ha ha. … ((approx. 40 seconds of talk about the ships and the battle)) 71 Evan: Dad, you only have three guns? 72 Dad: I don't know. You wait and see. There's a lot of surprises coming 73 for you mister. 74 Evan: Phuh huh huh 75 Marie: Ha ha ha. 76 Evan: Ha ha ha ha ha. 77 Dad: Ha ha ha? 78 Marie: Ha. 79 Evan: Blah blah blah. Your ship is really dumb. 80 Marie: Blah blah blah. [((makes "talking" gesture)) 81 Dad: [It's very very powerful. 82 Evan: Very very ugly. 83 Marie: [Very very ugly. 84 Dad: [You know why it's so powerful? Cuz you think it's weak but it's 85 actually ((engine sound, stands up and zooms ship through air)) The battle begins. Dad:

These naturally occurring sequences of talk and action do not provide evidence that learning has occurred, at least not in the form we are accustomed to finding it in reports of laboratory experiments or results of standardized tests. Rather, they provide a glimpse of the everyday activities through which children, over time, learn about their worlds and develop the interactional competence to participate in them (e.g. Schieffelin & Ochs, 1986). Furthermore, by examining these sequences closely, we can gain insight into how children might learn from these activities by identifying the specific opportunities for learning they provide. By analyzing the two instances described above, we find that the different forms of imitation occasioned by participation in these activities have different implications for learning. As Marie (Transcript 2) echoes her brother’s words and actions, she has the opportunity to “try on” new phrases and mannerisms, using her whole body to inhabit a role and enact its relationships with the roles taken on by her brother and father (ally and enemy). Though Marie mimics her brother quite closely, she seems to be taking up his actions and integrating them into her understanding of her own role and purpose, rather than simply “parroting” him – this can be seen in lines 79 and 80 where Marie directly imitates part of her brother’s utterance (“blah blah blah”) but “makes it her own” by adding an appropriate gesture (the fingers-to-thumb “talking hand” gesture). The learning opportunities for participants in the improvisational game in Transcript 1 are perhaps even more substantial. Designing a next move in the game requires parsing the syntax of previous moves and working out the overall structure and “rules” of the one-upping game. Children comment on one another’s moves, providing feedback on previous moves, as well as guidance and motivation for improvement of future moves. This extended play sequence provides a rich environment for building linguistic and interactional competence at lexical, syntactic, and discourse levels. Finally we note that in our data, a primary interactional environment for many types of imitation is the activity frame of play. Imitation is not just a move within these games, it can be the fundamental building block out of which these games are created. It is troubling to consider that the dominant social organizational conventions of formal schooling, with its emphasis on assessment of individual performance, may tend to shut down this powerful, child-driven, engine for social learning.

© ISLS

94

ICLS2012

Volume 2: Symposia

Block Building at Home and in Preschool
Danielle Keifert & Reed Stevens Prior research has shown that parents and teachers support children’s reasoning in different settings, including home, school, and museum (Callanan & Jipson, 2001; Callanan & Oakes, 1992; Crowley, et al., 2001; Hood & Bloom, 1979). While providing important insight into joint parent-child interactions, this work typically limited to one type of learning (parent-supported children’s learning) in single settings (designed science activities). Yet, the majority of day-to-day encounters with science phenomena likely occur independent of predetermined social arrangements and outside of designed settings (Crowley & Jacobs, 2002). What we need to better understand is how spontaneous interactional arrangements in multiple settings shape everyday practices that form the foundation of later STEM learning and how preschool children support each other’s learning during these practices. Through a comparative case study of two episodes of a preschooler building with blocks at home and preschool, this paper examines the interactional arrangements for a common activity in and across these settings. We adopt the lens of Newman, Griffin, and Cole (1989), who argue that the individual is not the correct unit of analysis to compare across settings, favoring instead the larger functional system within which tasks—or activities like building with blocks—are defined by the ways in which people orient to and accept different goals during interaction. Thus, while the comparison in this paper is of the same child building with blocks in two settings (home and preschool), the analysis of the differences in those episodes considers the interactional arrangements that constitute the system within which the task occurs (playing with Jenga blocks at home with mom nearby versus building a Bad-Guy block trap with peers at school). Following interaction analysis traditions, a turn-by-turn analysis provides the grounding for understanding both how the interactions and block-building activity unfold, as well as how the interactional arrangements shape that unfolding activity (Goodwin & Heritage, 1990; Heritage, 2008; McDermott, et al., 1978; Schegloff, 1972). The focus is on two episodes of four year-old Jamie building with blocks. At home, the interactional arrangement allows Jamie to self-direct his block-building activity with his mother nearby. The construction of the block structure co-emerges with an evolving narrative about the building’s function as verbalized by Jamie. He begins by narrating that he is “building a stack…a freight stack”, but quickly states that he is making “a building”. Jamie then encounters trouble with his building saying, “my building that’s kind of not staying up, Mommy” and his mother chimes in with “Well you know what I think the problem is sweetheart, is that your side pieces are a little bit taller than your stack in the middle.” Initially Jamie rejects this advice saying that if he switches the sidepieces to another orientation (so they are oriented like the blocks in the middle) “That, it would be too low!” When Mom suggests using more pieces he says he only wants to use three. By rejecting Mom’s advice Jamie remains the sole constructor of the block structure, so even when he later decides to rebuild following Mom’s advice, he still can claim ownership of the building. Throughout the 24 minutes of block play observed, Jamie’s story telling and building of his block-structure co-emerge, changing only when he, the sole constructor of both, chooses to make changes and based on Jamie’s own development as a problem-solver. In the school episode, Jamie participates in block building in an interactional arrangement consisting of several participating peers and building blocks. The video segment begins with Jamie at work with schoolmates building a “Bad-Guy Trap” (so named by the children). At school the block-building activity is also shaped by an overarching story, which the co-builders draw from cultural forms of storytelling (good guy versus bad guy, setting a trap for the bad guy) as well as a form of division of labor where each builder is responsible for a component of the building. This can be seen when a violation of the ownership of different parts of the structure occurs as when one child reaches over and takes blocks from Jamie’s area of work. Jamie quickly claims to the group “Noam‘s taking, was taking those o:::ff!” in a whiny tone. This form of policing of each other’s behavior is a form of maintaining the norms and rules established within the working group (McDermott, Gospondinoff, & Aron, 1978). The construction of both the story and the building structure also include opportunities for negotiation of roles and activity (“Let’s make half like that and half like that!”). Each of the components of the interactional arrangement (peers, story, available blocks, division of labor) contributes to the overall of the structure, which in turn shapes and is shaped by the co-authored story, all of which is must be negotiated. Across these two settings we see Jamie working on constructing both a story and a building structure made of blocks. However, the experience of “block building” for Jamie is very different when the larger functional system, the interactional arrangement, is considered. The interactional arrangements in each setting, particularly the sole ownership of the activity at home and the shared ownership of the activity with peers while at school, shapes the way in which story and building structure co-emerge. When building at home, Jamie’s talk focuses around his co-evolving story and building structure, and he is the sole decision maker regarding structural and narrative features. In this arrangements Jamie is able to focus on physical-spatial problem solving, choosing when, if at all, he takes his mother’s advice. However, when building at school, much of Jamie and his peers’ talk is about negotiating the story, the building structure, and access to materials. Jamie’s peers contribute significantly to the process making Jamie one of many contributors to specifying and constraining the activity. In fact, much of Jamie’s activity at school surrounds social negotiation with his co-builders. So although we

© ISLS

95

ICLS2012

Volume 2: Symposia

may say that Jamie is performing the same task of “building with blocks” in both settings, the nature of the task is strikingly different as a result of the differing interactional arrangements. The activity is effectively transformed by the nature of the interactional arrangement and through this transformation Jamie is exposed to very different forms of story and building construction, various problems solving techniques (physical and social), and how to effectively use the affordances of different arrangements to achieve his own goals. Recognizing the differences in opportunities for learning, problem solving, and negotiating available resources in differing interactional arrangements provides important considerations for researchers. As we look to better understand learning across multiple contexts (whether designed or non-designed), and how learning connects across these contexts to build foundations for STEM skills, we must consider the ways in which interactional arrangements shape these foundational experiences. Although in the comparison provided the experiences seem to provide complementary opportunities for Jamie to develop competence that may support later STEM learning, this may not be the case for all comparisons of similar activities. In continuing research, we will broaden our analysis of Jamie’s opportunities for learning in both home and school contexts, and consider how experiences may mutual enhance or inhibit the development of STEM reasoning across settings.

TV Viewing Practices and Children’s Reasoning
Pryce Davis & Reed Stevens Television is a controversial medium—criticized for wasting valuable time, promoting violence in children, and harming family relationships (Wartella & Robb, 2008). Despite possible negative effects, TV viewing dominates young people’s media consumption. From age two to age eleven, TV viewing among children increases from an average of one-and-a-half to five hours a day (Wartella, Richert, & Robb, 2010; Kaiser Family Foundation, 2010). If we accept the premise that television viewing is central aspect of preschool-age children’s lives, then uncovering the types of thinking and learning that accompany TV viewing should be a major subject of study. While many studies have focused on children’s learning from TV (e.g. Fisch, 2004; Uchikoshi, 2006; etc), these studies are interested in the aftereffects of TV viewing. Instead, we should try to understand what kinds of thinking happen[ during TV viewing. Rather than focusing on questions about whether kids watch too much TV (e.g. Condry, 1993), if TV viewing is positive or negative in and of itself (e.g. Zimmerman & Christakis, 2005), or how we can design interventions to reduce TV viewing (e.g. Dennison, et al., 2004), researchers should be asking: How do children make sense of what they see on television? How can we make visible and understand children’s television-mediated reasoning? In what contexts does reasoning occur? Only after we address these questions can we as educators encourage fruitful television watching or take advantage of the skills children develop while watching TV at home. In this study, we draw data from a corpus of videos and ethnographic field notes of children watching television and interacting with their families in their homes (see Dugan, Stevens, & Mehus, 2010). We analyze side-by-side synchronized recordings of kids watching TV and what they are viewing. This data, together with parental journal data about their kids’ everyday activities and viewing habits, forms our “in-room, in-show, inworld” framework. This framework allows us to accurately and completely as possible record children’s actual experience of watching and reasoning about television. For this paper we present a case study of two siblings—LeAnne (6) and Harrison (4). These siblings were selected as being generally representative of the data as a whole. They typically watch 4-5 hours of television per week. Their mother claims they are “interactive” and “talkative” television watchers, who often repeatedly watch the same episode. In this particular case study, we present these siblings viewing two different episodes of their favorite children’s science program (“Henry’s Amazing Animals”). The two vignettes represent the two relevant trends in the data. First, children often seem to passively view TV. Secondly, children do display active reasoning about TV under two particularly important practices: Co-viewing with adults and repeated viewing of single episodes. In our first vignette, the children watch an episode for the first time. While watching the show the children sit quietly by themselves and hardly take their eyes from the screen. There is scant activity or talk taking place in the room and little visible reasoning. Of course, the show is presenting a near constant stream of information about animal camouflage. Overall, this vignette reflects the stereotyped “couch potato” view of children watching television, and demonstrates none of the activity and reasoning claimed by the children’s mother. This isn’t to say that the children are not reasoning, but that their reasoning is invisible to researchers. In the second vignette, the children watch an episode that their mother says is one they have viewed many times as it is Harrison’s favorite. In this episode the kids talk almost constantly, both to each other and to the researcher in the room. Their talk reveals that the children are engaged in many types of reasoning. The children not only repeat information from the show, but also give information minutes before the show gives the same information. Furthermore, the show prompts them to share novel information as when LeAnne sees something on the screen that causes her to share how starfish eat (which is not presented in the show). In other instances, the children connect information from the show to their personal lives. For example, an image of the

© ISLS

96

ICLS2012

Volume 2: Symposia

ocean prompts LeAnne to share a personal story about a trip to the beach that gives her an emotional connection to the animals in the show and how she has researched the animals on her computer. At other times the kids build coherent explanations using causal inferences. In one case, the show talks about sunken ships making good homes for fish, “as long as they are not filled with oil” and LeAnne infers that fish cannot breathe oil. In other cases, the kids draw inferences to explain the structure of the show, the function of particular features of animals, and even the relationship between the show’s characters. Finally, the children share previous ideas about the show in order to generate new questions. Overall, children display complex reasoning practices while watching TV, but these typically only occur (or only become visible) during certain viewing practices and with certain kinds of social support. These practices are highlighted in the differences between the two vignettes. First of all, the children hadn’t seen the first episode before so the information may be novel to them; they stay silent because they are trying to comprehend what is being presented. However, they had seen the second episode multiple times and are familiar with the information. So, they are free to actively grapple with the information in order to construct their understanding. Little work exists on unprompted repeated viewing, although experimental work has found forced repeated viewing to increase comprehension without decreasing interest (Crawley, et al., 1999). Secondly, in the second vignette there is a more immediately present adult with the kids than in the first vignette. Previous research points to adult directed co-viewing as an important practice (Friedrich & Stein, 1975; Valkenburg, Krcmar, & de Roos, 1998). In the first vignette, there are two adults in the room with the children: The mom who is on the computer and the researcher manning the recording equipment, but they are both silent and out of view. So, the children do not interact with them. In the second vignette, the researcher is easily in the kids’ view. This researcher passively observes, and doesn’t ask questions or prompt the kids. However, the kids talk to him and treat him like a co-viewer. In a way, he is given the role of the student and the kids teach him about the animals on the show. So, unlike previous research on co-viewing the adult is not explicitly driving the kid’s attention, but is merely passive yet attentive to what the kids’ point out. To conclude, we return to the driving questions of this research. How do children make sense of what they see on television? How can make visible and understand children’s television-mediated reasoning? We contend that, although often passive, kids engage in complex reasoning while watching TV under certain circumstances. In fact, they display quite varied reasoning skills, from sharing novel knowledge to drawing causal inferences. In what contexts does reasoning occur? Certain TV viewing practices are more likely to support reasoning. Repeatedly viewing allows kids to continually engage with the same information, so they can actively construct knowledge about the show’s topics, which they take into their everyday lives. Co-viewing helps prompt reasoning, even when the adult is not explicitly driving the child’s attention, but is merely passive yet attentive to what the kids’ thinking. This work extends previous findings by demonstrating instances of child-directed co-viewing and repeated viewing, meaning that the child is requesting the repeated viewing and is the active agent in the co-viewing. Taken together, these findings give a conceptual foundation about the things we should attend to as we attempt to understand the role and effects of television in children’s lives.

References
Brown, R. (1968). The development of Wh- questions in child speech. Journal of Verbal Learning and Verbal Behavior, 7, 279-290. Callanan, M., & Jipson, J. (2001). Explanatory conversations & young children's developing scientific literacy In K. Crowley, C. Schunn & T. Okada (Eds.), Designing for science. Mahwah, NJ: Erlbaum. Callanan, M., & Oakes, L. (1992). Preschoolers' questions & parents' explanations: Causal thinking in everyday activity. Cognitive Development, 7, 213-233. Chouinard, M. (2007). Children's questions: a mechanism for cognitive development. Monographs of the Society for Research in Child Development, 72 (1), 1-129. Comstock, G., & Scharrer, E. (2007). Media and the American child. Burlington, MA: Academic Press. Condry, J. (1993). Thief of time, unfaithful servant: TV & the American child. Daedalus, 122(1), 259-278. Crawley, A., Anderson, D., Wilder, A., Williams, M., & Santomero, A. (1999). Effects of repeated exposures to a single episode of the television program Blue's Clues on the viewing behaviors & comprehension of preschool children. Journal of Educational Psychology, 91(4), 630-637. Crowley, K., Callanan, M., Jipson, J., Galco, J., Topping, K., & Shrager, J. (2001). Shared scientific thinking in everyday parent-child activity. Science Education, 85, 712-732. Crowley, K., & Jacobs, M. (2002). Building islands of expertise in everyday family activity. In G. Leinhardt, K. Crowley & K. Knutson (Eds.), Learning conversations in museums. Mahwah, NJ: Erlbaum. Davis, E. (1932). The Form and Function of Children's Questions. Child Development, 3(1), p57-74. Dennison B., Russo, T., Burdick P., & Jenkins, P. (2004). An intervention to reduce television viewing by preschool children. Archives of Pediatrics and Adolescent Medicine, 158(2), 170-176

© ISLS

97

ICLS2012

Volume 2: Symposia

Dugan, T., Stevens, R., & Mehus, S. (2010). From show, to room, to world: A Cross-Context Investigation of How Children Learn from Media Programming. Proceedings of the 9th international conference on Learning sciences (ICLS 2010), 992-999. Duveen, G. (2000). Piaget ethnographer: Qualitative methods in the study of culture & development. Social Science Information, 39(1), 79-97. Fisch, S. (2004). Children’s learning from educational TV: Sesame Street & beyond. Mahwah, NJ: Erlbaum. Frazier, B., Gelman, S., Wellman, H. (2009). Preschoolers’ search for explanatory information within adultchild conversation. Child Development, 80(6), 1592-1611. Frederich, L., & Stein, A. (1975). Prosocial television & young children: The effects of verbal labeling & role playing on learning and behavior. Child Development, 46(1), 27-38. Freed, A. (1994). The form & function of questions in informal dyadic conversation. Journal of Pragmatics, 21(6), 621-644. Gergely, G & Csibra, G. (2005). The social construction of the cultural mind: Imitative learning as a mechanism of human pedagogy. Interaction Studies, 6(3), 463–481. Goodwin, C., & Heritage, J. (1990). Conversation analysis. Annual Review of Anthropology, 19, 283-307. Heritage, J. (2008). Conversation analysis as social theory. In B. Turner (Ed.), The new Blackwell companion to social theory. Oxford: Blackwell. Hood, L., & Bloom, L. (1979). What, when, & how about why: A longitudinal study of early expressions of causality. Monographs of the Society for Research in Child Development, 44(6), 1-47. Kaiser Family Foundation. (2010). Generation M2: Media in the Lives of 8-to-18-Year-Olds. Menlo Park, CA: Kaiser. Kearsley, G. (1976). Questions & question asking in verbal discourse: A cross-disciplinary review. Journal of Psycholinguistic Research, 5(4), 355-375. McDermott, R., Gospondinoff, K., & Aron, J. (1978). Criteria for an ethnographically adequate description of concerted activities & their contexts. Semiotica, 24, 245-275. Meltzoff, A. (1988). Infant imitation and memory: Nine-month-olds in immediate and deferred tests. Child Development, 59, 217-225. Meltzoff, A., Kuhl, P., Movellan, J. & Sejnowski, T. (2009). Foundations for a new science of learning. Science, 325, 284-288). Meyer, W., & Shane. J. (1973). The form & function of children's questions. Journal of Genetic Psychology, 123(2), 285-296. Newman, D., Griffin, P., & Cole, M. (1989). The Construction Zone. NY: Cambridge. Rizzolatti, G., Fadiga, L., Fogassi, L. & Gallese, V. (2002). From mirror neurons to imitation: Facts & Speculations. In A. Meltzoff & W. Prinz (Eds.), The Imitative Mind. (pp. 246-266). Cambridge. Schegloff, E. (1972). Notes on a conversational practice: Formulating place. In D. Sudnow (Ed.), Studies in social interaction (pp. 75-119). NY: Free Press. Schieiffelin, B. & Ochs, E. (1986). Language socialization in two cultures. NY: Cambridge University Press. Stevens, R., Satwicz, T., & McCarthy, L. (2008). In game, in room, in world: Reconnecting video game play to the rest of kids’ lives. In K. Salen (Ed.), Ecology of games: MacArthur Foundation series on digital media and learning. Cambridge, MA: MIT. Uchikoshi, T. (2006). Early Reading in Bilingual Kindergartners: Can Educational Television Help? Scientific Studies of Reading, 10(1), 89-120. Tizard, B., Hughes, M., Carmichael, H., & Pinkerton, G. (1983). Children's questions and adults' answers. Journal of Child Psychology and Psychiatry, 24(2), 269-281. Tomasello, M. (1999). The cultural origins of human cognition. Cambridge: Cambridge University Press. Tyack, D., & Ingram, D. (1976). Children's production & comprehension of questions. Journal of Child Language, 4(2), 211-224. Valkenburg, M., Krcmar, M., & de Roos, S. (1998). The impact of a cultural children's program & adult mediation on children's knowledge & attitudes towards opera. J. of Broadcast. Elec. Media 42, 315-26. Van Hekken, S., & Roelofsen, W. (1981). More questions than answers: A study of question–answer sequences in a naturalistic setting. Journal of Child Language, 9(2), 445-460. Wartella, E., Richert, R. & Robb, M. (2010) Babies, Television & Videos: How Did We Get Here? Developmental Review, 30(2), 116-127. Wartella, E., & Robb, M. (2008). Historical & recurring concerns about children’s use of the mass media. In S. Calvert & B. Wilson (Eds.), The Handbook of Children, Media, & Development (pp. 7–26). Blackwell. Zimmerman, F., & Christakis, D. (2005). Children's television viewing & cognitive outcomes: A longitudinal analysis of national data. Archives of Pediatrics & Adolescent Medicine, 159(7), 619-625.

© ISLS

98

ICLS2012

Volume 2: Symposia

You’re It! Body, Action, and Object in STEM Learning
Dor Abrahamson (Chair), UC Berkeley, 4649 Tolman Hall, Berkeley, CA 94720-1670, dor@berkeley.edu Carmen J. Petrick & H. Taylor Martin, UT Austin, 1 University Station D5700, carmenpetrick@gmail.com David J. DeLiema, Noel Enyedy, Francis F. Sten, & Darin Hoyer, UC Los Angeles, ddeliema@gmail.com Mina C. Johnson–Glenberg, David Birchfield, Tatyana Koziupa, Caroline Savio-Ramos, & Julie Cruse, School of Arts, Media + Engineering, Arizona State U., Tempe, AZ, Mina.Johnson@asu.edu Robb Lindgren, Anthony Aakre, & J. Michael Moshell, University of Central Florida, Robb.Lindgren@ucf.edu Cameron L. Fadjo & John B. Black, Teachers College, Columbia, 525 W. 120 th St., NY, clf2110@columbia.edu Mike Eisenberg (Discussant), U. Colorado, Campus Box 430, Michael.Eisenberg@colorado.edu Abstract: In this special double symposium, sixteen established and emerging scholars from seven US universities, who share theoretical perspectives of grounded cognition, empirical contexts of design for STEM content domains, and analytic attention to nuances of multimodal expression, all gather to explore synergy and coherence across their diverging research questions, methodologies, and conclusions in light of the conference theme “Future of Learning.” Jointly we ask, What are the relations among embodiment, action, artifacts, and discourse in the development of mathematical, scientific, engineering, or computer-sciences concepts? The session offers emerging answers as well as implications for theory and practice. THERE was a child went forth every day; And the first object he look’d upon, that object he became; And that object became part of him for the day, or a certain part of the day, or for many years, or stretching cycles of years. (Walt Whitman)

Introduction: “You’re It!” Is More Than a Tagline
“You’re it!”, so mundane a playground exclamation, appears to capture much more of human experience than a game of tag. To each of us—16 established and emerging scholars from seven different universities across the US—“You’re it!” bears in profound and empirically substantiated ways on relations among action, embodiment, artifacts, reasoning, and discourse, as these relations pertain to developing competence in some STEM domain (Science, Technology, Engineering, Mathematics). Our distinct yet teaming ideas, we sense, should be gathered and shared among us and the larger learning-sciences community, because apparently these ideas collectively hone and point toward what might be the theoretical core and pedagogical promise of the embodiment approach. In this symposium, we attempt to shed light on the nature and dynamics of enactment, integration, and signification as naturalistic learning phases that can be recruited via pedagogical design for content instruction. Each study interprets cases of mediated interactions designed and facilitated with the objective of fostering knowledge (which we conceptualize and pin down from varying epistemological perspectives as professional perception, insight, models, skills, etc). Moreover, each presentation provides rich qualitative analyses of paradigmatic moments, in which study participants’ physical action—whether with, upon, or about objects that are either material, virtual, or imaginary—contributes to individual microgenesis of target subject matter. Yet more specifically, we are all interested in elaborating from a grounded-cognition perspective theoretical models pertaining to manipulation—whether actual, vicarious, or simulated—and how these operations contribute to learning. All the data discussed in this symposium were collected in instructional situations, writ large. Turning to instructional practice, we are also interested in determining and characterizing any unique pedagogical affordances of particular technology and interaction strategies vis-à-vis student cognition of focal content. Thus, in accord with the ICLS 2012 “The Future of Learning” theme, this symposium attempts to ground nextgeneration interaction design in established tenets of learning sciences theory. Our discussant, Mike Eisenberg, brings to this symposium his renowned expertise in cognitive science, computer science, mathematics, engineering, and integrated multimedia design for STEM content learning. In writing our individual sections, we chose to use a common format, so as to highlight our similarities and suggest our synergies. The sections present the proposed papers in their intended order of presentation. During the 120 min. session, these six individual papers (6 x 15 min.) will be followed by comments from our discussant (20 min.), and then we will engage in a general discussion with the audience (10 min.). To varying degrees, presenters will bring software, media, and artifacts from their respective research studies that will enhance post-symposium informal follow-up conversations with interested members of the audience.

© ISLS

99

ICLS2012

Volume 2: Symposia

You Made It! From Action to Object in Guided Embodied Interaction Design
Dor Abrahamson, Dragan Trninic, and José F. Gutiérrez, University of California at Berkeley
“Object” is supposed to mean....something that is either internally or externally present in a certain situation. Thus, not only external things like....wood blocks, or signs and persons, are objects, but it is also possible that a certain form of knowledge or a certain cognitive ability is the “object.” (Hoffmann, 2007, p. 189, original italics) What role might instructors play in scaffolding students’ generalizations from embodied interaction? Twentytwo Grade 4-6 students (ages 9-11) participated, either individually or in pairs, in a task-based, semi-structured, tutorial clinical interview. The intervention’s objective was to gather empirical data for a design-based research study investigating the emergence of conceptual knowledge from physical activity. Under the researcher’s guidance, participants engaged in an embodied-interaction problem-solving activity. Their task was to make a computer screen green by remote-controlling two virtual objects, one hand each. Unknown to them, the screen would be green only when their hands were at particular heights above the desk, relative to each other (see Figure 1). In Abrahamson, Trninic, Gutiérrez, Huth, and Lee (2011) we describe the following typical participation trajectory. Students first developed naïve qualitative strategies (e.g., “The higher you go, the bigger the distance”). Next, when we overlaid a virtual Cartesian grid on the screen, they used this mathematical resource to bootstrap an a-per-b form (e.g., “For every one unit I go up on the left, I got up two on the right”); and when we supplemented numerals, they determined a multiplicative relation (e.g., “The right hand is always double the left hand”). Here we report on a study of the tutor’s function in scaffolding these insights.

a. b. c. d. Figure 1. The Mathematical Imagery Trainer for Proportion (MIT-P) set at a 1:2 ratio, so that the right hand needs to be twice as high along the monitor as the left hand. In an empirically determined schematic interaction sequence, the student: (a) positions hands incorrectly (red feedback); (b) stumbles on a correct position (green); (c) raises hands maintaining constant distance between them (red); and (d) corrects position (green), infers rule. Compare 1b and 1d—note the different distances between the cursors. The following transcription from our empirical data is presented to illustrate the microgenesis of mathematical objects via guided, mediated embodied interaction. In particular, we attend to nuanced language features—sliding uses and substitution of pronouns—as indicators of how a dyad zigzags between embodied actions, relations, and rules, ultimately objectifying referents in a shared perceptuomotor field, all so as to repair “designed” practical and discursive vagueness (Abrahamson et al., 2009; Newman, Griffin, & Cole, 1989; Rowland, 1999). As per Hoffmann’s quotation in our motto, indeed our “object” is an externally present thing— an “invisible” distance between two points. This particular phenomenological object is of critical importance to learning via our design: noticing, controlling, and naming it is the first step to articulate a proto-ratio principle. Dor: Amira: D: A: D: A: D: A: D: A: D: A:
© ISLS

So what’s the rule? What makes it green? [= scaffolds reflection, requests generalization] Having this one [she indicates her right hand] be higher [...she indicates her left hand]. Hn’hn. Ok. So, any higher? [= implies a request for more specificity. Note: the grid is on.] ....about three squares higher. [= complies by offering quantitative relational locator] Three squares higher. Ok. So if you bring your hands down... [= launches generalization] I think it’s like, if it’s up here also... There’s like a few spaces where it’s.... In some spaces you have to be lower down. [= in some areas on the screen the interval is smaller] Aha. In some spaces you have to be lower down. [= echoes; implies positive valorization] Yeah. Right down here. But then when you go up here, you have to be higher. Ah! Ok, so some spaces you have to be down, but then when you go up there, it has to be... [= echoes, but switches “you” to “it”; elides “higher” to invite another, clearer descriptor] Yeah... A bigger distance. [= offers “distance” as an alternative completion of the assertion, and thus disambiguates that her “higher” had referred to the hands’ interval not elevation] A bigger distance. Ok. [= echoes; affirms with explicit positive valorization] Down here you only need about one. [“one” indicates absolute, not relational interval]
100

ICLS2012

Volume 2: Symposia

D: A:

Ok, so down there you only need about one. [echoes] .... As you keep going up, it has to be more..... Here it’s 1, ...2...3...4...5 [infers; applies]

We thus witness how goal-oriented interaction situated in discursive interaction “begets” a mathematical object. During the presentation, I will screen several samples of video footage, in which tutor–tutee dyads coconstruct mathematical referents using available semiotic means of objectification (Radford, 2003), including speech, gesture, gaze, and material–virtual resources. I will interpret those unique moments as culminating brief histories of localized discursive interaction around task-based pedagogical activity.

Learning Mathematics: You’re It vs. It’s It
Carmen J. Petrick and H. Taylor Martin, University of Texas at Austin
Children’s play is everywhere permeated by mimetic modes of behavior, and its realm is by no means limited to what one person can imitate in another. The child plays at being not only a shopkeeper or teacher but also as windmill and a train. Of what use to him is this schooling of his mimetic faculty? (Benjamin, 1986, p. 333) This presentation reports on part of a larger study comparing students learning geometry through either embodied or observational activities. There is evidence that embodying concepts is beneficial to learners (Abrahamson, 2004; Fadjo, Lu, & Black, 2009; Roth, 2001); yet, little is known about the differences in the learning process between students who physically enact concepts and students who do not. This study compares how high school students remembered an activity in which they explored the concept of ratio by either “being it” or “watching it.” By “being it,” we mean experiencing oneself as the mathematical object. In contrast, we refer to “watching it” as observing a mathematical object as being remote and separate from oneself. We use a theoretical framework of embodied cognition as we try to understand how being in a body and interacting with objects in the world can influence how students make meaning of mathematics (Anderson, 2003; Barsalou, 2008; Wilson, 2002). Being able to “get inside” a problem can be a helpful strategy for students learning mathematics (Wright, 2001). Embodiment is one way to help students do that (Noble, Nemirovsky, Wright, & Cornelia, 2001; Petrick, Berland, & Martin, 2011; Wilensky & Reisman, 2006). When a student becomes “it,” we hypothesize that he or she will have a very different kind of experience than a student who detachedly watches “it,” because embodiment promotes connections between physical actions and mathematics in a way that observing does not. In this study we examined students’ written responses to a survey asking them to write what they remembered about a learning activity, and we compared two groups of students: those who learned through embodiment (n = 69) and those who learned by observing (n = 59). We predicted that students in the embodied condition would remember more about the activity as a whole. Also, we predicted that students in the embodied condition would be more likely to write their responses in a first-person narrative, thus showing that they had realized they were it. In the larger study, fourteen classes of high school geometry students followed a two-week curriculum on similarity. Four teachers participated in the study, and half of each teachers’ classes were randomly assigned to either the embodied or observer condition. Students in the embodied condition participated in eight activities designed to promote direct embodiment of mathematical ideas, such as ratio and similar triangles. In these activities, the students became “it” by physically enacting the concepts. Students in the observer condition participated in eight activities that were very similar to the embodied condition, except these students observed or drew pictures of abstract symbols enacting the concepts rather than embodying them. Unit pre- and post-tests showed that students in the embodied condition had greater learning gains than students in the observer condition, and that these differences were found on the conceptual items of the test but not the procedural items (see Petrick & Martin, 2011). To look closer at how the students experienced the different types of activities, we administered a survey at the end of the unit asking students to write down everything they remembered about each of the eight activities. We have chosen to focus on the first activity of the unit, called Make the Screen Green, for this presentation. In the embodied condition, students interacted with a version of the Mathematical Imagery Trainer (“MIT”; Abrahamson & Howison, 2010), in which they used Wii remotes to control the heights of two blocks on a screen (see Figure 1, below). Students took turns moving the remotes to help them find the hidden rule that turned the screen from red to green, and every few minutes the teacher would ask students to use their hands to simulate the positions of the blocks that they thought would make the screen green. As they did this, each student’s partner would give feedback. Students in the observer condition did not manipulate the blocks or simulate the movements of the blocks with their hands. Instead, they watched a video showing the same blocks and screen that students in the embodied condition saw. Similarly, the screen would turn green whenever the ratio of the heights of the blocks was 2:1. Yet this video was a recording of other students’ interactions with the

© ISLS

101

ICLS2012

Volume 2: Symposia

MIT from an earlier pilot. Here, too, every few minutes the teacher would ask students to describe to their partner what made they thought made the screen turn green, and each student’s partner would offer feedback.

Figure 1. Classroom setup for “Make the Screen Green”: embodied condition (left) observer condition (right). After analyzing what all students wrote on their survey at the end of the unit, we found that students in the embodied condition wrote significantly more words overall in their response than students in the observer condition. Upon looking more deeply, we found that the embodied condition wrote significantly more mathematical details (e.g., mentioning a two-to-one relationship) than students in the observer condition, and they also wrote significantly more non-mathematical details (e.g., names of partners). In addition, we found that students in the embodied condition were significantly more likely to adopt first-person narratives, while students in the observer condition were significantly more likely to write from a third-person narrative. These differences appear even though there were no significant differences between conditions in student learning on procedural items on the unit pre- and post-tests. During the presentation, we will discuss further details about the two activities, information about the analyses, and implications of these results for the theory and practice of mathematics education.

Learning Science by Being You, Being It, Being Both
David J. DeLiema, Noel Enyedy, Francis F. Steen, and Darin Hoyer, U. of California at Los Angeles
When I observed phenomena in the laboratory that I did not understand, I would also ask questions as if interrogating myself: “Why would I do that if I were a virus or a cancer cell, or the immune system. (Salk, 1983, p. 7) This presentation reports on findings from an ongoing research project investigating the role of physical action, particularly gesture, in learning scientific content. It has been theorized that learning science involves the construction of mental models (cf. Frederiksen, White, & Gutwill, 1999; Gentner & Stevens, 1983; JohnsonLaird, 1983). The objective of my project is to revisit and potentially qualify these theories from the perspective of the rising paradigm of grounded cognition (Barsalou, 2010). The current study, by analyzing students’ gestures during the incipient moments of mental model construction, is a first step toward that goal. According to classical computationalism (Gentner & Stevens, 1983), modeling transpires entirely inside the head (Greca & Moreira, 2000). Yet this classical theory, which conceptualizes models as formed out of propositional mental symbols, bears the fundamental methodological disadvantage that these cognitive constructs cannot be seen or measured, such as when students study content (Rouse & Morris, 1986, p. 1). Regardless of whether or not these theories obtain, one methodological means of monitoring learning is asking people to represent their emerging understandings in speech and diagrams (Coll & Treagust, 2001; Harrison & Treagust, 1996; Niedderer & Goldberg, 1996). The present study looks to productions in another semiotic modality, hand gestures, as “physically instantiated mental models” (Schwartz & Black, 1996, p. 464; see also Clement & Steinberg, 2002; Crowder, 1996). A methodological advantage of studying gestures, in this respect, is their relatively unmediated presentation of thought process, as compared to verbally or diagrammatically instantiated mental models. For the current study, I selected the conceptual content of “packet switching,” the historical technological innovation that gave rise to the Internet. Study participants with no prior knowledge of packet switching were asked to perform interpretive gestures at the same time as they listened to verbal statements about this system. The term interpretive gesture refers to the student’s attempt to comprehend and then gesturally model the words of the conversation partner, as the discourse unfolds. Next, I had participants discuss packet switching more naturally (i.e. spontaneously using co-speech gestures). As I now elaborate, this non-

© ISLS

102

ICLS2012

Volume 2: Symposia

routine procedure offers a unique window onto the microevolution of a mental model. The following builds on intensive analysis of a case study consisting of a single participant (Cynthia [pseudonym], a college student). The analysis builds on the thesis that students reasoning about STEM phenomena naturally enact both Observer ViewPoints (O-VPT, seeing the phenomenon from outside) and first-person, anthropomorphized Character ViewPoints (C-VPT, “being” the phenomenon; see Nemirovsky & Monk, 2000; Ochs et al., 1996; Wilensky & Reisman, 2006). Indeed, the student in the present study routinely alternated between O-VPT of components and C-VPT as components. In several instances, the student coordinated static O-VPT representations of the component shape (i.e. box-like) with dynamical C-VPT representations of its actions (i.e. grasping or inscribing). (In the following example, the researcher’s speech is in inscribed in roman type, and Cynthia’s gestures, which follow each of the researcher’s speech turns, are in italics.) Each packet [left hand forms horizontal box shape and remains in place] grabs a few small pieces of the email message [right hand reaches out, plucks an imaginary small object from the air (see Figure 1] and stores those pieces inside itself [places the object “inside” the left hand—repeats three times].

Figure 1. Cynthia’s left hand is a packet; her right hand operates as a packet on itself. Cynthia’s left hand established narrative context by instantiating an O-VPT packet, even as the right hand, representing a “hand” of the same packet, acted upon it as a C-VPT element. By this token, “You’re it” means importing naturalistic interaction schemas into the inquiry process by literally being the phenomenon in question. Gestures, by concretizing your view when you dive in to be it, and concretizing its view when you dive out to observe it, support the dynamic coordination between the You and It viewpoints by blending into a single model traces of their respective allocentric and egocentric experiences. These same viewpoints for depictions of system events tended to reappear in Cynthia’s co-speech gestures during her later retelling of packet switching. During my presentation, I will show video clips of the learning interaction and provide moment-tomoment analyses of how Cynthia’s gestures build multiviewpoint models, establish context around action, and produce evolving representations, and I offer implications for the design of virtual STEM environments.

Seeing It versus Doing It: Lessons from Mixed Reality STEM Education
Mina Johnson–Glenberg, David Birchfield, Tatyana Koziupa, Caroline Savio-Ramos, and Julie Cruse School of Arts, Media + Engineering, and the Learning Sciences Institute, Arizona State University
Watching a child makes it obvious that the development of his mind comes about through his movements…. Mind and movement are parts of the same entity. (Maria Montessori, 1967) Mixed-reality embodied learning platforms are coming of age. We will present several studies that have demonstrated increased learning when students are randomly assigned to embodied, mixed-reality (MR) environments compared to learning in regular instruction environments, where teacher and content are held constant. The SMALLab (Situated Multimedia Arts Learning Lab) and Serious Games for Embodied Learning groups at ASU create and research content for K-12 education that is embedded in kinesthetic platforms. (See www.smallablearning.com for videos.) We explore the boundaries of environments that use the body as an interface for learning. The two most common platforms are rigid body motion- and skeletal-tracking cameras (e.g., Kinect). We co-design all lessons with classroom teachers to create content that engages the major sense

© ISLS

103

ICLS2012

Volume 2: Symposia

modalities (visual, auditory, and kinesthetic). SMALLab uses 12 infrared motion-tracking cameras to send to a computer information about where a student is in a 15 X 15 ft. floor-projected environment. Students step into the active space and hold a “wand” (a trackable object) that allows the physical body to function as a 3D cursor in the interactive space. Figure 1 shows two students using wands to manipulate elements in a chemistry lesson.

Figure 1. Two students adding molecules into a virtual flask in a chemistry titration scenario.

Figure 2. Comparative mean scores over time for chemistry scenario.

With turn-taking, entire classrooms with 30 students are able to physically experience a learning scenario within a typical class period. We contend that the more modalities (sensorimotor systems) are activated during the encoding of information, the crisper and more stable the knowledge representations will be in schematic storage. These crisper representations, with more modal associative overlap, will be more easily recalled. Better retrieval leads to better performance on assessment measures. If gestures are another modality— and they emerge from perceptual and motor simulations that underlie embodied cognition (Hostetter & Alibali, 2008)—then creating an embodied learning scenario that reifies the gestures should be a powerful teaching aid. Yet, it is not trivial to create “congruent” gestures that map to the lesson that is to be learned (we use congruent the way Segal, Black, Tversky, 2010, do). To this end, we will also present some design guidelines for creating meaningful embodied content and how to think about action as a method for deeper encoding. Several studies have been published supporting significantly larger learning gains when students are active in SMALLab versus regular instruction (Birchfield & Johnson–Glenberg, 2010; Johnson-Glenberg et al., 2009, Tolentino et al., 2009). We hypothesize the primary drivers for the learning changes are embodiment, collaboration, and novelty. In addition, the gains also may be mediated by increased peer-to-peer language use and gameplay. Results from several experiments will be presented, including the titration chemistry scenario. Our studies typically use a waitlist design and three invariant tests. Figure 2 shows that significant gains are observed each time the classes are assigned to the SMALLab condition. We have analyzed results from 200 students in a study designed to address the question of watching it versus being it. We label these two conditions as low vs. high embodiment. The two embodied levels are crossed with three learning platforms: SMALLab, an interactive whiteboard (IWB), or a desktop-and-mouse condition. Psychology undergraduates experience a onehour lesson on Centripetal Force. The research has led to several design principles intended to frame the realization of embodied learning experiences in computer-mediated environments (Birchfield, Johnson–Glenberg, Megowan–Romanowicz, Savvides, & Uysal, 2010). Specifically:  Direct Impact: Learners’ physical actions should have a direct, causal impact in the simulated environment;  Map to Function: A learner’s gesture should closely align with its function and role in the simulated environment (e.g., physical and simulated throwing gestures should align);  Human Scale: Computer interfaces should support movement on a human scale (e.g., degrees of freedom, size, and speed of a gesture);  Socio-Cultural Meaning: The communicative aspects of human presence and gesture should be accounted for (e.g., the cultural meaning of a gesture, the information conveyed by a gesture needs to be addressed).

© ISLS

104

ICLS2012

Volume 2: Symposia

You’re the Asteroid! Body-Based Metaphors in a Mixed Reality Simulation of Planetary Astronomy
Robb Lindgren, Anthony Aakre, and J. Michael Moshell, University of Central Florida
The brain’s sensorimotor representations of space “gain their coherence not by their subservience to some over-arching, mathematical definition of space but with respect to a repertoire of movement. (Arbib, 1991, p. 379, as quoted by Hagendoorn, 2012) In this paper we describe a specific approach to generating embodied learning, where users are embedded within a simulation and given the opportunity to learn the important relationships from the inside. A number of recent reports have highlighted the benefits of informal, simulation-based learning experience for science education (Bell et al., 2009; Honey & Hilton, 2011), but there has been fairly little specificity about how the interactions one has with these simulations affect learning. Building upon recent work where mixed reality (MR) environments (the merging of physical and virtual elements in interactive spaces) have been shown to have great potential for facilitating learning (e.g., Birchfield & Johnson-Glenberg, 2010; Hughes, Stapleton, Hughes, & Smith, 2005; Kirkley & Kirkley, 2005), we have developed an interaction approach we call “body-based metaphors.” Unlike the relational metaphors that drive certain kinds of knowledge construction described by Gentner (1988), body-based metaphors are functional metaphors where the source domain (S) functions (or is made to function) like the target domain (T). In the MR environment we have developed, learners enact functional metaphors by using their bodies to act out part of a simulation of planetary astronomy (see Figure 1). We believe that these body-based metaphors are particularly effective for young learners who may struggle with the structure mapping process associated with relational metaphors.

Figure 1. A middle school student uses their body to put an asteroid into orbit. The learning goal for this project is to develop intuitions about physics concepts related to planetary movement (orbits, gravity, etc.). Philosophers have argued that body activity serves as the basis of conceptual understanding (Gallagher, 2005; Johnson, 1987), and this may be especially true of understanding spatial relationships. The use of body activity to teach physics concepts has been met with mixed success previously, likely because Earth does not provide a “pure” environment for examining elements such as force. With MR, however, it is possible both to isolate these elements and connect bodily movement with the abstract representations (graphs, vector diagrams, etc.) that are typically used to convey knowledge of physics. We will describe data from research we have conducted to investigate whether the body-based metaphor approach of interacting with digital simulations has advantages over the traditional mouse-andkeyboard interface. To conduct this research we created a simulation game we call MEteor that can be run on both a 30-foot-by-10-foot interactive floor space and standard desktop computer. Participants work through a series of game levels that require a basic understanding of Newton’s and Kepler’s laws (e.g., hitting a target on the opposite side of a large planet). A separate display shows the participant their movement within a graph, allowing them to assess their predictions compared to the actual movement of objects as dictated by the laws of physics. We are applying a number of traditional metrics of learning (pre- and post- knowledge questions, questionnaires about science efficacy), but we are also utilizing a number of alternative measures to probe for effects that may be more commensurate with the modality of learning in this simulation. For example, in our preliminary research we found that participants’ sketches of the simulation included more dynamic elements

© ISLS

105

ICLS2012

Volume 2: Symposia

(arrows showing movement, etc.) and less surface features (textures, background objects) when using the MR simulation compared to the desktop simulation. A primary focus of this research is whether or not we can develop effective measures that specifically target embodied learning. We are interested, first, whether or not a learner’s experience and level of comfort with physical and “embodied activities” (e.g., sports, dance, 4H, girl scouts, etc.) predispose them to success with the type of interactive learning intervention we have developed. A recent paper on dance and spatial cognition provides a good example of how experiencing different and more types of movements creates a greater repertoire for understanding (Hagendoorn, 2012). To this end we are using a pre-questionnaire that surveys the participant’s experience in various physical activities and the types of things they do with their bodies to aid their thinking. A second measure we use is recording the degree to which a participant’s movements are consistent with the normative trajectories of simulation elements, and how these patterns of movement change over time. We hope to see, for example, that participants using the immersive MR simulation quickly adapt their movements to match how things actually move in space (e.g., slowing down when an orbiting asteroid is far away from the planet and speeding up when closer). Finally we observe the quantity and kind of gestures participants use both in the simulation and outside of it when explaining their reasoning about physics. Previous research has shown that gestures have a significant impact on cognition (Goldin-Meadow, 2003; Goldin-Meadow, Nusbaum, Kelly, & Wagner, 2001), and we are interested in whether a greater propensity to gesture is related to the other knowledge and performance measures being applied. We are using motion tracking to quantify the degree that participants are gesturing in the simulation and using video protocols to analyze gesture in pre- and post-interviews. These varied measures are allowing us to produce a more nuanced description and evaluation of embodied learning generally and body-based metaphors specifically.

You’re In the Game: Direct Embodiment and Computational Artifact Construction
Cameron L. Fadjo and John B. Black, Teachers College, Columbia University
Virtual characters have virtual minds and virtual bodies. They become the player’s surrogate mind and body. (Gee, 2010, p. 258) This presentation reports on findings from a field-based study investigating the learning of abstract computer science concepts and skills through physical action. Recent research in embodied and grounded cognition has examined the roles action, perception, and environment play in the teaching and learning of abstract concepts in mathematics (Abrahamson, 2010), science (Barab et al., 2007; Chan & Black, 2006), dance (Grafton & Cross, 2008), drama (Noice & Noice, 2006), and robotics (Lu, Kang, Huang, & Black, 2011; Petrick et al., 2011). The objective of this study is to examine if physical, or Direct (Fadjo & Black, 2011) and Imagined Embodiment (Fadjo & Black, 2011) during pedagogical activities improves the learning of abstract concepts in computer science. In particular, we are interested in examining if the embodiment of pre-defined dialogue-based scripts during classroom instruction improves the development of certain Computational Thinking skills and concept knowledge (Resnick & Brennan, 2011; Wing, 2006) during the construction of a computational artifact. This work is a first step toward defining a grounded embodied epistemology of pedagogy. Embodied (Glenberg, 2010) and grounded (Barsalou, 2010) perspectives of cognition have emerged over the past thirty years as the traditional Cartesian dualism view of cognition has been challenged (Gibbs, 2006). In particular, the problems of transduction (Barsalou, 1999) and grounding (Harnad, 1990; c.f. Pecher & Zwaan) have led cognitive scientists, cognitive psychologists, and philosophers to question the role the body plays in cognition. Further exploration of these problems has resulted in the emergence of two major developments in support of an ‘embodied’ cognition (Gibbs, 2006). The first development, dynamical systems theory, emerged from the fields of artificial intelligence and cognitive science as a way to define and explore the complex relationship between mind, body, and environment in cognition (Gibbs, 2006; Robbins & Aydede, 2009). The second development within embodied cognition comes primarily from work in the field of cognitive linguistics that emphasizes the importance of linguistic structures on “human conceptual knowledge, bodily experience, and the communicative functions of discourse” (Gibbs, 2006). Our study examines the development of Computational Thinking through an embodied approach to learning the language of computing. Recent studies by Glenberg and colleagues (2009, 2004) have demonstrated that an embodied approach to reading comprehension through physical and imagined manipulation is highly effective for learning abstract concepts in novel scenarios. This study extends previous work on embodied learning by immersing the learner in a scenario where she or he becomes the video game character. For the current study, we incorporated physical and imagined embodiment in the instruction of certain Computational Thinking concepts (conditional logic, loops, and parallelism) and skills (pattern recognition, abstraction, and decomposition) during the construction of video game artifacts using Scratch, a block-based, visual programming and design language developed at the MIT Media Lab (Resnick & Brennan, 2011). Sixth© ISLS 106

ICLS2012

Volume 2: Symposia

and seventh-grade students from a suburban public middle school with no prior programming experience were asked to use their body to physically enact pre-defined programming scripts during four 10-minute instructional sessions over a six-day period. Instruction began with a pair of instructors modeling the physical embodiment of the pre-defined scripts, while all students observed the teachers ‘being’ the characters. Then, depending on the group to which they had been randomly assigned, the students either physically embodied and imagined the same pre-defined scripts themselves (see Figure 1) or only imagined these interactions, without physical embodiment. By embodying the character’s actions and behaviors, the learners “became” the characters. So doing, they moved and interacted with one another in a traditional learning environment. At the same time, they created an imaged scenario that was informed by play schemas (Salen & Zimmerman, 2004). This scenario, in turn, was formed by previous game play experience and prior artifact construction (Fadjo & Black, 2011).

Figure 1. Direct Embodiment of a pre-defined Scratch Script. The learner is attempting to read the pre-defined script while physically enacting the sequence of actions and statements. In our presentation we will show split-screen video recordings of students becoming the character and engaging in the Direct Embodiment of pre-defined Scratch scripts with simultaneous tracking of sequential code structures, present findings from our recent study on using this grounded embodied approach to developing Computational Thinking and the effects these actions had on concept implementation, and discuss implications of Direct and Imagined Embodiment on the instruction of advanced computer science concepts and skills.

References
Abrahamson, D. (2004). Embodied spatial articulation: A gesture perspective on student negotiation between kinesthetic schemas and epistemic forms in learning mathematics. In D. E. McDougall & J. A. Ross (Eds.), Proceedings of the 26th Annual Meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (Vol. 2, pp. 791 – 797). Toronto, Ontario: Preney. Abrahamson, D. (2010). A tempest in a teapot is but a drop in the ocean: action-objects in analogical mathematical reasoning. In K. Gomez, L. Lyons, & J. Radinsky (Eds.), Learning in the Disciplines: Proceedings of the 9th International Conference of the Learning Sciences (ICLS 2010) (Vol. 1 [Full Papers], pp. 492-499). International Society of the Learning Sciences: Chicago IL. Abrahamson, D., Bryant, M. J., Gutiérrez, J. F., Mookerjee, A. V., Souchkova, D., & Thacker, I. E. (2009). Figuring it out: mathematical learning as guided semiotic disambiguation of useful yet initially entangled intuitions. In S. L. Swars, D. W. Stinson & S. Lemons-Smith (Eds.), Proceedings of the Thirty-First Annual Meeting of the North-American Chapter of the International Group for the Psychology of Mathematics Education (Vol. 5, pp. 662-670). Atlanta, GA: Georgia State University. Abrahamson, D., & Howison, M. (2010, April). Embodied artifacts: coordinated action as an object-to-thinkwith. Paper presented at the annual meeting of the American Educational Research Association. Abrahamson, D., Trninic, D., Gutiérrez, J. F., Huth, J., & Lee, R. G. (2011). Hooks and shifts: a dialectical study of mediated discovery. Technology, Knowledge, and Learning, 16(1), 55-85. Anderson, M. L. (2003). Embodied cognition: a field guide. Artificial Intelligence, 149, 91-130. Arbib, M. A. (1991). Interaction of multiple representations of space in the brain. In J. Paillard (Ed.), Brain and space (pp. 379-403). Oxford: Oxford University Press. Bakker, A., & Hoffmann, M. H. G. (2005). Diagrammatic reasoning as the basis for developing concepts: A semiotic analysis of students’ learning about statistical distribution. Educational Studies in Mathematics, 60(3), 333-358. Barab, S., Zuiker, S., Warren, S., Hickey, D., Ingram-Goble, A., Kwon, E.-J., et al. (2007). Situationally embodied curriculum: Relating formalisms and contexts. Science Education, 91, 750-782. Barsalou, L.W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22, 577-660. Barsalou, L. (2008). Grounded cognition. Annual Reviews Psychology, 59, 617-645. Barsalou, L.W. (2010). Grounded cognition: past, present, and future. Topics in Cog. Science, 2(4), 716-724. Bell, P., Lewenstein, B., Shouse, A., & Feder, M. (Eds.). (2009). Learning science in informal environments: people, places, and pursuits. Washington, DC: National Academy Press.
© ISLS 107

ICLS2012

Volume 2: Symposia

Benjamin, W. (1986). Reflections: essays, aphorisms, autobiographical writings (E. Jephcott, Trans.). New York: Schocken. Birchfield, D., & Johnson-Glenberg, M. C. (2010). A next gen interface for embodied learning: SMALLab and the geological layer cake. International J. of Gaming and Computer-mediated Simulation, 2(1) 49-58. Chan, M. S., & Black, J. B. (2006). Direct-manipulation animation: incorporating the haptic channel in the learning process to support middle school students in science learning and mental model acquisition. In S. Barab, K. Hay, & D. Hickey (Eds.), Proceedings of the 7th International Conference of the Learning Sciences (pp. 64-70). Mahwah, NJ: LEA. Clement, J. J., & Steinberg, M. S. (2002). Step-wise evolution of mental models of electric circuits: a “learningaloud” case study. The Journal of the Learning Sciences, 11(4), 389-452. Coll, R. K., & Treagust, D. F. (2001). Learners’ mental models of chemical bonding. Research in Science Education, 31, 357-382. Crowder, E. M. (1996). Gestures at work in sense-making science talk. The Journal of the Learning Sciences, 5(3), 173-208. Fadjo, C., Lu, M., & Black, J. B. (2009). Instructional embodiment and video game programming in an after school program. Paper presented at the World Conference on Educational Multimedia, Hypermedia and Telecommunications, Chesapeake, VA. Fadjo, C. L., & Black, J. B. (2011). A grounded embodied approach to the instruction of computational thinking. In Proceedings of the 42nd ACM Technical Symposium on Computer Science Education. New York: ACM. Frederiksen, J. R., White, B. Y., & Gutwill, J. (1999). Dynamic mental models in learning science: Journal of research in science teaching, 36(7), 806-836. Gallagher, S. (2005). How the body shapes the mind. Oxford University Press, Oxford, UK. Gee, J. P. (2008). Video games and embodiment. Games and Culture, 3(3-4), 253–263. Gentner, D. (1988). Metaphor as structure mapping: The relational shift. Child Development, 59, 47-59. Gentner, D., & Stevens, A. L. (1983) Mental models. Mahwah, NJ: Lawrence Erlbaum. Gibbs, R.W. (2005). Embodiment and cognitive science. New York: Cambridge University Press. Glenberg, A. M. (2010). Embodiment as a unifying perspective for psychology. Wiley Interdisciplinary Reviews: Cognitive Science, 1, 586–596. Glenberg, A. M., Goldberg, A. B., & Zhu, X. (2011). Improving early reading comprehension using embodied CAI. Instructional Science, 39(1), 1–13. Glenberg, A. M., Gutierrez, T., Levin, J. R., Japuntich, S., & Kaschak, M. P. (2004). Activity and imagined activity can enhance young children’s reading comprehension. Journal of Educational Psychology, 96(3), 424–436. Goldin-Meadow, S. (2003). Hearing gesture: how our hands help us think. Cambridge, MA: Harvard University Press. Goldin-Meadow, S., Nusbaum, H., Kelly, S., & Wagner, S. (2001). Explaining math: gesturing lightens the load. Psychological Science, 12, 516-522. Grafton, S., & Cross, E. (2008). Dance and the brain. The Dana Foundation. Retrieved October 23, 2009 from http://www.dana.org/printerfriendly.aspx?id=10744 Greca, I. M., & Moreira, A. (2000). Mental models, conceptual models, and modelling. International Journal of Science Education, 22(1), 1-11. Hagendoorn, I. (2012). Inscribing the body, exscribing space. Phenomenology and the Cognitive Sciences, 11(1), 69-78. Harnad, S. (1990). The symbol grounding problem. Physica D, 42, 335–346. Harrison, A. G., Treagust, D. F. (1996). Secondary students’ mental models of atoms and molecules: implications for teaching chemistry. Science Education, 80(5), 509-534. Hoffmann, M. H. G. (2007). Learning from people, things, and signs. Studies in Philosophy and Education, 26, 185-204. Honey, M. A., & Hilton, M. (Eds.). (2011). Learning science through computer games and simulations. Washington DC: National Academies Press. Hostetter, A. B., & Alibali, M. W. (2008). Visible embodiment: gestures as simulated action: Psychonomic Bulletin & Review, 15, 495-514. Hughes, C. E., Stapleton, C. B., Hughes, D. E., & Smith, E. (2005). Mixed reality in education, entertainment and training: An interdisciplinary approach. IEEE Computer Graphics & Applications, 26(6), 24-30. Johnson-Laird, P. N. (1983). Mental models. Cambridge, MA: Harvard University Press. Johnson, M. (1987). The body in the mind: The bodily basis of meaning, imagination, and reason. Chicago: University of Chicago Press.

© ISLS

108

ICLS2012

Volume 2: Symposia

Johnson-Glenberg, M. C., Birchfield, D., Megowan-Romanowicz, C., Tolentino, L., & Martinez, C. (2009) Embodied Games, Next Gen Interfaces, and Assessment of High School Physics, International Journal of Learning and Media,1(2). Access http://ijlm.net/knowinganddoing/10.1162/ijlm.2009.0017 Kirkley, S. and Kirkley, J. (2005). Creating next generation blended learning environments using mixed reality, video games and simulations. TechTrends, 49(3), 42-89. Lu, C. M., Kang, S., Huang, S., & Black, J. B. (2011). Building student understanding and interest in science through embodied experiences with LEGO Robotics. Educational Multimedia, Hypermedia, and Telecommunications. Charlottesville, VA: Association for Advancement of Computing in Education. McNeill, D. (1992). Hand and mind: what gestures reveal about thought. Chicago: University of Chicago Press. Nemirovsky, R., & Monk, S. (2000). “If you look at it the other way . . .”: An exploration into the nature of symbolizing. In P. Cobb, E. Yackel, & K. McClain (Eds.), Symbolizing and communicating in mathematics classrooms: perspectives on discourse, tools, and instructional design (pp. 177–221). Mahwah, NJ: Lawrence Erlbaum. Newman, D., Griffin, P., & Cole, M. (1989). The construction zone: working for cognitive change in school. New York: Cambridge University Press. Niedderer, H., & Goldberg, F. (1996). Learning processes in electric circuits. Paper presented at the annual meeting of the National Association for Research in Science Teaching, St. Louis, MO. Noble, T., Nemirovsky, R., Wright, T., & Tierney, C. (2001). Experiencing change: the mathematics of change in multiple environments. Journal for Research in Mathematics Education, 32(1), 85-108. Noice, H., & Noice, T. (2006). What studies of actors and acting can tell us about memory and cognitive functioning. Current Directions in Psychological Science, 15(1), 14-18. Ochs, E., Gonzalez, P., & Jacoby, S. (1996). When I come down, I'm in a domain state: grammar and graphic representation in the interpretive activity of physics. In E. Ochs, E. A. Schegloff, & S. Thompson (Eds.), Interaction and grammar (pp. 328-369). Cambridge, UK: Cambridge University Press. Pecher, D., & Zwaan, R. A. (2010). Grounding cognition: the role of perception and action in memory, language, and thinking. New York: Cambridge University Press. Petrick, C., Berland, M., & Martin, T. (2011). Allocentrism and computational thinking. In G. Stahl, H. Spada, & N. Miyake (Eds.), Connecting computer-supported collaborative learning to policy and practice: CSCL2011 Conference Proceedings (Vol. 2, pp. 666-670). Hong Kong: ISLS. Petrick, C., & Martin, T. (2011). Every body move: learning mathematics through embodied actions. Manuscript in progress (copy on file with author). Radford, L. (2003). Gestures, speech, and the sprouting of signs: a semiotic-cultural approach to students' types of generalization. Mathematical Thinking and Learning, 5(1), 37-70. Resnick, M., & Brennan, K. (2011, January 24). Four questions about Scratch. ScratchEd Webinar Series. Robbins, P., & Aydede, M. (Eds.) (2009). The Cambridge handbook of situated cognition. New York: Cambridge University Press. Roth, W.-M. (2001). Gestures: their role in teaching and learning. Review of Educational Research, 71(3), 365392. Rowland, T. (1999). Pronouns in mathematics talk: power, vagueness and generalisation. For the Learning of Mathematics, 19(2), 19-26. Rouse, W. B., & Morris, N. M. (1986). On looking into the black box: prospects and limits in the search for mental models. Psychological Bulletin, 100, 349–363. Salen, K., & Zimmerman, E. (2005). Rules of play: game design fundamentals. Cambridge, MA: MIT Press. Salk, J. (1983). Anatomy of reality: merging of intuition and reason. New York: Columbia University Press. Schwartz, D. L. & Black, J. B. (1996). Shuttling between depictive models and abstract rules: induction and fall-back. Cognitive Science, 20, 457–497. Segal, A., Black, J. & Tversky, B. (2010, November). Do gestural interfaces promote learning? Embodied interaction: Congruent gestures promote performance in math. Poster presented at the annual meeting of the Psychonomics Society, St. Louis, MI. Tolentino, L., Birchfield, D., Megowan-Romanowicz, C., Johnson-Glenberg, M. C., Kelliher, A., & Martinez, C. (2009). Teaching and learning in the mixed-reality science classroom. Journal of Science Education and Technology. 18, 6, 501-517. DOI: 10.1007/s10956-009-916. Wilensky, U., & Reisman, K. (2006). Thinking like a wolf, a sheep or a firefly: learning biology through constructing and testing computational theories—an embodied modeling approach. Cognition & Instruction, 24(2), 171-209. Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin & Review, 9(4), 625-636. Wing, J. (2006). Computational thinking. Communications of the ACM, 49(3), 33-35. Wright, T. (2001). Karen in motion: the role of physical enactment in developing an understanding of distance, time, and speed. Journal of Mathematical Behavior, 20, 145-162.

© ISLS

109

ICLS2012

Volume 2: Symposia

The Future of Assessment: Measuring Science Reasoning and Inquiry Skills Using Simulations and Immersive Environments
Jodi L. Davenport, WestEd, Oakland, CA USA, jdavenp@wested.org Edys S. Quellmalz, WestEd, Redwood City, CA USA, equellm@wested.org Jody Clarke-Midura, Harvard University, Cambridge, MA USA, jec294@mail.harvard.edu Chris Dede, Harvard University, Cambridge, MA USA, chris_dede@harvard.edu Janice D. Gobert, Worcester Polytechnic Institute, Worcester, MA USA, jgobert@wpi.edu Kenneth R. Koedinger, Carnegie Mellon University, Pittsburgh, PA USA, koedinger@cs.cmu.edu Marty McCall, Smarter Balanced Assessment Consortium, Portland, OR, mccall.marty@gmail.com Michael J. Timms, Australian Council for Educational Research, Melbourne, Australia, mtimms9@gmail.com Abstract: Simulations and immersive environments provide innovative ways to measure students’ science reasoning and inquiry skills. These computer-based assessments allow for dynamic displays of science systems that expand how phenomena, information, and data can be represented; they also allow for interactivity that provides new ways for learners to demonstrate their knowledge and skills. A number of groups have been working to create and evaluate next-generation assessments that both evaluate students on 21st Century scientific skills and provide evidence models for making inferences about student proficiency. In this symposium, researchers who are currently developing and testing simulation-based and immersive assessments to meaningfully assess science content and inquiry skills will share findings from classroom-based studies of students using the assessments. The presentations will be followed by a discussion from James Pellegrino, an expert in assessment design.

Symposium Objectives
Simulations and immersive environments provide innovative ways to measure students’ science reasoning and inquiry skills. However, do assessments using these environments provide more information than traditional tests about student proficiency of science reasoning and inquiry skills? Whereas factual knowledge can be easily assessed with traditional paper-and-pencil tests, more complex scientific reasoning and inquiry skills (e.g., systems thinking, designing investigations, gathering evidence, explaining observations) are more difficult to measure with static assessments. Dynamic and interactive assessment designs expand how phenomena, information, and data can be represented and increase the number of ways learners can show their knowledge and skills. As the field starts to integrate technology-based assessments, the challenges for K-12 educators and assessment developers are to create assessment tasks that allow students to demonstrate 21st century scientific skills and to create evidence models for making inferences about student progress and proficiency. We want to move beyond simply putting multiple-choice questions about declarative knowledge online. In this symposium, we bring together researchers who have developed simulation-based and immersive assessments to meaningfully assess science content and inquiry skills. The researchers will share both the designs of these technology-based science assessments using the conceptual framework of evidence-centered design and the findings from studies using the assessments with students. The goals of the current session are not only to describe next-generation assessments, but also to provide principled frameworks for their design, use, and evaluation.

Symposium Overview
The three papers in this session present research findings on innovative, computer-based assessments that are designed to measure scientific reasoning and inquiry skills. The papers report classroom-based findings that demonstrate novel ways to use student actions in open-ended environments to evaluate student proficiency. In the first paper, Gobert and Koedinger present data from a study of the innovative, simulation-based learning environment, Science Assistments (www.scienceassistments.org). The Science Assistments platform records each “move” as students engage in inquiry practices. The system then uses model tracing to evaluate the actions that students performed to determine what students know about science inquiry. As students create hypotheses and design experiments, the system updates a model that estimates student proficiency on inquiry and reasoning skills. The Science Assistments system leverages the affordances of model-tracing algorithms to detect different patterns of student behaviors, including genuine discovery and confirmation bias. In the second paper, Clarke-Midura, McCall, and Dede investigate the use of an immersive environment as a platform for assessing student inquiry and reasoning skills. As students move their avatars through a 3-D world, they are able to make observations, gather and analyze data, and draw conclusions. Clarke-Midura, McCall, and Dede found that the performance assessment using the immersive environment was

© ISLS

110

ICLS2012

Volume 2: Symposia

a reliable measure of inquiry and reasoning skills, though only a small number of students in the classroom studies demonstrated high-level inquiry skills. Finally, in the third paper, Davenport, Quellmalz, and Timms report the results of an empirical study to determine whether active, animations and interactive, simulation-based assessments are better than static assessments at distinguishing factual knowledge of scientific principles from meaningful inquiry and reasoning skills (e.g., generating predictions from observations, designing experiments, and drawing conclusions). A pool of 1566 students participated in a within-subjects design in their science classrooms. Students took science assessments in each of three different modalities; static, active (using dynamic animations), and interactive (using simulations). The results suggest that the dynamic and interactive assessments were more effective than the static assessment (most similar to traditional, paper-based tests) at distinguishing declarative, factual knowledge from deeper scientific reasoning and inquiry skills. After the presentations, James Pellegrino, an expert on technology and assessment, will provide a brief discussion and lead the question and answer session.

Significance
This symposium will bring together a variety of perspectives on the principled design and evaluation of nextgeneration assessments using simulation-based and immersive technologies. The symposium will create an opportunity for a broader discussion of the theoretical and practical considerations for leveraging emerging technologies to meaningfully assess complex science knowledge and inquiry practices.

Using Model-tracing to Conduct Performance Assessment of Students’ Science Inquiry Skill at Conducting Experiments Within a Microworld
Janice D. Gobert, Worcester Polytechnic Institute & Kenneth R. Koedinger, Carnegie-Mellon University

Introduction
Many national frameworks for science emphasize inquiry skills (e.g., NRC, 1996). However, in typical classroom practice, science instruction often focuses on rote learning in part because science process skills are difficult to assess (Fadel, Honey, & Pasnick, 2007) and rote knowledge is prioritized on high-stakes tests. Short answer assessments of inquiry have been used (cf., Alonzo & Aschbacher, 2004; Songer, 2006), however, these tend to not align well to current national frameworks (Quellmalz, Kreikemeier, DeBarger, & Haertel, 2006) and it is unclear whether they properly identify inquiry skills (Black, 1999; Pellegrino, 2001). Hands-on performance assessments are more authentic (Baxter and Shavelson 1994; Ruiz-Primo & Shavelson, 1996), however, these are seldom used in schools because of difficulty with reliable administration and the resulting high cost. Science Assistments (www.scienceassistments.org) learning environment assists and assesses (hence, “assistments”) middle school students on inquiry so teachers can assess their students’ skills during instruction-in the context in which they are developing (Mislevy et al, 2002).

Framework
As a proof of concept for automated assessment of scientific inquiry skills, we used model-tracing (Corbett & Anderson, 1995; Koedinger & Corbett, 2006) to develop a cognitive model of science inquiry skills, particularly, the control for variables strategy (Chen & Klahr, 1999) and warranting claims with data. This model provides a rich qualitative, process-oriented scoring of students’ inquiry “moves” within a guided scientific inquiry simulation for the domain of state change. We address the validity of this automated approach to performance assessment both quantitatively, in terms of reliability and predictive validity, and qualitatively, in terms of providing rich traces of student inquiry steps and “mis-steps” or haphazard inquiry (Buckley, Gobert et al, 2010). Additionally, we present Cronbach’s alphas as reliability measures for each of our variables, and correlations with other inquiry tasks as additional construct validity data.

Methods and Data Sources
Participants. 78 eighth grade students (aged 12-14 years) from a public school in Central MA participated. Students belonged to one of six class sections and had one of two science teachers. Materials. Pre- and post-tests for inquiry skills (n=12) and domain knowledge (n=7) were used. A Phase Change Microworld activity was used with which students engaged in a series of inquiry tasks. Data Collection and Analysis: By applying model-tracing to students’ log data from their interactions with microworlds, we use production rules to code for: 1) CVS-relevance for each of the trials (using the control for variables strategy and relevant to the student’s hypothesis), 2) tested-and-true hypotheses for each of the trials

© ISLS

111

ICLS2012

Volume 2: Symposia

(whether a claim is supported based on data), and 3) lastly, an average of these scores, referred to as %cvs+truetested for each of the trials. Our model tracer tracked whether: students’ initial hypotheses were scientifically accurate, whether the experimental trials they ran were relevant to their hypotheses, whether their trials used the control for variables strategy (Chen & Klahr, 1999), whether their final analysis entered was supported or unsupported by their data, and whether they had collected appropriate experimental evidence that supported their final conclusion (relevant controlled trials). Using data from the model-tracer, we calculated Cronbach’s alpha for our variables to ascertain the reliability across the 4 trials on each of the measures. The Cronbach’s alpha for the 4 CVS-relevant scores was 0.683; the Cronbach’s alpha for the 4 true-tested scores was 0.741; and lastly, the Cronbach’s alpha for the aggregate of the 2 inquiry scores across the 4 trials, %CVS+true-tested, was 0.774, indicating a high degree of internal consistency for each of the three measures. Correlations were calculated between our auto-scored performance measures of inquiry with specific post-test inquiry items that should, in theory, be related. We obtained moderate correlations between our performance measures of inquiry and our post-test items for identifying an independent variable, identifying a dependent variable, and demonstrating the control of variables strategy (CVS).

Findings
In this paper we have shown that we can use model-tracing as a method of performance assessment for science inquiry skills, an ill defined domain. This builds upon the extensive work that has been done to date for welldefined domains such as math (Corbett & Anderson, 1995; Koedinger & Corbett, 2006). Additionally: 1) the reliability of our machine-scored measures of inquiry are highly consistent across the 4 Assistment activities or “trials,” suggesting that we can reliably capture students’ inquiry performance on these rich inquiry tasks, and 2) our measures are moderately correlated with post-test measures of inquiry performance for analogous concepts. Lastly, our data show that model-tracing can detect interesting patterns of student inquiry such as confirmation bias and overcoming confirmation basis. These are important data with respect to demonstrating auto-scoring of rich inquiry behaviors, but are also important, particularly the former, in terms of its implications for adaptive scaffolding of student inquiry, such as that being done by the Science Assistments group (www.scienceassistments.org; Gobert et al, 2007, 2009; Sao Pedro et al, in press).

Significance
This work makes contribution to theoretical understanding of scientific inquiry, to its assessment, and to technical methods to auto-score inquiry. This represents an advance in this area since to date there has been difficulty in separating inquiry from the domain-specific context in which it was learned (Mislevy et al., 2002; Gobert, Pallant, & Daniels, 2010), and difficulty measuring inquiry skills due to their complexity and the amount of data required for reliable measurement (Shavelson et al, 1999).

Assessing Science Inquiry using Immersive Virtual Environments
Jody Clarke-Midura, Marty McCall, & Chris Dede, Harvard Graduate School of Education

Introduction
Scientific inquiry is the method by which scientists interact with and study the world. While detailed definitions of inquiry can be complex, at its core the process is hypothesized to involve theorizing and investigating. For example, Kuhn and colleagues define inquiry learning as investigations where students individually or collectively investigate a set of phenomena (virtual or real) and draw conclusions about it (Kuhn, Black, Keselman, & Kaplan, 2000). Similarly, White, Collins, & Frederiksen (2011) offer a definition of inquiry as a process that oscillates between theory and evidence within the practice of argumentation. Given the multifaceted and open-ended nature of inquiry, it is not surprising that research has found existing methods for assessing science inquiry learning to be limited (Quellmalz, Kreikemeier, DeBarger, & Haertel, 2006; USDE, 2010). Traditional science assessments either fail to align with the active nature of inquiry (Quellmalz et al. 2006) or have difficulty distinguishing what part of the complex reasoning process students do not understand (Gotwals & Songer, 2010). This is partly due to the difficulty of capturing students’ actions and behaviors as they perform a paper-based or hands-on task. Digital media, such as virtual environments and simulations, allow us to create tasks that are more characteristic of how students engage with inquiry in the real world; these processes and trajectories are unobtrusively captured as log data (Clarke, 2009). In doing so, these techologies allow us to create observations of student learning not possible via hands-on, paper-based, and multiple choice assessments (Clarke-Midura & Dede, 2010). In this paper, we discuss how virtual performance assessments can provide reliable observations of students’ inquiry knowledge. These virtual performance assessments are delivered via

© ISLS

112

ICLS2012

Volume 2: Symposia

an immersive environment and require students to solve an authentic scientific problem in a virtual context (see vpa.gse.harvard.edu).

Framework
We used the Evidence Centered Design framework (ECD; Mislevy, Steinberg, & Almond, 2003) to develop our assessments. ECD “provides a framework for developing assessment tasks that elicit evidence (scores) that bears directly on the claims that one wants to make about what a student knows and can do” (Shute, et al., 2007, p. 6). Using this framework, we reframed our science inquiry constructs into specific knowledge, skills and abilities (KSAs) aligned with national frameworks. We then converted the KSAs into proficiencies: data gathering, evaluating evidence, experimenting, and reasoning from evidence. Through the process of articulating the exact details of what is being measured and how it is being measured, it is easy to link the KSAs to evidence of student learning. Linking KSAs like this provides a high degree of validity that research has found often lacking in performance assessments (e.g. Linn, Baker, & Dunbar, 1991).

Methods and Data Sources
In the virtual performance assessments, students have the ability to walk around the virtual environment, make observations, gather data, and run tests in a laboratory, in order to solve a scientific problem (see figures 1 and 2 below). They then build a claim and support it with evidence they gathered.

Figures 1 and 2: Screenshots of the Virtual Assessments. As mentioned above, we compiled our inquiry skills into proficiencies: data gathering, evaluating evidence, experimenting, and reasoning from evidence (making claims). Observations of students demonstrating these skills as they participated in the assessment were compiled into groupings. Unlike traditional tests that characterize student performance in terms of response correctness, the interactive environment records whether or not students engaged in an activity. Students have varying levels of understanding about the scientific process. They may, for example, know that scientists collect evidence and make observations, but may not understand the role of control data or how to interpret observations in the light of previous research. The pattern of engagement in the interactive environment and the examinees’ final demonstration of evidence and reasoning from this evidence provide information about understanding of the scientific process. In order to assess the reliability of these observations as evidence of a students’ proficiency, we asked the following research question: Are observations of student’s skills (i.e. reasoning from evidence) behaving as if they are governed by a coherent skill? This study involved 643 middle school students in two states in the US (females=341). In order to answer the research question, we used Item Response Theory (IRT). IRT is a type of latent trait analysis that can be viewed as non-linear factor analysis. A trait is any skill or ability that determines the likelihood of a specific response to a test question. Observations generated from the log data were recoded from raw data as degrees of correctness (no, partial, or full credit for demonstrating the skill). Both the ability of the examinee and the difficulty of observation levels are on the same scale. Data were analyzed using WINSTEPS (Linacre, 2003) with a partial credit one-parameter model.

Findings
IRT analysis found the observations were providing evidence of a solid trait. Due to limited space, this paper presents results on students’ ability to “reason from evidence.” Results showed moderate fit statistics (Table 1). The item-total correlation coefficients are very high—they should be over .25—indicating that student who did well on the overall skill also did well on each observation. This is an indicator that observations are performing as expected by the model, supporting the claim that actions are governed by a coherent trait.

© ISLS

113

ICLS2012 Table 1. Item Fit Statistics Overall difficulty Infit Outfit 0.07 0.58 -0.08 -0.2 -0.42 0.23 0.55 -0.73 0.64 0.72 0.67 1.08 1.26 1.12 1.23 1.29 0.61 0.67 0.63 1.11 1.57 1.17 1.3 1.28

Volume 2: Symposia

Question Observation1 Observation2 Observation3 Observation4 Observation5 Observation6 Observation7 Observation8

Item-total correlation 0.76 0.77 0.74 0.42 0.27 0.39 0.49 0.50

While we found that we were able to model latent traits based on observations of students actions in the world, there were surprising findings about how students were responding to the multivariate nature of the problem. Our findings were in line with Kuhn et al’s research (2000) on students’ misconceptions around multivariate systems: • • • students had difficulty teasing out causal and non-causal factors on the outcome. students were not able to distinguish the additive effects that individual features contributed to the respective effects on the outcome. students focused on surface level features while solving the problem.

We are in the process of exploring various analytic approaches to modeling the log data that will provide further insight and diagnostic data on what actions lead to lower and higher inquiry abilities.

Significance
Science inquiry is a complex process. Our research shows that we can use virtual environments to simulate the complexity of inquiry, while reliably assessing student learning. We demonstrated the reliability of our measure using traditional methods (IRT). However, we are also exploring how we can model student observational data from the virtual assessment using Bayesian networks and cognitive diagnostic models. Only through deep understanding of how to measure and model inquiry will we better understand the best methods for teaching it and for preparing our students to understand the complexity of multivariate systems.

Affordances of Dynamic and Interactive Assessments for Measuring Science Inquiry and Reasoning
Jodi L. Davenport, Edys S. Quellmalz, WestEd, and Michael Timms, ACER

Introduction
What are the affordances of simulation-based assessments for eliciting science inquiry skills? Computer-based assessments can portray dynamic information and allow for simulations that are interactive and responsive to student input. Technology-based science tests have been piloted in international testing programs including the Programme for International Student Assessment (PISA) (NGB, 2006; Koomen, 2006). These interactive assessments appear to be ideally suited to measuring student proficiency on science inquiry and reasoning skills. However, computer-based assessments are substantially more costly to develop and have many technical requirements. In the current study, we investigate whether dynamic and interactive assessments are more effective than traditional, static assessments at discriminating student proficiency on three types of science practices: identifying scientific principles (e.g., stating or recognizing principles), using science principles (e.g., predicting or explaining), and conducting inquiry (e.g., designing experiments).

Framework
Our project uses the theoretical frameworks of evidence-centered design and model-based learning to identify the interconnected knowledge and skills that form the student models in our assessments. The evidence-centered assessment design framework shapes the development of the assessments in our study. The process begins with domain analysis, then specification of the student models (content and inquiry to be tested), task models (designs of assessment tasks and items) and evidence models (data for content and inquiry learning) (Mislevy et al., 2003). Research on model-based learning suggests that effective science learners form, use, evaluate, and revise their mental models of phenomena in a recursive process that results in more complete, accurate, and

© ISLS

114

ICLS2012

Volume 2: Symposia

useful mental models of a science system (Gobert & Buckley, 2000). Further, cognitive research shows that learners who internalize schema of complex system organization—structure, functions, and emergent behaviors—can transfer this heuristic understanding across systems (e.g., Goldstone & Wilensky, 2008). The assessment tasks were designed to elicit evidence of science practices described in the 2009 National Assessment of Educational Progress (NGB, 2006). The assessments require students to students carry out the science practices in the context of the content domain of ecosystems, and to explicitly consider the components, interactions, and emergent behaviors characteristic of all complex systems. To ensure the construct validity of the items, we carried out expert reviews with 3 independent reviewers and student think-alouds with 10 middle school students. Both expert reviews and student think-alouds revealed that the items elicited the targeted science practices.

Methods and Data Sources
A sample of 1566 students from the classrooms of 22 teachers in 12 states in the United States participated in the study. Each student completed 3 versions of computer-based Ecosystems assessments that varied in the level of interactivity (static, active, and interactive). In the static version, students viewed still images. In the active version, students viewed animations, but did not conduct active investigations. In the interactive version, students designed and ran their own experiments. Items were aligned across the assessments to tap into the same science practice skills (identifying scientific principles, using science principles, and conducting inquiry) and subskills (e.g., predicting or explaining, drawing conclusions). See Figure 1 for examples of the different modalities of assessment. The assessments were given in three consecutive sessions and the order of the sessions was counterbalanced (e.g., some students receive the static versions during the first session and others receiving the interactive versions first). To analyze the results, we used a generalizability study (G-Study) analysis which estimates the correlations between items designed to tap into the three science practices, and can determine which sets of items appear to be measuring distinct constructs versus the same construct. Our hypothesis was that the interactive assessment would be the most effective at distinguishing student abilities on conducting inquiry from the other science practices, as only the interactive assessment allows students to actively engage in designing experiments and gathering data based on their designs. We also hypothesized that the active, dynamic assessment would be more effective than the static assessment at distinguishing between using science principle and identifying principles, as the active, dynamic assessment allows students to watch animations and make observations of dynamic systems to generate predictions and explanations.

Figure 1. top left, static; top right, active; bottom, interactive

Findings
© ISLS 115

ICLS2012

Volume 2: Symposia

Consistent with our hypotheses, our G-study analysis found that the active, dynamic assessment was the most effective at distinguishing student performance between identifying science practices (e.g., declarative facts) and using science principles (e.g., making predictions and explanations). The interactive, simulation-based assessment was the most effective at distinguishing student performance on conducting inquiry from either of the other science practices. Table 1 shows the correlations between each of the science practices for each of the assessment modalities. Notice that the active, dynamic assessment had the lowest correlation between identifying and using principles, and that the interactive assessment had the lowest correlation between conducting inquiry and either identifying or using principles. Table 1: G-study correlations between science practices across assessment modalities. Correlation between Identifying Principles and Using Principles 0.92 0.80 0.82 Correlation between Identifying Principles and Conducting Inquiry 0.80 0.80 0.72 Correlation between Using Principles and Conducting Inquiry 0.91 1 0.84

Static Active/Dynamic Interactive

Significance
The current study provides the first large-scale empirical evidence of the affordances of dynamic and interactive assessments for discriminating among science reasoning and inquiry skills. Dynamic assessments are more effective than static assessments at differentiating between factual knowledge and the ability to apply that knowledge in meaningful contexts. Simulation-based, interactive assessments, with environments that are responsive to student actions, were more effective than either static or active assessments at uniquely measuring students’ ability to conduct inquiry.

References
Black, P. (1999). Testing: Friend or Foe? Theory and Practice of Assessment and Testing. New York, NY: Falmer Press. Chen, Z., & Klahr, D. (1999). All Other Things Being Equal: Acquisition and Transfer of the Control of Variables Strategy. Child Development, 70 (5), 1098-1120. Clarke, J. (2009). Exploring the Complexity of Inquiry in an Open-Ended Problem Space. Doctoral dissertation presented to the Fellows of Harvard University. Cambridge: Harvard Graduate School of Education. Clarke-Midura, J., & Dede, C. (2010). Assessment, technology, and change. Journal of Research on Technology in Education, 42(3), 309-328. Corbett, A., & Anderson, J. (1995). Knowledge-Tracing: Modeling the Acquisition of Procedural Knowledge. User Modeling and User-Adapted Interaction, 4, 253-278. Fadel, C., Honey, M., and Pasnick, S. (2007). Assessment in the Age of Innovation, Education Week, Volume 26 (38), 34-40. Gobert, J. D., & Buckley, B. C. (2000). Introduction to model-based teaching and learning in science education. International Journal of Science Education, 22(9), 891–894. Gobert, J.; Heffernan, N.; Koedinger, K.; Beck, J. (2009). ASSISTments Meets Science Learning (AMSL). Proposal (R305A090170) funded by the U.S. Dept. of Education. Gobert, J.; Heffernan, N.; Ruiz, C.; Kim, R. (2007). AMI: ASSISTments Meets Inquiry. Proposal NSF-DRL# 0733286 funded by the National Science Foundation. Gobert, J.D, Pallant, A.R., & Daniels, J.T.M. (2010). Unpacking inquiry skills from content knowledge in Geoscience: A research perspective with implications for assessment design. International Journal of Learning Technologies, 5(3), 310-334. Goldstone, R. L., & Wilensky, U. (2008). Promoting transfer through complex systems principles. Journal of the Learning Sciences, 17, 465–516. Gotwals, A.W. and Songer, N.B. (2010) Reasoning Up and Down a Food Chain: Using an Assessment Framework to Investigate Students' Middle Knowledge. Science Education (94)2. P. 259-281. Koedinger, K., & Corbett, A. (2006). Cognitive Tutors: Technology Bringing Learning Sciences to the Classroom. In R. Sawyer, The Cambridge Handbook of the Learning Sciences (pp. 61-77). New York, NY: Cambridge University Press. Koedinger, K. R., Suthers, D. D., & Forbus, K. D. (1999). Component-based construction of a science learning space. International Journal of Artificial Intelligence in Education, 10, 292-313.

© ISLS

116

ICLS2012

Volume 2: Symposia

Koomen, M. (2006, April). The development and implementation of a computer-based assessment of science literacy in PISA 2006. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA. Kuhn, D., Black, J., Keselman, A., Kaplan, D. (2000). The Development of Cognitive Skills to Support Inquiry Learning. Cognition & Instruction, 18(4), 495-523. Linacre, M. J. (2003). Winsteps Rasch measurementsoftware [Computer software]. Chicago: Winsteps. Mislevy, R., & Haertel, G. (2006). Implications of evidence centered design for educational testing PADI Technical Report 17. Menlo Park, CA: SRI Interantional. Mislevy, R. J., Chudowsky, N., Draney, K., Fried, R., Gaffney, T., and Haertel, G. (2002). Design patterns for assessing science inquiry. Unpublished manuscript, Washington, D.C. Mislevy, R., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessment. Measurement: Interdisciplinary Research & Perspective, 1(1), 3-62. Mislevy, R., Almond, R., Yan, D., & Steinberg, L. (1999). Bayes nets in educational assessment: Where do the numbers come from? In Kathryn B. Laskey and Henri Prade, editors, Proceedings of the Fifteenth Conference on Uncertainty in AI, San Francisco. Morgan Kaufmann Publishers, Inc. National Committee on Science Education Standards and Assessment. (1996). National Science Education Standards, Washington, D.C., National Academy Press. National Governing Board (NGB). (2006). Science framework for the 2009 National Assessment of Educational Progress. Washington, DC: Author. National Research Council. (2000). How People Learn: Brain, Mind, Experience, and School: Expanded Edition. Committee on Learning Research and Educational Practice. M.Suzanne Donovan, John D.Bransford, and James W.Pellegrino, editors. Washington, DC: The National Academies Press. Pellegrino, J. (2001). Rethinking and redesigning educational assessment: Preschool through postsecondary. Denver, CO: Education Commission of the States. Quellmalz, E., Kreikmeier, P., DeBarger, A. H., & Haertel, G. (2006). A study of the alignment of the NAEP, TIMSS, and New Standards Science Assessments with the inquiry abilities in the National Science Education Standards. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA. Sao Pedro, M., Baker, R., Gobert, J., Montalvo, O., & Nakama, A. (in press). Using Machine-Learned Detectors of Systematic Inquiry Behavior to Predict Gains in Inquiry Skills. To appear in User Modeling and User-Adapted Interaction. Shute, V. J., Hansen, E. G., & Almond, R. G. (2007). An assessment for learning system called ACED: Designing for learning effectiveness and accessibility. (RR-07-26). Princeton, NJ: Educational Testing Service. Strauss A, Corbin J. (1990). Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Newbury Park, CA: Sage. United States Department of Education (USDE). (2010). National Education Technology Plan 2010. Washington, DC: US Department of Education. White, B., Collins, A., & Frederiksen, J. (2011). The nature of scientific meta-knowledge. In M. S. Khine & I. Saleh (Eds.), Dynamic modeling: Cognitive tool for scientific enquiry. London: Spinger.

Acknowledgments
The research in the paper by Clarke-Midura and Dede was supported by grants from the Institute of Education Sciences (IES#R305A080141) and the Bill and Melinda Gates Foundation to Harvard University. Any findings, interpretations, and conclusions do not constitute an official position of the U. S. Department of Education. The authors gratefully acknowledge the contributions of Drs. Jillianne Code, Geordie Dukas, Michael Mayrath, Nick Zap, as well as the assistance of numerous students at the Harvard Graduate School of Education. The material in the paper by Davenport, Quellmalz, and Timms is based upon work supported by the National Science Foundation under Grant No. DRL-0814776 awarded to WestEd, Edys Quellmalz, Principal Investigator. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

© ISLS

117

ICLS2012

Volume 2: Symposia

Scripting Science Inquiry Learning in CSCL Classrooms
Annelies Raes, Tammy Schellens, & Bram De Wever, Dept. of Educational Studies, Ghent University, Belgium Email: annelies.raes@ugent.be, tammy.schellens@ugent.be, bram.dewever@ugent.be Ingo Kollar, Christof Wecker, Sybille Langer, & Frank Fischer, Ludwig-Maximilians-Universität, Germany ingo.kollar@psy.lmu.de, christof.wecker@psy.lmu.de, sybille.langer@psy.lmu.de, frank.fischer@psy.lmu.de Mike Tissenbaum, James D. Slotta, Ontario Institute for Studies in Education, University of Toronto, Canada Email: miketissenbaum@gmail.com, jslotta@oise.utoronto.ca Vanessa L. Peters & Nancy Butler Songer, University of Michigan, School of Education, USA Email: vlpeters@umich.edu, songer@umich.edu Chair: Bram De Wever, Ghent University, Belgium, bram.dewever@ugent.be Discussant: Pierre Dillenbourg, Ecole Polytechnique Fédérale de Lausanne, Switzerland, pierre.dillenbourg@epfl.ch Abstract: Research on scripting computer-supported collaborative learning (CSCL) has recently received a lot of attention. However, most findings within this research grew out of studies focusing scripting online collaborative learning activities that often had an asynchronous nature and were conducted in artificial settings. This symposium includes an international set of presenters from Belgium, Canada, Germany, and the USA and brings together four studies that focus on scripting face-to-face “classroom” activities, seeing the “classroom” as a formal physical learning environment. The common denominator of the contributions is that they are all field studies focusing on computer-supported science inquiry learning, aiming to investigate the optimal conditions for organizing these inquiry learning environments. Each paper will present the research context, method, data, and conclusions on how scripting can be implemented to support science inquiry learning. Broader implications of the findings of these studies will be discussed with the audience.

Aim and Structure of this Symposium
The main aim of this symposium is to highlight the added value of scripting CSCL activities in authentic science classrooms. Within the field of Computer-Supported Collaborative Learning, a lot of attention has been paid to how learning environments need to be organized in order to promote students’ learning. There is widespread agreement that technology in itself does not support learning (Lai, 2008) and that technology should be well integrated into a learning environment in order to realize its full potential (Weinberger, Reiserer, Ertl, Fischer, & Mandl, 2005). In this respect, the introduction of scripts (see Kobbe et al., 2007) has been proven useful. More specifically, earlier research showed that “scripts have proven to be a powerful instructional approach to foster specific collaborative activities and interaction patterns, such as identifying conceptual differences, asking thought-provoking questions, integrating multiple perspectives, and/or constructing arguments and counter-arguments” (Weinberger, Stegmann, & Fischer, 2010; cf. a recent discussion at CSCL 2011, see Gijlers, van Dijk, & Weinberger, 2011, p. 1071). While a lot of research towards scripting has been focusing on online – usually asynchronous – learning environments, the present symposium focuses on studying scripting in CSCL authentic classroom settings. Within these settings, the collaboration script is functioning next to, or in addition with, the teacher in the learning environment or classroom. Through bringing together the work of scholars from four distinct research groups from four different countries, we hope to provide a diversity of perspectives and empirical data to widen the discussion on the issue of implementing scripting in authentic CSCL situations. This symposium comprises four papers, which all focus on (1) science inquiry learning, (2) in authentic situations, (3) supported by technology, (4) in secondary education, and (5) the role of scripting in these learning environments. The first two studies focus more on collaboration scripts within the classrooms, in order to find ways how to introduce small group collaboration scripts and potentially combine them with the teacher as another potential source of support for the learners. The third study investigates an emergent script introducing user-generated content in a physics learning environment. The fourth study centers on implementing a mobile technology application to script students’ activities while performing field observations. The discussant, Pierre Dillenbourg, who is not affiliated with any of these research programs, but has studied scripting in high detail, will synthesize the ideas and advances in this work and lead a discussion amongst the participants and the audience. He will invite (a) the participants to connect and compare their findings and put forward specific implications for inquiry based science learning in TEL classrooms and (b) the audience to relate to and critically comment on these findings and scientific and practical implications.

© ISLS

118

ICLS2012

Volume 2: Symposia

Paper 1: Scripting Computer-Supported Collaborative Information Problem Solving on the Web
Annelies Raes, Tammy Schellens, & Bram De Wever

Introduction
Previous research has indicated that learning science content by means of a web-based inquiry project is effective in enhancing learners’ knowledge acquisition (Slotta & Linn, 2009) and their metacognitive awareness while solving information problems on the web (Raes, Schellens, De Wever, 2011). However, since students in a web-based inquiry project usually work together in small groups, it is vital to pay attention to the collaboration process and students’ (socially) shared regulation, in addition to processes on an individual level (Spada & Rummel, 2005; Vauras, Iiskala, Kajamies, Kinnunen, & Lehtinen, 2003). In this respect, this study focuses on both the individual and collaborative processes playing when students learn by means of a web-based inquiry science project and deal with information problems in an authentic classroom setting. Moreover, it is questioned whether the quality of collaboration and shared regulation can be improved by integrating a collaboration script and subsequently, if this also leads to higher individual learning outcomes. Providing students with a collaboration script is put forth as a way to support computer-supported collaborative learning (CSCL) since collaboration scripts facilitate social and cognitive processes of collaborative learning by shaping the way learners interact with each other (Kobbe et al., 2007; Kollar, Fischer & Slotta, 2007). The script developed for this study particularly focused on the aspect of roles and the mechanisms task distribution and sequencing aiming at prompting the regulatory skills that critical information problem solving on the web entails (BrandGruwel, Wopereis, & Walraven, 2009). The one student was the ‘executer’ assigned to operate the computer and typing the answers, the other student was the ‘web detective’ assigned to critically supervise the online search activities. Students were prompted to switch roles during the project. The effects of the integration of such a collaboration script are investigated through a quasiexperimental field study. In total 220 students from 13 different secondary school classes were involved in this study. The intervention consisted of the implementation of a web-based collaborative inquiry project lasting 4 sessions of 50 minutes. During the first session, secondary students completed an individual pretest and were introduced to the Web-based project provided on the Web-based Inquiry Science Environment (WISE) (Slotta & Linn, 2009). They were free to choose their partner and started the first introductory activity of the WISEproject in dyads. Students worked in the same dyads during the whole intervention and navigated through the sequence of inquiry activities using the inquiry map in the online learning environment. During the project, students were asked to write their answers down in reflection notes. Finally, all students completed an individual posttest. Seven classes were provided with a collaboration script embedded in the WISE-project (experimental condition, N = 97 students), six classes were not provided with this collaboration script (control condition, N = 107 students).

Data and Methods
The study used a mixed-methods approach in which both quantitative and qualitative sources of evidence are triangulated. The quantitative part of the study consisted of pretest-posttest-comparisons regarding the outcome measures knowledge acquisition and metacognitive awareness during information problem solving on the Web. The latter variable was measured by facing students with an unfamiliar information problem in pretest and posttest, more specifically a scientific controversy. They were assigned to take up a particular position that they needed to justify with appropriate evidence from the web to support their claim. After performing this IPS-task students were asked to fill out an adapted version of the Metacognitive Awareness Inventory (MAI) (Schraw & Dennison, 1994). This self-report inventory was used to measure students’ perception about their metacognitive and strategic activities while performing the task. Moreover, students’ answers during the web-based projects were scored and summed to result in a process score for every dyad. Quality of collaboration was measured by means of a self-perception questionnaire filled out by the students after finishing the project. In addition, the collaborative processes of a subset of these students (35 dyads) were videotaped and the quality of collaboration was rated based on the scheme of Meier, Spada and Rummel (2007). In this study the students worked together in small groups within classes, and as such, the problem under investigation has a clear hierarchical structure. In this respect, the analysis of test data at an individual level raises a methodological issue frequently discussed in research on group learning and collaborative problem-solving (e.g. Cress, 2008; De Wever, Van Keer, Schellens, & Valcke, 2007). Because of this, the Hierarchical Linear Modeling is used for testing main effects and interaction effects of predictor variables on different levels. The independent variables that were taken into account were gender, achievement level and perceived quality of collaboration at student level; group composition based on gender, group composition based on achievement and students’ process score at group level; and academic track (science track vs. general track) and condition (collaboration script present vs. absent) at class level.

© ISLS

119

ICLS2012

Volume 2: Symposia

The software MLwiN 2.23 for multilevel analysis was used to analyze the hierarchical data. A threestep procedure has been followed to analyze the effects of the presented explanatory variables on the dependent variables. The first step consists of the estimation of a four-level conceptual null model, which serves as a baseline model. This unconditional null model without any predictor variables provides both the overall pretest score and the overall learning gain for all students across all groups and classes. Moreover this null model will, by means of the Intra-Class-Correlation (ICC), answer the question if the outcome measures vary among students, across groups and across classes. The second step concerns the input of the main explanatory variables in the fixed part of the model and allows cross-level interactions between student and class characteristics. This will give insight in the differential effects for different groups of students with different student and class characteristics. Finally, in the third step, the aggregated characteristics based on gender and achievement level, i.e. group composition, was added to the model. The results of these quantitative multilevel analyses will be presented and discussed at the conference in addition with qualitative analyses on the collaborative inquiry activities of the 35 different groups. These in-depth analyses will deepen the quantitative results and will give insight in how students deal or deal not with the embedded collaboration script and how this influences their learning performances and outcomes.

Paper 2: Using Small Group and Classroom Scripts to Foster High School Students’ Acquisition of Online Search Competence
Ingo Kollar, Christof Wecker, Sybille Langer, & Frank Fischer

Introduction
An important strategy in developing well-grounded positions in societal debates on science issues, such as whether pre-implantation diagnostics should be allowed or not, is to search the Internet for relevant information. However, finding information that is relevant, impartial, credible and scientifically sound is a challenging task for high school students. One promising approach to foster high school students’ online search competence is (web-based) collaborative inquiry learning. Yet, without appropriate scaffolding, inquiry learning is likely to fail (de Jong, 2006). This contribution looks at the effects of two kinds of scaffolds that can be used during collaborative inquiry learning. First, it investigates small group collaboration scripts that specify, sequence and distribute learning activities among roles that are filled by the members of a small group (e.g., Kollar, Fischer & Hesse, 2006). With respect to online search, such a script may for example have one learner suggest certain search terms, while her learning partner is asked to anticipate what unwanted search results will likely be triggered by these terms. Similar task distributions could be realized for further steps of the online search process (e.g., selecting links from a hit list or finding information on a selected website). Although prior research has shown that small group collaboration scripts can be designed to foster quite some variety of domain-general skills (e.g., argumentation; see Stegmann, Weinberger, & Fischer, 2007), it is still an open question whether they can also foster students’ online search competence. As a second kind of scaffold, we look at classroom scripts that distribute learning activities over the different social planes of the classroom (Dillenbourg & Jermann, 2007). With respect to online search, one could imagine a classroom script that first has the teacher model successful online search behavior (plenary activity), followed by dyadic and then individualized Internet search. We compared the effects of two classroom scripts (variant 1: all search activities were carried out on the small group level, i.e. in dyads, vs. variant 2: search activities changed between the plenary level, i.e. through teacher-student or student-student modeling, and the small group level) that were either enriched or not enriched by a small-group collaboration script. With respect to classroom scripts, we expected that the script that included plenary (i.e. modeling) in addition to small group activities (variant 2) would work better than the script that located all search activities on the small group level (variant 1). We also expected that when all search activities are conducted on the small group level (variant 1 of the classroom script), a small group collaboration script would effectively scaffold learners’ acquisition of online search competence. Most effective, however, should be the combination of a small-group collaboration script and a classroom script that included search activities on the plenary and the small-group level to be.

Data and Methods
Participants were 174 9th graders from eight classes from urban high schools (90 female, 84 male). Within regular Biology lessons led by the students’ regular Biology teachers, who were trained to implement the experimental condition, their class was assigned to one of the four experimental conditions of a 2x2 design (see below). Learners were equipped with laptops and worked on a web-based inquiry learning curriculum unit on Genetic Engineering. The main task was to develop a well-grounded position towards the question whether Genetic Engineering should be allowed or not. The learning phase spanned seven regular Biology lessons and involved considerable time to search the Internet for arguments and evidence. During these search phases, we realized a 2x2-factorial design with the factors “small-group collaboration script” (present vs. not present) and “type of classroom script” (small-group level only vs. small-group and plenary level). In all groups, we

© ISLS

120

ICLS2012

Volume 2: Symposia

implemented collaborative browsing, i.e., during their online search processes, the laptops of the two co-present partners of a dyad were connected so that the browser displayed identical web pages on both screens, no matter who of the two learners had accessed them. The small-group collaboration script was implemented as a browser plug-in that segmented the browser window into two frames (a scaffolding frame and the regular browser frame). The scaffolding frame displayed prompts related to the actual step in the search process (e.g., Google start page, Google hit list, chosen web site) and distributed them between the two learning partners. For example, accompanying a Google hit list, the scaffolding frame displayed prompts such as “Suggest a link which from your perspective is likely to contain valuable information” to one learner and “Listen to your partner’s suggestion and estimate whether the link s/he suggested is (a) impartial, (b) relevant and (c) scientifically grounded” to his/her learning partner. In the conditions without the small-group collaboration script, no prompts were displayed. The type of classroom script was also manipulated during the online search phases. In the small-group level only classroom script, online search was to be conducted in dyads. In the classroom script with both the small-group and the plenary level, single steps were modeled in front of the class before this activity was to be conducted on a dyadic level. Online search competence was measured by the students’ performance in an individual test that asked them to describe in as much detail as possible how they would use the Internet to form a position concerning the question whether nuclear power plants should be abandoned. 15% of the data included in the analysis was analyzed by two independent coders. Coding was based on a scheme that captured adequate steps and important quality characteristics during successful online search (e.g., judge the credibility of a website). Intra-class correlation was sufficient (ICC = .83). An analogous test on a different topic was used as a pretest (ICC = .51).

Results and Discussion
Results indicated that as expected, when no small-group collaboration script was provided, the classroom script with online search activities both on the plenary and the small group level led to higher levels of online search competence than the classroom script that located all search activities on the small group level, F(1,66) = 23.89, p < .01, partial Eta² = .27. Also confirming our expectations, the small group collaboration script proved more effective than no small group script, when all search activities were located on the small group level, F(1,90) = 10.06, p < .01, partial Eta² = .10). Adding to these results, first results from further in depth process analyses revealed that both the small group collaboration script and the classroom script comprising activities on both the plenary and the small group level led to higher quality levels of online search behavior than no such support. In contrast to our hypothesis, however, the combination of the small group collaboration script and the classroom script that alternated plenary and small group search phases was not the most effective concerning the learners’ acquisition of online search competence. When the classroom script covered both the small-group and the plenary level, adding the small-group collaboration script had a slight negative effect, F(1,74) = 4.27, p < .05, partial Eta² = .06. Thus, it appears that modeling as part of a classroom script can be regarded as a viable alternative to scripting small group collaboration (see also Rummel & Spada, 2005).

Paper 3: Scripting Collective Inquiry in High School Physics
Mike Tissenbaum & James D. Slotta

Introduction
Our work focuses on developing a smart classroom for knowledge communities where all students are actively involved in the production, aggregation, and assessment of science content (Ulrich et al., 2008; Ito et al., 2009). Having students contribute their own educational content provides opportunities for them to understand connections amongst often disparate pieces of information within a domain. The most common way of creating such connections is by assigning meta-data, or “tags”, to individual content (Mathes, 2004; Wiley, 2000)., allowing individuals to assign descriptors without needing to know about every other piece of content that shares the same designation. They can rely on the emergence of a collective data set, guided by the tags, to reveal meaningful connections and increased usefulness of content elements (Hayman & Lothian, 2007). To define and coordinate the flow of materials, activities and interactions within such complex collaborative inquiry, learning scientists have advanced the notions of scripting and orchestration (Dillenbourg, Jarvela & Fischer, 2009; Dimitriadis, 2001), where specified learning and interaction designs (i.e., “the script”) are enacted (“the orchestration”). The script can be seen as a formalism that captures the pedagogical structure of a learning design (e.g., student must upload two relevant videos) and collaboration patterns between students, their peers, and the teacher. When user-contributed materials are introduced, the script becomes more openended, and any design must be left somewhat “unbound” allowing specific themes, directions or content collections to emerge during its enactment (Peters & Slotta, 2010). Technology-supported inquiry learning environments offer a proven means of scaffolding student learning, encouraging reflection, providing timely access to tools and materials, and engaging learners collaboratively (Slotta & Linn, 2009).

© ISLS

121

ICLS2012

Volume 2: Symposia

This present paper reports on our efforts to formalize scripts, in terms of the roles, groupings, learning contexts, technology, materials, and interaction patterns for inquiry and knowledge communities. Towards developing a formal concept of scripting, we conducted a series of quasi-experimental studies, beginning with investigations of small, controlled “micro” scripts, followed by more complex “macro” scripts, where students contribute content and actively use the emerging aggregate knowledge base. Below, we report on two studies of micro scripts, and a larger macro script to support high school physics students as they collectively upload content, solve, tag and explain physics problems, then re-use those resources in scripted inquiry activities. Students were scaffolded in all aspects the script by a smart classroom technology framework (Slotta, 2010).

Data and Methods: Three Successive Designs of Collaborative Inquiry Scripts
Our first iteration investigated a collaborative inquiry script called Tag, Answer, and Reflect (TAR) where students uploaded their responses to homework problems, including their selection of “physics principle tags,” then reflected on the aggregate set of tags and explanations. We investigated the following questions: (1) Did TAR help students make connections between physics principles? (2) How did different technologies (large format displays, customized views) help the teacher to understand students’ knowledge, for adjustments to the script before, and orchestration of the class during its enactment? (3) Did a visualization of student work help students make connections between physics principles? Two different comparisons were examined: First, individuals were compared with small groups as they examined aggregated responses from the community; second, the effect of large-format displays of a groups’ work vs. a single shared laptop was examined in terms of improved student performance. Overall, groups faired significantly better at solving problems (97% overall accuracy) than individuals working at home (80% overall accuracy), with t=2.02, and p<0.05. The connections made by the groups (80.94%) were also more accurate than individuals (76.57%), although only marginally significant (p=.081). The large-format displays were seen to aid the teacher in orchestrating class activities, allowing the detection of student errors, and supporting spontaneous short lectures and discussions with small groups. Our second iteration sought to extend the TAR script, in order to better aid students in their reflections and investigate the impact of the aggregated knowledge on group performance. Our quasi-experimental study compared two enactments: one where students were provided with access to the aggregated individual responses during their group reflection phase, and one where no aggregates were provided. Groups that received the aggregated responses outscored those who did not (t=4.13, p<0.01), as measured by a rubric developed with the teacher. These outcomes further our ideas of TAR, including the timing of supports for the teacher. The third study builds on the findings from the previous two, establishing a “macro” script to address a persistent, year-long curriculum engaging students in tagging and reflecting about homework problems while also uploading and tagging “found examples” from everyday life, including photos, Web URLs, and physics problems. We thus introduce a notion of a “macro” script specifying longer-term patterns, supported by “micro” scripts like TAR. The macro script in this study, called the “collective inquiry cycle,” (CIC) involves five steps: (1) Students submit individual inquiry items to the knowledge base; (2) Collectively (at home or in class) students examine and tag peers’ work, adding comments to explanations; (3) Teacher reviews the community knowledge, to prepare a discussion; (4) In-class activity engages students with collective knowledge artifacts chosen by teacher; (5) Students reflect individually. Macro scripts must allow for adaptations by the teacher, who inserts new materials and activities in response to micro script data (e.g., students’ uploads or TAR data). The visualization of student work was expanded to represent student ideas as a complex interconnected web of social and semantic relations, allowing students to filter the information to match their own interests and learning needs - making the knowledge not only collective but also personal. The visualization also became the focal point of real-time smart classroom activities where students leveraged the products of the class in creating physics “challenge problems.” Customized views allowed the teacher to gain an understanding of students’ collective knowledge and adjust class activities to better address student needs. Tools were also developed to allow the teacher to assess student work, review individual student progress, and add new materials to the script. We are currently collecting data from two grade 11 physics classes (n=20, n=25), over a six-month curriculum (Sept, 2011 – March, 2012), divided into three units: Kinematics; Force and Motion; and Energy, Work, and Power. The units are thematically connected, allowing content to carry over between each. We are orchestrating CIC and TAR, including design-based variations of activities and technologies, within each unit. Our final paper will present a pre- post comparison of students’ reasoning about physics principles, an analysis of student contributions to the knowledge base (e.g., the challenge problems they create), and progress in student explanation patterns over time. We will connect these analyses to our notions of a scripting framework.

Conclusions
These studies have addressed two key aspects of a scripting framework for collaborative inquiry. First is the notion of a macro-script for coordinating student activities and teacher interventions over a substantial curricular scope. This includes the capability for student created artifacts, peer exchange, voting and discourse, and

© ISLS

122

ICLS2012

Volume 2: Symposia

teacher adaptations such as additional resources (ie. questions, examples, assessments), as well as the role for emergent artifacts as resources within the script. Second is the development of micro-scripts (e.g. TAR) that can be supported with a smart classroom infrastructure to orchestrate and coordinate student inquiry activities that make use of, and contribute to the community knowledge base These studies have informed our progress toward a scripting framework for engaging students in a knowledge community while providing teachers with the tools to adjust the script in response to the emergent ideas within that community. Further we have begun to formalize an understanding of the informational needs of the teacher in the execution of these scripts.

Paper 4: Development of a Mobile Scripting Application to Support Students’ Collection and Analysis of Scientific Data
Vanessa L. Peters & Nancy Butler Songer

Introduction
Research in the learning sciences suggests that even with carefully-designed curricular materials, students develop only a partial understanding of scientific inquiry. While students are able to recognize certain features of inquiry thinking, few students can systematically demonstrate the difference between claim, evidence and reasoning for making predictions or constructing explanations about science (Jeong, Songer, & Lee, 2007). Previous research suggests that more cohesion and scripting are needed to support students’ development of complex reasoning skills when collecting and interpreting data for scientific investigations (Metz, 2000). The U.S. College Board has identified the following data-related practices as belonging to an essential skill set for answering questions about scientific problems: justifying the selection of the data type that are needed, designing a plan for data collection and documentation, and evaluating the quality and sources of data (College Board, 2009). Pedagogical scripts can support aspects of scientific inquiry by specifying parts of the learning process such as work phases, task distribution, and the sequencing of activities (Dillenbourg & Jermann, 2007; Peters & Slotta, 2010). Scripts can be communicated through initial instructions by the teacher, integrated into the learning environment (Kollar, Fischer, & Slotta, 2007), or integrated into the design of learning technologies. This study builds on the strong foundation of empirically-tested activities developed for BioKIDS: Kids’ Inquiry of Diverse Species (Songer, 2006) with the development of a new iPod Touch application for supporting students’ collection and analysis of scientific data. The goal of the BioKIDS program is to support elementary and middle school students’ development of complex reasoning and technological fluency in science through curricular activities that fuse scientific content knowledge with scientific practices. In the BioKIDS curricular units, students work in small groups to collect data on animal or plant species from several different areas in their schoolyard. Students take part in recording field observations that are later used for constructing scientific explanations and making predictions that use their collected data as evidence.

Data and Methods
Working with a technology developer, we developed a customized iPod Touch application that scripts students’ recordings of field observations when collecting and classifying data about plant and animal species (Figure 1). The script guides students in data collection through a sequence of panels that asks for specific information about the characteristics of their observed plant or animal, including the habitat, energy role and location of the species. The application’s intuitive and icon-driven interface was well-suited for middle school students, including those whose first language was not English. The BioKIDS application did not require that students enter data in one panel before progressing to the next, thus, it was possible for them to omit entering certain information. After students recorded their field observations, the data were aggregated in a repository that organized the information according to biodiversity richness and abundance. Data collected with the BioKIDS application were synched automatically to an online database; data collected with the clipboard were added manually to the repository using the same organizational structure as the iPod Touch application. The participants in this study were 41 seventh graders from an urban middle school in the Midwestern United States. Students were divided into two groups for data collection: one group used the iPod Touches (n = 21), and the other group used printed worksheets and a clipboard (n = 20). Both groups were taught by the same teacher and completed the same activities. Curriculum implementation differed only in the method students used for recording field observations in their schoolyard. Both groups collected data on the same day and under the same conditions. The teacher was provided with instructions on using the iPod Touch application prior to the study; students were not provided with any such instruction. All field observations from both the iPod Touch and clipboard were analyzed for their completeness and usefulness for providing data-based evidence that could be used for answering scientific questions.

© ISLS

123

ICLS2012

Volume 2: Symposia

Figure 1. Panel sequence in BioKIDS iPod Touch application for scripting students’ observations of field data.

Results and Discussion
Both the iPod Touch and clipboard groups recorded approximately the same number of field observations (79 and 77, respectively), however, there were marked differences between the two groups in terms of the completeness and volume of the collected data. Students using the scripted BioKIDS application noted the habitat and energy role for 100% of their observations, while the clipboard group noted this information for only 5.19% of their observations. Fisher’s Exact Test was used to compare the frequency of complete observations for each organism class. An observation was considered complete if it included both the abundance (number of animals/plants) and the richness (type of animal/plant) of the observed organism. Between the two groups, there were significant differences in the recorded observations for mammals (p = .0001), birds (p = .0004), anthropods (p = .0119) and plants (p = .0001). Only the gastropods and annelids (p = .4286) were found to be non-significant between the two groups. Aside from the novelty and fun-factor, the BioKIDS iPod Touch application presented a significant advantage over the clipboard as a method for data collection when engaging students in inquiry learning. To be meaningful, students need to collect and analyze data in sufficient volume in order to use it productively as evidence for supporting scientific claims. A field observation in this activity was useful if included both the type of animal or plant observed (e.g., prairie falcon, shrub) and the number observed. Unlike traditional pencil-andpaper approaches, technology makes it easy to illustrate the power of large data sets, providing students with enriched opportunities for data analysis that would otherwise be difficult or impossible with smaller sets of data. In the iPod group, the scripted application made it easier for students to record their observations when collecting data, which resulted in a more voluminous and complete repository of field data. One of the goals of BioKIDS is to develop strong learning tools, including technological tools, that support students when collecting and analyzing data, providing them with opportunities for using data-based evidence for answering scientific questions (Songer, 2006). The findings from this study demonstrate the value of technology scripts for supporting a systematic approach to data collection and analysis, and have implications for both technology developers and learning scientists. New technologies bring with them new ways of interacting and understanding different forms of knowledge. Future development work will consider additional scripting capabilities for supporting students’ data collection and learning with mobile technologies.

References
Brand-Gruwel, S., Wopereis, I., & Walraven, A. (2009). A descriptive model of information problem solving while using internet. Computers & Education, 53(4), 1207-1217. doi: 10.1016/j.compedu.2009.06.004 College Board. (2009). Science: College board standards for college success. New York, NY: The College Board Press. Cress, U. (2008). The need for considering multilevel analysis in CSCL research-An appeal for the use of more advanced statistical methods. International Journal of Computer-Supported Collaborative Learning, 3(1), 69-84. doi: 10.1007/s11412-007-9032-2 de Jong, T. (2006). Technological advances in inquiry learning. Science, 312, 532-533. De Wever, B., Van Keer, H., Schellens, T., & Valcke, M. (2007). Applying multilevel modelling to content analysis data: Methodological issues in the study of role assignment in asynchronous discussion groups. Learning and Instruction, 17(4), 436-447. doi: DOI 10.1016/j.learninstruc.2007.04.001 Dillenbourg, P. & Jerman, P. (2007). Designing integrative scripts. In F. Fischer, I. Kollar, H. Mandl, & J. M. Haake (Eds.), Scripting Computer-Supported Collaborative Learning: Cognitive, computational and educational perspectives (pp. 275-301). New York: Springer. Dimitriadis, Y. (2011). Supporting teachers in orchestrating CSCL classrooms. Research on E-Learning and ICT in Education, (September), 33-40. Gijlers, H., van Dijk, A. & Weinberger, A. (2011). How Can Scripts and Awareness Tools Orchestrate Individual and Collaborative Drawing of Elementary Students for Learning Science? In H. Spada, G. Stahl, N. Miyake, & N. Law (Eds.), Connecting Computer-Supported Collaborative Learning to Policy and Practice: CSCL2011 Conference Proceedings (Vol. III, pp. 1065-1072). Hong Kong: ISLS.
© ISLS 124

ICLS2012

Volume 2: Symposia

Hayman, S., & Lothian, N. (2007). Taxonomy directed folksonomies: Integrating user tagging and controlled vocabularies for australian education networks. World Library and Information Congress: 73RD IFLA General Conference and Council, (August) 1-27. Ito, M., Baumer, S., Bittanti, M., Boyd, D., Cody, R., Herr-Stephenson, B., et al. (2009). Hanging out, messing around, and geeking out: Kids living and learning with new media (john D. and catherine T. MacArthur foundation series on digital media and learning) (1st ed.) The MIT Press. Jeong, H., Songer, N. B., & Lee, S.Y. (2007) Evidentiary competence: Sixth graders’ understanding for gathering and interpreting evidence in scientific investigations. Research in Science Education, 37(1), 75-97. Kobbe, L., Weinberger, A., Dillenbourg, P., Harrer, A., Hämäläinen, R., Häkkinen, P., & Fischer, F. (2007). Specifying computer-supported collaboration scripts. Computer Supported Collaborative Learning, 2, 211-223. Kollar, I., Fischer, F., & Hesse, F. W. (2006). Computer-supported collaboration scripts - a conceptual analysis. Educational Psychology Review, 18(2), 159-185. Kollar, I., Fischer, F. & Slotta, J. D. (2007). Internal and external scripts in computer-supported collaborative learning. Learning and Instruction, 17(6), 708-721. Lai, K.W. (2008). ICT supporting the learning process: The premise, reality, and promise. In J. Voogt & G. Knezek (Eds.), International handbook of information technology in primary and secondary education. Part one. (pp. 215-230). New York: Springer. Mathes, A. (2004). Folksonomies - cooperative classification and communication through shared metadata. Computer Mediated Communication, , 1-13. Meier, A., Spada, H., & Rummel, N. (2007). A rating scheme for assessing the quality of computer-supported collaboration processes. International Journal of Computer-Supported Collaborative Learning, 2(1), 63-86. doi: DOI 10.1007/s11412-006-9005-x Metz, K.E. (2000). Young children's inquiry in biology: Building the knowledge bases to empower independent inquiry. In J. Minstrell & E. van Zee (Eds.), Inquiring into inquiry in science learning and teaching (pp. 371-404). Washington, DC: American Association for the Advancement of Science. Peters, V. L., & Slotta, J. D. (2010). Scaffolding knowledge communities in the classroom: New opportunities in the Web 2.0 era. In M. J. Jacobson & P. Reimann (Eds.), Designs for learning environments of the future: International perspectives from the learning sciences (pp. 205-232). New York, NY: Springer. Raes, A., Schellens, T., & De Wever, B. (2011). Multiple modes of scaffolding to enhance web-based inquiry. In H. Spada, G. Stahl, N. Miyake, & N. Law, N. (Eds.), Connecting Computer-Supported Collaborative Learning to Policy and Practice: CSCL Conference Proceedings, Vol I. ISLS. Rummel, N., & Spada, H. (2005). Learning to collaborate: An instructional approach to promoting collaborative problem solving in computer-mediated settings. Journal of the Learning Sciences, 14(2), 201-241. Schraw, G., & Dennison, R. S. (1994). Assessing Metacognitive Awareness. Contemporary Educational Psychology, 19(4), 460-475. Slotta, J. D. (2010). Evolving the classrooms of the future: The interplay of pedagogy, technology and community. In K. Makitalo-Siegl, F. Kaplan, J. Zottmann & F. Fischer (Eds.), The classroom of the future orchestrating collaborative learning spaces (pp. 215-242). Rotterdam: SensePublisher. Slotta, J. D., & Linn, M. C. (2009). WISE Science, Web-Based Inquiry in the Classroom. New York: Teachers College Press. Songer, N. B. (2006). BioKIDS: An animated conversation on the development of complex reasoning in science. In R. Keith Sawyer, (Ed.) Cambridge handbook of the learning sciences (pp. 355-369). New York: Cambridge University Press. Stegmann, K., Weinberger, A., & Fischer, F. (2007). Facilitating argumentative knowledge construction with computer-supported collaboration scripts. International Journal of Computer-Supported Collaborative Learning, 2(4), 421-447. Vauras, M., Iiskala, T., Kajamies, A., Kinnunen, R., & Lehtinen, E. (2003). Shared-regulation and motivation of collaborating peers: A case analysis. Psychologia, 46(1), 19-37. Ullrich, C., Borau, K., Luo, H., & Tan, X. (2008). Why web 2.0 is good for learning and for research : Principles and prototypes. International Conference on World Wide Web, 705-714. Weinberger, A., Reiserer, M., Ertl, B., Fischer, F., & Mandl, H. (2005). Facilitating collaborative knowledge construction in computer-mediated learning environments with cooperation scripts. In R. Bromme, F. W. Hesse, & H. Spada (Eds.), Barriers and biases in computer-mediated knowledge communication (pp. 15-38). Boston: Kluwer. Weinberger, A., Stegmann, K., & Fischer, F. (2010). Learning to argue online: Scripted groups surpass individuals (unscripted groups do not), Computers in Human Behavior, 26, 506-515. Wiley, D. A. (2000). Connecting learning objects to instructional design theory: A definition, a metaphor, and a taxonomy. Learning Technology, 2830(435), 1-35.

© ISLS

125

ICLS2012

Volume 2: Symposia

Learning as Identity Formation: Implications for Design, Research, and Practice
Jochen Rick, EduTech, Saarland University, Campus C5 4, Saarbrücken, D-66123, j.rick@mx.uni-saarland.de Ben DeVane, University of Florida, P.O.Box 115810, 101 Norman Gym, Gainesville, Florida 32611-5810, ben@digitalworlds.ufl.edu Tamara Clegg, College of Information Studies, University of Maryland, 2117A Hornbake, South Wing, College Park, MD 20742, tclegg@umd.edu Vanessa L. Peters & Nancy Butler Songer, University of Michigan, 610 East University Avenue, Ann Arbor, MI 48109, {vlpeters,songer}@umich.edu Susan R. Goldman, Learning Sciences Research Institute (MC 057), 1007 W. Harrison Street, Room 2048, University of Illinois, Chicago, IL 60607-7137, sgoldman@uic.edu Cindy E. Hmelo-Silver, Dept. of Educational Psychology, Rutgers University, 10 Seminary Place (Room 320), New Brunswick, NJ 08901, cindy.hmelo-silver@gse.rutgers.edu Abstract: Increasingly, research is acknowledging the importance of identity for understanding and supporting learning. Identity is a particularly useful concept for highlighting extraordinary individual experiences, interpreting rich social contexts, and viewing learning at multiple timescales, from an instant reaction to lifelong psychosocial development. However, identity is complex—it is conceptualized differently by various academic traditions. This inherent diversity can lead to a confusing or murky notion of identity. In this symposium, we clarify the concept, integrating the relevant theoretical perspectives of social and psychological identity to assess its implications to design, research, and practice. We provide concrete examples from our own research of the benefits of viewing learning as identity formation and the challenges to conducting such research.

Learning as Identity Formation
The learning sciences are manifold, acknowledging contributions from numerous academic traditions (education, psychology, computer science, design, etc.) and recognizing multiple perspectives on learning, both their power and limitations. Bransford and Schwartz (1999) argue that having a more fine-grained notion of transfer has powerful consequences for how we understand and promote learning. In his keynote at ICLS 2002, Roy Pea noted the limitations of the learning model imposed by the then-new “No Child Left Behind” legislation: that learning gains can be quantified through testing and thereby directly compared. In this symposium, we examine the perspective of learning as identity formation—people use their personal experience to actively construct their identity in relation to ideas, others, and to themselves. We provide concrete examples from our own work on the value of this perspective to design, research, and practice. As with any lens, there are limitations: While there is significant overlap, learning is not the same as identity formation (Nasir & Cook, 2009). On the other hand, important aspects of learning come into clearer focus. In the preface of his seminal book “Mindstorms: Children, Computers, and Powerful Ideas,” Seymour Papert (1980) describes his early fascination with the differential gear. “I believe that working with differentials did more for my mathematical development than anything I was taught in elementary school” (p. xviii). He viewed the multiplication tables as gears. His first brush with two variable equations (e.g., 3x + 4y = 10) immediately evoked the differential. The gears of his childhood served him as useful model to assimilate the world and to form his identity as a mathematician. While the cognitive component was significant, Papert credits more power to the affective component (his “love” of the gear) in shaping his life. Problematically, such a profound example of learning cannot be easily understood through conventional means: “A ‘pre- and post-‘ test at age two would have missed them” (p. xx). How can we study such experiences and use our understanding to facilitate them? To better understand the challenges involved, it is worthwhile noting the peculiarities of Papert’s experience. First, the learning took place largely outside a classroom. Second, the full story takes place over a long time, spanning from playing with toy trucks at age two to using his relationship with gears to inform his studies with Piaget and ultimately his theories of how the computer can support learning. Third, it was a highly individual story of identity formation. Papert admits that others would not have taken away the same things from those experiences. Fourth, it takes place in a larger social context, where he was able to use his passions to inform his school studies and his professional career. Characterizing learning in a larger social context has been well represented inside the learning sciences. In particular, communities of practice has been an influential framework to incorporate social aspects of learning. That theory originated with observations of how traditional methods of career training (e.g., tailors, butchers) have substantially different properties than school-based education. Lave and Wenger (1991)

© ISLS

126

ICLS2012

Volume 2: Symposia

document how successful implementations of legitimate peripheral participation can allow newcomers to join a community of practice and progress to being central to that community. Wenger (1998) further theoretically expands the implications to identity formation through the twin processes of becoming and belonging. Over time, an apprentice will learn new skills and take over more responsibility to become a tailor. Concurrently, both the apprentice and the tailoring community will increasingly acknowledge that he belongs to that community. There is both an individual and a social component. There has been substantial work on studying the social components of identity in evocative situations, pointing out the key differences to classroom learning, and then integrating the positive aspects into other learning settings (Gee, 2000). For example, Bryant, Forte, and Bruckman (2005) studied how contributors to Wikipedia became part of that community. They found that a particularly powerful incentive for people to freely contribute their time was that others would see their work (i.e., that it made a difference to a sizable audience). This contrast sharply with standard classwork, which is usually only viewed by the teacher. To exploit the advantage of an audience, Forte and Bruckman (2006) had learners construct a publically available wiki based on their in-class learning and independent research. Studies of identity in meaningful social contexts can help us reflect on existing research and practice. When studying how high schoolers played dominoes in their free time, Nasir (2005) found that players were not just trying to win the game; they also used sophisticated and subtle strategies, often bending the rules of the game, to scaffold weaker players. This stands in contrast to Barron’s (2003) work documenting common problems in group-work (e.g., ignoring contributions by a peripheral group member). It suggests that children do have the inherent ability to support each other given the right social context. The design challenge is creating the right context. In another study, Nasir and Cook (2009) focused on learning in after-school athletic activities, such as track and basketball. As voluntary activities, there was a different social impetus and individual commitment to these activities. Coaches explicitly facilitated learning as identity formation. One track coach informed a skeptical student, “you are a hurdler.” That statement indicated a specific learning trajectory: While she was struggling at the time, she would become a successful hurdler—a position of value in this context. Similar sentiments (e.g., “you are a scientist”) are much more rare in a conventional classroom. Independent of the social and skill-based aspects of identity, there is a core identity—an identity so core to the person that it is important in all situations (Erikson, 1963). In Freudian psychology, this core identity roughly equates to the ego. Whether it be in outfitting a home (Csikszentmihalyi & Rochberg-Halton, 1981), writing a computer program (Turkle, 1984), participating in a MUD (Bers, 2001; Turkle, 1995), or working on a simple worksheet (Wenger, 1998), people consciously or unconsciously “play with” their core identity. The tools we provide learners are thus not neutral in that regard. Bers (2001) has explicitly aimed at creating identity construction environments, technologies and technologically-rich psychoeducational interventions that support explorations of self. She designed the Zora virtual community to allow children to create and share different things (virtual places, objects, heroes, and villains) that reflect their core values. Bers (2006) later expands the term to include an explicit focus on promoting positive youth development. Positive youth development focuses on promoting six aspects of identity for youths: competence (cognitive abilities and healthy behavioral skills), positive bonds with people and institutions, character (integrity and moral centeredness), confidence, caring, and contribution to a civil society. Such vital components of psychosocial development are often neglected in school curricula and by evaluations that focus on cognitive aspects of learning.

Complimentary Perspectives on Identity Formation
This symposium brings together insights from four separate research projects to demonstrate how learning as identity formation can inform the design of learning environments, learning sciences research, and educational practice. Rick will present how different notions of identity overlap and how, to understand the rich learning in such complex situation, certain theories may be more applicable to one person’s experience than another’s. Devane and Clegg will provide two perspectives on how to study identity formation at different temporal scales, from the instance of an “aha” moment to lifelong psychosocial development. Peters will present on the role of disciplinary identity in science education research—how the knowledge, methods, and culture associated with specific academic domains shapes both our approach to research (i.e., the “practice”) as well as the products and outcomes that educational research seeks to achieve.

Presentation 1 — A Confluence of Identity
Jochen Rick Personal home pages are outgrowing their playful beginnings to serve serious purposes. At the forefront of this emergence is academia, where they are becoming a meaningful way for researchers to engage each other. In computer science, it is common for researchers to visit their colleagues’ home pages to find research articles and contact information. Additionally, a personal home page acts as an informal curriculum vitae or an überbusiness card. Graduate students are advised to maintain a professional page. Many include a link to their home

© ISLS

127

ICLS2012

Volume 2: Symposia

page when applying for a faculty position. Likewise, some faculty members polish their pages before promotion. Yet, the medium is still in its infancy: The medium, its adopters, and their practices are unduly constrained by current technology. To better study the meaning and use of personal home pages in academia, I created a better personal-home-page system to loosen these constraints. That system applies wiki technology to facilitate easy editing, to enable interaction, and to focus the user on content creation. To honor its origins in the WikiWikiWeb, the software was named AniAniWeb. Whereas “wiki wiki” means quick in Hawaiian Creole (the quickest way to create a website is to ask any visitor to be an author), “aniani” means mirror (an appropriate metaphor for a tool to write about oneself). Mirrors, literal and metaphorical, play an important role in human development. In literature, music, visual art, or computer programming, they allow us to see ourselves from the outside, and to objectify aspects of ourselves we had perceived only from within. (Turkle, 1995, p. 155) The dissertation research presents a case study of six graduate students and their experience with AniAniWeb over a period of two years (Rick, 2007). Their practices are viewed through three analytical lenses: media theory, communities of practice, and core identity theory. When combined, these frameworks led to a rich understanding of personal home pages in academia. This work is not unique in combining these frameworks to understand a new medium. Turkle (1995) combines aspects of each of these perspectives to understand identity in MUDs (i.e., text-based multiplayer real-time virtual worlds). As a clinical psychologist, her main focus was on the individual—how does the technology affect the individual? Meyrowitz (1985) combines Goffman’s (1959) framework on social interaction with a McLuhan (1964) perspective on technology to understand television. His main focus is on understanding the societal changes that television has caused. Going into the case study, I hypothesized that these frameworks would be useful in explaining the use and meaning of personal home pages in academia. The results have confirmed that. All three frameworks contribute to a better understanding of the whole. At times, the frameworks are orthogonal—they explain different things. For some adopters, their experience hardly reflects their core identity; AniAniWeb was simply a useful tool for them. For others, AniAniWeb does act as a mirror allowing adopters to play with who they are. At other times, the frameworks are complementary—illuminating different aspects of the same phenomenon. Graduate students are on a learning trajectory inside of the academic community of practice. When they use their home page to address that audience, the meaning is necessarily social in nature. However, their reflection on how successful they have been in that goal can strongly impact their core identity. As self-presentation environments, authoring a personal home page is naturally about presenting oneself to others. But, who are these others? At times, such as in the middle of a job search, there is a clear primary audience and the home page can be tailored to that audience. More frequently, multiple audiences are simultaneously served by the same home page. A home page can be visited by friends, family, colleagues, fellow academics trying to find out more about you, potential employers, your boss, etc. Compounding the problem, AniAniWeb so facilitated content creation that several adopters primarily used the technology for themselves—to keep notes or to record their lives. Authoring a personal home page that serves these different audiences to the author's satisfaction is the multiple audience problem. Authors in asynchronous media only have a hazy awareness of who their audience is; often they even fail to realize that they have one (Forte & Bruckman, 2006). Authoring a personal home page without useful feedback on how others receive the page is the audience awareness problem. For prolific users, such as those studied in this work, negotiating these two problems can be a challenge. In face-to-face conversation, we can (and do) change how we present ourselves based on our audience (Goffman, 1959). A waiter acts differently while in the kitchen than in the dining room. People act differently at home and at work. When these barriers break down (e.g., introducing your boss to your parents at your wedding), figuring out how to integrate the multiple roles can be tricky. People who simultaneously inhabit multiple spheres, such as teleworkers (Nippert-Eng, 1996), must continually negotiate which roles they play. Electronic technologies have a tendency to change such interpersonal dynamics (McLuhan, 1964; Meyrowitz, 1985). They can radically alter our self epistemologies—how we view ourselves. Because the virtual persona is not tied to an actual body, the Internet is a particularly fertile ground for playing characters radically different than your physical self. In a MUD, an able-bodied man can convincingly play a crippled woman in a chat room (Van Gelder, 1985). Turkle (1995) concludes that electronic media foster a fragmented self epistemology, where it is possible to be radically different in different settings; however, there are several electronic media where it is unacceptable and uncommon for the presented self to be substantially different from the physical self. Personal home pages are one such case. As only one site comes at the top of a Google query for your name, it is nearly impossible to separate content to different audiences. Personal home pages encourage an integrating self epistemology, emphasizing a core self that exists across multiple aspects of the adopter's life.

© ISLS

128

ICLS2012

Volume 2: Symposia

Presentation 2 — Identity in Time: Theoretical and methodological approaches
Ben Devane Studies in the learning sciences focused on the importance of identity to social learning have increased in number and prominence (e.g., Bers, 2001; Sfard & Prusak, 2005; Barab et al., 2007; Nasir & Hand, 2008). At the same time that identity has become an increasingly important construct in much of learning science research, what learning science researchers mean when they discuss ‘identity’ can often be indistinct, uncertain or vague. In the broader social science research literature, two major conflicting theories of identity predominate, developmental (Erikson, 1959) and interactionist (Mead, 1934). Learning sciences research often uses the term identity without making clear what theory of identity is intended to be understood, leading to unnecessary confusion and disagreement. In this presentation, I argue that these two theories of identity do not necessarily represent conflicting views of social identity. Instead I argue that these differing theories are the result of disciplinary methods that focus on distinct timescales, the chronological units used to examine social processes (Lemke, 2000). In doing so, I briefly review the major social sciences literature on the two theories of identity, describe the major methodological differences between the two approaches, and outline the relevance of timescales to the particular methodological approach. This presentation pays particular attention to the relationship between short, “moment-to-moment” timescales of identification, commonly called identity enactment or performance, and longer timescales of identification, known as identity formation or development.

Perspectives on identity – interactionist vs. developmental
Developmental and interactionist perspectives on identity are each associated with a formative scholar for the theory, Erik Erikson and G.H. Mead respectively. Developmental perspectives typically understand identity as a normally singular and continuous process of mental and emotional development, emphasizing the importance of both social institutions and social groups like families in said developmental process (see Erikson, 1959). Erikson wrote that identity “connotes both a persistent sameness within oneself (selfsameness) and a persistent sharing of some kind of essential character with others” (1959, pg. 102). In the learning sciences, developmental perspectives on identity have examined the relationship between learning and long-term, core aspects of identification like sexuality, ethnicity, morality and self-esteem (Halverson, 2009; Kirshner, 2008). Rooted in Pragmatist scholarship in psychology and sociology, interactionist perspectives on identity place emphasis on a person’s unique social experiences and social roles. G.H. Mead, often recognized as a key figure in symbolic interactionist school, wrote that identity resulted from a dialogue of an individual’s unique set of experiences with the “importation of the social process” from society as a whole (Mead, 1934, p. 186). Identity in this understanding results from a negotiation between social norms, social context and social structure, and is often referenced as being performed or enacted (e.g. Goffman, 1956). In the learning sciences, interactionist perspectives often interpret identity as learning to perform of a knowledgeable social role, like developing an identity as a scientist (e.g., Brown et al., 2005; Barab et al., 2007).

Timescales, method and identification
The differences between these two approaches to identity can be understood in terms of their methodological relationship to collecting and interpreting data at a given timescale. Interactionist paradigms focus on understanding identity at smaller timescales. Research in interactional sociolinguistics, for example, may use methods from discourse analysis or conversation analysis examine a few minutes of talk to understand how a person is enacting a particular ethnic, gender or role-based identity. Interactional perspective in the learning sciences may use microethnography to examine the development of a given role-based identity, say as a scientist or mathematician, over a period of months in a classroom. Conversely, identity research from a developmental perspective may use ethnographic methods to look at identity development of the course of a year or more. The different theories of identity are rooted in the distinct “analytic primacy” of their methodological approaches, and not conflicting phenomena (see Penuel & Wertsch, 1995). Research on identity needs analytic methods that put data on identity enactment at shorter timescales into dialogue with data on identity formation at longer timescales. Going further, this paper will examine methodological approaches focused on understanding the relationship between timescales and processes of identification–such as connective ethnography (Leander & McKim, 2003), ethnographic discourse analysis (Gee & Green, 1998) and microethnography (Erickson & Schutz, 1982)–and discuss the theoretical perspectives connected to each.

Presentation 3 — Understanding Identity Development From Day-to-Day Experiences In Kitchen Science Investigators
Tamara Clegg Extending from the previous presentation, this presentation also looks at identity across timescales, but with respect to science learning. It is different in that our goal is to promote science learning and identity
© ISLS 129

ICLS2012

Volume 2: Symposia

development. Also drawing from socio-cultural perspectives of identity (Wenger, 1998), we analyzed learners’ day-to-day experiences using Gee’s (2000) Discourse framework. In order to understand how learners’ day-today experiences are becoming more stable aspects of their identity, we analyzed learners’ scientific dispositions. We define disposition as values of, ideas about, and ways of participating in a particular discipline (in this case, scientific reasoning) that come frequently, consciously, and voluntarily (Gresalfi & Cobb, 2006; Katz, 1993). Simply put, scientific disposition is the initiative learners take to use scientific practices. We see disposition in the increased amount and complexity of learners’ scientific practices and in their use of those practices in different contexts (Bereiter, 1995). In this presentation, I look at learners’ development of disposition in the context of Kitchen Science Investigators (KSI). KSI is an after-school or summer camp program we designed to help learners begin to have the types of day-to-day experiences that would promote their scientific identity development. We designed KSI to promote development of scientific disposition through using science to make and perfect recipes. In this afterschool program, participants learn science content and design experiments to learn more about ingredients and how to use them to accomplish cooking and baking goals. In such a context, there are opportunities to learn much about chemistry, biology, physics, and arithmetic and to develop competence in a variety of scientific practices. In this presentation, I look at the case studies of two KSI participants in a 9-month implementation of KSI who began to develop scientific dispositions as a result of KSI experiences. We analyzed those cases to identify how that development progressed, and the components of the learning environment (and other experiences in their lives) that promoted that development. We analyzed learners’ scientific participation in KSI and their reports of scientific participation outside of the program, in other settings of their lives. Specifically, we found the disposition development process was initiated as learners used science to accomplish personally meaningful goals in the context of making and eating tasty dishes. As learners began to use science to accomplish their goals, they began to take on more scientific roles. As learners had more scientific experiences and as they continued to participate scientifically, their values shifted. While learners’ experiences, participation, and values were different, each of these aspects of their participation became more scientific, leading to their development of more scientific dispositions, or stable aspects of themselves. In this presentation, I will describe these shifts in each case across the contexts of learners’ lives (e.g., school, home, KSI). I will then present aspects of the learning environment that promoted that development.

Presentation 4 — Blurring the boundaries: Understanding disciplinary identities in science education research
Vanessa L. Peters & Nancy Butler Songer In STEM education, improvements to teaching and learning depend on the collective efforts of scholars who bring disciplinary expertise to research collaborations. To date, research studies have provided limited understanding of how specific individuals – such as learning scientists and natural scientists – exchange and synthesize expertise towards project deliverables and outcomes. For STEM education research to be truly transformative, scholars must collaborate deeply and effectually not only within and across academic domains, but also with other learning organizations including schools, community programs and industry. In this presentation, I report on a study that examined the knowledge exchange and creation process of an interdisciplinary science education research team, and describe how disciplinary identities shaped the interactions and work practices of collaborators. Interdisciplinarity: Are we there yet? Promoting effective interdisciplinary collaboration has been a top priority for policy makers and federal agencies. In 2005, the US National Academy of Sciences (NAS) published a comprehensive report that summarized the current scope of interdisciplinary scholarship, with the goal of providing findings and recommendations to funding organizations and academic institutions for how such research could be facilitated. The findings of the report discussed at length the institutional-level challenges facing interdisciplinary collaboration, such as the disciplinary-specific rules that govern hiring, promotion and the allocation of resources. Yet the top challenge reported by researchers about interdisciplinary work involved the development of effective strategies for enhancing communication among project members in disciplinary disparate research teams. Guidelines for communication have yet to be developed, pending an analysis of the interactions and discourse strategies used during interdisciplinary exchanges. As Leshner (2011) observes, “The need for interdisciplinary approaches has increased tremendously… although we have been discussing it for 40 years, collectively we never seem to get it right. If we could come up with a series of distilled lessons learned, principles, and action steps that count be taken, then I think we could make tremendous progress” (AAAS, 2011). Interdisciplinarity has been defined in many ways. Although the terms inter-, multi-, cross- and transdisciplinary all refer to pluralistic disciplinary approaches, there are differences in how the disciplines are

© ISLS

130

ICLS2012

Volume 2: Symposia

combined. These differences are perhaps best described has having some placement on a continuum of disciplinary integration. At one end of the continuum is monodisciplinary, the traditional approach that employs the theories, methodologies and epistemologies of a single domain (Yang, 2011). Next are multi- and crossdisciplinary approaches that consider the dimensions of different disciplines in parallel, with little attempt to synthesize theories, methods or research findings (Klein, 2010). Interdisciplinary – the most broadly used term – is a problem-oriented approach that seeks to establish a common terminology and epistemological framework among the disciplines for academic inquiry (Knights & Willmott, 1997). At the far end of the continuum is transdisciplinary, an emerging approach that transcends disciplinary boundaries through the integration of knowledge and the fusion of methodologies, often with the aim of developing new theoretical paradigms (Hadorn, Pohl, & Bammer, 2010). Characterizing interdisciplinarity This research used a case study approach to examine how knowledge is exchanged and synthesized in a large interdisciplinary research team collaborating on curriculum development work for science education. Data were collected from team research meetings (including both in-person and videoconferencing), interviews and project documents. Project meetings are a fundamental component of research collaborations, and often the only forum in which all members participate simultaneously. Thus, their inclusion as a data source is necessary for an indepth understanding of the complexities of interdisciplinary collaboration. In this presentation, I discuss the role of disciplinary identities in science education research—how the knowledge, methods and language associated with traditional academic domains shapes both our approach to research (i.e., our “practice”) as well as the products and outcomes that educational research seeks to achieve. I will describe a model of communicative behavior that explains the process of disciplinary knowledge exchange in educational research, and provide recommendations for a research approach that promotes transdisciplinary perspectives over traditional domainbased approaches.

Participants (alphabetical order)
Tamara Clegg — Presenter 3 Tamara Clegg received her Ph.D. from Georgia Tech’s School of Interactive Computing specializing in the Learning Sciences. She is currently a postdoctoral fellow at the University of Maryland with the Computing Innovations Fellows program. Her work currently focuses on developing technology to support life-relevant learning environments where children engage in science in the context of achieving goals relevant to their lives. She is using participatory design with children to design these new technologies and new uses of existing technologies. Her work currently includes creating new life-relevant learning environments to understand how identity development happens across these environments. From this analysis, she aims to draw out design guidelines for life-relevant learning activities and technology in various contexts (e.g., sports, gardening). Ben DeVane — Presenter 2 Ben DeVane is assistant professor of Digital Arts & Sciences in the University of Florida Digital Worlds Institute. Trained at the University of Wisconsin Games, Learning & Society initiative, his dissertation research was a four-year ethnographic study of identity and learning in a game-based learning community for workingclass youth. His design research has focused on developing and assessing learning games about topics like public health, environmental science and financial literacy. His scholarship been published in E-Learning, Games & Culture, the International Journal of Learning and Media, and Theory into Practice. Susan R. Goldman — Discussant Susan R. Goldman (PhD., University of Pittsburgh) is Distinguished Professor of Liberal Arts and Sciences, Psychology, and Education and Co-Director of the Learning Sciences Research Institute at the University of Illinois at Chicago. She conducts research on subject matter learning, instruction, assessment, and roles for technology, especially in literacy and mathematics. A particular focus of her current research is on understanding the literacy demands in different disciplinary contexts and the implications of these demands for supporting learning. She is Executive Editor for Cognition & Instruction and Associate Editor for Journal of Educational Psychology. Goldman is a board member and President of the International Society of the Learning Sciences (2011–2012). Cindy E. Hmelo-Silver — Session Chair Cindy E. Hmelo Silver is Professor of Educational Psychology at Rutgers University. She received an M.S. in Educational Technology from SUNY at Stony Brook and a Ph.D. in Cognitive Studies from Vanderbilt University and served postdoctoral fellowships at Georgia Institute of Technology and the University of Pittsburgh's Learning Research and Development Center. She is Editor-in-Chief of the Journal of the Learning Sciences.

© ISLS

131

ICLS2012

Volume 2: Symposia

Vanessa L. Peters — Presenter 4 Vanessa is a postdoctoral research fellow at the University of Michigan, where she is collaborating with an interdisciplinary research team to develop curricular units for teaching middle and high-school students about the ecological impacts of climate change. Her work on the project has focused on the development of SPECIES, an online learning environment with an embedded predictive distribution modeling tool that has been repurposed for younger audiences. Vanessa completed her Ph.D. at the University of Toronto, where she conducted a design-based study on knowledge community and inquiry practices in secondary science education. Her research interests include scripted collaboration, discourse processes in teaching and learning social computing. Jochen “Jeff” Rick — Presenter 1 Jeff’s research interests are in designing innovative applications for leading-edge technologies to support collaborative learning through active inquiry, exploration, and construction. As a design-based researcher, he has found it particularly useful to employ qualitative methods (e.g., ethnographic-style interviews, detailed video analysis) to understand the relation that an individual or a specific learning group has to a system. He received a Ph.D. in Computer Science (specialization of Learning Sciences and Technology) from Georgia Tech in 2007. From 2007 to 2010, he worked as a research fellow on the ShareIT project, investigating how new shareable interfaces (e.g., interactive tabletops) can support co-located collaboration. Since then, he has been on the faculty of the Department of Educational Technology, Saarland University. His latest research focuses on two students working with one iPad; he is particularly interested in how children combine physical gestures and verbal communication to collaborate.

References
American Association for the Advancement of Science. (2011, April). Experts at U.S. symposium urge efforts to encourage and support interdisciplinary research – News archive. Retrieved August 6, 2011 from http://www.aaas.org/news/releases/2011/0418science_on_fire.shtml. Barab, S., Dodge, T., Thomas, M., Jackson, C., & Tuzun, H. (2007). Our designs and the social agendas they carry. Journal of the Learning Sciences, 16(2), 263–305. Barron, B. (2003). When smart groups fail. Journal of the Learning Sciences, 12(3), 207–359. Bereiter, C. (1995). A dispositional view of transfer. In A. McKeough, J. Lupart & A. Marini (Eds.), Teaching for transfer: Fostering generalization in learning (pp. 21-34). Mahwah, NJ: Lawrence Erlbaum Associates. Bers, M. U. (2001). Identity construction environments: Developing personal and moral values through the design of a virtual city. Journal of the Learning Sciences, 10(4), 365–415. Bers, M. U. (2006). The role of new technologies to foster positive youth development. Applied Developmental Science, 10(4), 200–219. Bransford, J. D., & Schwartz, D. L. (1999). Rethinking transfer: A simple proposal with multiple implications. Review of Research in Education, 24, 61–100. Brown, B. A., Reveles, J. M., & Kelly, G. J. (2005). Scientific literacy and discursive identity: A theoretical framework for understanding science learning. Science Education, 89(5), 779–802. Bryant, S. L., Forte, A., & Bruckman, A. (2005). Becoming wikipedian: Transformation of participating in a collaborative online encyclopedia. In Proceedings of GROUP ’05, pp. 1–10. New York: ACM Press. Csikszentmihalyi, M., & Rochberg-Halton, E. (1981). The meaning of things: Domestic symbols and the self. Cambridge, UK: Cambridge University Press. Erickson, F., & Schultz, J. (1981). When is a context? Some issues and methods in the analysis of social competence. In J. Green & C. Wallat (Eds.), Ethnography and language in educational settings (pp. 147–159). Norwood, NJ: Ablex. Erikson, E. (1956). The problem of ego identity. Journal of the American Psychoanalytic Association, 4(1), 56. Erikson, E. H. (1963). Childhood and society (second ed.). New York: W. W. Norton & Company. Forte, A., & Bruckman, A. (2006). From Wikipedia to the classroom: Exploring online publication and learning. In Proceedings of ICLS 2006, pp. 182–188. ISLS. Gee, J. P. (2000). Identity as an analytic lens for research in education. Review of Research in Education, 25, 99–125. Gee, J., & Green, J. (1998). Discourse analysis, learning, and social practice: A methodological study. Review of research in education, 23(1), 119–63. Goffman, E. (1959). The presentation of self in everyday life. New York: Anchor Books. Gresalfi, M. S., & Cobb, P. (2006). Cultivating students' discipline-specific dispositions as a critical goal for pedagogy and equity. Pedagogies: An International Journal, 1(1), 49–57.

© ISLS

132

ICLS2012

Volume 2: Symposia

Hadorn, G. H., Pohl, C., & Bammer, G. (2010). Solving problems through transdisciplinary research. In R. Frodeman, J. T. Klein, C. Mitcham, & J. B. Holbrook (Eds.), The Oxford handbook on interdisciplinarity (431–452). Oxford, UK: Oxford University Press. Halverson, E. (2005). InsideOut: Facilitating gay youth identity development through a performance-based youth organization. Identity, 5(1), 67–90. Katz, L. (1993). Dispositions as educational goals. ERIC Clearinghouse on Elementary and Early Childhood Education: ERIC Digest. Kirshner, B. (2008). Guided participation in three youth activism organizations: Facilitation, apprenticeship, and joint work. Journal of the Learning Sciences, 17(1), 60–101. Klein, J. T. (2010). A taxonomy of interdisciplinarity. In R. Frodeman, J. T. Klein, C. Mitcham, & J. B. Holbrook (Eds.), The Oxford handbook on interdisciplinarity (pp. 15–30). Oxford, UK: Oxford University Press. Knights D., & Willmott, H. (1997). The hype and hope of interdisciplinary management studies. British Journal of Management, 8(1), 9–22. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge, UK: Cambridge University Press. Leander, K., & McKim, K. (2003). Tracing the everyday ‘sitings’ of adolescents on the Internet: A strategic adaptation of ethnography across online and offline spaces. Education, Communication & Information, 3(2), 211–240. Lemke, J. L. (2000). Across the scales of time: Artifacts, activities, and meanings in ecosocial systems. Mind, Culture & Activity, 7(4), 273–290. McLuhan, H. M. (1964). Understanding media: The extensions of man. Cambridge, MA: MIT Press. Meyrowitz, J. (1985). No sense of place: The impact of electronic media on social behavior. New York: Oxford University Press. Nasir, N. S. (2005). Individual cognitive structuring and the sociocultural context: Strategy shifts in the game of dominoes. Journal of the Learning Sciences, 14(1), 5–34. Nasir, N. S., & Cooks, J. (1999). Becoming a hurdler: How learning settings afford identities. Anthropology & Education Quarterly, 40(1), 41–61. Nasir, N. S., & Hand, V. (2008). From the court to the classroom: Opportunities for engagement, learning, and identity in basketball and classroom mathematics. Journal of the Learning Sciences, 17(2), 143–179. National Academy of Sciences (2005). Facilitating interdisciplinary research. Washington, DC: National Academies Press. Nippert-Eng, C. E. (1996). Home and work. Chicago: The University of Chicago Press. Papert, S. (1980). Mindstorms: Children, computers, and powerful ideas (second ed.). New York: Basic Books. Penuel, W., & Wertsch, J. (1995). Vygotsky and identity formation: A sociocultural approach. Educational Psychologist, 30(2), 83–92. Rick, J. (2007). Personal home pages in academia: The medium, its adopters, and their practices. Unpublished doctoral dissertation, Georgia Institute of Technology, Atlanta, GA. Sfard, A., & Prusak, A. (2005). Telling identities: In search of an analytic tool for investigating learning as a culturally shaped activity. Educational Researcher, 34(4), 14–22. Turkle, S. (1984). The second self: Computers and the human spirit. New York: Simon & Schuster. Turkle, S. (1995). Life on the screen: Identity in the age of the Internet. New York: Simon & Schuster. Van Gelder, L. (1985, October). The strange case of the electronic lover: A real-life story of deception, seduction, and technology. Ms. Magazine, XIV(4), 94–124. Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge, UK: Cambridge University Press. Yang, A. S., (2011). Interdisciplinary and critical inquiry: Visualizing at the Art/Bioscience interface. Interdisciplinary Science Reviews, 36(1), 43–55.

Acknowledgements
This symposium originated at the early career workshop of CSCL 2011. There, we discovered our mutual interest in identity formation and that we had different insights to contribute. We would like to thank the organizers and mentors of that workshop for encouraging us to collaborate on an ICLS 2012 symposium.

© ISLS

133

ICLS2012

Volume 2: Symposia

Building (Timely) Bridges between Learning Analytics, Educational Data Mining and Core Learning Sciences Perspectives
Ido Roll, University of British Columbia, 6224 Agricultural Road, Vancouver, BC V6T 1Z1, ido@phas.ubc.ca Vincent Aleven, Kenneth R. Koedinger, Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213 {aleven, koedinger}@cs.cmu.edu Matthew Berland, University of Texas at San Antonio, 1 UTSA Circle San Antonio, TX 78249, matthew@berland.org Taylor Martin, Tom Benton, Carmen Petrick, University of Texas at Austin, 1 University Station, Austin, Texas, 78712 {taylormartin, tbenton, cpetrick}@mail.utexas.edu Arnon Hershkovitz, Michael Wixon, Ryan Baker, Janice Gobert, Michael Sao Pedro 100 Institute Rd, Worcester Polytechnic Institute, Worcester, MA {arnonh, mwixon, rsbaker, jgobert, mikesp}@wpi.edu. Bruce Sherin, Northwestern University, 2120 Campus Drive, Evanston, IL, bsherin@northwestern.edu Paulo Blikstein, Marcelo Worsley, 520 Galvez Mall, Stanford, CA, 94305 {paulob, marcelo.worsley}@stanford.edu, Paulo Blikstein, Organizer and Chair Roy Pea, Stanford University, Discussant Abstract: Despite the exponential growth of the research on Learning Analytics (LA) and Educational Data Mining (EDM) over the last few years, the work has been still distant from the core Learning Sciences methods, theoretical constructs, and literature. At the same time, over the last 15 years, Learning Sciences as a field has been quite innovative, eclectic, and effective in incorporating new methodological stances, such as micro-genetic methods, microethnographies, and design-based research. It seems that the time has come to build sound connections between these traditions. The goal of this symposium is to bring together researchers coming from different academic perspectives, to explore and examine common LA/EDM methodological and theoretical threads with wide applicability within the Learning Sciences. The papers presented explore text mining in clinical interviews, moment-bymoment learning curves and traces, data mining of programming logs, and cognitive tutors, representing the main perspectives and methodological approaches in the field.

Overall focus of the symposium
Despite the exponential growth of the research on Learning Analytics (LA) and Educational Data Mining (EDM) (Baker & Yacef, 2009) over the next few years, the work has been still distant from the core Learning Sciences methods, theoretical constructs, and literature. At the same time, over the last 20 years, the Learning Sciences as a field has been quite innovative, eclectic, and effective in incorporating new methodological stances, such as micro-genetic methods (Siegler & Crowley, 1991), micro-ethnographies (Nemirovsky, 2011), and design-based research (Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003; Confrey, 2005; Edelson, 2002). However, in the last years, as computational technologies became truly ubiquitous, the possibilities of data collection opened up incredible new opportunities, such as the capture of students’ interaction with computer software, websites, as well as gesture, speech, position, mood, etc. There is clear interest in these techniques in the Learning Sciences for various reasons – for one, those data seem like a somewhat natural complement to micro-genetic methods, since they allow for an even greater sampling rate, and to design-based research, since they are accepting of unstructured learning activities and rapid-prototyping. Second, many learning scientists themselves have a computer science background, so they are familiar with computational modeling tools. However, by its very nature, data-mining and machine learning tend to be agnostic of mechanism – in many fields of application (outside of education), gargantuan datasets and fast computers provide researchers with answers oftentimes much more advanced than the theoretical or explanatory models they have—what matters are correct predictions. This approach, however, is problematic in education, where understanding the causal chain is crucial for prescribing sound novel policies or educational designs. Thus, we are at a crossroad in the history of both fields, one in search of new methods, and another searching for sound theoretical models to explain its findings. The question posed for the LS community is—

© ISLS

134

ICLS2012

Volume 2: Symposia

how to make a productive connection and avoid ‘big data’ from drifting towards a-theoretical, data-oriented work? The goal of this symposium is answer that questions by bringing together researchers coming from different academic perspectives, to explore and examine common LA/EDM methodological and theoretical threads with wide applicability within the Learning Sciences. The papers presented explore text mining in clinical interviews, data mining of programming logs, cognitive tutors, moment-by-moment learning traces, coming from scholars from different traditions but with a strong presence in the LS community.

Presentations
Hershkovitz et al. examine “disengaged behavior” and how to identify it using an automated detector. They will also relate that data with data from the Patterns of Adaptive Learning Scale and try to find out if different behavioral and epistemological issues relate to disengaged behavior. Overall, they show that students characterized by mastery or performance goal orientation have (on average) double the probability of carelessness as compared to students characterized by low scores for these goal orientations. Berland et al. present a study with students in which they use visual programming to create virtual robots. Patterns of collaboration affected students’ success at programming – students who shared complex code created better programs immediately afterwards. The study suggests that sharing is beneficial in the short term, but over the entire implementation, the average sharing measure was not correlated with code quality. Blikstein et al. paper follows a group of undergraduate students learning computer programming, as snapshots of their code are stored in a central server. Students fill a questionnaire about programming style and previous experience. The researchers use machine learning techniques to find patterns in how more and less experienced learners progressed towards expertise. Roll et al. demonstrate the potential of using students’ moment-by-moment traces to offer domain-level support in scientific inquiry tasks, suggesting that adapting the task can be a productive mean for giving support in a constructivist environments. Log and assessment data from an evaluation of the Invention Lab in six grade nine classes (N = 92) was used to validate the modeling approach of the lab. Roll suggests that students’ trace data can be used to offer adaptive, individualized support in a manner that does not reduce critical elements of inquiry learning. Sherin’s paper examines techniques from statistical natural language processing (SNLP) and their use to analyze data produced by clinical interviews with middle-school students about scientific topics. In particular, he introduces Latent Dirichlet Allocation as an alternative to the more popular Latest Semantic Analysis, and uses it to segment transcripts of student’s conceptions in openended interviews about the seasons, verifying the algorithm against human coders.

Using Dynamic Time Warping and Cluster Analysis to Analyze the Learning of Computer Programming
Paulo Blikstein, Marcelo Worsley, Stanford University Educational data mining and Learning Analytics (EDM; Amershi & Conati, 2009; Baker, Corbett, Koedinger, & Wagner, 2004) has grown rapidly over the last years, profiting from the vast availability of logfile data from elearning systems, instrumented computer applications (Blikstein, 2009, 2011, Berland, 2008), computer vision systems, and web logs. The majority of the work focuses on standardized tasks delivered by cognitive tutors or clickstreams of online courses (Baker & Yacef, 2009). However, there is increasing interest in promoting types of learning activities and outcomes that are non-standardized, engaging students in constructionist (Papert, 1980) open-ended tasks such as designing robots or programming computers. In this work, we present a machine-learning-based framework to predict the performance and level of expertise of students based on their coding style. We will show results from a study in which we capture and analyze logs of undergraduate students (n=150) learning to program in an introductory Java course. In addition to logfiles, we also had students fill a questionnaire about their programming style, motivation, and previous experience. As students learn to write program, they develop their own distinctive style of coding (top-down vs. bottom-up, “planners” vs. “tinkerers,” Turkle & Papert, 1991). The capture of these styles and their evolution during a computer science course could point to cognitive changes as well, and point to optimal points of intervention. Therefore, we analyze code fragments written by students for various assignments, based on fundamental concepts such as recursion, functional programming, object-oriented programming, error-handling, looping, etc. The code fragments committed to a global repository, giving us access to the each incremental time-stamped step in the process. First, we calculate differences between each successive commits (or “diff” statistics) by a student, in terms of the number of lines and characters added/deleted/reordered/modified, control-flow blocks added/deleted, and comments added. This gives us an approximate idea of the increments made by the student in each successive step and the time interval between steps. The individual code fragments are separately compiled and executed to capture their runtime errors and compiler errors. This results in several time-series of various diff statistics at each commit step, along-with the compiler or run-time errors. Using Dynamic Time Warping to calculate the distance between the time-series, we hierarchically cluster the timeseries data of students, and use silhouette values to decide the cutoff for the number of clusters. We then analyze
© ISLS 135

ICLS2012

Volume 2: Symposia

the clusters for certain underlying “coding trends” which all students in that cluster share. The overall idea is to first identify the features influencing the coding style of a student, and then use them as features to group students into clusters. Finally, we cross those results with self-reported questionnaires on style, motivation, and experience. Preliminary results show distinctive changes in the style and learning pattern for self-declared “topdown” and “bottom-up” programmers, as well as connections between the evolution of coding styles and expertise.

AMOEBA: Mining how students learn to program together
Matthew Berland, University of Texas at San Antonio Taylor Martin, Tom Benton, Carmen Petrick, University of Texas at Austin This work details findings from AMOEBA, a new system to describe from a large dataset how students learn to program collaboratively. Work such as Hancock (2003) and Guzdial (2003) have shown key processes in how students learn to program through collaboration, but human observation is limited in the scale at which process data can be collected, the time frame over which it can be considered. Based on a single 90-minute session with students ranging from beginning to more experienced programmers, we were able examine student programming at a fine grain to produce a meaningful description of how students collectively learned to program, where similarities arose, and how students’ programs differed. To explore student patterns of programming, we built the IPRO programming environment to collect those fine-grained data. The IPRO programming language was designed to foster quick programming on the fly (“tinkering”); it utilizes a restricted library of sensors, logical operators, mathematical operators, and actions. IPRO programs are immediately executable, and no syntax errors are possible in the environment. We designed IPRO with these features so that: student programs can be easily categorized and analyzed; it is easy to start programming with no background or prior knowledge; and students can share code easily. Berland (2008) and Hancock (2003) both describe how features such as reusable code and easy testability engage novices by allowing them to learn as they make rewarding progress on their programs. We deployed the system to 53 students across 16 teams (of 3-4 virtual robots each) on the first day of a girls’ programming summer camp. Each student made between 100-500 edits to her programs over the course of 90 minutes. The new programs produced by each edit were automatically evaluated by the system in terms of how successful that program was at scoring goals. Metrics were created to evaluate similarity of any two edits as well as uniqueness of a program at a particular time. To investigate these relationships, we built AMOEBA. AMOEBA explores the log data by visualizing patterns of similarity across programs. AMOEBA tracks the similarity of all programs of each of the 53 students on 16 teams over the course of 90 minutes. In short, our similarity metric reports how many subcomponents of the program are found to be in common with all of the other students' programs in existence at that moment during the class. Similarity is scored by the number of components in common times the inverse frequency of that subcomponent in the corpus as a whole (similar to tf-idf as per Salton, 1989). AMOEBA in Figure 1 (left), below, shows “connections” between students where a connection is any similarity between two programs that is unlikely to be random. Those connections are marked with the most unlikely subcomponent that the two students' programs have in common. Note that the students are groups by team in the diagram, to highlight when similarity is more common on teams and when it is not. We also report, as per Figure 1 (right), a “running correlation” of similarity to quality.

Figure 1 – AMOEBA system (left) and running correlation of similarity to program quality (right) Our results show that students routinely wrote remarkably similar code at the same time. As seen in Figure 2, the correlation between similarity and quality also decreased. That is, the benefit gained by writing similar program code to your peers (through sharing or through mutual understanding of a topic) decreased as the situation became more complex. By matching AMOEBA log data to video, we noted that most of the
© ISLS 136

ICLS2012

Volume 2: Symposia

unlikely similarities were not due to students sharing much program code. Indeed, this is borne out by the data, which show few working programs replicated in their entirety across the classroom in the last half of the class. These results have two interesting implications for learning. The first implication is that not all sharing and collaboration are created equal: that collaboration needs to be situated in necessity (as per Brown, Collins, & Duguid, 1989) appears to bear out even at a fine-grained level. Second, students learning to program in a complex system can react to that system in ways that are similar enough as to be very unlikely.

Student Attributes and Carelessness in Science Microworld
Arnon Hershkovitz, Michael Wixon, Ryan Baker, Janice Gobert, Michael Sao Pedro, Worcester Polytechnic Institute Student attributes – including student goals, attitudes, and beliefs – play a key role in their learning outcomes. These attributes might impact learning by creating different forms of disengagement. One disengaged behavior is carelessness, i.e., when a student fails to answer an item correctly despite possessing the required skills (Clements, 1982). We have operationalized carelessness using an automated detector of contextual slip, i.e., the probability that the student performed incorrectly at a specific time despite knowing the needed skill (Baker, Corbett, & Aleven, 2008). The notion of contextual slip matches previous carelessness definitions, but is easier to apply than previous operational definitions. Our detector uses a log-based machine-learned model, hence can be scaled without being overly time-consuming. We study carelessness in demonstrating science inquiry skills (e.g., control for variable strategy) in the context of a phase change activity using microworlds. Participants are 148 eighth grade students, aged 12-14 years old, from a public middle school in Central Massachusetts. Before taking the activity, the participants took the Patterns of Adaptive Learning Scale (PALS) survey. All students’ fine-grained actions were logged and then analyzed at the “clip” level; a clip is a consecutive set of a student’s actions describing activity in its context. Carelessness detector was developed based on 73 features describing the clips (N=2114), and using REPTree, a regression tree classifier (resulted in a 6-fold cross-validation correlation of r=0.63). Overall, mean carelessness was low with a value of 0.05 across clips (SD=0.16) and 0.06 (SD= 0.05) across students (N=130). Students were then clustered based on their PALS measures, resulting in three clusters: Learning Goals, Performance Goals, and Lack of Goals. Surprisingly, it was found that mean carelessness in the Lack of Goals cluster was significantly lower from its mean in both Learning Goals and Performance Goals clusters. Furthermore, we studied how carelessness is being manifested over three consecutive activities. Overall, carelessness increased over all trials (as students have more practice opportunities for the same skill), however there were important differences among the clusters: In the Learning Goal cluster mean carelessness does not significantly increase from activity 1 to 2, nor from activity 2 to 3, but it is significantly higher in activity 3 compared to activity 1; in the Performance Goal cluster mean carelessness significantly increases between activities 1 to 2, but does not significantly increase between activities 2 to 3; and in the Lack of Goals cluster there is no significant differences were found in mean carelessness for either pair of activities. This work shows that students characterized by mastery or performance goal orientation have (on average) double the probability of carelessness as compared to students characterized by low scores for these goal orientations. It is possible that students with higher amounts of mastery or performance goals succeed in learning and correspondingly become more confident, and following that they tend to carelessness despite their goal orientation. Differences in manifestation of carelessness over consecutive trials demonstrate important individual differences which might imply on timely scaffolding for those students who need to be encouraged to be more focused during learning. The methodology used in this research demonstrated the strength of using student log files and data mining methods, as the current operationalization of carelessness might be relatively easily (under limitation of, e.g., transferability) large-scaled.

Computing student science conceptions with Latent Dirichlet Allocation
Bruce Sherin, Northwestern University Computationally-based analytic methods are becoming part of the standard toolkit employed by researchers in the learning sciences, and education more broadly. But, as a field, we have only scratched the surface of what is possible. Some techniques, such as Latent Semantic Analysis, have seen extremely wide use (Landauer, Foltz, & Laham, 1998). But the popularity of these techniques might be accidental; they were employed, with great success, by a few researchers (e.g., Graesser, Wiemer-Hastings, & Wiemer-Hastings, 2000; Magliano, WiemerHastings, Millis, Munoz, & McNamara, 2002; Wade-Stein & Kintsch, 2004). This initial success then led to their wide adoption. But we should be aware that there are many other existing methods, that are sometimes more appropriate to the task at hand. Here I will discuss one such method Latent Dirichlet Allocation (LDA), that can

© ISLS

137

ICLS2012

Volume 2: Symposia

be applied to solve problems that have been solved by LSA (Blei, Ng, & Jordan, 2003). Although these two methods can be applied to some similar tasks – and although they both have the word “Latent” in their names – they are actually quite different. Furthermore, in some cases, LDA is more powerful and more appropriate. In the remainder of this summary, I will first explain the research context. Then I will briefly explain LDA and how it can be employed in this context. Over the last few decades, there have been literally thousands of research studies devoted to the study of students’ alternative conceptions in science (Duit, 2009). In many cases this research proceeds as follows: (1) We interview some students about a target domain or phenomenon. These interviews are videotaped. (2) We transcribe the videos. (3) We somehow extract a set of student conceptions from the transcripts. This generally requires repeated reading of the transcript and viewing of the video. And it usually involves some type of hand coding of the transcripts. What I would like to do is to employ learning analytic techniques to perform the tasks in step (3); I want to give transcripts to a computer algorithm, and have it extract student conceptions. In particular, the data I will discuss is drawn from a corpus of clinical interviews in which middle students were asked to explain the Earth’s seasons. Thus, I would like my learning analytic algorithms to extract students’ seasons-related conceptions from transcripts of these interviews. There are two additional complications. First, I adopt a knowledge-in-pieces perspective (diSessa, 1993). This means that I do not assume that students have existing models of the seasons. Instead, in many cases, they construct models of the seasons from fragments of knowledge. Thus, I actually want my analysis to identify these fragments, rather than full-blown models. Second, I expect that the fragments that a given student draws on may change as an interview unfolds. For this reason, I will not want to analyze transcripts, viewed as a whole. Instead, I must break each transcript into segments and determine the knowledge fragments associated with each segment. LDA is well-suited to this task. Although the computational algorithms that perform LDA are quite complicated, the underlying conception is straightforward. In LDA, a corpus of text is modeled by a set of topics. In my case, these topics are the fragments. Each of these topics/fragments is associated with a probability distribution over a set of words. If a particular fragment is active, then it will generate words according to this probability distribution. Finally, each segment of a transcript is modeled as a mixture of the fragments. This means that, within the segment, a particular fragment is chosen with a probability determined by this mixture. Then a word is generated according to the probability distribution associated with the fragment that was selected. This imaginary process iterates, once for each word, generating a text. This leads to a computational problem that must be solved. We have to discover the topics/fragments, the probability distribution of terms associated with each fragment, and the mixture of fragments associated with each segment of text. This is a difficult computational problem, but it is one that has been solved for us (Blei, et al., 2003). The bottom line, as I will describe in my talk, is that the application of LDA to the seasons corpus produces an analysis that is sensible, and that aligns extremely closely with the work of human analysts; it discovers similar fragments, and it also produces a sensible account of the fragments of knowledge that are drawn on as each interview unfolds. I will conclude with a comparison to a similar analyses I performed using LSA.

Automated Task Adaptation to Support Students’ inquiry Learning
Ido Roll, University of British Columbia, Vincent Aleven, Kenneth R. Koedinger, Carnegie Mellon University. Constructivist instructional activities hold the promise of helping students acquire transferable knowledge (Duffy & Cunningham, 1996; Tobias & Duffy, 2009). However, research often shows that students fail to acquire the desired learning goals when learning from constructivist instruction (Kirschner, Sweller, & Clark, 2006; Tobias & Duffy, 2009). The challenges of learning from constructivist instruction are most apparent in exploratory learning environments and scientific inquiry activities. In these activities students are expected to reveal an underlying model that governs the behavior of given data or simulation (de Jong & van Joolingen, 1998). One source of difficulty in scientific inquiry tasks is the relative lack of explicit support at the domain level. Since students are expected to discover the deep structure of the domain and learn by a process of exploration, inquiry environments usually withhold domain knowledge. For example, the Invention Lab (Figure 2) asks students to invent a method for calculating spread based on given data (Roll, Aleven, & Koedinger, 2010). Students are not given direct instruction on how to use standard deviation (or other alternatives), as this will short-circuit the reasoning process. At the same time, in the absence of explicit guidance, students often default to solutions that are partial at best. For example, many students simply settle for range (that is, max minus min), to measure spread (Roll, Aleven, & Koedinger, 2011).

© ISLS

138

ICLS2012

Volume 2: Symposia

A common solution to the lack-of-support problem is often given in the form of process scaffolding or the use of cognitive tools. For example, students are often prompted to raise hypotheses and test them. However, it seems that process support alone, without complementary domain-level support, is insufficient for novice students (Mulder, Lazonder, & de Jong, 2009). Two main challenges limit the ability to give domain-level support during scientific inquiry tasks. First, as suggested above, explicit support is likely to short-circuit the desired sense-making processes. Second, given the high agency that students have in scientific inquiry tasks, students often explore the domain in different directions. Thus, support cannot be “one size fits all”, and canned responses cannot address many of the situations in which students are in need for support.

Figure 2: A Typical Task in the Invention Lab asks students to invent a general method for calculating the variability of the given data. In this talk we demonstrate the potential of using students’ moment-by-moment traces to offer domainlevel support in scientific inquiry tasks. First, we suggest that adapting the task can be a productive mean for giving support in a constructivist environment. By adapting the task to students’ demonstrated difficulties one can create tasks that are within students’ ZPD, thus addressing the need for explicit support in overly-complex situations. Second, adapting the task does not reduce the autonomy and agency of the learner, thus adhering to the constructivist principles of inquiry learning. Last, task adaptation does not short-circuit the reasoning behavior. Putting it more figuratively, while explicit support helps learners reach the goalpost, task adaptation brings the goalpost closer to the learner. Second, we suggest that the combination of constrained based modeling (Mitrovic, Koedinger, & Martin, 2003) and symbolic modeling (Anderson et al., 2004) is best suited for adapting the task to individual students. Specifically, the Invention Lab analyzes students’ invented models in real time. Since no two models are identical, the Invention Lab looks for pre-defined desired features in these models. For example, does the model uses all the given data? (While range uses only the extreme data points, using all data points is essential to capture the true distribution of the data). By identifying the shortcomings of students’ invented models, the Invention Lab creates new sets of data for students to analyze. The new data targets these knowledge gaps one by one. For example, students who “invent” range may be asked to compare two data sets with distinct distributions and a common range (e.g., {1,4,4,5,5,8} vs {1,2,3,6,7,8}). This iterative process of adapting available data to students’ invented methods is assumed to achieve two goals. First, it breaks down the problem and allows students to tackle the deep concepts of the domain one-by-one. Second, the iterative invention using adaptive data sets encourages students to integrate the different features of the data into a unified schema. In addition, the invention lab uses its knowledge of students’ moment-by-moment actions to offer feedback on students general inquiry behaviors. Log and assessment data from an evaluation of the Invention Lab in six grade nine classes (n = 92) was used to evaluate the modeling approach of the lab. Overall, this work suggests that students’ trace data can be used to offer adaptive, individualized support in a manner that does not reduce critical elements of inquiry learning.

References
Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., & Qin, Y. (2004). An integrated theory of the mind. Psychological Review, 111(4), 1036–1060

© ISLS

139

ICLS2012

Volume 2: Symposia

Baker, R.S.J.d., Corbett, A.T., Aleven, V. (2008). More accurate student modeling through contextual estimation of slip and guess probabilities in Bayesian Knowledge Tracing. In Proceedings of the 9th International Conference on Intelligent Tutoring Systems, 406-415. Baker, R.S.J.d., Yacef, K. (2009) The State of Educational Data Mining in 2009: A Review and Future Visions. Journal of Educational Data Mining, 1 (1), 3-17. Berland, M., Martin, T., Benton, T., & Petrick, C. (2011). Programming on the move: design lessons from IPRO. Proceedings of the 2011 annual conference extended abstracts on Human factors in computing systems (pp. 2149–2154). Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet Allocation. Journal of Machine Learning Research, 3(4/5), 993-1022. Blikstein, P. & Worsley, M. (2011). Learning Analytics: Assessing Constructionist Learning Using Machine Learning. Paper presented at the Annual meeting of the American Research Education Association, New Orleans, LA. Blikstein, P. (2009). An Atom is Known by the Company it Keeps: Content, Representation and Pedagogy Within the Epistemic Revolution of the Complexity Sciences. Unpublished PhD. dissertation, Northwestern University, Evanston, IL. Blikstein, P. (2011). Using learning analytics to assess students’ behavior in open-ended programming tasks. Paper presented at the I Learning Analytics and Knowledge Conference. Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18 (1), 32-41. Bruner, J. (1999). Postscript: Some reflections on education research. In E. C. Lagemann & L. S. Clements, M.A. (1982). Careless errors made by sixth-grade children on written mathematical tasks. Journal for Research in Mathematics Education, 13, 136-144. Cobb, P., Confrey, J., diSessa, A. A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9-13. Confrey, J. (2005). The evolution of design studies as methodology. The Cambridge handbook of the learning sciences, 135-151. de Jong, T., & van Joolingen, W. R. (1998). Scientific discovery learning with computer simulations of conceptual domains . Review of Educational Research, 68, 179-201. diSessa, A. A. (1993). Toward an epistemology of physics. Cognition and Instruction, 10(2 & 3), 165-255. Duffy, T. M., & Cunningham, D. J. (1996). Constructivism: Implications for the design and delivery of instruction. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology. (pp. 170-98). New York: Macmillan. Duit, R. (2009). Bibliography: Students' and Teachers' Conceptions and Science Education. Kiel, Germany: Leibniz Institute for Science Education. Edelson, D. C. (2002). Design research: What we learn when we engage in design. The Journal of the Learning Sciences, 11(1), 105-121. Graesser, A. C., Wiemer-Hastings, P., & Wiemer-Hastings, K. (2000). Using Latent Semantic Analysis to Evaluate the Contributions of Students in AutoTutor. Interactive Learning Environments, 129-147. Guzdial, M. (2003). A media computation course for non-majors. Proceedings of the 34th ACM SIGCSE Technical Symposium on Compuer Science Education. Hancock, C. M. (2003). Real-time programming and the big ideas of computational literacy. MIT. Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75-86. Landauer, T., Foltz, P. W., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25(2-3), 259-284. Magliano, J. P., Wiemer-Hastings, K., Millis, K. K., Munoz, B. D., & McNamara, D. (2002). Using latent semantic analysis to assess reader strategies. [Empirical Study]. Behavior Research Methods, Instruments & Computers, 34(2), 181-188. Mitrovic, A., Koedinger, K.R., Martin, B.: A comparative analysis of cognitive tutoring and constraint-based modeling. In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS (LNAI), vol. 2702, pp. 313–322. Springer, Heidelberg (2003) Mulder, Y. G., Lazonder, A. W., & de Jong, T. (2009). Finding out how they find it out: An empirical analysis of inquiry learners' need for support. International Journal of Science Education, 1-21. Nemirovsky, R. (2011). Episodic Feelings and Transfer of Learning. The Journal of the Learning Sciences, 20(2), 308-337. Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N., Eastmond, E., Brennan, K., Millner, A., et al. (2009). Scratch: programming for all. Communications of the ACM, 52(11), 60–67.

© ISLS

140

ICLS2012

Volume 2: Symposia

Roll, I., Aleven, V., & Koedinger, K. R. (2010). The invention lab: Using a hybrid of model tracing and constraint-based modeling to offer intelligent support in inquiry environments. In V. Aleven, J. Kay, & J. Mostow (Eds.), Proceedings of the international conference on intelligent tutoring systems. (pp. 11524). Berlin: Springer Verlag. Roll, I., Aleven, V., & Koedinger, K. R. (2011). Outcomes and mechanisms of transfer in invention activities. In L. Carlson, C. Hölscher, & T. Shipley (Eds.), Proceedings of the 33rd Annual Conference of the Cognitive Science Society (pp. 2824-2829). Austin, TX: Cognitive Science Society. Salton, G. (1989). Automatic Text Processing: the Transformation, Analysis, and Retrieval of Information by Computer. Boston: Addison-Wesley Longman. Shulman (Eds.), Issues in education research: Problems and possibilities (pp. 399-409). San Francisco: Jossey-Bass Publishers. Siegler, R. S., & Crowley, K. (1991). The microgenetic method: A direct means for studying cognitive development. American Psychologist, 46(6), 606-620. Tobias, S., & Duffy, T. M. (2009). Constructivist instruction: Success or failure? New York: Taylor & Francis. Wade-Stein, D., & Kintsch, E. (2004). Summary Street: Interactive computer support for writing. Cognition and Instruction, 22(3), 333-362.

Acknowledgments
For “Automated Task Adaptation to Support Students’ inquiry Learning”: This work was supported by the Pittsburgh Science of Learning Center, which is supported by the National Science Foundation (#SBE0836012), and by the University of British Columbia through the Carl Wieman Science Education Initiative.

© ISLS

141

ICLS2012

Volume 2: Symposia

The Use of Game Design, Social Learning Networks, and Everyday Expertise to Engage Youth with Contemporary Science
Philip Bell, Leah A. Bricker, Katie Van Horne, Theresa Horstman University of Washington, 1100 NE 45th St., Suite 200, Seattle, WA 98105, Email: pbell@uw.edu, lbricker@uw.edu, katievh@uw.edu, thorst@uw.edu Nichole Pinkard (Discussant), DePaul University, 243 South Wabash Avenue, Chicago, IL 60604 npinkard@cdm.depaul.edu

Abstract: Developments in information technology and digital media offer the opportunity to dramatically reorganize the curricular experiences available to students. This symposium explores elements of a next-generation high school biology course design, development, and implementation project that utilizes a social networking/media platform, facilitates youth participation in contemporary science, and networks youth with scientists and professionals. Specifically, we aim to study aspects of the platform that allow us to investigate learning related to: (a) the integration of game mechanics into the design of instructional materials, (b) youths’ engagement in contemporary scientific practices, and (c) platform features that make youths’ interests, expertise, and experiences visible so that they can be leveraged in instruction. We explore how to promote generative conditions for expertise development within these technology-mediated social learning networks and the effects of those experiences on student participation, identification, and learning of knowledge, skills, and practices.

Introduction
This symposium explores elements of a next-generation high school biology course design, development, and implementation project that utilizes a social networking/media platform, facilitates youth participation in contemporary science, and networks youth with scientists and other relevant professionals. Disciplinary experts give students feedback on their projects and share their career and educational trajectories. In this symposium, researchers present conceptual frameworks associated with the various aspects of the project model, as well as preliminary data, using the course’s infectious disease and genetics units as a case study. Specifically, we aim to study aspects of the social networking/media platform (henceforth referred to as the “learning platform”) that allow us to investigate learning related to: (a) the integration of game mechanics into the design of instructional materials, (b) youths’ engagement in contemporary scientific practices, and (c) platform features that make youths’ interests, expertise, and experiences visible so that they can be leveraged in instruction. After an overview of the project writ large from which this case study stems, we present three elements of the case study units, including the associated conceptual frameworks guiding design and development, and some preliminary analyses. The first presentation examines the conceptual terrain of a badge system designed for both the infectious disease and genetics units. The second presentation examines the conceptual terrain associated with youths’ engagement in contemporary scientific issues and practices and presents a preliminary analysis of participating youths’ explorations in this arena.. The third presentation examines the conceptual terrain associated with design strategies that attempt to bridge youths’ out-of-school interest-driven pathways with their in-school mandate-driven trajectories and how the learning platform can serve as a bridging object.

Developing and Studying a Social Learning Network Model in the Context of Next Generation High School Courses
Developments in information technology and digital media offer the opportunity to dramatically reorganize the curricular experiences available to students (United States Department of Education, 2010). Collectives of students and teachers can engage in sustained, collaborative projects through cultivated social learning networks. Students can use social media to more easily connect to disciplinary experts and receive feedback and mentoring. They can use information technology to meaningfully participate in contemporary disciplinary pursuits, allowing for more participatory forms of project-based learning (e.g., citizen science, participatory youth models). Digital media technologies allow students to consume and produce a broad variety of rich media sources that relate to their learning investigations. Students can also use pervasive technologies to extend their learning experiences across the settings of their lives. We are developing an educational enterprise that integrates these learning affordances into three next-generation courses in the subject areas of biology, English language arts, and algebra. We have leveraged and customized the Remix platform to provide a cloud-based technological infrastructure for our social learning network (cf. Nichole Pinkard, 2007,

© ISLS

142

ICLS2012

Volume 2: Symposia

http://remixlearning.com/). In this symposium we use two units in the biology course (infectious disease and genetics) as case studies to explore theoretical terrains associated with some of our design principles. Building upon a history of research on leveraging Internet technologies in instruction in the science classroom (e.g., Linn, Davis & Bell, 2004), we approach the cultivation, study, and refinement of social learning networks as design-based research (Bell, 2004) that sets the stage for large-scale design-based implementation research in the near future (see Penuel, et al., 2011). We pursue a sociocultural learning perspective to attend to the development of everyday expertise across social settings and networks of actors over developmental timescales (Bell et al., in press). We build upon theoretical traditions that highlight sophisticated learning as participation in repertoires of practice (Gutiérrez & Rogoff, 2003) and apprenticeship processes, which occur within a nexus of structures of critical social practice (Dreier, 2009; Lave & Wenger, 1991). Youth are positioned as developing experts who express agency through extended projects designed to overlap with practices and interests of their home communities. We leverage the sense-making practices of youth and relate them to the practices of disciplinary fields. Youths are brought into sustained social interaction with networks of disciplinary experts who model expertise and guide youths’ project work. Youth authentically engage in interdisciplinary intellectual work and report their results to peers, teachers, disciplinary experts, and community members through multiple mechanisms.

Research Program and Scholarly Significance
A multifaceted research program explores: (a) how to promote generative conditions for expertise development within these technology-mediated social learning networks across classrooms and (b) the effects of those experiences on student participation, identification, and learning of knowledge, skills, and practices. We leverage the affordances of the technology platform to develop case studies that document and help visualize relevant phenomena, supplemented with video recording of classroom life and artifacts associated with the project work of youth. Through these efforts, we hope to develop an educational model for promoting sustained disciplinary learning across settings that builds on the social and intellectual capital of youth and extends their educational pathways.

Paper 1: Designing Badges for Use in a Project-Based Learning Curriculum Facilitated by a Social Media Platform
The use of video games, games for learning, and game-based education is increasing in formal and informal educational settings (Salen, 2008; Gee, 2009). Also garnering more attention is the related gamification movement, using game mechanics in typically non-game learning environments to increase motivation and participation (Deterding, et al. 2011). Though games and game mechanics have prompted innovative applications in education (most recently the Mozilla Open Badges https://wiki.mozilla.org/Badges and Digital Media and Learning competition http://dmlcompetition.net/Competition/4/badges-competition-cfp.php), the practices for how to do so successfully remain elusive and can benefit from further investigation. Designing games for learning is a complicated endeavor that leverages the practices of game design and instructional design combined to create engaging opportunities for students to learn without compromising the depth of instruction or the authenticity of the disciplines. Though this sounds ideal, actual implementation is filled with complex challenges. This paper examines one such combination of game and instructional design through the conceptual design and partial implementation of a badge system in a project-based curriculum facilitated by a social networking/media platform. This paper attempts to address issues which may contribute to answering the following questions: (a) How can we successfully design a badge system for educational settings?, and (b) What are the strategies for successfully reconciling differences between game and instructional design without compromising the qualities of game play or the opportunities for students to make meaningful contributions to contemporary scientific issues?

Research Focus
To answer these questions, the focus of this research examines three key components, which informed the conceptual design of the badge structure. First, we detail an analysis of existing badging systems in order to glean the appropriate design strategies to meet the needs specific to the curriculum. This includes an account of military and scout badges and the structure of achievement systems and badges in different video games to better understand the purpose of badges in specific settings. Second, we explicate a detailed framing of the instructional strategies and learning objectives that the infectious disease and genetics units are designed to address. In addition, the project has a commitment to provide students learning opportunities that put the learners at the forefront of contemporary scientific issues and knowledge production by positioning learners as developing experts. These components need to be accounted for in the design of the badge system and the question that remains is how. Lastly, we examine

© ISLS

143

ICLS2012

Volume 2: Symposia

constraints and affordances of the social media platform itself including the technical capabilities as it informs the design of the badge system relative to the curricular demands.

Conceptual Framework and Study Design
The conceptual framework for this paper is rooted in game design (Crawford, 2003; Rollings & Adams, 2003; Salen & Zimmerman, 2003; Schell, 2008). This sets the foundation for considering the tensions created by combining game design with instructional design practices in designing a badge system. Though there are many methods and multi-faceted purposes to both game design and instructional design, this conceptual framework is limited to considering general purposes of each and the implications to design decisions. A comparison of the general purposes of game and instructional design leads to discussions on approaches to systems design, the tension between innovation and standardization, and designing a badge system that serves the unique needs of the curriculum. The study design is a combination of design based research (Design Based Research Collective, 2003) and conceptual analysis. The research consists of iterative cycles of integrating game design theory with instructional design practices in designing and partial creation of a badge system and then enacting it in the infectious disease unit and genetics unit. The infectious disease unit is a five-week unit that was piloted twice; in both the Spring and Fall of 2011. In the unit students designed and carried out investigations of local or global transmission of infectious disease using contemporary scientific software tools and the learning platform. The genetics unit is a six-week unit piloted in the Winter of 2012. The conceptual analysis includes a closer examination of the anticipated purposes of the badge system (inspire, mark interest and student identification, provide diagnostic feedback, acknowledge accomplishments, and make learning pathways visible).

Preliminary Data and Badge System Specifications to Date
This paper presents preliminary data and system specifications given the unit and badge system are still in enactment at the time of writing. The conference paper will expand on this analysis and discuss preliminary results. The design of the current badge system in development strategically awards badges to students (a) at significant moments in project progression, (b) when they make connections across disciplines, and (c) during instances of practice related to learning platform use and out-of-school community involvement. At the core are four main badges that represent culminating moments in the curriculum. For example, the scientific investigator badge is awarded to students for participating and completing project work in either global epidemic modeling or local social network analysis. To earn this badge, students must generate a testable question and propose a research design to progress in their investigation and reflect on how they would pursue future studies at the conclusion of the unit. The culminating scientific argumentation badge involves incremental levels of badges awarded during all aspects of argumentation work built into the unit. These levels include: (a) identifying and describing how argumentation is built into an activity with which they claim they have deep expertise, (b) identifying the “rules” of argumentation in their self-identified activity, and (c) utilizing the class-generated set of scientific argumentation rules to craft a scientific argument related to his/her project research and findings. There are four additional badges that act as support structures to these core badges and reinforce learning platform and community involvement. Two additional badges requiring work above and beyond the coursework are designed to be particularly difficult to acquire. With the exception of these last two difficult badges, all groups of badges consist of mandatory tasks (required to achieve the culminating badge) and optional badges (as opportunities to pursue additional interests). Other badge attributes such as hidden, automatic, peer-awarded, teacher-awarded, expert-awarded, repeatable, unlocking, emergent, limited, and expired are distributed through the badge infrastructure. Iterative design and conceptual rework of the badge system as we move through enactment cycles will continue to inform the system design.

Implications for Designing Educational Badges
We anticipate the process of interpreting previous badge systems, instructional framing of learning objectives through core ideas and practices in contemporary science, and the technical support and restrictions of the platform as central to understanding the success of a badge system. At the conclusion of enactment and through the iterative design of the badge system refined by student feedback, analysis of student work and badge artifacts, and design analysis, we will report upon the following conditions for designing educational badges: Defining a clear purpose of the badge system, determining which pieces of student activity are badge worthy, complicating the learning space to bridge connections not otherwise visible to learners, and lastly, examining design consequences of keeping within the technical restrictions of the platform functionality. Calling attention to specific attributes of learning through the incentive of a badge system offers a unique opportunity to make explicit learning that may otherwise go unnoticed. It is also an opportunity to shape and influence the way in which students engage in the curriculum both for individual activities and with the curriculum unit as a whole. However, it is important to recognize that neither game design nor instructional

© ISLS

144

ICLS2012

Volume 2: Symposia

design approaches, independent of each other, meet the design needs of an educational badge system. It is the integration of both approaches, unique to each context that will most likely produce the desired results.

Paper 2: Networked Learning and Engagement in Contemporary Scientific Practices
Student participation in contemporary scientific practices, such as global epidemic modeling, requires intensive engagement within networked learning contexts. Learning analytics provide a set of tools to allow educators to trace students’ learning pathways as they gain competence with the scientific practices. The field of learning analytics represents a new analytic direction in education, its roots stemming from web analytics and business intelligence (Shum & Ferguson, 2011). Researchers collect data such as learners’ interests, learning preferences, interaction dynamics with resources, disciplinary experts, educators, and fellow learners, and sense-making strategies and routines (e.g., Retalis et al., 2006). Because this is a new analytic direction in education, few researchers in K-12 education are working in this arena to date. We highlight the conceptual terrain of a learning analytic approach used to study student learning of contemporary scientific practices in a six-week infectious disease unit and a six-week genetics unit. In addition, we present preliminary analysis guided by two research questions: (a) How does participation in the social learning network provide opportunities for deepening participation in disciplinary practices? (b) How is exploration, sharing, and deepening of interests evident in the social learning network? (Nacu & Pinkard, 2011)

Theoretical Framework and Methodological Approach
Emphasizing mastery of contemporary scientific practices helps foster deep understanding of the construction of scientific knowledge (NRC, 2011). Learning trajectories represent the available sets of learning experiences students can engage in to gain expertise relative to their interests. Exploration of trajectories within online learning environments allows for scaffolding integration of students’ experiences and expertise while making visible pathways for students to increase their domain-specific competencies and at the same time resourcing those pathways by connecting students to disciplinary experts, literature, materials, and tools (Pinkard & Schmidt, in preparation). This framework emphasizes the need to instructionally integrate a focus on scientific practices with possible learning trajectories in order to understand how to support learners’ engagement in communities of contemporary scientific practices. The six-week infectious disease unit, in which students investigated local or global transmission of infectious disease, was piloted twice; in both the Spring and Fall of 2011. In the six-week genetics unit piloted in Winter of 2012 students investigated and identified species using DNA barcoding and participated in a Foldit protein folding contest. This study reports on fieldnotes of classroom activity, criterion-based analysis of student products, artifacts of student – expert communication, and metrics tracking students’ interactions with online and offline technologies, their peers, and curricular resources. We employ both qualitative accounts and quantitative analytics to examine learning trajectories over the unit.

Conceptual Constructs of Learning Trajectories & Preliminary Analysis
In order to map the conceptual terrain, we lay out constructs specifically developed to direct learning analytics research on students’ expertise development in interdisciplinary science. Given a focus on fostering student engagement in contemporary scientific practices with the goal of producing knowledge and artifacts that make meaningful contributions to the disciplines, the constructs outline possible ways to account for student participation and learning in social learning network contexts. We hope to contribute to thinking about the applications of these constructs to the analysis of learner behavior on social learning networks specifically related to disciplinary science learning over relatively long timescales (months to years). The development of these constructs was modeled after Nacu & Pinkard’s (2011) conceptual framework for analyzing social learning networks that support youth’s practices of critical consumption and active production of digital media. Here we focus on some similar constructs and other constructs that are uniquely associated with science learning including: interest, participation in practices, knowledge-linked expertise, identity, engagement in learning, networked learning, production and contribution. In the preliminary analysis of the infectious disease unit, students engaged in a practice of iterative model building and simulation execution using software designed to model global disease spread. They drove this practice through use of online resources, scientific journal articles, and consultation with peers, educators, and experts. In addition, they began to see biology as a predominately interdisciplinary science (NRC, 2009) and through participation in these practices and interfacing with experts, students recognized career opportunities that require expertise at the intersection of disciplines.

© ISLS

145

ICLS2012

Volume 2: Symposia

Two representative examples demonstrate student engagement in the intellectual work of contemporary science as they interface with disciplinary experts in ways that direct their present and future learning trajectories. First, Julie talked about her interests in mathematics and computer science. She was on the robotics team at the high school and used her involvement with the team to develop her computer programming skills. She reported that her mother, a biologist, often talked with her about pursuing a career in the biological sciences. Julie did not see the connection between biology and her interests in computer science and mathematics until she began to work on computational modeling of infectious disease spread across the globe. Through iterative development and tweaking of her team’s epidemic models, she worked to identify the underlying mathematical expressions that fuel the software-generated simulations. Her daily project work always focused on understanding how her team’s research about the effects of airline travel on disease spread could be explained mathematically. She guided her team in comparing their infection curves generated from the modeling work with published data from research at the Centers for Disease Control she found online. After interacting with a computer scientist who uses computational models to study infectious disease scenarios, Julie told the researchers about how her mom has always pushed her toward biology but she did not see the connection between biology and her interests in programming until she participated in this unit. Through deep engagement with technological tools, Julie’s learning trajectory was driven by her interests and resourced by the curriculum, educators, and disciplinary experts in ways that allowed her to access knowledge about extended learning pathways that lead into careers at the intersection of biology and computer science. Second, the unit allows students to engage in collaborative knowledge production at the edges of the current research in the disciplines through access to the software tools and expert users of those tools. During the Spring 2011 unit enactment, a team chose to study the effects of poverty on the spread of infectious disease but the software’s design did not offer a straightforward way to accomplish this. Instead, the team decided to alter parameters in the model to simulate poverty. They shared this work with the principal investigator and designer of the modeling software, and he told the team that this was an excellent choice of study design given the limitations of the system. He also let them know that his team is addressing this limitation and is working on building this capability into the software. During his final review of their research he commented, “The inclusion of economic and health infrastructure indicators in large-scale simulations is an interesting direction at the forefront in the research field.” The technology platform allows students to connect with disciplinary experts and resources as they deepen their participation in contemporary scientific practices while tackling relevant problems in the field.

Implications for Sustained Disciplinary Learning
Learning analytics have affordances for researchers, educators, and learners. Online learning technologies that foreground social networking capabilities allow for data collection and analysis related to learning in ways that are powerful for (a) documenting learning, and (b) formatively giving insight to educators and learners about the learning process and personal progress. This is especially important in learning environments that support complicated and complex participation in contemporary scientific practices. In addition, aggregate learning analytics will highlight patterns and can be used to understand differences in learning influences based on instructional supports and specific learning cultures. These patterns will help identify the variety of pathways students can pursue towards expertise in contemporary scientific practices which in turn can lead to improved design and implementation of learning experiences embedded in social learning networks. Additionally, it is the first-step toward tracing the development of student identities by understanding how social learning networks connect students to scientists and other professionals and resources that provide them social capital and degrees of freedom to pursue learning pathways related to developing contemporary and interdisciplinary scientific expertise.

Paper 3: Leveraging Youths’ Everyday Expertise in Service of Engagement with Contemporary Scientific Practice: The Case of Argumentation
This paper highlights how researchers used the learning platform to surface youths’ interests, experiences, and expertise, which researchers then leveraged in the infectious disease and genetics units in service of engaging youth in scientific practices, such as the construction of scientific arguments. As previously discussed, one goal of the biology course design effort from which the papers in this symposium stem is to engage youths with contemporary scientific practices. One such practice is argumentation given its knowledge-shaping function in the sciences (e.g., Latour and Woolgar, 1986; Toulmin, Rieke, and Janik, 1984). In the infectious disease and genetics units, youths construct written products (e.g., a scientific research abstract, a proposal for funding) in which they argue scientifically using various rhetorical forms common in the sciences. Researchers working on this aspect of the biology units have a history of engaging youth with how to argue scientifically, as well as documenting youths’ everyday argumentative practices (see Bricker and Bell, 2011). Scholars have reported that it is quite difficult for youth to engage in scientific argumentation (e.g.,

© ISLS

146

ICLS2012

Volume 2: Symposia

coordinate evidence with theory) (e.g., Kuhn, 1993) and in the first iteration of the infectious disease unit, we saw evidence of this difficulty. When writing their research abstracts, youths had no difficulty constructing claims relative to their research but they did not always utilize applicable evidence from their research to support their claims. Given our research on youths’ everyday argumentation, we have argued elsewhere (Bricker and Bell, 2008) that a helpful design strategy might be to surface youths’ everyday argumentation practices and then help them code switch in order to craft scientific arguments (cf. Gumperez and Hymes, 1986). We designed opportunities into the infectious disease and genetics units to test our assertions relative to engaging youth with scientific argumentation through the use of their everyday argumentation expertise. In this paper, we outline the conceptual terrain associated with these design strategies and report preliminary data.

Using the Learning Platform to Surface Interest and Everyday Expertise
At the beginning of both the infectious disease and genetics units, youths created “About Me” profiles. They selected an avatar to represent themselves (e.g., a symbol, photographs that represented a particular interest, drawings that the youths themselves created). In addition to whatever they wanted to write about themselves, we asked them to address the following question when creating their profiles: What are you an expert at doing and/or what do you do often as a hobby/hobbies? Youth noted activities such as playing sports, playing instruments, reading, writing, socializing with family and friends, watching TV, and playing videogames. In prior research – an ethnography of youths’ science and technology learning across settings and timescales (see Bell, et al., 2006) – once we identified youths’ areas of significant interest and expertise, we asked them about argumentative practices that were embedded within their activity. The youth participating in the ethnography identified specific argumentative strategies that they utilized within any given activity context. For example, youth identified strategies for arguing with parents in order to obtain a desired object or outcome (tell the parents only the affordances of the object or outcome and leave out any cons). In another example, they identified strategies for argumentation in their sports play (e.g., make certain that argumentation contains elements of critique so that it serves a learning function in service of helping self and others improve). In addition, we gave ethnography participants digital cameras and asked them to document the various meanings they associated with argumentation-in-context-of-everyday-expertise (cf. Clark-Ibanez, 2004). We embedded many of these same activities into the design of the infectious disease and genetics units. Using areas of interest and expertise culled from youths’ “About Me” profiles, we asked youth to explicate the rules of argumentation, particularly rules related to discourse, as argumentation takes place within their areas of everyday expertise. We also asked them to photograph images related to their areas of everyday expertise and the argumentation within those areas. Once we (the community of learners participating in the unit – youth, teachers, and researchers) fully explicated the details of our everyday argumentation practices as they are situated within specific activity, we created rules for scientific argumentation, which we coded as simply another type of argumentation practice situated within a specific activity (science). We utilized the platform for this activity as well by using it to communicate with participating scientists and other professionals about their scientific argumentation practices. We report preliminary findings as part of this paper.

Implications
Utilizing youths’ interests, expertise, cultural practices, and experiences in the designs of learning environments is a well-documented learning strategy (e.g., González, Moll, and Amanti, 2005; McIntyre, Rosebery, and González, 2001; Nasir, et al., 2006). The use of social networking and media platforms enable learners to easily surface, share, compile, and trace everyday expertise. Learners can then utilize this as a springboard to learn complex disciplinary ideas and practices.

References
Bell, P. (2004). On the theoretical breadth of design-based research in education. Educational Psychologist, 39(4), 243-253. Bell, P., Bricker, L.A., Lee, T.R., Reeve, S., & Zimmerman, H.T. (2006). Understanding the cultural foundations of children’s biological knowledge: insights from everyday cognition research. In S.A. Barab, K.E. Hay, and D.T. Hickey (Eds.), Proceedings of the Seventh International Conference of the Learning Sciences (pp. 1029-1035). Mahwah, NJ: Lawrence Erlbaum Associates. Bell, P., Bricker, L.A., Reeve, S., Zimmerman, H.T., & Tzou, C. (in press). Discovering and supporting successful learning pathways of youth in and out of school: Accounting for the development of everyday expertise across settings. In B. Bevan, P. Bell, & R. Stevens (Eds.), Learning about out of school time (LOST) learning opportunities. New York: Springer. Bricker, L.A., & Bell, P. (2008). Conceptualizations of argumentation from science studies and the learning sciences and their implications for the practices of science education. Science Education, 92(3), 473498.

© ISLS

147

ICLS2012

Volume 2: Symposia

Bricker, L.A., & Bell, P. (2011). Argumentation and reasoning in life and in school: Implications for the design of school science learning environments. In M.S. Khine (Ed.), Perspectives on scientific argumentation: Theory, practice, and research. New York, NY: Springer. Clark-Ibanez, M. (2004). Framing the social work with photo-elicitation interviews. The American Behavioral Scientist, 47(12), 1507-1527. Crawford, C. (2003). Chris Crawford on game design. Indianapolis, IN: New Riders. Deterding, S., Sicart, M., Nacke, L., O’Hara, K., & Dixon, D. (2011, May 7-12). Gamification: Using game design elements in non-gaming contexts. Workshop presented at the ACM CHI Conference. Retrieved September 2, 2011, from http://gamification-research.org/wp-content/uploads/2011/04/01-DeterdingSicart-Nacke-OHara-Dixon.pdf Design Based Research Collective. (2002). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8. Dreier, O. (2009). Persons in structures of social practice. Theory Psychology, 19(2), 193-212. Gee, J. P. (2009). Digital media and learning as an emerging field, Part I: How we got here. International Journal of Learning and Media, 1(2), 13-23. González, N., Moll, L.C., & Amanti, C. (Eds.). (2005). Funds of knowledge: Theorizing practices in households, communities, and classrooms. Mahwah, NJ: Lawrence Erlbaum Associates. Gumperz, J.J., & Hymes, D. (Eds.) (1986). Directions in sociolinguistics: The ethnography of communication. Oxford, UK: Basil Blackwell. Gutiérrez, K., & Rogoff, B. (2003). Cultural ways of learning: Individual traits or repertoires of practice. Educational Researcher, 22(5), 19-25. Kuhn, D. (1993). Connecting scientific and informal reasoning. Merrill-Palmer Quarterly, 39(1), 74-103. Latour, B., & Woolgar, S. (1986). Laboratory life: the construction of scientific facts. Princeton, NJ: Princeton University Press. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. New York, NY: Cambridge University Press. Linn, M. C., Davis, E. A., & Bell, P. (2004). Internet environments for science education. Mahwah, NJ: Erlbaum. McIntyre, E., Rosebery, A., & González, N. (2001). Classroom diversity: Connecting curriculum to students’ lives. Portsmouth, NH: Heinemann. Nacu, D., & Pinkard, N. (2011) iRemix platform: Reports and visualizations. A Progress Update for the Bill and Melinda Gates Foundation. Nasir, N.S., Rosebery, A., Warren, B., & Lee, C.D. (2006). Learning as a cultural process: Achieving equity through diversity. In R.K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 489504). Cambridge, UK: Cambridge University Press. National Research Council. (2009). A new biology for the 21st century. Washington, DC: National Academies Press. National Research Council. (2011). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC: National Academies Press. Penuel, W.R., Fishman, B.J., Cheng, B.H., & Sabelli, N. (2011). Organizing research and development at the intersection of learning, implementation, and design. Educational Researcher, 40(7), 331-337. Pinkard, N. (2007). Preparing urban youth to be multiliterate. Spotlight on Digital Media and Learning, from http://spotlight.macfound.org/blog/entry/nichole-pinkard-youth-multiliterate-learners Pinkard, N., & Schmidt, R. (in preparation). Using role based pathways to ignite student learning identities. Retalis, S., Papasalouros, A., Psaromiligkos, Y., Siscos, S., & Kargidis, T. (2006). Towards networked learning analytics: A concept and a tool. In S. Banks et al. (Eds). Proceedings of the Fifth International Conference on Networked Learning 2006 (pp. 1-8). Lancaster, UK: Lancaster University. Rollings, A., & Adams, E. (2003). Andrew Rollings and Ernest Adams on game design. Berkeley, CA: New Riders. Salen, K. (2008). Toward an ecology of gaming. In K. Salen (Ed.), The ecology of games: Connecting youth, games, and learning (pp. 1-20). Cambridge, MA: The MIT Press. Salen, K., & Zimmerman, E. (2003). Rules of Play: Game Design Fundamentals. Cambridge, MA: The MIT Press. Schell, J. (2008). The Art of Game Design: A book of lenses. Waltham, MA: Morgan Kaufmann. Shum, S.B., & Ferguson, R. (2011). Social learning analytics. United Kingdom: Knowledge Media Institute, The Open University. Toulmin, S.E., Rieke, R., & Janik, A. (1984). An introduction to reasoning (2nd ed.). New York, NY: Macmillan Publishing Company, Inc. United States Department of Education. (2010). Transforming American education: Learning powered by technology. Washington, D.C.: Government Printing Office.

© ISLS

148

ICLS2012

Volume 2: Short Papers

Developing Primary Students’ Argumentation Skills in Inquiry-Based Mathematics Classrooms
Jill Fielding-Wells, Katie Makar, The University of Queensland, Australia Email: j.wells2@uq.edu.au, k.makar@uq.edu.au Abstract: Most educational research on argumentation comes from science, with argumentation in mathematics tending to focus on proof. We contend that argumentation can be used productively in learning mathematics even at the primary level. A research study was designed to explore children’s development of argumentation in an Australian primary mathematics classroom. The classroom of 23 children (aged 9-10) had regularly used an inquiry-based approach to address extended, complex, ill-structured problems. The children’s discussions and use of evidence is reported as they considered contentious media claims. The results of the design-based research study suggest that the children became proficient with Toulmin’s argument framework (simplified). They were able to use this framework to plan, implement and defend the outcomes of a mathematical investigation they designed to provide evidence for or against the media claims. The paper highlights benefits and challenges with which student grappled while making and substantiating their final claims.

Why Study Argumentation in Mathematics?
There is no natural or logical necessity to the state of mathematical knowledge at present. For as long as our results are useful, appropriate and consistent within our current understanding of the term, and given the existential position of our experience, we call them true. At any particular time, however, this is controversial, as there are generally rival theories claiming truth. What characterizes mathematics is the activity: engaging in interesting problems; making imaginative conjectures; testing; reflecting; examining results informally; formalizing and testing results formally; publishing ideas for criticism and development by the mathematical community, etc. (Lerman, 1990, p. 55). Contrast Lerman’s description of mathematical practice with the way in which mathematics is largely taught in schools. New understandings routinely come through the repetition of increasingly complex concepts and learnt procedures as students demonstrate growing competence; there are few opportunities for the practical application of mathematics through addressing ill-structured problems in real-life contexts. It is unsurprising then that students report that they fail to see the sense in mathematics or find it difficult to connect it to their lives (McPhan, Moroney, Pegg, Cooksey, & Lynch, 2008). In response to these issues, a focus on mathematical reasoning has been incorporated into many curriculum documents (ACARA, 2012; National Council of Teachers of Mathematics, 2000) with investigative and inquiry approaches frequently suggested as a way forward. Mathematical inquiry is used to address ill-structured problems, those in which the problem statement or solution strategies contain ambiguities (Makar, 2010; Reitman, 1965). Their inherent ambiguity allows for multiple interpretations of a question, a range of pathways, and numerous solutions with varying degrees of efficiency, applicability and elegance. This requires students to focus on decision making, analysis and justification. Rather than a ‘correct’ answer or strategy, there is a claim which requires evidence, explanation and defense—in short, an argument. Potential benefits to argumentation in the classroom have been welldocumented in science education where inquiry-based learning (IBL) has been promoted for some time. For example, explanation and argumentation provide students with opportunities to develop high levels of subject specific literacy (Jimenez-Aleixandre & Erduran, 2007) and enculturate them into the practices of the subject (Duschl, 2007). Research informs us however, that students must be explicitly taught the skills of argumentation (Kuhn, 1991) through suitable instruction, task structuring and modeling (Jiménez-Aleixandre & Erduran, 2007). Skills to be taught include knowing the meaning of argument, positioning, justifying with evidence, constructing arguments, evaluating arguments and reflecting on the argument process (Simon & Richardson, 2009). While argumentation is not a new concept in mathematics, argumentation research in the discipline has largely been associated with mathematical proof (see, for example, Conner, 2007; Lampert, 1990). Lampert (1990, p. 40) for example, focused on students’ use of hypothesis testing, asserting that it is “the strategies used for figuring out, rather than the answers, that are the site of the mathematical argument”. However, the approach to argumentation adopted in this paper differs from this, and describes a process by which students use IBL to examine a contextualized problem, use mathematics to obtain evidence, and then make and justify claims within

© ISLS

149

ICLS2012

Volume 2: Short Papers

the question context. In IBL, numerous solutions are possible and therefore the answer itself must be justified and defended, as well as the choice of solution strategies. The nature of the problem is such that the students are supported to determine their own method(s) of approaching and solving their problems; the mathematical solutions and strategies employed serve to provide the grounds and justification for their claims.

Structure and evaluation of arguments
One of the mostly widely used frameworks for examining argument structure is that of Toulmin, Rieke and Janik (1984). Their argument framework describes multiple elements of an argument, among which are claim, grounds, and qualifiers. The claim is the initial assertion that identifies the stance and position taken. The grounds provide the underlying support, or primary evidence, that is required to enable the claim to be accepted as reliable and valid. In practice, claims are often based on imperfect evidence and it is necessary to indicate the strength of the argument through the use of a qualifier which communicates the strength of the link between the grounds and the claim. Toulmin et al. describes breakdown in the grounds or warrants as fallacy, resulting in “arguments that can seem persuasive despite being unsound” (1984:132). Toulmin et al. identify broad categories of fallacy which can be adopted to assist in the determination of common fallacious thinking, for example, missing grounds, defective grounds or irrelevant grounds. Sampson and Clark (2006) took a broader approach and provided five criteria for teachers to assess the quality of students’ scientific arguments: (1) The nature and quality of the claim (Are students able to coordinate claims with available evidence?); (2) How far the claim is justified (Was evidence provided? Was it the right kind of evidence?); (3) If the claim accounts for all the available evidence (Was the evidence complete and comprehensive?); (4) How the argument attempts to discount alternatives (Were multiple perspectives considered?); (5) How epistemological references are used to coordinate claims and evidence (How did students gather and interpret data? Did they consider the design or methodology when weighing the evidence?). Further research into argumentation practices in IBL in primary mathematics is warranted; much of the research into argumentation and IBL has come from school and tertiary science, which cannot be assumed to be directly transferrable to learning in mathematics. IBL has been introduced relatively recently as a pedagogical approach in mathematics and, as such, there has been little research into its development in primary mathematics and still less into the development of argumentation within this context. Likewise, there is little reported in the literature in terms of what is possible in developing argumentation with young children or what supports and structures may facilitate this. This paper attempts to explore what children may be capable of in terms of the development and use of evidence in argumentation as well as supports that may assist them.

Design and Method
While learners often initially respond to inquiry questions with rapid, intuitive responses that are typically unsupported, scaffolding them through a cycle of collection, and organization and analysis of evidence, enables them to plan increasingly sophisticated statistical investigations (Fielding-Wells, 2010). This was achieved by focusing students on the inquiry question and having them collaboratively envisage evidence they might need and conclusions that could result. The current study adopted a design-experiment approach (Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003) as a methodological stance because of its aim to develop theory through multiple iterations of reflective-prospective cycles of improvement. The focus in this paper is on one cycle of a project which seeks to deepen understanding of primary age children’s developing argumentation practices. Participants were from a class of twenty-three 9-10 year olds in a large, co-educational, suburban primary school in Australia who had been involved in learning mathematics through inquiry for one and a half years. A mathematical inquiry unit designed and taught by the first author provided a topical and contentious context for the use of statistics. As a result of ongoing drought, the state water authority had spent $1.7M to purchase and issue shower timers to all households in an attempt to encourage decreased water usage through limiting shower duration, a move which received both public support and criticism. The five-week unit was implemented with approximately two 60 minute sessions per week. Prior to this unit, specific modeling and discussion of a simplified structure of an argument was undertaken. The model used by the students incorporated claim, grounds and qualification. The learning sequence was videotaped, students’ work samples were collected, and research logs maintained. The results reported here focus on excerpts from the students’ final written arguments. An adapted grounded theory approach (Corbin & Strauss, 2008; Flick, 2009) was taken to analyze the student artifacts. Each argument was subjected to open coding to identify unanticipated insights (Corbin & Strauss, 2008), sensitize the researcher to overall patterns and nuances in the data, and enable development of codes to classify aspects of the study (Flick, 2009). Next, each argument was analytically deconstructed into its individual components using Toulmin et al.’s (1984) structure of argumentation. Toulmin et al.’s fallacy schema was used to identify the frequency and nature of categories of fallacy. This level of deconstruction enabled an overall view of the developing abilities of the students as well as a closer consideration of any patterns in fallacious thinking that would indicate areas for the development of additional supports. Finally, students’

© ISLS

150

ICLS2012

Volume 2: Short Papers

responses were compared with Sampson and Clarke’s (2006) criteria for quality arguments to further suggest areas in which students were generally strong and weak in their presented arguments.

Results
In the initial phase of the inquiry, students were challenged to determine whether a tax-payer funded distribution of shower timers to all households was a cost effective decision, particularly in light of media criticism of their quality. The students collectively negotiated and refined the inquiry question: Are the [state] water commission shower timers accurate to within 10% of 4 minutes? The class designed a statistical investigation in which thirty shower timers were tested five times each. Working in collaborative groups, students timed a small number of shower timers with the results recorded, collated and distributed among the class for analysis. Their results found that 18 of the 30 shower timers came within 10% of four minutes (3:36 – 4:24) on all five tests. The students were generally able to successfully plan and conduct the inquiry and interpret their data accurately. This was anticipated in consideration of the students’ extensive prior experiences with inquiry learning. In this section, we focus on students’ final arguments of the shower timer inquiry. After completing their data collection, students were asked to write a letter to the water commissioner and report what they had found. The extent to which an individual student’s argument exhibited a claim, evidence and qualification was categorized as Evident, Developing-Inconsistent and Not Evident. As distinguishing factors, the main codes from the analysis (Table 1) frame our reporting and discussion around student work. Table 1: Categorization of claims, evidence and qualifications by students (n = 23) Evident Claim Clearly stated position Consistent throughout Foregrounded Addresses the inquiry question Ground within the scope of the inquiry question Grounds support the claim Ground acceptable within the community of practice Indicates the strength of the grounds 21 20 19 6 19 13 13 3 Developing Inconsistent 1 2 2 15 3 9 8 3 Not Evident 1 1 2 2 1 1 2 17

Grounds

Qualification

Claim
Few students experienced difficulty in producing an argument in which their claim was foregrounded, which positioned the reader clearly and which maintained consistency of position throughout the argument (Table 1). Below, we illustrate the diversity of responses from students across the more challenging area of reporting claims. In terms of students addressing the inquiry question when making their claim, some students did articulate a response specifically related to the inquiry question: Shana: The four minute shower timers produced to help introduce the campaign are faulty. Leticia: Shower timers are clarey (sic) faulty. However, it was notable that the majority of students experienced difficulty in doing so. Without exception, each of the students assigned to the Developing-Inconsistent category made a claim consistent with the broader context of the inquiry. The specific inquiry question related to the accuracy of the timers, however, and these students’ claims addressed tangential contextual issues that were not directly addressed, nor could be supported through the inquiry: Dominica: The 1.7 million dollars spent on shower timers … was a waste of money and only helped saving water a bit. In order to justify such a claim, it would be necessary to have data on the financial benefit of the water saved and the actual amount of water saved as a direct result of the shower timers. Dominica’s response was included in the Developing category as it was situated within the broader context of the inquiry rather than considering only the data gathered to address the refined inquiry question.

Grounds
Most students were able to provide sufficient grounds from the inquiry findings to enable the reader to be satisfied that their claim was justified, reasoned and supported. Notably, these students were able to demonstrate

© ISLS

151

ICLS2012

Volume 2: Short Papers

that their selection and application of mathematical processes was appropriate and efficient, and that they were able to interpret the resultant answers. Dominica: My class has provided grounds from this test. We tested every shower timer 5 tim es we worked out that 18/30 were accurate and ran in the range of 3:36 – 4:24. Fo r the other 12 shower timers they were faulty because they were out of the range … [or] they stopped during timing. Not all students were proficient in providing evidence mathematically. Despite all students having an amalgamated list of times, some students incorrectly interpreted the mathematics, or relied on non-mathematical sources for their evidence. These arguments were further examined using Toulmin et al.’s (1984) categorization of fallacy to identify areas of difficulty. The fallacious grounds put forward by students were segregated fairly evenly between the provision of irrelevant grounds (Leanne, Samuel) or defective grounds (Geneva, Blain): Leanne: Shower timers help the environment a lot because…if you have a hot shower you could use up all the hot water. Samuel: There are different amounts of sand in the timers. Geneva: They would still shorten showers that can go up to 30 minuites (sic). [No evidence was provided or discovered that gave 30 minutes as a time for a shower]. Blain: One of the timers we tested was 2:00-2:25 over 4:00. [Focused on a single outlier rather than aggregate results]

Qualification
Finally, consideration was given to whether the students had used a qualifier to indicate the strength of their findings, specifically based on the statistical acceptability of inferring numbers for a population of 1.1 million from a sample of 30. Several of the students recognized that this was a limitation despite this being a concept that is not expected to be appreciated this early in formal schooling. Andrea: From the evidence that has been collected I’m not completely sure because we only timed thirty shower timers out of 1100000. There is (sic) 3666.6 groups of 30 in 1100000 and we tested one group.

Discussion and Conclusion
In this paper, we explored potential benefits of adopting argumentation practices for primary children working in a classroom which engages in mathematical inquiry. In particular, we sought to understand the ways in which argumentation structures could be adapted into the context of mathematics education. Because mathematics is often considered to be a discipline which eschews ambiguity and lacks contention, we saw important opportunities for creating student learning experiences which might dispel this perspective. In this study, children as young as nine years old were able to take advantage of argumentation structures to develop understanding in multiple areas of mathematics. In this particular inquiry, students worked with concepts in measurement, data collection and analysis, financial mathematics and logical reasoning. Although students still found many aspects of formal argumentation challenging, what they were able to accomplish was encouraging. As an exploratory study, we were interested in what students were able to achieve as well as what they found challenging. Building on previous work (Wells, 2010), it appeared that students found the simplified Toulmin et al. framework generally useful when structuring and articulating the process of inquiry. Conversely, experience with inquiry provided the familiarity with gathering evidence and drawing conclusions that enabled argumentation practices to be developed. In designing their inquiry question, for example, the students found it beneficial to consider from the outset both the potential claims that could emerge and the evidence that would be needed in order to support and defend those claims. These were practices that had been emphasized throughout their experiences with conducting mathematical inquiry. In considering Sampson and Clark’s (2006) criteria for quality arguments, many of the students were at least partially able to appropriately coordinate their claims with the available evidence, go beyond reliance on beliefs and emotion in justifying their findings, and recognize the need to include aspects of their methodology into the reporting of findings. Being able to acknowledge limitations or qualify their findings were areas that students across the board found difficult and may be an area that is more appropriately addressed in later years. The results also suggest that students had some difficulty in stating relevant claims that addressed the specific inquiry question, an issue that several researchers have argued students find difficult when working with data in mathematics (Hancock, Kaput, & Goldsmith, 1992; Lavigne & Lajoie, 2007). However, all students that made claims that went beyond the inquiry question remained focused within the broader context of the inquiry. Four elements of the design likely contributed to students’ success in engaging with argumentation practices in mathematics. First, the students were enculturated in an inquiry-based environment that valued discussion, meaning-making and multiple perspectives. Second, the topic of the inquiry was designed around a

© ISLS

152

ICLS2012

Volume 2: Short Papers

contentious issue which challenged and provoked the students’ into providing an effective argument. Third, students were asked to respond to an authentic audience beyond the school. This aspect likely raised the need to be explicit about their method and inclusive of detail as the audience was unaware of their processes and procedures (Berland & Forte, 2010). Finally, students were scaffolded through the inquiry and the argumentation process by the teacher to ensure that they were explicitly taught foundational concepts of argumentation. This point has been raised by multiple researchers working in argumentation—that these skills need to be explicitly taught (Simon & Richardson, 2009). This study provides initial evidence that the benefits of argumentation further extend practices valued in mathematical inquiry. Argumentation works well to enculturate learners into a community of practice which values negotiation, making connections and applying mathematics beyond a procedural level. Finally, embedding argumentation into mathematical inquiry strengthens the conception of mathematics as more than a discipline of certainty, particularly when using mathematical concepts within authentic contexts.

References
ACARA. (2012). Australian Curriculum: Mathematics v3.0. Retrieved from www.australiancurriculum.edu.au. Berland, L. K., & Forte, A. (2010). When students speak, who listens? Constructing arguments in classroom argumentation. Paper presented at the 9th International Conference of the Learning Sciences, Chicago. Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9-13. Conner, A. (2007). Student teachers' conceptions of proof and facilitation of argumentation in secondary mathematics classrooms. Ph.D. dissertation, The Pennsylvania State University, Pennsylvania. Dissertations & Theses: Full Text database. Corbin, J. M., & Strauss, A. (2008). Basics of qualitative research: Techniques and procedures for developing grounded theory. Thousand Oaks, CA: SAGE. Duschl, R. A. (2007). Quality argumentation and epistemic criteria. In S. Erduran & M. P. Jimenez-Aleixandre (Eds.), Argumentation in science education. New York: Springer. Fielding-Wells, J. (2010). Linking problems, conclusions and evidence: Primary students’ early experiences of planning statistical investigations. Paper presented at the Eighth International Conference on Teaching Statistics, Ljubljana, Slovenia. Flick, U. (2009). An introduction to qualitative research (4th ed.). London: SAGE. Hancock, C., Kaput, J. J., & Goldsmith, L. T. (1992). Authentic inquiry with data: Critical barriers to classroom implementation. Educational Psychologist, 27(3), 337-364. Jimenez-Aleixandre, M. P., & Erduran, S. (2007). Argumentation in science education: An overview. In S. Erduran & M. P. Jimenez-Aleixandre (Eds.), Argumentation in science education (pp. 3-27): Springer. Kuhn, D. ( 1991). The skills of argument. Cambridge: Cambridge University Press. Lampert, M. (1990). When the problem is not the question and the solution is not the answer: Mathematical knowing and teaching. American Educational Research Journal, 27(1), 29-63. Lavigne, N. C., & Lajoie, S. P. (2007). Statistical reasoning of middle school children engaged in survey inquiry. Contemporary Educational Psychology, 32(4), 630-666. Lerman, S. (1990). Alternative perspectives of the nature of mathematics and their influence on the teaching of mathematics. British Educational Research Journal, 16(1), 53-61. doi: 10.1080/0141192900160105 Makar, K. (2010). Teaching primary teachers to teach statistical investigations: the uniqueness of intial experiences. Paper presented at the Eighth International Conference on Teaching Statistics, Ljubljana. McPhan, G., Moroney, W., Pegg, J., Cooksey, R., & Lynch, T. (2008). Maths? Why not? AAMT: Canberra. National Council of Teachers of Mathematics (2000). Principles and standards for school mathematics. Reston, VA: author. Retrieved from www.nctm.org/standards/content.aspx?id=4294967312. Reitman, W. (1965). Cognition and thought: An information-processing approach. New York: Wiley. Sampson, V., & Clarke, D. (2006). Assessment of argument in science education: A critical review of the literature. Paper presented at the International Conference of the Learning Sciences, Indiana. Simon, S., & Richardson, K. (2009). Argumentation in school science: Breaking the tradition of authoritative exposition through a pedagogy that promotes discussion and reasoning. Argumentation, 23(4), 469. Toulmin, S., Rieke, R., & Janik, A. (1984). An introduction to reasoning (2nd ed.). New York: Macmillan. Wells, J. (2010). Developing argumentation practices in inquiry based mathematics classrooms. PhD Confirmation Document. School of Education. University of Queensland. Brisbane.

Acknowledgments
This research was funded by the Australian Research Council (LP0990184) in partnership with Education Queensland and The University of Queensland. The first author is in receipt of an Australian Postgraduate Award Scholarship and also wishes to acknowledge the financial support of the Commonwealth Government.

© ISLS

153

ICLS2012

Volume 2: Short Papers

Language Formality, Learning Environments, and Student Achievement
Emily Schoerning, Brian Hand, University of Iowa, N297A Lindquist Center, Teaching and Learning, Iowa City, IA 52242 Email: emily-schoerning@uiowa.edu; brian-hand@uiowa.edu Abstract: This study examines learning environments in traditional elementary science classrooms and elementary science classrooms that utilize Argument Based Inquiry (ABI) via the Science Writing Heuristic (SWH). The degree of formality used in classroom spoken language is characterized, and its impacts on learning environments and student outcomes are explored. It is found that distinctly different styles of language characterize traditional vs. SWH classrooms, and proposed that informal language has a positive impact on student achievement.

Major Issue Addressed
This study seeks to examine and characterize the linguistic styles utilized in traditional and Science Writing Heuristic (SWH) elementary science classrooms, with a particular focus on spoken language formality. Characteristics utilized to describe language formality include fluidity of dialog, prevalence of honorifics, use of slang and popular expressions, presence of non-verbal relaxation markers, summarization of others’ speech during dialog, as well as several others. Both students’ and teachers’ use of spoken language are examined. Once the language styles typical of traditional and SWH classrooms have been characterized their relationship to the classroom’s learning environment will be explored. Connections between language formality, learning environments, and student outcomes will be identified via quantitative and qualitative methods.

Potential Significance of the Work
It has been seen that Argument Based Inquiry (ABI) and specifically the SWH increases student achievement on both tests of basic skills and critical thinking tests (Author, 2007; Lord, 1999, Mergendoller et al., 2006; HlemoSilver 2007; Dochy et al, 2003). This study will contribute to our theoretical understanding of SWH. The question of why the SWH increases student achievement, and specifically student achievement in hard to reach groups such as students with special needs, has yet to be fully unpacked (Author, 2009). It is theorized that one of the reasons the SWH is so effective is related to the learning environment it generates. Characterization of this learning environment is ongoing. As we examine the differences between SWH and control classrooms in terms of argument structures we have begun to consider that one element may underlie many of the differences we see: linguistic formality (Author, 2010). The use of formal and informal language may be different in SWH and traditional classrooms. While there are occasional mentions of language formality as a possible contributor to learning environments in the literature, little has been done to characterize what makes language formal or informal (Piirto, 2000; Carrell & Willmington 1996; Rubin, 1982; Krashen, 1976). An instrument to accomplish this goal has been developed for this study. In the course of this study the instrument’s validity and reliability will be established to allow its use by other researchers. Language formality greatly impacts our perceptions and our actions. Allowing clear characterization of this element of the classroom environment will allow for exploration of such topics as discourse space and non-threatening learning environments. The generation of a reliable and valid instrument for the characterization of linguistic formality is potentially significant to those researchers who are interested in exploring learning environments. The study’s contributions to our understanding of what factors underlie the effectiveness of inquiry-based teaching and learning would also be a potentially significant in terms of theory development.

Theoretical and Methodological Approaches Pursued
Both quantitative and qualitative methods were utilized in this research. The qualitative work consists of indepth interviews with teachers and detailed notes regarding classroom observations made at five schools: three treatment and two control. Quantitative assessment of language formality was also done during classroom observations using the Spoken Language Style Characterization Tool (SLSCT), which consists of 18 Likertscale measures assessing elements of student and teacher language. Face validity of this instrument has been determined by a panel of experts. The reliability of the instrument has yet to be determined due to ongoing data collection; internal reliability will be determined once a sufficiently large data set has been generated. If reliability is insufficient the instrument will be revised and the process repeated until reasonable reliability is

© ISLS

154

ICLS2012

Volume 2: Short Papers

achieved. Quantitative assessment of student achievement will be accomplished via two instruments; the Cornell Critical Thinking Test (CCT) and the Iowa Test of Basic Skills (ITBS). These instruments are both well-characterized and have been shown to have clear correlations with student achievement, (Ennis et al., 2005; Hoover et al., 2003). Classroom’s SLSCT scores will be compared to their ITBS and CCT scores to determine if correlations exist between language formality and student achievement. SLSCT scores will also be analyzed in the context of observation notes and teacher interviews to qualitatively describe the effect of language formality on learning environments.

Preliminary Findings, Conclusions, and Implications
Preliminary findings indicate that there are clear differences in the language styles of traditional and SWH classrooms. Teachers in traditional classrooms are more likely to emphasize the acquisition and use of formal vocabulary and require that students make formal requests to speak. Dialog between students is rare, and both students and teachers infrequently summarize each other’s contributions before adding to the dialog. These characteristics all indicate formal language use, while SWH classrooms can be characterized by their utilization of aspects of informal language. Dialog interchange is more often frequent and fluid, both students and teachers are likely to restate or summarize other’s thoughts before beginning their own, and all participants in the classroom are more likely to display non-verbal relaxation markers such as smiling, laughter, or relaxed body language. Importantly, dialog between students emerges as a valid mode of discourse within SWH classrooms. Language is the medium through which we explore and engage in science. Differences in spoken language style both reflect and form distinctly different learning environments. The clear characterization of these language styles allows us to describe and quantify differences in learning environments that before we may only have intuited or vaguely felt. Through this characterization we may be able to unpack why the SWH and other forms of ABI are beneficial to so many types of students that traditionally struggle to perform in science class (Author, 2009). It is possible that language informality gives traditionally disadvantaged students both the comfort and the confidence that allows them to meaningfully engage in science education (Duran, 1998; Rakow & Bermudez, 1993). The use of a language style common to their community and everyday lives may reduce their perceptions of science as an exclusive field, increasing participation. Implications of this study include suggestions for practitioners in regards to what language styles contribute to learning environments that foster student achievement. The study will be of interest to researchers who want to understand why and how factors in the classroom environment influence student learning, particularly as these relate to sociocultural practices, language, and science-related modes of discourse such as argumentation.

References
Akkus R., Gunel M., Hand B. (2007). Comparing an Inquiry-based Approach known as the Science Writing Heuristic to Traditional Science Teaching Practices: Are there Differences? International Journal of Science Education. 29(14), 1745-1765. Choi, A., Notebaert, A., Diaz, J, Hand, B. (2010). Examining Arguments Generated by Year 5, 7, and 10 Students in Science Classrooms. Research in Science Education. 40:149-169. Carrel, L.J, Willmington, S.C. (1996). A Comparison of Self-Report and Performance Data in Assessing Speaking and Listening Competence. Communication Reports. 9(2), 185-191 Dochy,F., M. Segers, P. Van den Bossche, D. Gijbels. (2003). Effects of problem-based learning: A metaanalysis. Learning and Instruction. 13: 533-568. Duran, B.J. (1998). Language minority students in high school: the role of language in learning biology concepts. Science Education, 82(3), 311-341. Ennis, R.H, Millman, J. (1985) Cornell critical thinking tests level X and level Z manual. Pacific Grove: Midwest Publications. Hand, B., Norton-Meier L., Staker J. (2009). Negotiating Science: The Critical Role of Argument in Science Inquiry, Grades 5-10. Portsmouth, NH: Heinemann. Hoover, H.D. et al. (2003). The Iowa tests: guide to research and development. Itasca, Illinois: Riverside Publishing Company. Hmelo-Silver, C.E. (2007). Scaffolding and achievement in Problem-Based and Inquiry Learning: A response to Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42: 99-107. Krashen, S.D. (1976). Formal and Informal Linguistic Environments in Language Acquisition and Language Learning. TESOL Quarterly. 10(2), 157-168. Lord, T. R. (1999). A comparison between traditional and constructivist teaching in environmental science. Journal of Environmental Education, 30(3), 22–28.

© ISLS

155

ICLS2012

Volume 2: Short Papers

Mergendoller, J.R, N.L. Maxwell, Y. Bellisimo. (2006). The effectiveness of problem-based instruction: A comparative study of instructional method and student characteristics. Interdisciplinary Journal of Problem-based Learning. 1: 49-69. Norman G.R, H.G. Schmidt. (2000). Effectiveness of problem-based learning curricula: theory, practice and paper darts. Medical Education. 34: 721-728. Piirto, J. (2000). Speech: An Enhancement to (Technical) Writing. Journal of Engineering Education. 21-23. Rakow, S.J., & Bermudez, A.B. (1993). Science is “Ciencia”: Meeting the needs of Hispanic American students. Science Education, 77(6), 669-683. Rubin, R.B. (1982). Assessing Speaking and Listening Competence at the College Level: The Communication Competency Assessment Instrument. Communication Education, 31, 19-32.

© ISLS

156

ICLS2012

Volume 2: Short Papers

Memetic Processes as Conceptual Framework for Idea Improvement in Knowledge Building
Karsten Krauskopf, Johanna Bertram, Stefan Huber, Knowledge Media Research Center, Schleichstr. 6, 72076 Tübingen, Germany, k.krauskopf@iwm-kmrc.de, j.bertram@iwm-kmrc.de, s.huber@iwm-kmrc.de Ya Ping Hsiao, Open University of the Netherlands, Valkenburgerweg 177, 6419 AT, Heerlen , The Netherlands, amy.hsiao@ou.nlKatherine Panciera, University of Minnesota, 4-192A KHKH, 200 Union St SE, Minneapolis, MN 55455, USA, katpa@cs.umn.edu Nicole Sträfling, University of Duisburg-Essen, Forsthausweg 2, 47057 Duisburg, Germany, nicole.straefling@uni-due.de Astrid Wichmann, Ruhr University Bochum, Universitätsstraße 150, 44801 Bochum, Germany, astrid.wichmann@rub.de Jan van Aalst, Faculty of Education, University of Hong Kong, 323 Runme Shaw Building, Pokfulam Road, Hong Kong, China, vanaalst@hku.hk Abstract: Idea improvement is a key characteristic of knowledge building where ideas are conceptualized as improvable and epistemic objects to sustained inquiry. However, despite its importance in this theory, little research exists that focuses on the process of idea improvement. In this paper we examine the development of ideas within a community of learners by proposing a conceptual framework to understand ideas as memes and investigate the memetic processes affecting them. We apply this framework to students’ discourse through a three week unit from a Knowledge Forum® data set by following the survival paths (fitness) of memes. We use a mixed methods approach to identify memes, define quantitative indicators to calculate the memes' fitness and elaborate on the improvement of the fittest memes. Benefits and open questions related to our suggested framework are discussed.

Introduction
Knowledge building focuses on the production and continual improvement of ideas of value to a community (Bereiter, 2002). The principle of idea improvement is a feature of knowledge building, where ideas are conceptualized as “real,” improvable, and epistemic objects; once an idea has been contributed to a shared knowledge space, students can ask about the assumptions underlying the idea, and modify it in various ways (Bereiter, 2002; van Aalst, 2006). One influential aspect in this process lies with the learners and their original ideas. Another influential aspect is independent from their creators: the expressed ideas can also be described as something with an “out-in-the-world existence” and a public life (Zhang, Scardamalia, Lamon, Messina, & Reeve, 2007). Once an idea has been made public it changes based on the input of the whole community. In short, the development of an idea becomes an evolutionary process with the learners as the main driving force of idea improvement (Hong & Sullivan, 2009). Research on knowledge building has recently been focussing on assessing aspects of idea improvement by observing students’ learning processes. Zhang and colleagues (2007), for instance, assess idea improvement by judging the scientific acceptability of an idea. The authors identify “inquiry threads”, which are sequences of notes that address the same problem or topic, and measure scientific levels of ideas. But assessing idea improvement in terms of scientificness does not allow judgement of relevance of ideas for the community. We propose a framework that allows us to focus on the relevance of an idea in a community of learners. This framework conceptualizes ideas as memes and then analyzes the development of these ideas through the learners’ discourse. The goal of this study is to explore idea development in analogy to the development of memes within the context of Knowledge Forum®, an online learning environment (Scardamalia & Bereiter, 2003). Therefore, we examine the development of ideas within a community of learners through the lens of memes and memetic processes. We analyze students’ discourse by following the survival paths of memes through a three week unit in the Knowledge Forum® data. Based on this examination, memes are identified in the students’ discourse and described by quantitative indicators for these memes’ fitness. Second, we qualitatively describe the developmental paths for the fittest memes. Finally, benefits, limitations, and open questions related to our suggested framework are discussed.

Ideas and Memetic Processes
Memes, as defined by Dawkins (1976), are units of cultural transmission and imitation that spread within a culture, as an equivalent of genes. Behavior, ideas, knowledge or fashion can be considered memes that propagate from person to person via imitation and variation (Dawkins, 1976). In some ways, the culture itself can be seen as a meme pool. Bereiter (2002) describes the cultural meme-pool as frequently changing over time, while new memes emerge, others mutate or their frequency changes. These processes can be seen as a cultural

© ISLS

157

ICLS2012

Volume 2: Short Papers

evolution. This concept of the cultural meme-pool contributes to Scardamalia and Bereiters’ (2006) description of knowledge as an advancing concept with ideas emerging, others dying out, problems being solved and new problems coming up. Because knowledge can thus be compared to the cultural meme pool which underlies evolutionary processes, idea improvement can be analyzed by comparing it to memes. The quality of an idea can be defined by evolutionary indicators such as its likelihood to survive. The overall survival rate of a meme can be understood as its fitness (Heylighen, 1999). This notion includes, however, that idea survival determined for an expressed meme by its repetition and variation as in evolution, does not necessarily progress but adapts. In analogy, the fitness of a meme cannot be used as a sole measure of quality but is a complementary step for focused qualitative analyses. With regard to the memetic processes co-determining a meme’s fitness transmission, variation, and the resulting selection pressure are of relevance to our framework. Following the overview by Nye (2011) these can be defined as follows: “The core information of a meme is its semantic information. When semantic information changes, the meme has mutated or a new meme has been created. A meme reproduces when semantic information is replicated from one agent to another. [...] Conversely, identical physical transmissions change semantic meaning based on context and interpretation.” (Nye, 2011, p. 14). Thus, a prerequisite for considering semantic information to be a meme is its ability to reproduce recursively within the respective environment. First, this implies that a meme must be expressed in behavior, or in the case investigated here, in written language. The transmission of this expressed meme can be considered most important to its reproduction. As Nye (2011, p. 18) puts it: “This definition is ontologically complete: semantic information is a meme within a society and environment if and only if it can recursively reproduce in that society and environment.” Variation can be affected during expression and/or transmission by external factors, such as time pressure, or internal factors, such as limits to cognitive and motivational resources. Even though following a code, language, or procedure can reduce misunderstandings thanks to a given syntax, reducing the complexity in to a specific form of notation fosters ambiguous statements. This is why semantic variation appears likely to occur for symbolically expressed memes. Overall, this “noise” included in the transmission of memes and resulting variation puts the memes under a selection pressure and only certain memes survive. Therefore, analyzing the development of ideas, instead of the correctness of ideas, focuses more on the processes involved and less on the content. In sum, we assume that identifying memetic processes in knowledge building is a complementary approach that can help to tap more directly into the central process of idea improvement, i.e. how ideas spread and survive over time in a learners’ discourse. We expect to (1) quantitatively describe indicators for idea improvement that can be inferred from the notion of memes and memetic processes. Furthermore, we expect, (2) based on this, to be able to qualitatively describe how ideas develop over time and to more directly describe their improvement as a process.

Method
Data
We analyzed a subset of the data from a study conducted by Niu and van Aalst (2009). Two classes of a tenth grade social studies course participated in that study. Each class was divided into groups of eight persons. In a short inquiry unit (three weeks) the students investigated general environmental problems such as pine beetle infestation. For this investigation they used the asynchronous online discourse environment Knowledge Forum® (Version 4.5, Scardamalia, 2003, see www.knowledgeforum.com). In Knowledge Forum®, students can contribute their ideas to the database in the form of written notes which make up discussion threads. Other students who have access to the database can revise these notes, reply to them, or contribute their own reflection. For this study we used a data set of eight students (one female, seven male). The participants worked on the topic of how to free a forest from a pine beetle infestation. They worked irregularly on the problem over a period of eight consecutive days and five additional posts were added a month later. A total of 128 posts were analyzed, consisting of all the posts from this group.

Data Analysis
The analytic procedure consisted of three steps. First, to identify the memes emerging in the data set a coding scheme was inductively developed and all notes were coded accordingly. Individual notes served as the unit of analysis. Secondly, quantitative information was extracted from the data in order to calculate a fitness score for each meme monitoring when memes were expressed and reproduced. Finally, after calculating and plotting the fitness scores over time, we “zoomed” in qualitatively on the fittest memes and followed their path in order to describe the development of the complexity and quality of the meme, and therefore, understand its variation.

Qualitative Analysis - Identifying Memes
We based our analysis on the notion that a meme can be expressed in written language, e.g. the notes posted by students. To identify the memes emerging in the students’ discussion two independent raters (two of the

© ISLS

158

ICLS2012

Volume 2: Short Papers

authors) performed a qualitative content analysis following Mayring (2000) to inductively develop categories that would capture the central memes emerging in the data set. These initial sets of categories were discussed by the two raters and converged. Then, a third independent rater (first author) double checked the categories. Disagreements were resolved by discussion resulting in the final 18 categories, example categories/memes are described in Table 1. Finally, all posted notes were coded by applying one or more of these categories. We differentiated between the repetition of the essential meme and variation, i.e. emergence of a different meme. Repetition (the same category applied) was coded when the analyzed note resembled the core idea of the previous note. Variation within a meme was also coded as repetition if the main message had not been changed. If two or more memes were contained in a note, multiple codes were applied. Coders followed the rule to apply the same code again if the content of the note resembled (copied) the same idea as the previous note. Variation (new category applied) was coded when different ideas emerged spontaneously or existing ones were integrated in a way that the original idea did not resemble the final idea. Coders followed the rule to apply a different code if individual ideas mutated (at the group level) or the new change to existing elements was introduced, also if the preceding idea was lost during the developmental process and the following idea did not resemble the preceding idea. Table 1: Coding scheme - examples of memes. Meme Forest ecosystem Pest control Predators Definition Impact of beetle infestation and counter measures on the forest ecosystem, question of balance between environmental costs and benefits Repetition of initial task or problem, the question of how to resolve the beetle infestation is discussed Beetles should be killed by predators, from within (parasites) or outside (woodpeckers) and discussion about these predators

Quantitative Analysis
After we had identified memes in the qualitative coding procedure described above, we extracted the respective notes in which each meme occurred. Because the notes were posted over a period of 8 days at varying points in time, we defined a fixed time interval for which we aggregated the quantitative indicators. We performed the analysis with MS Excel. Due to the fact that the results do not differ meaningfully, we chose a 15 minutes interval, mainly to represent our analyses in an economic way (see Figure 1). The results did not differ meaningfully between smaller time intervals, but contained more specific information than larger time intervals. However, we are aware that the issue of conceptualizing time in asynchronous communication is a complex issue, in part because the time scale is a different one for every participant (Suthers, Dwyer, Medina, & Vatrapu, 2010). Thus, the time intervals created here are defined by our analysis not by the original time line of the data and therefore not readily interpretable. Quantitative Indicators. As mentioned above, we adopted the formula proposed by Heylighen (1999) for the two external stages: expression and transmission (see Formula 1). For each of the two components a value can be computed and then combined into a fitness score for the respective meme a predefined time interval. The simplified formula for our combined fitness measure f has the following form f(m, t) = E(m, t) * T(m, t) (1)

The fitness f of a meme m for the expression stage in time interval t is denoted by E (m, t) (expression) and the fitness in the transmission stage by T (m, t) (transmission). E (m, t) describes expression, or how often a meme has been expressed by saving a new note or changes in a note that contain the respective meme m for time interval t. T (m, t) describes transmission, or how often a posted note containing the respective meme m has been read by others during time interval t. The values of f are not interpretable in an absolute sense but only relative to other memes. Both terms can be larger than 1 and if one the terms reaches 0, the meme has been eliminated. This happens when a meme is not replicated further or when posted messages containing a meme are not read by other students anymore. In our case, however, notes were not deleted and stayed present in the Knowledge Forum® database to potentially be read. Therefore, in this study a meme can only be eliminated when T (m, t) equals 0. To get a global fitness measure f , we calculated the mean fitness of a meme (Formula 2). In this formula the fitness of all time intervals is summed up and divided by the number of intervals n:
g

fg(m) =

(Σ E(m, t ) * T(m, t ))
i=1
i i

n

(2)

© ISLS

159

ICLS2012

Volume 2: Short Papers

Results
Quantitative – Describing the Paths of Memes
Figure 1 depicts the results for the analysis applying Formula 1 to three sample memes (see Table 1). The last time interval available was excluded from analysis, because reading activity had ceased at the end of the course. What is most prominent in the data is that most of the memes have low fitness values over all the students’ discourse and that there are three dominant memes, which also repeatedly show higher spikes than any other meme: Forest Ecosystem, Pest Control, and Predators. Towards the end of the discussion, however, after showing two peaks clearly higher than any other meme (time intervals 20 and 24), the forest ecosystem becomes visible as the fittest meme, which is also the only meme that is still “alive” at the very end. In contrast, in the beginning of the discussion other memes - concerned with describing pine beetles in general and as a threat - show higher fitness values (not included in Figure 1).

Figure 1. Fitness of sample memes within predefined 15 minute time intervals (see sect. quantitative analysis). The average fitness values derived from Formula 2 mirror this data pattern. Here, also forest ecosystem, and pest control show high mean fitness. Additionally, the Predator meme had a high mean fitness, however as shown in Figure 1 it did not survive until the end of the discussion. Overall, we see from the quantitative data that the different solutions to the infestation problem are the fittest memes but rather we find a “struggle” between the elaboration on the initial task (pest control) and a counter argument; the impact of any counter measure on the forest ecosystem.

Qualitative Interpretation – How do the Fittest Memes Develop (Variation)
To qualitatively zoom in on the idea development in our data, we tried to describe the variation within a meme and the interactions between memes. The meme Pest Control was coded 25 times in the data. Different kinds of pest control mechanism were discussed, starting with a list of various pest control options, which were not discussed any further by the students. Some options from this list of measures were discussed and were judged to be no solution to the pine beetle problem, mainly again due to their impact on the forest as a whole (i.e. forest ecosystem). The most prominently discussed solution was the idea of having woodpeckers prey on the beetles (i.e. predators). Furthermore, it was mentioned that these measures can kill the beetles, but can not prevent further epidemics. In other words, it was discussed that long-term solutions must be sought; for the current pest control there will be no solution which not has an impact on other animals or environment - the forest ecosystem. The meme Forest Ecosystem was coded 32 times in the data and it clearly interacts with pest control. The students’ notes show that the students tried to optimize their ideas regarding the problem of the pine beetles. They not only tried to eliminate the beetles, but also considered the environmental and economic impact of the possible solutions. Every single idea regarding the elimination of the pine beetles was tested against the impact on the environment and most were also tested against the cost of the solution. Thus showing that the students wanted to improve their ideas. Overall, the qualitative results show that the development of the solution of the pine beetle problem was not a straightforward one. The idea for the solution improved from simple versions like “kill beetles” to sophisticated ones like looking for long-term solutions. Along this development it seems that in particular the initial task of pest control and the counter argument of the impact on the rest of the (forest) ecosystem contend with each other. This contest seems mutually beneficial to the improvement of the solution. As above mentioned, the various pest control ideas had been made public and were changed based on the input of the whole community – in this case – the aforementioned impact on the forest ecosystem.

© ISLS

160

ICLS2012

Volume 2: Short Papers

Discussion
The goal of this paper was to contribute to the study of idea improvement, a core component of knowledge building. Assessing idea improvement is of complex nature. One aspect that has been neglected so far is to determine idea improvement by assessing the relevance of ideas in the community. The current results show that the way of data preparation and analysis can be a powerful tool to complement the summative appraisal of learners’ discourse. We revealed that the group had broadened their discussion and improved their ideas in the sense of understanding the complexity of the initial problem by relating it to adjacent issues (e.g. forest ecosystem), which here were the fittest in the end. In sum, we found that the few memes which survived the discourse were also more complex ones. So in this case, the fitness of the memes was related to their quality in terms of complexity. However, in respect to the potential of the described framework, there are also some limitations: first, our analyses zoomed in on just one aspect that could help assess idea improvement. In order to gain a complete understanding of what determines idea improvement, various facets need to be taken into account. Hence, future research should focus on combining several aspects (e.g. such as specificness, see Zhang at al., 2007) when assessing idea improvement. A second limitation lies in the reductionist approach taken: evolutionary processes only allow a very limited view on ideas; possibly it even neglects important aspects of ideas that are characteristic for idea development. We do not suggest that our framework alone is sufficient, but it can help to focus the questions for deeper analysis. Overall, we conclude that our framework provides a way to more directly research idea improvement as a process in which the community is a driving force.

References
Bereiter, C. (2002). Education and mind in the knowledge age. Mahwah, NJ: L. Erlbaum Associates. Dawkins, R. (1976). The selfish gene. Oxford: Oxford University Press. Hong, H.-Y., & Sullivan, F. R. (2009). Towards an idea-centered, principle-based design approach to support learning as knowledge creation. Educational Technology Research and Development, 57, 613-627. Heylighen, F. (1999). What makes a meme successful? Selection criteria for cultural evolution. Proceedings of the 15th International Congress on Cybernetics, Namur, p. 418- 423. Mayring, P. (2000). Qualitative content analysis. Forum: Qualitative Social Research, 1(2). Retrieved August 25, 2011, from http://www.qualitative-research.net/fqs-texte/2-00/02-00mayring-e.htm. Niu, H., & van Aalst, J. (2009). Participation in Knowledge-Building Discourse: An analysis of online discussions in mainstream and honors social studies courses. Canadian Journal of Learning and Technology 35(1), http://www.cjlt.ca/index.php/cjlt/article/viewArticle/515/245. Nye, B. D. (2011). Modeling Memes: A memetic view of affordance learning. Publicly accessible Penn Dissertations, paper 336. Retrieved August 25, from http://repository.upenn.edu/edissertations/336. Scardamalia, M. (2003). Knowledge building environment: Extending the limits of the possible in education and knowledge work. In. A. DiStefano, K. E. Rudestam, & R. Silverman (Eds.), Encyclopedia of distributed learning (pp. 269 - 272). Thousand Oaks, CA: Sage Publications. Scardamalia, M., & Bereiter, C. (2003). Knowledge building. In Encyclopedia of education, second edition. (pp. 1370-1373). New York: Macmillan Reference, USA Scardamalia, M., & Bereiter, C. (2006). Knowledge building: theory, pedagogy, and technology. In R. K. Sawyer (Eds.), Cambridge handbook of the learning sciences. (p. 97-115). New York: Cambridge University Press. Suthers, D. D., Dwyer, N., Medina, R., & Vatrapu, R. (2010). A framework for conceptualizing, representing, and analyzing distributed interaction. International Journal of Computer Supported Collaborative Learning, 5(1), 5-42. van Aalst, J. (2006). Rethinking the nature of online work in asynchronous learning networks. British Journal of Educational Technology, 37, 279-288. Zhang, J., Scardamalia, M., Lamon, M., Messina, R., & Reeve, R. (2007). Socio-cognitive dynamics of knowledge building in the work of 9- and 10-year-olds. Educational Technology Research & Development, 55, 117-145.

Acknowledgments
This study resulted from an exploration the authors began during the Summer School “Making Sense of Social Media”, co-sponsored by the Knowledge Media Research Center and Tuebingen ScienceCampus, August 1-4, 2011. The data were originally collected as part of a New Economy Collaborative Research grant from the Social Sciences and Humanities Research Council of Canada to Marlene Scardamalia (OISE/University of Toronto). The authors thank Hui Niu (Simon Fraser University) for permission to reanalyze the Knowledge Forum® database.

© ISLS

161

ICLS2012

Volume 2: Short Papers

Unraveling Idea Development in Discourse Trajectories
Iassen Halatchliyski1, Aileen Oeberst1, Martina Bientzle1, Franziska Bokhorst1, Jan van Aalst2 1 Knowledge Media Research Center, Schleichstr. 6, 72076 Tübingen, Germany 2 Faculty of Education, University of Hong Kong, 323 R S Building, Pokfulam Road, Hong Kong, China Emails: {i.halatchliyski, a.oeberst, m.bientzle, f.bokhorst} @iwm-kmrc.de, vanaalst@hkucc.hku.hk, Abstract: With the present paper we want to shed light onto an issue that is central within the knowledge building theory but only little studied – the development of ideas in collaborative learning discourse. Starting from the construction of a network of explicit and implicit relations between ideas, we apply a scientometric method to tackle the temporality of collaborative processes based on the structure of successive ideas. The resulting discourse trajectories are shown to give a holistic and also a detailed view on how knowledge advances when their interpretation is combined with a qualitative analysis of the content of the ideas and their relations. The weighted relevance of relations between ideas enables the identification of sub-topics in the discourse, important ideas, and influence or uptake events.

Introduction
How is knowledge about the world created and advanced? Knowledge building is an approach from the learning sciences that builds on contemporary philosophical views and research on expertise (Bereiter, 2002; Scardamalia & Bereiter, 2006) to engage students in modern knowledge work, including ability to collaborate, deal with novelty, and solve ill-structured problems. At the heart of knowledge building is a computer-mediated collaborative discourse that is oriented toward idea improvement. Following Popper’s (1972) theory of objective knowledge, knowledge-building theory considers ideas as “real” objects that can be critiqued, tested and modified, much like how real objects like bicycles undergo these processes. Hence, the process of idea development is fundamental for understanding knowledge building. However, despite this acknowledged role, there is a dearth of analytical approaches for investigating the dynamic development of ideas in knowledge-building discourse. Therefore, the main goal of this paper is to present a methodological technique from sceintometrics for studying what we call a discourse trajectory, i.e. the genuine process characteristics of a discourse based on idea development over time.

Related Research
Most studies of knowledge building have followed a content analysis approach to studying a discourse process (Gunawardena, Lowe, & Anderson, 1997; Henri, 1992), where qualitative data is segmented into idea units and these are coded for their cognitive, social and other aspects. The frequency of the assigned codes is then statistically regarded for comparisons of different students, discussion groups or phases. Students’ contributions can also be categorized as different discourse activities, on the basis of which a sequential analysis (Jeong, 2005) of their temporal ordering can reveal patterns and facilitate deeper insight into the collaborative discourse as a process. However, as is becoming increasingly clear in CSCL research, the coding and counting technique neglects some important discourse qualities as it takes statements out of their context and generally addresses actions of individuals instead of the group as a whole (Strijbos & Stahl, 2007). Stahl, Koschmann and Suthers (2006) defined collaborative learning as an interactive process of shared meaning-making in a group and pleaded for exploratory and interpretative conversation analysis of case study narratives. Henri (1992) addressed interactivity in a discourse process distinguishing independent from implicit interactive and explicit interactive contributions depending on if and how they refer to other contributions in the discourse. Later, Gunawardena et al. (1997) extended this view noting that in a knowledge constructing discourse all contributions can be linked to one or more other contributions and to the discussed topic. This marks a change from understanding interactivity as reference between contributions to treating it as diffusion of knowledge and other more general forms of uptake or influence. Suthers, Dwyer, Medina and Vatrapu (2010) introduced and employed contingency graphs for uptake analysis with the goal to identify and map also very subtle kinds of contribution uptakes that often remain unnoted. As noted by Lipponen, Rahikainen, Lallimo and Hakkarainen (2003) a discussion thread structure conceals the majority of semantic and conceptual relations between contributions. Another approach to studying interactivity in discourse processes presents the social network analysis (SNA), which is a well established methodology in the CSCL research field (e.g. de Laat, Lally, Lipponen, & Simons, 2007; Reffay, & Chanier, 2002). It has been used for studying relations between persons embedded in a network determining cohesiveness of the learning group and students’ relative positions. Following the reasoning of Stahl (2006) on intersubjective meaning-making in CSCL in terms of networks of references the discourse process can also be approached from the perspective of a network of collaboratively created artifacts.

© ISLS

162

ICLS2012

Volume 2: Short Papers

There are accordingly a few notable examples of studies of automatically detected semantic networks of related contributions (Sha, Teplovs, & van Aalst, 2010). Because network analysis is based on indicators of relations that lack deeper meaning in the discourse, it should be applied in combination with other in-depth and content related methods. In sum, an appropriate methodology for studying discourse as a process has to be multi-faceted and address: the temporal dimension of development; the interactivity between contributions and between participants; the content of the discourse (Arvaja et al., 2007).

Main Path Analysis of Discourse Trajectory
The main goal of our study is to provide a primer for a novel methodology that tackles the trajectory of a discourse process analyzing the collaborative development of ideas over time. The approach presented here is rather simple and grounds on a network analysis of interrelated ideas. Based on a set of identified relations a main path analysis algorithm (Hummon & Doreian, 1989) assigns different weights to the relations according to their position in the network. This weighted network can be interpreted in order to identify the most influential ideas, idea paths, i.e. successions of related contributions, discourse trajectory, i.e. the overall structure of the paths over time, etc. The procedure of main path analysis stems from the scientometric research tradition that deals with citations between scholarly publications (e.g. Carley, Hummon, & Harty, 1993). In the present application of the analysis again relations between authored content are of interest. To the best of our knowledge, the present work is a pioneering attempt of applying scientometric methodology in the field of CSCL, and knowledge building in particular. Main path analysis calculates a weight for each relation in a citation network according to the number of times the relation is used while tracing all chronological connections between all possible pairs of contributions (De Nooy, Mrvar, & Batagelj, 2005). So, a relation on idea paths is more important, the more often it serves as a link between preceding and subsequent ideas. Contributions that establishing such important relations often synthesize old ideas, add new knowledge and represent a basis for developing new ideas. A main path in the discourse trajectory is produced by the links with the highest weights. De Nooy et al. (2005) reason that networks of scientific citations represent systems of knowledge flows. This fits our goal to analyze development of ideas in a knowledge building discourse. The resulting discourse trajectory is comparable to the development of a scientific field and can be analyzed for instances of integration, fragmentation, specialization or paradigmatic changes (De Nooy et al., 2005). Moreover, we see very promising possibilities for combining it with in-depth and content analysis methods.

Method
Data Set
We reanalyzed Knowledge Forum data from a study of two Grade 10 classes — one regular and the other honors — who investigated aspects of environmental issues; see Niu & van Aalst (2009) for details. The original study showed that both high and low-achieving students can sustain a knowledge building discourse. The students worked on the Knowledge Forum software in groups of approximately eight over a period of three weeks. For the present study we compared the discourses of one group from each class. Both groups separately discussed ideas for handling the problems of the nuclear accident at Chernobyl. The honors group wrote 60 notes and the regular group 164 notes on this issue. We analyzed the log data and the content of the computer notes; the log files contain writing (saving a note) and reading (opening a note) events, which were recorded with a timestamp while using Knowledge Forum.

Analysis
As we were interested in collaborative learning at the level of idea development in discourse it was important to take all ideas and all paths of influence over time into account. We adopted Lipponen’s (2000, p.185) definition that an “idea is a set of propositions that formed a coherent unit of meaning” and determined that a single note represents a suitable unit of analysis for our data (cf. Rourke, Anderson, Garrison, & Archer, 2001). Even though a note may present new information taken from an external source, almost always it is also connected to some previously stated ideas, as put forward by Gunawardena et al. (1997). Direct links between notes in a thread capture only a small part of all existing relations between notes, however. Thus, in order to capture relations between ideas, we made use of both — explicit links and implicit connections — by analyzing data from two different sources, the log files of the software as well as the content of the contributed notes. Based on the timestamps two independent raters considered all the notes each student had read before writing a new note. The raters then compared the content of each note the student had read and the content of her own new note. Any disagreements between the raters about the evidence of connection between notes were discussed and solved unanimously. Implicit relations in the discussions of the Chernobyl issue were found for example between notes that addressed the same point: the fault of the designer of the reactor vs. that of the

© ISLS

163

ICLS2012

Volume 2: Short Papers

technical operator; the radiation outburst; the increase in the incidence of cancer; the old covering of the remains of the reactor; financial support for the Ukraine; and many more other points. The complete qualitative analysis of the content relations between the notes is still in progress. The discourse trajectory results reported here are only based on the detection of connections. We then applied the main path analysis procedure (Hummon & Doreian, 1989; Carley, Hummon, & Harty, 1993) to the resulting networks of both group discourses and obtained weights for each link based on its relative importance for all paths of idea development. This was done with a standard procedure for main path analysis in the Pajek software (Batagelj & Mrvar, 1998) for network analysis.

Results
Figures 1 and 2 depict the results of the main path analysis of both group discourses. The vertically layered view illustrates idea development during the different days of activity in class and at home. The numbered points represent different notes. Arrows represent some of the relatively important relations between notes with thicker arrows denoting more important relations, i.e. higher weights calculated with the main path analysis. The arrows are directed from an older to a newer note. It is important to mind that the isolated notes positioned on the left and on the right of the figures were also found by the raters to be related to other notes, but these relations received very low weights in the main path analysis. In order to identify the main idea paths in the discourse trajectories more clearly, Figure 1 and 2 show only relations over the arbitrary threshold weight of 0.05 for normalized weights between 0 and 1. Both figures display single connected main idea paths, because there were no disparate discussions around different topics in the groups. However, both figures show different discourse trajectories at a first glance that remain to be characterized with the help of the contributed content. Honors Group (Figure 1) The notes from the first day on (top of the figure) generate concurrent idea paths interrelating and stimulating one another; addressing background information on causes and on effects of the accident. On April 4th the development of these ideas seems to get focused into one path dealing with technological issues based on the causes of the accident. After April 4th the idea paths in the honors group separate resulting in four largely independent lines of inquiry that are pursued until the last days of the course. The thickest path deals with futuristic solutions of neutralizing radiation and inspires the largest amount of participation. The remaining three paths discuss the covering of the destroyed reactor, the use of nuclear energy in general and the politics behind the accident.

April 1 | --April 2 -home --April 3 -home | --April 4 | --April 4 -home | --April 5 --April 6 --| April 6 -home | --| April 7 --April 11 -home --| April 15 --April 18 -home

Figure 1. Discourse trajectory of the honors group: notes and main idea relations

© ISLS

164

ICLS2012

Volume 2: Short Papers

Regular Group (Figure 2) All the initial idea paths merge into a single note, 316, on April 1st that brings up the need for solutions after the basic facts are known. Further tracking of the discourse trajectory suggests that this pattern repeats within the regular group. Concurrent idea developments are short-lived and continuously meld into a single note on the same or the next day. On April 4th note 488 proposes a solution for the polluted soil around the old reactor, i.e. digging it and disposing of it in outer space. Then, again, this central idea gives rise to various related idea paths concerning difficulties of the solution – most importantly – the money problem. Additional solutions and refinements emerge on April 5th and are then put to a vote by note 686. Note 843 bundles the focus to the solution of building a new covering for the old reactor. The game of proposing solutions and recognizing difficulties continues yielding tightly interrelated idea paths.

March 31 | | | --April 1 | --| April 4 | --| April 5 | --| April 6 | | --| April 7 --April 13 -home --| April 14 | | --April 15

Figure 2. Discourse trajectory of the regular group: notes and main idea relations

Conclusion
The short comments on the results illustrate some initial ways of interpreting discourse trajectories obtained through main path analysis. With the present paper we pursued the goal to open up a field of possibilities for studying collaborative learning processes as we introduced a new method to the field. We showed that it handles the temporal perspective of idea development very well providing an objective measure of the relevance of ideas and their relations. These can then be examined more closely with regard to their contents in a mixed methods approach (see also Carley et al., 1993). The obtained discourse trajectory also provides a holistic view on the collaborative process. Although both analyzed groups were successful in knowledge building, they showed very different styles of collaboration regarding the convergence-divergence polarity (Halatchliyski, Kimmerle, & Cress, 2011). The regular group produced a large number of notes, and their main ideas were very tightly interrelated; convergence was maintained over the whole discourse trajectory. The honors group achieved a selforganized “division of labor” by building up a common understanding of the problems and then following divergent solution paths. It remains to be shown to what extend this is due to individual differences. Our next goal is to complete the qualitative analysis of the data set in order to determine what kind of contributions and relations receive higher or lower weights, i.e. are more or less important in the discourse.

References
Arvaja, M., Salovaara, H., Häkkinen, P. & Järvela, S. (2007). Combining individual and group-level perspectives for studying collaborative knowledge construction in context. Learning & Instruction, 17,

© ISLS

165

ICLS2012

Volume 2: Short Papers

448-459. Batagelj, V., & Mrvar, A. (1998). Pajek – Program for Large Network Analysis. Connections, 21, 47-57. Bereiter, C. (2002). Education and mind in the knowledge age. Mahwah, NJ: Lawrence Erlbaum Associates. Carley, K. M., Hummon, N., & Harty, M. (1993). Scientific Influence: An Analysis of the Main Path Structure in the Journal of Conflict Resolution. Knowledge: Creation, Diffusion, Utilization, 14, 417-447. de Laat, M., Lally, V., Lipponen, L., & Simons, R.-J. (2007). Investigating patterns of interaction in networked learning and computer-supported collaborative learning: A role for Social Network Analysis. International Journal of Computer-Supported Collaborative Learning, 2, 87-103. de Nooy, W., Mrvar, A., & Batagelj, V. (2005). Exploratory Social Network Analysis with Pajek, Cambridge University Press, 2005. Gunawardena, C. N., Lowe, C. A., & Anderson, T. (1997). Analysis of A Global Online Debate and The Development of an Interaction Analysis Model for Examining Social Construction of Knowledge in Computer Conferencing. Journal of Educational Computing Research, 17, 397-431. Halatchliyski, I., Kimmerle, J., & Cress, U. (2011). Divergent and convergent knowledge processes on Wikipedia. In H. Spada, G. Stahl, N. Miyake, & N. Law (Eds.), Connecting Computer-Supported Collaborative Learning to Policy and Practice: CSCL2011 Conference Proceedings (Vol. II, pp. 566570). Hong Kong: International Society of the Learning Sciences. Henri, F. (1992). Computer Conferencing and Content Analysis. In: A. Kaye (Ed.), Collaborative Learning through Computer Conferencing: The Najaden Papers (pp. 117-136). Berlin: Springer-Verlag. Hummon, N. P., & Doreian, P. (1989). Connectivity in a Citation Network: The Development of DNA Theory. Social Networks, 11, 39-63. Jeong, A. (2005). A guide to analyzing message-response sequences and group interaction patterns in computermediated communication. Distance Education, 26, 367-383. Lipponen, L. (2000). Towards knowledge building: from facts to explanations in primary students’ computer mediated discourse. Learning Environments Research, 3, 179-199. Lipponen, L., Rahikainen, M., Lallimo, J., & Hakkarainen, K. (2003). Patterns of participation and discourse in elementary students computer-supported collaborative learning. Learning and Instruction, 13, 487-509. Niu, H., & van Aalst, J. (2009). Participation in knowledge-building discourse: An analysis of online discussions in mainstream and honours social studies courses. Canadian Journal of Learning and Technology 35(1). http://www.cjlt.ca/index.php/cjlt/article (Retrieved November 5, 2011). Popper, K. R. (1972). Objective knowledge: An evolutionary approach. Oxford, UK: Clarendon Press. Reffay, C. & Chanier, T. (2002) Social Network Analysis used for modelling collaboration in distance learning groups, In S. A. Cerri, G. Gouardères, and F. Paraguaçu (Eds.): ITS 2002 Lecture Notes in Computer Science, 2363, 31–40. Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2000) Methodological issues in the content analysis of computer conference transcripts. International Journal of Artificial Intelligence in Education, 11, 8-22. Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and technology. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 97-115). New York, NY: Cambridge University Press. Sha, L., Teplovs, C., & van Aalst, (2010). A visualization of group cognition: Semantic network analysis of a CSCL community. In K. Gomez, L. Lyons, and J. Radinsky (Eds.) Proceedings of the 9th International Conference of the Learning Sciences - Volume 1 (ICLS '10) (pp. 929-936). Stahl, G. (2006). Group cognition: Computer support for building collaborative knowledge. Cambridge, MA: MIT Press. Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-supported collaborative learning: An historical perspective. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences. Cambridge, UK: Cambridge University Press. Strijbos, J.-W., & Stahl, G. (2007). Methodological issues in developing a multi-dimensional coding procedure for small group chat communication. Learning & Instruction. Special issue on measurement challenges in collaborative learning research, 17, 394-404. Suthers, D. D., Dwyer, N., Medina, R., & Vatrapu, R. (2010). A framework for conceptualizing, representing, and analyzing distributed interaction. International Journal of Computer-Supported Collaborative Learning, 5(1), 5-42.

Acknowledgments
This study resulted from an exploration the authors began during the Summer School “Making Sense of Social Media”, co-sponsored by the Knowledge Media Research Center and Tuebingen ScienceCampus, August 1-4, 2011. The data were originally collected as part of a New Economy Collaborative Research grant from the Social Sciences and Humanities Research Council of Canada to Marlene Scardamalia (OISE/University of Toronto). The authors thank Hui Niu (Simon Fraser University) for permission to reanalyze the database.

© ISLS

166

ICLS2012

Volume 2: Short Papers

Investigating Emergent Dynamics to Understand Interactions in Small Professional Development Groups
Susan A. Yoon, University of Pennsylvania, Philadelphia, PA, yoonsa@gse.upenn.edu Abstract: In this study a dynamic systems approach and analyses is used to identify dynamics that constrained the collaboration of a group of teachers to complete a reform-oriented curriculum task in a professional development setting. A central goal was to identify challenges to achieving emic or self-organized structure necessary for sustaining innovative educational programs. The study found that a local social and emotional dynamic influenced the emergence of a strong global dynamic of social support that ultimately produced the contextual outcome of low productivity. The paper asserts that teachers need intentional training to gain skills for working autonomously. Helping teachers become aware of the dynamics needed for successful collaboration would assist in achieving self-organized participation. An important contribution of this study is the application of a dynamic systems lens, which enabled the location of levels of micro and macroscopic influences that can be used to inform future professional development activities.

Introduction
The research reported here follows on a series of studies that have investigated the complex or non-linear nature of teacher interactions in professional development activities (Yoon et al., 2010). Previous research has revealed differences in the way that teacher groups display more or less adaptive collaborative activity (e.g., Horn & Little, 2010) and how collective approaches in professional development influence teacher growth and student learning (Desimone, 2009). Some research has suggested that differences between how teacher groups function may be attributed to several variables which include a lack of understanding of how to self-organize effectively as a team (Main, in press). However, the general consensus in the literature about teacher collaborative groups is that we still know every little about how they operate and the interactional processes that lead to problem solving and decision making (Havnes, 2009; Meirink et al., 2007; Scribner et al., 2007). The often non-linear nature of team formation and development (Ito & Brotheridge, 2008) coupled with the need to promote collectivity suggests that investigating the interaction of teacher groups requires methods that can accommodate these characteristics and capture critical impacts on collaborative activities as they emerge. To date there appear to be few methodological tools that can do this systematically. Thus, in this small group case study, I use a dynamic systems approach as a methodological tool to investigate the interactions of six teachers as they collaborated to construct reform-oriented curricula. I wanted to examine the utility of this systematic process in revealing the complex nature of interactions that enabled or constrained self-organization. I was also interested in understanding the developmental dynamics that shaped the group's ability to accomplish a systemic reform task.

A Dynamic Systems Approach to Understanding Group Activity
Dynamic systems are complex organizations of interacting parts that work together to give rise to patterns of behavior over time (Churchill, 2007). Through these lower level interactions that are initially unstable, more stable patterns emerge and solidify into attractor states that the entire system tends toward (Granic et al., 2007). For example, Martin et al. (2005) have studied how children's play partners, while initially variable, organize into coherent friendship clusters due to individual differences in personalities and other behavioral characteristics. While perturbations can shift the configuration of the system temporarily (e.g., a new person monopolizes a friend’s attention), depending on the strength of the attractor (e.g., friendship bond), the system will form very stable behavioral states (e.g., friendship resumes). How attractor states form and settle over time are known as self-organization and emergence in dynamic systems and can be tracked over time to explain the evolution of system trajectories (Lewis, 2000; Steenbeek & van Geert, 2007; Thelen & Smith, 1994). The self-organization or emic nature of dynamic systems aligns well with the local or peer-to-peer goal for our professional development activities. Methods for which to understand these dynamics also connect well with this study’s focal interest in how teacher interactions can shape outcomes. For example, Farmer et al. (2007) and Lichtwarck-Aschoff et al. (2008) identify developmental factors that work together as a system of correlated constraints. They discuss how system components mutually influence each other and become aggregated in macroscopic patterns to constrain or control the entire system activity. A coherent model that categorizes such local and global dynamics with contextual outcomes can be found in Arrow et al. (2000) in which small groups are viewed as dynamic systems. The case study in the present research uses this model and defines local dynamics as rules of activity for parts of the system, i.e., the teachers, and global dynamics as rules

© ISLS

167

ICLS2012

Volume 2: Short Papers

of activity for system-level properties that emerge out of local dynamics. These are the patterns of behavior or system attractors that stabilize in the professional development group and influence contextual outcomes.

Methods
Participants and Context
The study investigates the interactions among a group of six teachers who worked together for 10 months. The group was comprised of three males and three females from six different high schools in a large urban school district. Teaching experiences ranged from 5–26 years. Teachers taught across different science disciplines and all high-school grades. More details about selected participants follow. Teachers participated in a ten-day summer PD workshop in August of 2010 (50 hours), and then in monthly meetings from September 2010 – May 2011 (35 hours). Their task was to create a publishable high school science curriculum book using units they already constructed for a larger project on 21st century problem-based learning and digital participation. Teachers were selected due to their demonstrated commitment in the larger project, their pedagogical skills in delivering their curricular units, their perceived abilities for leadership, and their perceived ability to collaborate. Teachers were told that the project was ultimately aimed at producing curriculum that was vetted and constructed by teachers for teachers for implementation in real-world urban classrooms thereby increasing the potential that new teachers would be able to use the curriculum successfully. The goals of teacher ownership, collaboration and decision making were greatly emphasized and teachers were given the message that selforganization was a major expectation, i.e., that there would be little input from researchers as to how the book would be constructed.

Data Sources, Data Coding and Analyses
Data sources and analyses included: 1) Initial surveys of participant demographics that collected experiences and goals for participation; 2) Six individual interviews at the end of the summer workshop 2010; three individual interviews at the end of May 2011. Interview questions asked how participants felt about the collaborative effort, what their role in the book project was, and reasons for why particular project and group outcomes emerged; 3) Sixty hours of recordings from both the summer workshop and monthly sessions documenting group activities and dynamics; and 4) Five surveys of collaboration rankings during the summer workshop in which participants were asked to rank other members of the group from 1-5 in terms of who was the most central person in the collaboration task, a rank of 1 being most central. Initial surveys were used to establish an understanding of each participant's unique set of qualities and experiences that potentially influenced local dynamics such as having less teaching experience than others in the group. Interviews and recordings of group meetings were qualitatively mined by the researcher and two doctoral research assistants to look for local and global dynamics and contextual outcomes as they emerged over time. To validate the findings, the global dynamics and contextual outcomes were reviewed with participants in the May 2011 interviews. For collaboration rankings, an in-degree score for each teacher was calculated from the average of teachers' collaboration rankings for each time sample. In-degree scores can be used to represent an actor’s prestige or status in a system (Wasserman & Faust, 1994). In this study, the in-degree score was used to determine who might have influence on group dynamics at particular points of the study. The lower the average number a teacher received, the higher the degree of centrality. A sample of the findings is presented below.

Results
Local dynamics (rules of activity for the teachers): Table 1 shows the in-degree scores of the participants in the study. Participant trajectories illustrate who had influence in the system at specific points. Don ranked near the top or at the top during the first week of the workshop while Isabel’s ranking was at the middle or below. During the second week, Isabel and Don’s rankings switch on Day 7 where Isabel had the highest in-degree score. At the end on Day 9, Isabel returned to her normal middle spot while Don’s position fell to nearly the bottom.
Table 1: Teacher's in-degree scores based on a ranking of like-mindedness Ranking In-degree Score In-degree Score In-degree Score In-degree Score In-degree Score Day 1 Day 3 Day 5 Day 7 Day 9 1 2 3 4 5 6 Carol (1.8) Don (2.0) Isabel (3.0) Stan (4.3) Shelley (4.4) Bill (4.5) Don (2.6) Carol (3) Bill (3.3) Isabel (4.0) Stan (4.0) Shelley (5.0) Don (3.0) Bill (3.0) Carol (3.4) Isabel (3.6) Stan (4.2) Shelley (4.8) Isabel (1.8) Carol (2.3) Don (3) Bill (3.3) Shelley (5.6) Stan (5.6) Bill (1.8) Carol (2.2) Isabel (2.4) Stan (4.2) Don (4.3) Shelley (5.5)

© ISLS

168

ICLS2012

Volume 2: Short Papers

A review of their individual qualities, experiences and goals for participation helps to make sense of these trajectories over time as detailed in the next section. Don was the oldest teacher in the group who came to teaching after a series of jobs in the corporate world because he wanted to make a difference in society. His goals for participation were to help other teachers feel comfortable with new pedagogical approaches. At the time of the summer workshop, he was working on a Masters degree in family therapy and intended to make another career change from teaching. In the workshop interview, he described his contribution as keeping things light and social to ensure that the group had fun. He often discussed the difficult bureaucratic and social issues that impacted students’ and teachers’ lives in the district. In contrast, Isabel was the youngest teacher in the group with five years of experience. Her goals for participation included an interest in contributing to science education and to impact the practice of other teachers by constructing good curricula. She expressed an interest in eventually doing a doctorate in education. She wanted to participate in the summer workshop to collaborate with different teachers. In the May 2011 interview, she said she felt that she had many good ideas to share but didn't feel that she had the authority to push them in the group.

Global dynamics (rules of activity for the group): To investigate the global dynamics that emerged at
the system level, workshop recordings were mined to identify patterns in group interactions that may have been influenced by the above local teacher dynamics. Since the task required them to make decisions about how the curricular book would be constructed the analysis concentrated on instances in the discourse where the group had to make a decision. From this analysis, three global dynamics emerged, i.e., social support, anti-work; and don’t rock the boat. Evidence of the first dynamic is presented below.

The social support dynamic: This dynamic was about group members gaining moral support from each other and sharing experiences about the challenges they faced as teachers working in a dysfunctional urban school district. Don’s personal dynamics and goals being aligned with social and emotional support, continually influenced others in the group to participate in social sharing rather than completing the curriculum construction. Don also had a strong and forceful personality, which he immediately exerted. These dynamics can be observed in the following excerpt of Day 1 discourse. Before this excerpt, the group was asked to begin making decisions about what activities they wanted to include in the curriculum. Bill started on a line of discussion about getting parents on board. He talked about how some of his students went home after an impactful demo and told their parents.
Excerpt 1: August 2, 2010, Day 1 hour 1: 1:10:031 1. Bill: So the parents [were interested] 2. Don: [That is a great demo] 3. Bill: [Right. Parents were] 4. Don: [that really is.] 5. Bill: Parents were interested in what was going on so that told me right then 6. that the kids went home and told their [parents] 7. Don: [EXACTLY.] 8. Bill: well we’re going to be doing this. It was something different and 9. And when my parents came in on parents night, I’d say, you know well, 10. if your children don’t know exactly what field they want to go into, 11. this maybe a field that your children could go into 12. so by putting in a hook with your parent that made them interested which made 13. the children a little more interested [in what’s going on] 14. Don: [So if we are going] to do that 15. at least now we can’t speak for you because you are outside the school district, 15. 16. Carol: [Right] 17. Don: thinking about us, we have the first week, Tuesday and Wednesday 18. ((knocking the table)) we’re off for two days, then we have four days and that, 19. and that Thursday night, that’s back to school night. So if we’re going to do it 20. Bill: I mean that’s a, [I mean] 21. Don: [I mean] it’s a good idea 22. Bill [Right and] 23. Don: [but we gotta] do it early. 24. Bill: Right we’d have to, we’d have to do [something like that early.] 25. Don: [Because we literally have] 26. 6 days until back to school night.

© ISLS

169

ICLS2012

Volume 2: Short Papers

Then Stan explained that he continued to have trouble getting his students to sign study consent forms. From there he talked about how his teaching roster had also changed mid-semester which prompted Shelley to share her own issues with the teaching roster. By this time, the conversation had spiraled into one that moved away from decision making about what activities to include in the curriculum. From the discourse excerpt, the tone of Don’s comments were cautionary due to the fact that teachers needed to stay in line with the district’s often irrational decisions (such as the timing of back to school night). This opened the floor for Stan and Shelley to discuss their professional issues to garner social support from the group. This global dynamic of social support often dominated the discourse throughout the workshop and in combination with other emergent global dynamics, constrained the group’s ability to complete the curriculum construction. This strong attractor was initiated and sustained by Don’s local dynamics as evidenced from the discourse and his high degree of centrality in the first week. Isabel, whose local dynamics represented contribution and collaboration, rarely participated in social sharing. In the discussions she often attempted to redirect the group toward focusing on the curriculum construction task. Despite her efforts, several decision-making episodes Isabel initiated were left unresolved. When asked to reflect on the kinds of group dynamics that emerged over the 10 month collaboration during the May 2011 interview, she said the group socialized a lot and hypothesized that this dynamic in part emerged due to the research facilitator’s inability to manage Don’s dynamics. It’s important to note from Table 1 that the day Isabel ranked #1 in the in-degree scores of the summer workshop (Day 7), the project PI spent a great deal of time in the morning discussing with the group, the limited time they had to finish and urged them to work more efficiently and to make greater strides in the remaining part of the week. However, due to the stability of the strong social support attractor, Isabel’s position fell again two days later (Day 9). These dynamics ultimately produced two contextual outcomes, i.e., a lack of self-organization and incomplete curriculum, which caused the project to fall short of its goals.

Discussion and Implications
Similar to previous studies (e.g., Havnes, 2009), this research investigated conversational processes that emerge in groups to understand the dynamics that impact collaboration (Scribner et al., 2007) and productivity. Using Arrow et al’s (2000) framework the findings demonstrate how Don’s local dynamics influenced the emergence of a strong global dynamic of social support that ultimately produced the contextual outcomes of the lack of self-organization and incomplete curriculum construction. By following the dynamics that emerged over time and by using in-degree rankings this method located potential source(s) of influence, evaluated the comparative strength of the influence and hypothesized some reasons why the influence persisted. Between the two focal teachers in this paper, Isabel who represented the dynamics of contribution and collaboration clearly had the disposition and goals that were ideal for participation. However, her local dynamics did not influence the group where Don’s appeared to dominate. Differences between the two teachers in terms of their experience and personalities may have contributed to their abilities to influence the group. Another important contributor as revealed in Isabel’s interview was the fact that teachers appeared to rely on the researchers to exert control, despite the strong emphasis on leadership, collaboration, teachers as decision makers, and ownership. But what is the reason for why this group operated in the manner that it did and how can knowing about this help to achieve self-organization in teacher groups? The interactional patterns shown in the group’s discourse can be explained through a dynamic systems lens. Initially, varying unstable local dynamics interacted and the confluence of variables gave rise to patterns of behavior over time (Churchill, 2007). Don’s passionate personality coupled with the expectation of external control solidified into attractor states (e.g., the social support dynamic) that the entire system tended toward (Granic et al., 2007). Although perturbations (e.g., the PI urging completion) had a temporary effect, the strength of the social support attractor continued to win out. From this assessment, it seems that the perturbation needs to be stronger than the stable attractors the system settles into. Other implications pertain to creating professional development structures that simultaneously address the local dynamics and needs of teachers and goals of the project. We saw that teachers wanted social and emotional support of the collaborative group, but this function did not fulfill the curriculum construction activity. A collaborative teacher group should succeed on both professional and personal levels to ensure positive membership and productivity. Another implication of the findings is to provide opportunities for teachers to become skilled collaborators. Teachers need intentional training and modeling on collaboration to gain self-organization skills. This point supports assertions made by Main (in press) in that teachers may lack understanding of how to work effectively as a team. Helping teachers become aware of the dynamics needed for successful collaboration, as Scribner et al. (2007) suggests, would assist in achieving the goal of self-organized participation. In order to identify these dynamics, an important contribution of this study is the application of a dynamic systems approach and analysis that revealed local influences on global dynamics that can ultimately inform future professional development activities.

© ISLS

170

ICLS2012

Volume 2: Short Papers

Endnotes
(1) Transcript convention used in excerpts follow the Jefferson Transcription System [ ]: start and end of overlapping speech ((laugh)): gestures or comments Underlining: emphasis in speech (.): hearable micro pause (2): seconds of pause in speech CAPS: rise in volume

References
Arrow, H., McGrath, J., & Berdahl, J. (2000). Small groups as complex systems. Thousand Oaks, CA: Sage Publications. Desimone, L. (2009). Improving impact studies of teachers professional development: Toward better conceptualizations and measures. Educational Researcher, 38(3), 181-199. doi: 10.3102/0013189X08331140 Farmer, T., Farmer, E., Estell, D., & Hutchins, B. (2007). The developmental dynamics of aggression and the prevention of school violence. Journal of Emotional and Behavioral Disorders, 15(4), 197-208. doi:10.1177/10634266070150040201 Granic, I., O’Hara, A., Pepler, D. & Lewis, M. (2007). A dynamic systems analysis of parent-child changes associated with successful “real-world” interventions for aggressive children. Journal of Abnormal Child Psychology, 35, 845-857. doi: 10.1007/s10802-007-9133-4 Havnes, A. (2009). Talk, planning and decision-making in interdisciplinary teacher teams; a case study. Teachers and Teaching: theory and practice, 15(1), 155-176. doi: 10.1080/13540600802661360. Horn, I.S., & Little, J.W. (2010). Attending to problems of practice: Routines and resources for professional learning in teachers’ workplace interactions. American Educational Research Journal, 47(1), 181-217. doi:10.3102/0002831209345158 Ito, J.K., & Brotheridge, C.M. (2008). Do teams grow up one stage at a time? Exploring the complexity of group development models. Team Performance Management, 14(5/6), 214-232. doi:10.1108/13527590810898491 Lewis, M. (2000). The promise of dynamic systems approaches for an integrated account of human development. Child Development, 71(1), 36-43. doi:10.1111/1467-8624.00116 Main (in press). Effective middle school teacher teams: A ternary model of interdependency rather than a catch phrase. Teachers and Teaching–Theory and Practice, 18(1). Martin, C., Fabes, R., Hanish, L., & Hollenstein, T. (2005). Social dynamics in the preschool. Developmental Review, 25, 299-327. doi:10.1016/j.dr.2005.10.001 Meirink, J.A., Meijer, P.C., & Verloop, N. (2007). A closer look at teachers’ individual learning in collaborative settings. Teachers and Teaching: theory and practice, 13(2), 145–164. doi:10.1080/13540600601152496 Scribner, J.P., Sawyer, R.K., Watson, S.T., & Myers, V.L. (2007). Teacher teams and distributed leadership: A study of group discourse and collaboration. Educational Administration Quarterly, 43(19), 67–100. doi:10.1177/0013161X06293631 Steenbeek, H., & van Geert, P. (2007). A theory and dynamic model of dyadic interaction: Concerns, appraisals, and contagiousness in a developmental context. Developmental Review, 27, 1-40. doi:10.1016/j.dr.2006.06.002 Thelan, E., & Smith, L. (1994). A Dynamic Systems Approach to the Development of Cognition and Action. Cambridge: MIT Press. Wasserman, S. & Faust, K. (1994). Social Network Analysis: Methods and Applications. Cambridge: Cambridge University Press. Yoon, S., Liu, L., & Goh, S. (2010). Convergent adaptation in small groups: Understanding professional development activities through a complex systems lens. Journal of Technology and Teacher Education, 18(2), 319-344. doi: p/31362.

Acknowledgment
This research is supported by the National Science Foundation under Grant No. 0741659.

© ISLS

171

ICLS2012

Volume 2: Short Papers

What a Long Strange Trip It’s Been: A Comparison of Authors, Abstracts, and References in the 1991 and 2010 ICLS Proceedings
Victor R. Lee, Lei Ye, & Mimi Recker, Department of Instructional Technology and Learning Sciences, Utah State University, 2830 Old Main Hill, Logan, UT 84322-2830 USA, {victor.lee, lei.ye, mimi.recker}@usu.edu Abstract: We examine differences in authorship, word usage, and references in full papers from the 1991 and 2010 ICLS proceedings. Through a series of analyses, we observe that, while authors largely hail from the US, national and regional participation in the LS community has broadened. Word usage suggests a shift in emphasis from cognitive issues to ones that are both cognitive and cultural. Reference analysis indicates a shift in core literatures and influential authors.

Introduction
While the dawn of an academic discipline is usually not heralded by a birth announcement, 1991 was certainly marked by three signature moments with respect to the publication and presentation of Learning Sciences research: the release of the first issue of the Journal of the Learning Sciences (JLS), edited by Janet Kolodner; the first proceedings of ICLS, which was held in Evanston, Illinois (USA), and the first Computer-Supported Collaborative Learning (CSCL) workshop in Carbondale, Illinois (USA) (as stated on the ISLS website). Today, JLS continues to thrive as a highly influential education research journal. ICLS continues as a respected conference venue, with its present iteration taking place in Sydney, Australia. A vibrant CSCL community continues to grow with a series of ongoing conferences and, most recently, the creation of another high-impact academic journal (ijCSCL) that began printing in 2004. Considering those research outlets alone, there are now at least four formally recognized venues for publishing innovative work related to the Learning Sciences. In this paper, we focus on changes in one of those publication venues (ICLS proceedings) at two points in time (Birnbaum, 1991; Gomez, Lyons & Radinsky, 2010). We adopt this more narrow focus for reasons of tractability and systematicity. On the one hand, we were eager to explore whether fairly simple tabulation procedures could offer us a glimpse into the nature of our field. At the same time, and considering there are only a limited number of printed copies of the 1991 proceedings and no public electronic versions, we were well aware that doing systematic counts of selected items within the proceedings would require a great deal of data preparation. However, we believe that this endeavor was appropriately timed and the two texts were well selected, as the two conferences were in the same metropolitan area (and thus should have enabled comparable geographic participation) and the time span was over the equivalent of a human generation. Moreover, these proceedings were also the oldest and most recent data points from a venue that has maintained the same name, even when additional relevant publication venues (such as ijCSCL) have emerged and established shared, but still distinct identities.

Analytical Precedents
Within the past decade, members of the Learning Sciences community have used tools from the information sciences to better understand participation in relevant journals and conferences. For example Kirby, Hoadley & Carr-Chellman (2005) conducted a citation analysis of six Learning Sciences (LS) and Instructional Systems Design (ISD) publications published through 2001. They sought to determine if overlap existed between two fields that have been understood by some as pursuing similar goals. In their study, they found that very few scholars (less than 0.5%) published in flagship journals for both fields and that cross-citations between LS and ISD publications did not exceed 0.5% of total references in either direction. Hoadley (2005) extended this work in an analysis of CSCL conference participation from 1995 to 2003. That study identified disciplinary and national affiliations of CSCL paper presenters and international collaborations over time. Kienle & Wessner (2006) provided another analysis of CSCL conference proceedings that included 2003 and 2005. There analysis showed greater international diversity and collaboration over time. For ICLS, however, the picture is less clear. Kirby, et al’s work included analysis of ICLS proceedings but that analysis has not considered the five meeting since 2000. All analyses of later conferences have maintained CSCL as the focus. While the contributions of scholars involved in the CSCL community is central to the growth of the Learning Sciences, it still remains the case that the individuals participating and the topics discussed have some areas of individual distinction. We intend to fill the ICLS gap by using some of the same analytical tools that were used in studies of CSCL proceedings. Also, we are considering the simple metric of word frequency as a potentially telling attribute for a proceedings analysis.
© ISLS 172

ICLS2012

Volume 2: Short Papers

Simply stated, we wanted to know which words were used most often in ICLS papers. While frequency alone can be a fairly crude measure for characterizing text contents, simple word frequencies from text corpora have still been recognized in high profile journals as a surprisingly powerful tool through which one might understand changes within cultures and communities (Michel et al., 2011). Our hope is to elucidate the topics and issues that were of primary concern at ICLS meetings.

Questions, Data Sources, and Methods
We ask four questions about the two sets of proceedings: 1) Which authors were contributing to these two conferences? 2) What nations were being represented? 3) What topics were being presented?, and 4) What sources were most cited? To answer these questions, we obtained copies of all full papers (up to 8 pages each) from both the ICLS 1991 and 2010 proceedings. We excluded posters, symposia, and keynote abstracts in order to maintain comparable data sets (i.e., the 1991 proceedings had no listed posters and keynote abstracts were highly variable in their lengths). In total, there were 58 full papers in the 1991 proceedings and 149 from 2010. The first page, with author information and abstract, and the complete reference lists were manually scanned from printed copies or downloaded from the ACM digital library. From those, we extracted the following information: Paper authors. Unique authors for each paper were extracted, segmented, and tallied. First author’s Geographic Location. Because of the high cost of data preparation and variability of presentation, only the first author’s geographic location (at the time of publication) was considered. Abstracts. Abstracts are meant to be concise and comprehensive descriptions of the contents of an article. We used these as a proxy for paper content that could help avoid possible inflation of word frequencies due to excessive word repetition within the paper. We excluded common stop words (such as “the” or “it”) and words that are generic to research papers (e.g., “results”, “study”). Referenced works. The complete reference lists were automatically parsed to extract the full set of authors, the date of publication, and the publication venue (i.e., journal titles).

Results & Analysis
Which authors were contributing to these conferences?
The 2010 proceedings contained almost three times as many papers as the 1991 proceedings (2010: 149 papers, 1991: 58 papers) from more unique authors (2010: 364 authors, 1991: 113 authors). Papers in the 2010 proceedings overall had a greater tendency to have more co-authors (2010: 2.44 co-authors, 1991: 1.95). Perhaps surprising is the observation that there were only four individuals who were listed as contributors to full papers in both conferences: Katerine Bielaczyc, John M. Carroll, Kenneth Koedinger, and Janet Kolodner. From firsthand observations by the authors of this current paper, we remain certain that there were more than four individuals who participated or attended both conferences. Most likely, these individuals had a change in status with respect to their conference involvement. They were presenters within symposia or posters, serving as discussants, participating in workshops, or simply interested community members who were attending and meeting with friends and colleagues. Comparable changes in participation status at conferences were documented by Hoadley (2005) with respect to five CSCL conferences.

What nations were being represented?
Because of the relatively high cost of data preparation (scanning, converting, editing, etc.) and a large amount of variability in how authors listed their information (thus making it difficult to automatically parse all author affiliations), a geo-analysis of authorship was conducted only on first authors. While we are aware of the limits associated with considering only one contributor to a paper, we did expect that this would be telling in that first authors are often the presenters and primary attendees for conference papers. With respect to national representation at ICLS in both meetings, there was a clear majority of first authors who came from North America (Figure 1), and in particular the United States (69% in 1991 and 62% in 2010). In some respects, this is not surprising as conference location influences who will submit papers and ultimately attend (Kienle & Wesser, 2006). The most visible changes in national participation took place beyond North America. The 1991 proceedings had first author representation in Europe and Australia only. In 2010, Asia represented 15% of the first authors and the other regions declined. In neither year were there any first authored papers from South America. An additional analysis of countries of percent representation of first authors in specific nations was conducted. These analyses suggest a general increase in the number of nations represented in 2010 in comparison to 1991 (2010: 18 nations, 1991: 8 nations). The greatest percent increase appeared from

© ISLS

173

ICLS2012

Volume 2: Short Papers

Germany (2010: 1.7%, 1991: 8.7%) and the greatest percent decrease coming from the United Kingdom (2010: 8.6% 1991: 0.7%).

Figure 1: Percent of first authors by world region in 1991(left) and in 2010 (right) In making sense of these differences and changes over time, it is important to note that when ICLS was first held in 1991, it was actually organized as a special session of a conference normally held by the Artificial Intelligence in Education community, a research community where a number of Learning Scientists had been originally trained. Thus, we should not be too surprised that there are differences. Still, the increase in representation from Asian nations is also paralleled by a greater global prominence that has been noted over the past two decades. Yet, even with this global shift, there is a striking pervasiveness of papers from the US (over 60% in both years) in comparison to CSCL. This can be understood as partially due to the fact that the conferences were held in the same metropolitan area. However, as is the case for many large nations, a variety of regions and a number of institutions comprise that large percentage. Given the large number of contributions from US authors, we chose to analyze the distribution of US-based first authors by state. Three of the more populous states, California, Illinois, and Pennsylvania, were highly represented in 1991 (CA = 20%, IL = 28%, PA = 15%). These states have been known to have prominent institutions conducting research related to Artificial Intelligence (AI). Those three states continued to have a relatively high percentage in 2010 (CA = 19%, IL = 9%, PA = 9%)., but were also accompanied by other states such as Indiana (6%), Maryland (6%), Washington (5%), and Wisconsin (9%). Sixteen states previously unrepresented in 1991 had first authors in 2010 and two states represented in 1991 (Connecticut and New Mexico) were not represented in 2010. To determine if population was the biggest predictor of author location, we extracted U.S. population data by state from the 1990 and 2010 censuses. A Spearman rank order correlation between first author location and U.S. state population was not significant in neither the 1991 (rs = .36) nor the 2010 (rs = .31) proceedings. Note also that paper contribution rates by state across conference proceedings were significantly correlated (rs = .70, p < .05), showing similar rates across time periods. Thus, state participation appears to be broadening, but it seems to be highly dependent on the presence and location of particular individuals and institutions with research resources (e.g., the LIFE Center with University of Washington as a partner institution, the GLS group at University of Wisconsin) rather than a uniform change due to demographic shifts.

What topics were being presented?
We used word frequency in abstracts as a proxy for the content that was being presented in each paper. We deliberately conflated the counts of words that would be the same except for small variations, such as plural form (e.g., “case” and “cases”), change in tense (e.g., “model” and “modeled”), or comparable adjective and noun forms (e.g., “mathematics” and “mathematical”). While these conflations could have led to groupings where there are subtle nuances in senses of words that reflect different research agendas, there were enough overlaps that consolidation was deemed appropriate by the authors. Full tables of words and word frequencies were produced, but due to space limitations, we present word clouds of the top 20% of words that appeared in the abstracts (Figure 2). In the word clouds, larger and darker fonts represent higher frequencies of occurrence.

© ISLS

174

ICLS2012

Volume 2: Short Papers

Figure 2: Word clouds from 1991 (left) and 2010 (right). In 1991, the most frequently used words were: model, cognition, conceptual, domain, environment, model, strategy, training, and tutor. The most frequently used words in 2010 were: case, conceptual, epistemic, examine, inquiry, interaction, mathematics, practices, representation, and strategies. The intersection of this set includes: case, cognition, conceptual, representation, theory, and strategy. While used frequently, “case” was often used in 1991 to refer to cases as used in case-based reasoning (Kolodner, 1993) and in 2010 it was used for case-based learning and for case studies. “Representation” had been used in 1991 often to refer to knowledge representation, and increasingly in 2010 to refer to external representations and representational practices (such as creating inscriptions). 1991 involved unique terminology that involved information processing models and constructs such as “training”, “tutor”, and “instruction”. Unique terminology in 2010 suggested a contingency of scholarship geared toward sociocultural constructs such as “discourse”, “participants”, and “practices”. While sociocultural constructs emerged and gained prominence, cognitive terminology such as “cognition” and “conceptual” still appeared in both conferences, often associated with research related to tutoring systems, conceptual change, and artificial intelligence.

What sources were being used?
Our final question in this investigation related to the sources – in particular, the authors and reference sources – that appeared in the two sets of proceedings. We were hoping to find out what journals figured most prominently (e.g., did JLS indeed play a prominent role in this community after it was established?) and whose writings were considered influential. As there were many more references than there were number of papers or contributing authors in the proceedings (2010: 24.9 refs/paper, 1991: 12.7 refs/paper) and given the aforementioned difficulties of data preparation, we focused strictly on automated analyses of references. While we ran several such analyses, we presently report just on journals and authors (due to space limitations). In analyzing the journals that were cited, we chose to focus on the journals that comprised at most the top 20% of cited journal articles. Only two journals (2.2% of all unique journals) were the source of 20% of journal articles cited in 1991. These included Cognitive Science (13.3%) and Artificial Intelligence (6.9%). In 2010, four journals served as the source of 20% of cited articles. These included: Journal of the Learning Sciences (7.1%), Cognition & Instruction (4.9%), Journal of Research in Science Teaching (4.7%), and Science Education (3.3%). Cognitive Science was still cited in 2010 but it made up a much smaller percentage of the total share (1.6%). The most highly cited AI journal in 2010 was the International Journal of Artificial Intelligence in Education (0.1%). Based on the journal share percentage, the foundational literature emphasis changed substantially as newer journals tuned to issues of learning and instruction were established and more widely cited. Finally, we considered authors of cited works, regardless of publication type. Although selfcitations may have played a role, we did not exclude instances of self-citation. Our underlying assumption was that, when an author was cited so heavily, even had s/he cited a lot of their own work, s/he were still likely to have been cited in at least some articles in which s/he were not an author. Space limitations prevent us from showing a longer list. Therefore, we list simply the top five cited authors (or organizations, as is the case with the US National Research Council) and the frequency of their name in the entire corpus of references. diSessa, A. A. was the only author to appear in both top-five lists. Table 3: Most cited authors and percent frequency within each set of proceedings.
1991 Author Anderson, J.R. Schank, R.C. Breuker, J.A. diSessa, A.A. Papert, S. % 3.1% 2.5% 1.2% 1.2% 1.2% 2010 Author US NRC Scardamalia, M. Hammer, D. diSessa, A. A. Brown, A. L. % 1.1% 1.0% 0.8% 0.8% 0.7%

© ISLS

175

ICLS2012

Volume 2: Short Papers

Discussion and Conclusion
What sort of journey has it been from Evanston, Illinois in 1991 to Chicago, Illinois in 2010? In terms of distance and distance, it was roughly 19 miles over 19 years. However, the field of Learning Sciences, as reflected in its conference proceedings, has shown a number of shifts. Participation generally broadened across geographic locales. A new core body of journals served as the source of a peer-reviewed knowledge and the authors whose work was acknowledged and cited largely changed. The research emphasis appeared to have moved from a cognitive approach in which training with systems using Artificial Intelligence was used to provide instruction to a view of learning that involved enculturation into disciplinary practices and a variety of interactions within a complex, socially-mediated learning ecology. Cognitive Science and Artificial Intelligence continue to play a role in the Learning Sciences, but that body of research is now accompanied with investigations of inquiry, science and mathematics learning, and a growing interest in issues of external representation. Undoubtedly, these findings are limited in their generalizability due to the sampling strategies and selected analytical techniques. More analyses and even more sophisticated techniques could have been used. However, these snapshots are still useful because they can serve as touchstones to help us reflect upon the ongoing intellectual trip that has been and continues to be the Learning Sciences.

References
Birnbaum, L. (Ed.). 1991 The International Conference on the Learning Sciences: Proceedings of the 1991 Conference, Association for the Advancement of Computing in Education, Charlottesville, VA. Gomez, K., Lyons, L., & Radinsky, J. (Eds.). Learning in the Disciplines: Proceedings of the 9th International Conference of the Learning Sciences (ICLS 2010). International Society of the Learning Sciences: Chicago. Hoadley, C. M. (2005). The shape of the elephant: Scope and membership of the CSCL Community. Paper presented at the 2005 Conference on Computer Supported Collaborative learning. Kienle, A., & Wessner, M. (2006). The CSCL community in its first decade: Development, continuity, connectivity. International Journal of Computer-Supported Collaborative Learning, 1(1), 9-33. Kirby, J., Hoadley, C., & Carr-Chellman, A. A. (2005). Instructional systems design and the learning sciences: A citation analysis. Educational Technology Research and Development, 53(1), 37-48. Kolodner, J. (1993). Case-Based Reasoning. San Mateo: Morgan Kaufmann. Michel, J.-B., Shein, Y. K., Aiden, A. P., Veres, A., Gray, M. K., The Google Books Team, . . . Aiden, E. L. (2011). Quantitative analysis of culture using millions of digitized books. Science, 331, 176-182.

Acknowledgments
We gratefully acknowledge Min Yuan, Jon Thomas, and Anne Diekema for their comments and assistance. A slightly longer version of this paper with additional figures and analyses can be obtained from the lead author.

© ISLS

176

ICLS2012

Volume 2: Short Papers

Scaffolding Collaborative Sensemaking during Critique of Explanations in Technology-Enhanced Science Curriculum
Elissa Sato, University of California at Berkeley, 4523 Tolman Hall, Berkeley CA 94720, elissa.sato@berkeley.edu Vanessa Svihla, University of New Mexico, 1 University of New Mexico, Albuquerque, NM 87131, vsvihla@unm.edu Abstract: This qualitative study extends a comparison study investigating critique and feedback in a technology-enhanced sixth-grade earth science curriculum unit. The unit was designed to elicit and scaffold collaborative sensemaking of criteria for explanations in science. Quantitative analysis suggested the activity had limited success in terms of prompting revision, but the qualitative analysis illustrated different dimensions of engagement in critique, demonstrating the rich potential of critique for collaborative sensemaking. Implications for instructional design are discussed.

Introduction
Generating explanations is central in science (National Research Council, 2007) and important in learning (Chi et al. 1994; NRC, 2007), yet non-trivial for students learning complex topics. Typical instruction does not support students in the practice of generating explanations, yet prior research has shown the value of scaffolding students to construct and evaluate explanations (e.g., McNeill, Lizotte, Krajcik, & Marx, 2006; Sandoval & Reiser, 2004). We situate the generation of explanations within the knowledge integration (KI) framework (Linn & Eylon, 2006), which identifies four processes: eliciting ideas, adding new ideas, distinguishing among ideas, and refining the repertoire of ideas through reflection. From this perspective, learning to generate quality explanations involves, in addition to synthesizing relevant domain knowledge, making sense of and applying criteria for what constitutes a good explanation. We conceptualize scaffolded critique of explanations as generative activities in that they support students in a) distinguishing among criteria in the context of specific explanations, and b) sorting through and refining the ideas captured by the targeted explanations. We investigate how critiquing explanations might impact students’ ability to distinguish and refine their repertoire of ideas. We present two cases to highlight ways students engaged with a particular critique activity, examining how students make sense of commonly used, yet vague criteria for good scientific explanations.

Methodological Approach Building on Prior Work
This paper presents findings that extend a prior comparison study investigating the source of feedback (i.e., from the teacher or a peer) and the quality of the explanation being critiqued (i.e., high or low KI score, cf. Linn, Lee, Tinker, Husic, & Chiu, 2006) and their impact on students’ revised explanations. Quantitative analysis found a significant advantage for instructor feedback on gains after controlling for the quality of the original explanation. A comparison of instructor and peer feedback revealed that instructor feedback was more prompt-specific than peer feedback, although students still struggled to apply specific feedback during revision (for a more detailed discussion of the study findings, please refer to Sato and Linn, 2011; in preparation). Although this may suggest that the activity did not help students distinguish between their ideas, our video data suggest otherwise and indicate elements in activity design that have implications for instruction. In this qualitative analysis, we present two cases that illustrate the kinds of engagement during critique that were observed in the data corpus as a whole. Although their orientations were neither one-dimensional nor fully consistent throughout the critique activity, the orientations were generally representative of each dyad’s engagement with the task. Due to space limitations, we primarily focus on one case.

Context
A sixth-grade technology-enhanced earth science curriculum unit, Global Climate Change (GCC), was developed using the Web-based Inquiry Science Environment (WISE, Linn, Davis, & Bell, 2004) based on the KI perspective (Linn & Eylon, 2006). Students worked in pairs throughout the unit. They generated an explanation for a phenomenon, then their ideas about criteria were elicited in a first critique of two sample explanations chosen from student responses in a previous implementation of the project (Table 1). We focus on this activity as it affords insight into how students make sense of the criteria presented in the task. Because students tend to focus on surface features rather than the underlying ideas, explanations were selected to represent these: one was stylistically sound in terms of spelling and grammar, but sparse in terms of science content; the other had imperfect spelling and grammar, but described the phenomenon in detail. Students critiqued the two in succession. Our goal was to implicitly prompt students to compare the two and help them distinguish between their existing ideas about what makes a good explanation. Students rated each explanation

© ISLS

177

ICLS2012

Volume 2: Short Papers

for both surface (spelling and grammar) and science content (e.g., the response needs more evidence; the science ideas are wrong or vague; the science ideas could be described in more detail) from a list of criteria, then explained their choice for science content. Our goal was to prompt students to discuss criteria commonly encountered during instruction; there was no correct choice per se among the list of science content criteria, nor were they mutually exclusive. By asking students to choose the criterion they felt best captured the science content in the explanation, we hoped to motivate students to engage in sensemaking of the criteria. Table 1: Preselected Explanations Critiqued by Students during the First Critique Prompt: What happens to global temperature in an environment with low albedo? What happens to solar radiation (SR) in the model in step 4.2 that supports your answer? Characteristics Preselected Explanation Good style (spelling/grammar), It changed a lot. It went down then bounced. vague science content Imperfect style (spelling/grammar), THe globle tempeture went down when the albedo was low, like for the detailed science content ocean, or when the albeod reflected only 5 percent of the solar radiation. When the soler radiatoin was reflecting, it coud not change into heat energy it just went back to space.

Case 1: The Sense-Making Pair
Kostas and Ted studied the peer feedback version of the GCC unit. Though they began the unit in the bottom quartile of their treatment group based on pretest measures, they were both highly motivated students who engaged with the unit and held extended discussions before deciding on responses to explanation prompts. It was common for the two to raise questions and objections if one did not share the other’s opinion or understanding of a particular activity. Although they did not generate detailed feedback, their discussions illustrated rich instances of both surface and content criteria sensemaking throughout critique. Kostas and Ted negotiated whether accurate spelling and grammar can be judged independently of science content during surface critique of both explanations. Kostas appeared to grapple with the distinction between stylistic clarity and its impact on the clarity of science content. In the transcript below, the pair has begun surface critique of the first explanation (“It changed a lot. It went down then bounced,” Table 1). Ted asserted his rating (“good: few errors”), and after an initial assent, Kostas had reversed his position. T: K: T: K: T: K: T: What would you give it? Out of three? On a scale //of//— //One?// One? ((incredulously)) Yeah, this is like, so bad. ((sweeps finger across the explanation)) No, it’s not— It’s not good English, you know? Like, “It changed a lot. It //bounced and//”—((reading out loud in a singsong)) //That’s in// “scientific content and clarity.” ((pointing to next critique prompt for science content and clarity)) That’s what it means. Here, “the science idea could be described in,” “some ideas are wrong or vague.” ((reading some of the science content criteria; K leans in toward the monitor as T reads)) Like, what changed, though? ((pointing to “It changed” portion of first explanation))

K:

Although Kostas did not specify his misgivings other than that they were not about spelling, he elaborated that the explanation had “bad English” (lines 7-8). Ted countered that Kostas was referring to science content and clarity, not spelling and grammar (lines 9-12), and oriented Kostas to the next critique prompt and its list of criteria targeting science content and clarity. By asking Ted, “what changed?” (line 14) Kostas seemed to have been trying to identify what was bothering him on a more specific level. Given that the explanation did not specify what “it” was that changed, this problem can be viewed simultaneously as an issue of vagueness in language and of lack of clarity in content. Here, Kostas was focusing on the vagueness of the language. In effect, Kostas may have been alluding to sophisticated ways in which the surface and content criteria can be considered to overlap. Although they eventually chose “good” as their response, the discussion re-emerged during the surface critique of the second explanation, indicating that making sense of the surface and content criteria was important for Kostas. Kostas and Ted continued to engage in sensemaking discussions during the science content critique for both explanations. In the example below, they had formed initial impressions about whether each criterion was applicable to the particular explanation before reaching the last criterion on the list,

© ISLS

178

ICLS2012

Volume 2: Short Papers

“the science ideas could be described in more detail.” The ensuing discussion illustrates the rich conversations they had as they negotiated the meanings of the criteria: K: This is... This is it. ((points to “needs more detail” criterion)) In more detail... 'cause [the explanation] just said, "It changed a lot. It went down then bounced." So it needs like, more support... evidence [another criterion], and like, more detail. Yeah, it's more evidence. It's... No, it doesn't need more detail. 'Cause, I mean... [it] has a LOT of detail. Well, not that... well—((sighs)) It has like, a 4 out of 5 detail. It said, "It went down, then bounced." That's kind of describing it in detail. But it's not describing it in like, proper grammar. I think this is right. ((points to “needs more detail”)) I think… no. This is “need more detail.” I think it’s “needs to have evidence.” I’m really confused between both of those. ((points to “needs more detail” and “needs to have evidence” criteria)) I don’t know. [inaudible] I think what you chose is right. I don’t think there’s, like, a wrong answer in this question.

T:

K: T: K:

T:

Although the target explanation prompt explicitly asked for evidence from a previously explored model (Table 1), Kostas and Ted did not discuss whether the explanation contained explicit references to the model. Instead, they discussed how to distinguish evidence from detail. Ted struggled to clarify why he believed that “needs to have evidence” was the more appropriate critique over “needs more detail (lines 19-21).” Ted’s concluding remark, “But it’s not describing it in like, proper grammar (line 23),” is notable in light of the fact that they had already evaluated the grammar and spelling of the explanation to be “good.” It may suggest that he had some ideas about what counts as evidence in science explanations as opposed to simply descriptive detail. His struggle is similar to Kostas’ earlier attempt to distinguish between surface stylistic clarity and content clarity in that Ted also struggles to articulate why he believes one criterion is more applicable than the other (lines 19-21). In this case, however, Kostas and Ted did not pursue this line of thought to the same extent as they did previously for Kostas. After Kostas professed his uncertainty about what evidence and detail meant (line 27-28), Ted responded that any of the criteria may be acceptable answers (line 31). Unable to pin down what exactly they felt was lacking i