You are on page 1of 11

Procedia - Social and Behavioral Sciences 00 (2012) 000–000

Procedia Social and Behavioral Sciences


Enriching multimedia expression with tangible objects: the learning benefits for preschoolers
Chau Kien Tsonga,1, Toh Seong Chongb, Zarina Samsudinc
a,, Centre For Instructional Technology and Multimedia,Universiti Sains Malaysia, 11800 USM, Pulau Pinang, Malaysia

Abstract The pedagogical benefits of tangible objects are abundant. However, the adaptation of tangible objects into multimedia learning for preschoolers is still remained absent. In view of this, this paper is produced with objective to discuss the advantages of using tangible objects by way of literature review. For tangible objects embraced in multimedia realm, multimedia objects can complement them meaningfully by providing fascinating multimedia expression and effects like games and versatile clues for preschoolers to know what kind of interaction is expected for tangible objects during the course of learning. A relevant prototype named TangiLearn has been developed for a case study. Its finding is briefly reported.

© 2011 Published by Elsevier Ltd. Selection and/or peer-review under responsibility of [name organizer]
multimedia; multimedia object; tangible object; tangible multimedia.

1 Corresponding author. Tel.: 010-5059150. E-mail address:

Toh. 2005). 2002). For example. & Zarina / Procedia . and their reasoning facilities are not yet fully developed. and underlying cognitive developmental abilities has to be sorted out. including education (Hole & Schull. Spreen. 2005). hand motor skills. As tangible objects surrounding children are characterized by offering full range of sensory qualities. . Although the efficacy of the multimedia systems nowadays is confirmed in many different lines of research (Vickneasvari. Majid. However. 2009. we observe a learning gap between the preschoolers and multimedia system. they can serve as an excellent candidate to bridge the learning gap. Smith. & Ong. & Casey. 2008. Many systems are overwhelmed by a large quantity of instructions that often exceed the learning abilities of the children (Mohamad. They are at the “preoperational state” of cognitive development structure where their cognitive state develops through senses stimulated by external concrete stimuli (Piaget.2 Chau. Souriyavongsa. They have limited vocabulary. they may not be equally suitable for preschoolers who have very varied needs and limitations described above. 2007. 1952. navigating a scene in VR is apparently a formidable task for preschoolers. As such. & Jones. Introduction Developing a truly beneficial multimedia learning system for preschoolers need extra considerations. Chen. virtual reality (VR) has been established for sometimes as a tool efficient for many aspects. MacFarlane. 1972). limited degree of motor acuity (Huang. Preschoolers are in a category where much behavior is described by its own specific characteristics.Social and Behavioral Sciences 00 (2012) 000–000 1. Read. 2011). Chin. 1 Gap between multimedia learning system and preschoolers A new paradigm shift in multimedia system that truly adapted to their characteristics. Chen. Fig.

Chau. Toh.Social and Behavioral Sciences 00 (2012) 000–000 3 Fig. & Zarina / Procedia . 2 Tangible multimedia bridges learning gap .

1997). Zuckerman & Resnick (2003) asked. In defining the term “abstract”.. who were unable to solve an addition problem presented symbolically. Accordingly. Children learn through hands as natural motor response in human daily living. We propose to name such multimedia “tangible multimedia learning system”. It gives different view between concrete and abstract. Children can touch it. as they are simultaneously a means for complex expression and sensation …” (p. p. children learn most when they are engaged with concrete activities meaningfully to them. with physical handling. There exists a theoretical viewpoint called “embodied cognition”. and then discuss why the merging of tangible objects and multimedia objects is required. accessible and learnable (Piaget.Social and Behavioral Sciences 00 (2012) 000–000 However. 2004). & Balakrishnan. tangible interface can help.4 Chau. 2007). 2010). This idea is also reflected in Montessori’s education where she stated that “Children build their mental image of the world. Prior to actual experimental research (Chau. concrete and abstract are inseparable. & Takayama. Tangible objects can give rise to mental images in the children mind. or in short. 2011). the children can concretize the abstract concepts that they never see before. they become conscious of reality” (Burnett. as cited in Marco. 2006). were able to do so when the same question was presented with concrete referents (Manches. can tinker with it …. Reeve. The Learning Benefits of Incorporating Tangible Objects for Preschoolers Tangible object denotes an intuitive and natural ways of learning (Ishii & Ullmer. various cognitive operations will be degraded. & Zarina / Procedia . 105). Indeed. Morsella and Krauss (2004) said. Hengeveld. 2009. tangible multimedia. Antle. some may doubt about the advantages and pedagogical possibility of tangible objects in multimedia realm. Similar outcomes produced in research conducted by Canobi. 143). particularly hands. Droumeva. “How do we ‘concretize the abstract’? A physical. Abstract thinking is actually grounded in perceptual and sensory experiences (Manches.” (p. if hands are prevented from learning. and Takayama (2006) said “A natural place to start is with our hands. Hartmann. they like to touch things. move. Wigdor. Empirical researches showed that children were able to perform symbol manipulation tasks with concrete physical objects when they failed to perform using abstract representations (O’Malley & Fraser. & Ha. Toh. . & Overbeeke. According to Papert (1991). just like toddlers. For example. Hummels. point. we have conducted a relevant case study and this paper also provides brief report of the case study. Tangible objects bring tangibility experience into learning. 72). & Zarina. Hartmann. and release. Shen. 2009. through the action and motor responses. Manches (2010) said. 1962. Cerezo. and. With tangible objects. “abstract is in the sense of being disconnected from perceptual experience” (p. they learn. & Pattison (2002) on additive composition on 50 children aged 5 to 6 using physical objects. Hughes’ (1986) research found that young children. Klemmer. It is only the concrete objects that can give perceptual experience. Mazzone. grasp. 2010). thus making abstract concepts more visible. 1962. & Read. one can be aware that it has something to do with the engagement of primary senses of the human. hands are crucial for children in learning (Klemmer. and without realizing. 811). Even though the so-called “natural” way is difficult to measure objectively. Baldassarri. Toh. 2. and this further supports the need of using tangible objects for concrete learning in multimedia realm. and is subject to interpretation (Forlines. 2009). the preschoolers will have to directly hold objects with one or both hands. To convince them of the need of such adaptation. With tangible objects. we thus produce this paper with an objective to discuss the learning benefits of using tangible objects by way of literature.

and perform spatial organization with more degree of freedom. Xu (2005) contended that GUI blocks human natural abilities (Xu. In their research. Tangible objects evoke space for trial-and-error activity. and Takayama (2006) said “. All these open up window for enhancing the understanding of the preschoolers (Kelly. Toh. Touch can convey additional meaningful information about its context (Wimmer. Singer. combine and recombine things. Kitamura. & Zarina / Procedia . & Itoh. 2008). Picture.. Hicks. Kishino. & Benford. Hartmann. Watson. 2008). on the other hand. 2009). Kelly. 143). many of the TUI users were found to use the spatial relationship between testing blocks and surrounding to assist them to recite the blocks content. a person can recognize an object merely through touching (Jetsu. 1968). 2008). Spatiality is human innate ability to interact with concrete objects in physical space (Sharlin. Klemmer. Hicks. Spatiality in our context also means space for hand gestural operation. they (hands) allow for complicated movement but their skin also has the highest tactile acuity of our extremities” (p. 2002. can be recognized. Cognition does not exist only in mind. like in the way the digital multimedia systems nowadays deliver information. spatiality means spatial information such as depth and position of physical objects required in the holistic processing of working memory system (Xie. 3). 2005). Piaget (1968) stated. O'Malley. 1992). & Goldin-Meadow (2002) discovered that children who need to gesture retain the mathematical knowledge better than children who only speak. Singer. the children cognition process to make sense of the concepts outside of children’s immediate context will be weaker. & Goldin-Meadow. but not gesture. Patten and Ishii’s (2000) comparative research on space usage in a TUI (Tangible user interface) and GUI (Graphical user interface) showed that TUI users outperformed GUI users in the location recall task. Fig. but have no substantial spatial organization (Piaget. Tangible objects offer the only way for tactile emotional experiences.Chau. Learning merely through visual and auditory channels. 2004). Alibali and DiRusso (1999) in a research on 25 children discovered that children count objects most accurately if they gesture when counting. “A world without objects would not present the character of spatial homogeneity and of coherence in displacements that marks our universe” (p. Manches. In a study conducted on 84 children.. 2011) not accessible visually such as the softness and weight of materials. 3 Engagement of vision and touch via tangible object . In our context.Social and Behavioral Sciences 00 (2012) 000–000 5 Tangible objects are a logical choice for spatial activities (Xie. but also during interaction between the materials and mind (Crook. This can be evidenced from a situation where without the aid of visual.

& Martin (1999) called this mental model the “tool of mental sight” (Marco. due to long use of abacus. the sole adoption of tangible objects in learning imposes certain constraints that hinder learning. invisible abstract concepts or materials in “intangible” form (like water molecule. and behavior of tangible objects changed against physics laws (making the effects on objects discreet. O'Malley. quizzes. or have their gravitational forces degraded. tangible objects cannot be passed trough.6 Chau. such suggestion is not viable because replay and search for relevant video files consume time whereas the demand of writing for written log exceeds the linguistic ability of young children (Manches. Baldassarri. 2009). Haxby. As a natural form of material that have substance. compare the past and present action. chemical reaction) can be made visible. moved on their own. 2009). Tangible objects do not provide any means for children to revisit any record of the previous solution for revision purposes. tangible objects are obviously a good choice. Multimedia expression overcome the aforesaid constraints by providing various multimedia effects such as pop up message. where its effects are gradual rather than discrete (Manches. 2010. & Read. Accordingly. Chao.Social and Behavioral Sciences 00 (2012) 000–000 Another important advantage of using tangible objects is their potential of supporting the working memory of a learner (Manches. 2009). and to examine the relationship between different solutions for the tasks that they are attempting (Manches. 3. Droumeva. However. The manipulation of tangible objects is continuous. and versatile clue so that the preschoolers know what kind of action or interaction expected for the tangible objects. Mazzone. 2009). This can be evidenced from the capability of abacus experts. hints. hence. 1968). colliding objects made passed through). O'Malley. 2010). games. the preschoolers may not be able to observe the outcomes clearly. This may be due to the fact that tactile information helps children to offload some cognitive burden and mental operation in learning (Manches. Fig. Another constraint is that actions performed on tangible objects are not traceable. 2010). Complementing tangible objects with multimedia If the preschoolers are to learn the real world. & Ha. Using various memory tasks. that are permanent and of constant dimensions (Piaget. However. Hatano and Osawa (1983) tested 3 abacus experts and found that they had a very large memory span of numerical digit strings and arithmetic abilities than the average. Cerezo. made disappear. One such constraint is the behavior of tangible objects that is governed by the laws of physics. the experts have internalized a mental model called “mental abacus” that mimics the structure of an abacus to keep visuo-spatially a number comprising long digits. There is suggestion of using video recording and written log of action. Through animation. Toh. & Benford. & Zarina / Procedia . & Benford. 4 Tangible and multimedia objects complement each other . animated feedback. Antle.

To gather preliminary evidence to support the assumption that tangible multimedia is beneficial to preschoolers. tangible objects are rightfully cognitive-friendly system to preschool whereas tangible systems are more as interaction-friendly system to users. digital multimedia objects are “externalized” meaningfully. TangiLearn was a learning environment consisted of many virtual and tangible learning objects. & Zarina. TangiLearn for preschoolers (Chau. 2012b).Social and Behavioral Sciences 00 (2012) 000–000 7 Our proposed tangible multimedia is different from many tangible systems that aimed for facilitating human-computer interaction. The tangible and multimedia objects binding were implemented using Quick Response (QR) code marker and ActionScripts 3. Findings of case study for TangiLearn Evidence for the pedagogical benefits of the tangible objects is abundant. Via such real-life objects. 2012a). For example. and thereby gain their “tangibility”. Fig.0 library. & Zarina. Toh. Toh. This drives us to develop a prototype of tangible multimedia. actual tangible spoon is used to represent virtual spoon in an animated movie. a one-day case study on six preschoolers aged six from a kindergarten in Kuala Lumpur has been conducted (Chau. 5 Making digital multimedia objects tangible 4.Chau. such as animals and household items placed randomly in front of computer (Figure 6). Unstructured observation and questionnaires were utilized. . & Zarina / Procedia . For this reason. Toh. Tangible multimedia in our context is that tangible objects complement digital multimedia objects for the purpose of making the multimedia contents tangible.

The study revealed that 4 participants rated TangiLearn “enjoyed very much” and 3 participants rated that the quiz was very easy. By average. they were very curious about the markers. Problems encountered above prompted us to explore alternative technology for implementing the tangible-multimedia binding. they were not deployed in the case study. For the purpose of reducing the need of precise alignment. Considering the choice of technology should rest on its usefulness to the students as learning aids. the participants could not move the tangible objects too close or too far from the camera. & Zarina / Procedia . The most obvious problem was related to the issue of physical alignment of markers. and align it to the webcam to play corresponding animations and videos about the object on the computer screen. RFID tag is inserted into a tangible object. 6 Randomly-placed learning objects in TangiLearn using QR marker in case study The participants were required to grasp a tangible object. Upon completion of learning session. All these problems have to be addressed in the full-scale experimental research. they took 35 seconds to get the visual marker recognized. They grasped the tangible objects firmly and spent some time tinkered with them. the chip in the tag will transmit the stored information to the reader. were attributed to iterative tangibility sense of learning experiences. the participants would need to answer the quiz by identifying and picking up the correct tangible object. the learning activities commenced. we noticed a more natural form of learning between the participants and multimedia. we plan to deploy RFID and sensors technology in the final experimental research. Because of its cost. Most of the participants have difficulties in orientating the visual markers to the camera precisely. To do this. We believed that the good performance of the participants in quiz. .Social and Behavioral Sciences 00 (2012) 000–000 Fig. Toh. The markers were huge. RFID tags and reader are deployed for objects recognition. thus they tended to tear them down for fun. When the tangible object is moved towards the radio wave field generated by a compatible reader. which require recitation of key terms learned in the system. Besides. thereby establishing mutual communication that allows the computer to identify the object (Figure 7).8 Chau. With this. and thereby blocked some parts of the tangible objects. Through observation. We also observed that using such markers. A few technical problems arose during the case study.

A. The more the preschooler grasps and presses the sensor via the tangible objects. Fig.. such as movie clip moves further in the learning scene. a type of sensor called force sensor is utilized. In the case study. .Social and Behavioral Sciences 00 (2012) 000–000 9 Fig. 63. Taking the strength of physical sensation. With excessive use of RFID and sensors technology. slider. A. 7 Implementation of RFID technology To further enhance the tactile attribute of digital objects in physical space. & Ha. Burnett. & DiRusso. A. Proceedings from IDC ’09: The 8th International Conference on Interaction Design and Children. is designed. W. M. M. Hands on what?: Comparing children’s mouse-based and tangible-based Interaction.. 80-88.g. wii mote controller) in TangiLearn. Conclusion In this paper. D. (1962). References Alibali. a manifestation of tangible multimedia. (2009).Chau. it is expected that the aforesaid learning benefits of tangible objects in multimedia realm can be garnered in TangiLearn. N. & Zarina / Procedia . A. from QR code to RFID technology. TangiLearn. we discuss how tangible objects are beneficial to preschoolers by way of literature review. Cognitive Development. 7177. Antle. Toh. Droumeva. The Elementary School Journal.. We also plan to use various types of sensors and controller (e. 8 Implementation of force sensor technology 5. The function of gesture in learning to count: More than keeping track. The force sensor is designed in a way that the sensor is glued on the tangible objects. the more the digital multimedia objects in TangiLearn will react. Evidence of improving self-report of enjoyment and learning outcomes in the case study reflect the need of such system for preschoolers. (1999). Montessori education today and yesterday. we were also informed of the change required for the technical implementation of the TangiLearn system. 37-56. 14(1).

. T. & Jones. 140-149. Penang. UK: Lawrence Earlbaum Associates. Kuantan. Childhood Social Development: Contemporary Perspectives . Inexpensive Immersive Environment. Nature Neurosci. 2012. Proceedings of CHI ’07 (2007). 513-532. Educational Psychology: An International Journal of Experimental Educational Psychology. (2002). Ishii. T.. C. Amsterdam. G.. 207-231. October 16-19. & Pattison. K. Proceedings of the International Conference for Academic Research 2012. M. S. (2009). Reeve. C.. K. Klemmer. Direct-touch vs. Hughes. How bodies matter: Five themes for Interaction Design. R. (1999). Proceedings from CHI’97: The SIGCHI Conference on Human Factors in Computing Systems. Young children’s understanding of addition concepts. Department of Computer Science and Statistics. B. Taoyuan. K. 2011. Jetsu I. Chao. (2008). Hole.. J. 647-656. H. Singer. & Osawa. Proceedings from Immerscom ’09: The 2nd International Conference on Immersive Telecommunications . Hatano. R. Finland. East Sussex. Chen C. (1997). Oxford: Blackwell Publishers. Unpublished doctoral dissertation. & Takayama. K. development and evaluation of a virtual reality (VR)-based learning environment: Its efficacy in novice car driver instruction. Proceedings from TEI’08: The Second International Conference on Tangible and Embedded Interaction. J. & Goldin-Meadow. S. Haxby. Toh. Tangible bits: Towards seamless interfaces between people. bits and atoms. P. 95-110. C. Cognition. University of Joensuu. C. Breaking the sound barrier: Designing an interactive tool for language acquisition in preschool deaf children. Proceedings from DIS ’06: The 6th Conference on Designing Interactive Systems.. 20 (1). (2002). K.. (2007). S.. L. (2008). Malaysia. Digit memory of grand experts in abacus-derived mental calculation. Proceedings of the 5th International Malaysian Educational Technology Convention (IMETC 2011). & Zarina / Procedia . 16. (2012b). V. J. C... Toh. Tangible multimedia: A case study for bringing tangibility into multimedia learning. D. (1983). & Zarina Samsudin. Chau. Children and number.. Wigdor. Kuala Lumpur. & Ullmer. Toh. Chau. 11 – 13. & Balakrishnan. K. 15(1-3). The design. Proceedings from IDC ’08: The 7th International Conference on Interaction Design and Children. F.. M. Chau. Chau. L. Shen. Attribute-based neural substrates in posterior temporal cortex for perceiving and knowing about objects. Hummels. Toh. M. 2012. & Schull J. A helping hand in assessing children's knowledge: Instructing adults to attend to gesture. (2012a). Proceedings of the International Educational Technology (IETC 2012).. & Overbeeke..Social and Behavioral Sciences 00 (2012) 000–000 . A. 234–241. 22(5).. R. Hartmann. Hengeveld. 159-166. & Zarina Samsudin. Netherlands: Elsevier Science. E. Malaysia June 8 – 11. (2011). K. Taiwan. Let Me Actuate You. Hicks. L.. J. Tangible multimedia technology: A research proposal for bringing tangibility into multimedia learning amongst preschool children. (2005). B. 1-26. Master Dissertation.. & Zarina Samsudin. Spreen. & Martin. Crook. Cultural Artefacts in social development: The case of computers. S. A. T. S. (1986). H.. (Eds). 210-216. Cognition and Instruction.. 913–919. Forlines. Smith. C. (2006). K. H. Huang.. Kelly. D. (2008). Universiti Sains Malaysia. Designing tangible multimedia for preschool children based on multimedia design theories. S. mouse input for tabletop displays.. Tangible User Interfaces and Programming.10 Canobi. J. (1992).. B. C.. In McGurk.

J. Kishino. Cambridge. Piaget. American Journal of Psychology 117(3). Wimmer. Tangible user interface for children: An overview [Technical report].. Norton. S. Chin Da. MA: MIT Press. human and spatiality. C. Piaget. New York. J. & Casey. 159-164. Chau.Social and Behavioral Sciences 00 (2012) 000–000 11 Marco. On tangible user interfaces. & Zarina / Procedia .org.C. Toh. & Fraser D. 77-84. Situating constructionism. Proceedings from TEI ’09: The 3rd International Conference on Tangible and Embedded Interaction.. Universiti Sains Malaysia. Watson.. L.. The effect of physical manipulation on children’s numerical strategies: Evaluating the potential for tangible technology. (1952). Bristol. Morsella. (2004). (2011). [Online]. Vickneasvari A/P Krishnasamy. New York: W. Preston. (2009). USA: University Press. S. S. (1968). E. & Read. International Journal of Humanities and Social Science. Piaget. System blocks: A physical interface for system dynamics learning. The construction of reality in the child. . J. Engagement and Expectations: Measuring Children's Fun. (1962). USA: ACM Press. The origins of intelligence in children. D.. (2011). J. (2007). The principles of genetic epistemology.346. (2000). F.. (2004).. Sharlin. MacFarlane. J. Improving Listening Comprehension among Malay Preschool Children Using Digital Stories. S.. 338 .. A. Xu. R. K. Grasp sensing for human-computer interaction. Proceedings of the 21st International System Dynamics Conference .pdf Papert. B. ACM Press. Y. In Papert & Harel (Eds. UK. The role of gestures in spatial working memory and speech. p. Master dissertation. Piaget. Z. Proceedings of DARE 2000 on Designing Augmented Reality Environments. Read. J. Personal and Ubiquitous Computing (8) 5. Patten. (2005). 103-111. New York: Basic Books. dreams. (2002). & Itoh. Majid Pour-Mohammadi. (1972). (2004). M. & Resnick M.. J. 810-811. A comparison of spatial organization strategies in graphical and tangible user interfaces. Available: http://www. Futurelab Series Report 12: Literature review in learning with tangible technologies. Souriyavongsa. 105 Mohamad Jafre Zainol Abidin. Unpublished doctoral dissertation. (2003). A. and Embodied Interaction. W. Cerezo. (2008). Comparing children's enjoyment and engagement using C. E. United Kingdom: NESTA Futurelab. 221-228. Manches. (1991). C. Baldassarri. The effects of a multimedia constructivist environment on students’ achievement and motivation in the learning of chemistry. Simon Fraser University. Canada. graphical and tangible user interfaces.Manches. Bringing tabletop technologies to kindergarten children. 411. Y. Endurability.futurelab.. Physical manipulation: Evaluating the potential for tangible designs. T. Proceedings from TEI ’11: The Fifth International Conference on Tangible. Proceedings from IDC'02. E. 41-50. C. Mazzone. (2010). & Ishii. R. S. J.424. O. (2009). UK: University of Central Lancashire.). Embedded. and imitation in childhood. School of Interactive Arts and Technology. & Krauss. Unpublished doctoral dissertation. H. Constructionism. Play. 1(14). Zuckerman. O'Malley. & Benford. Xie. HCI 2009 – People and Computers XXIII. J. O’Malley.. Penang.. & Ong. E. London: Routledge & Kegan Paul Ltd. Kitamura. University of Nottingham. UK: British Computer Society.